Moving to modern identity management systems can seem like a daunting task if you have an existing legacy user database to migrate from, moreso when the new one doesn’t provide any out-of-the-box solutions for doing so. Thankfully, with Keycloak, it’s just a matter of writing an adapter.
High-level overview
I have written such an adapter, called a User Federation Provider, and published it here:
https://github.com/daniel-frak/keycloak-user-migration
It is heavily inspired by the solution proposed by Smartling in their article:
https://tech.smartling.com/migrate-to-keycloak-with-zero-downtime-8dcab9e7cb2c
Unfortunately, their provider was written using a deprecated API which is no longer supported in new versions of Keycloak, which is why I was forced to write my own.
My provider expects a single REST URI with GET and POST endpoints, for providing user information and verifying passwords. When a user tries to log into Keycloak and isn’t found in its local database, calls will be made to those endpoints to authenticate him in the legacy system and migrate his data. This process will be completely invisible to the user.
The downside of this solution is that both authentication systems will have to be deployed at the same time, until every user has logged in at least once. Unfortunately, there is no other way to achieve our goal, assuming that we are responsibly encrypting our users’ passwords. Thankfully, though, on the application-side we only ever have to deal with Keycloak, so we can just conveniently forget about our old system after we enable the provider.
Basic migration
In most cases, you might not have to modify the provider code at all. All you need to do is supply a REST service which verifies passwords and supplies user information from the legacy system. That service can either be part of the legacy system or a separate application acting as a facade for it. When you enable the provider in Keycloak, you just need to input the URI of that service and everything should start working automatically.
Information regarding the REST service specifics and basic configuration of the provider can be found in the repository’s README.md.
Provider code explained
Depending on your needs, you might need to modify the provider code to a greater or lesser extent. In this section I will do my best to explain the structure and logic of its most important classes, to give you a better understanding of how it works under the hood.
The LegacyProvider class
This is the main provider class. It implements UserStorageProvider, UserLookupProvider and CredentialInputValidator to provide user migration features. You should not have to modify it but if you do, it’s best to familiarize yourself with these interfaces.
The ConfigurationProperties class
This class, as the name suggests, stores the properties which are configurable via the Keycloak admin interface. This is the place to go if you want to extend the provider’s flexibility.
The LegacyUserService interface and its implementation
This interface defines the API for connecting to an external legacy authentication system. It is implemented by RestUserService. If a REST service is not a good solution for you, you could easily write your own implementation of this service, e.g. one which would communicate with your legacy system via SOAP.
If REST is good enough for you, but the endpoints don’t quite fit, consider modifying the RestUserService class, instead.
how do i run this without a docker, do i need make changes in the keycloak source files as stated ?
I am using version 9.0 with JDK 1.8.
Hi Sam,
To run this without Docker, build the .jar and copy it to
/opt/jboss/keycloak/standalone/deployments(or wherever your Keycloak instance is installed).Keycloak should recognize it automatically.
The Dockerfile should be fairly self-explanatory in terms of the operations performed on Keycloak, but let me know if you need further help:
https://github.com/daniel-frak/keycloak-user-migration/blob/master/docker/keycloak/Dockerfile
If you don’t have Maven installed, you can use the provided wrapper, using this command:
./mvnw clean package && mv target/*.jar /opt/jboss/keycloak/standalone/deployments/app.jarThis should build the project and move the file to your Keycloak instance.
All in all, though, installing this in your Keycloak instance should just be matter of copying a file 🙂
Thanks it works 🙂
im deploying in the jboss 19.0.3 standalone. After generate jar file and move to deployments folder when i running jboss throws :
Failed to define class com.danielfrak.code.keycloak.providers.rest.LegacyProviderFactory in Module “deployment.keycloak-rest-provider-2.0.1-SNAPSHOT.jar” from Service Module Loader: java.lang.NoClassDefFoundError: Failed to link com/danielfrak/code/keycloak/providers/rest/LegacyProviderFactory (Module “deployment.keycloak-rest-provider-2.0.1-SNAPSHOT.jar” from Service Module Loader): org/keycloak/storage/UserStorageProviderFactory
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1090)
at org.jboss.modules.ModuleClassLoader.doDefineOrLoadClass(ModuleClassLoader.java:351)
at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:482)
at org.jboss.modules.ModuleClassLoader.loadClassLocal(ModuleClassLoader.java:276)
at org.jboss.modules.ModuleClassLoader$1.loadClassLocal(ModuleClassLoader.java:79)
at org.jboss.modules.Module.loadModuleClass(Module.java:765)
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:192)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:410)
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:116)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1217)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1228)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
at org.keycloak.keycloak-services@19.0.3//org.keycloak.provider.DefaultProviderLoader.load(DefaultProviderLoader.java:60)
at org.keycloak.keycloak-services@19.0.3//org.keycloak.provider.ProviderManager.load(ProviderManager.java:94)
at org.keycloak.keycloak-services@19.0.3//org.keycloak.services.DefaultKeycloakSessionFactory.loadFactories(DefaultKeycloakSessionFactory.java:294)
at org.keycloak.keycloak-services@19.0.3//org.keycloak.services.DefaultKeycloakSessionFactory.init(DefaultKeycloakSessionFactory.java:110)
at org.keycloak.keycloak-services@19.0.3//org.keycloak.services.resources.KeycloakApplication.createSessionFactory(KeycloakApplication.java:232)
at org.keycloak.keycloak-services@19.0.3//org.keycloak.services.resources.KeycloakApplication.startup(KeycloakApplication.java:125)
at org.keycloak.keycloak-wildfly-extensions@19.0.3//org.keycloak.provider.wildfly.WildflyPlatform.onStartup(WildflyPlatform.java:47)
at org.keycloak.keycloak-services@19.0.3//org.keycloak.services.resources.KeycloakApplication.(KeycloakApplication.java:115)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.core.ConstructorInjectorImpl.constructOutsideRequest(ConstructorInjectorImpl.java:225)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:209)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.core.providerfactory.Utils.createProviderInstance(Utils.java:102)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.core.providerfactory.ResteasyProviderFactoryImpl.createProviderInstance(ResteasyProviderFactoryImpl.java:1385)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.core.ResteasyDeploymentImpl.createApplication(ResteasyDeploymentImpl.java:418)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.core.ResteasyDeploymentImpl.initializeObjects(ResteasyDeploymentImpl.java:265)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.core.ResteasyDeploymentImpl.startInternal(ResteasyDeploymentImpl.java:137)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.core.ResteasyDeploymentImpl.start(ResteasyDeploymentImpl.java:121)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:144)
at org.jboss.resteasy.resteasy-core@4.7.6.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:42)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:117)
at org.wildfly.security.elytron-web.undertow-server-servlet@1.10.1.Final//org.wildfly.elytron.web.undertow.server.servlet.RunAsLifecycleInterceptor.doIt(RunAsLifecycleInterceptor.java:70)
at org.wildfly.security.elytron-web.undertow-server-servlet@1.10.1.Final//org.wildfly.elytron.web.undertow.server.servlet.RunAsLifecycleInterceptor.init(RunAsLifecycleInterceptor.java:76)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:103)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.start(ManagedServlet.java:309)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.ManagedServlet.createServlet(ManagedServlet.java:145)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:588)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:559)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
at org.wildfly.extension.undertow@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
at org.wildfly.extension.undertow@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
at org.wildfly.extension.undertow@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
at io.undertow.servlet@2.2.17.Final//io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:601)
at org.wildfly.extension.undertow@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:106)
at org.wildfly.extension.undertow@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:87)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.base/java.lang.Thread.run(Thread.java:833)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.JBossThread.run(JBossThread.java:513)
11:39:10,367 FATAL [org.keycloak.services] (ServerService Thread Pool — 59) Error during startup: java.lang.NoClassDefFoundError: Failed to link com/danielfrak/code/keycloak/providers/rest/LegacyProviderFactory (Module “deployment.keycloak-rest-provider-2.0.1-SNAPSHOT.jar” from Service Module Loader): org/keycloak/storage/UserStorageProviderFactory
Hi, if you have found bugs in the code, please create issues on the project’s Github page.
Remember to follow the contributing guidelines:
https://github.com/daniel-frak/keycloak-user-migration/blob/master/CONTRIBUTING.md#write-bug-reports-with-detail-background-and-sample-code
Thanks!! Daniel for pointing me in the right direction, I was able to deploy successfully using the JAR and will let you know how things will go on migration side,
And Daniel, could i use your adapter code in the production environment?
Yes, of course, you’re free to use and modify it however you want 🙂
I encourage you to create issues on the project’s Github repository if you encounter bugs and submit Pull Requests if you want to contribute to the adapter’s public code but that is all optional and even if you do modify the code, you don’t have to share it with anyone.
.
Hi Daniel, i tried to migrate the user by providing the user-endpoints in my legacy system(working when checked by not integrating with keycloak) and the uri in the user federation in keycloak.
When i try to login using the legacy user credentials i get error as, User not found in external repository.
However,for new users I am able to login successfully but not the legacy users.
Hi Sam,
The error comes from this place in the code:
https://github.com/daniel-frak/keycloak-user-migration/blob/2b216b9e45314979cd5c7e243f7add556b9fd7f8/src/main/java/com/danielfrak/code/keycloak/providers/rest/LegacyProvider.java#L48
It seems that the
legacyUserService.findByUsername(username)method call is not returning the user. The most probable cause is that your endpoint is not returning the user information along with a 200 HTTP status. Perhaps the endpoint is not getting hit at all or maybe it can’t read the username properly from the payload? My advice to you is to debug the endpoints in the legacy system, check:1) If you’re getting requests at all
2) If you’re properly retrieving the data
3) If you’re sending back the data in the correct format, with the correct HTTP status.
Hope that helps!
Hi Daniel Frąk,
I have setup in local env and it shows :
.keycloak.providers.rest.LegacyProvider] (default task-10) User not found in external repository: lucy
and the http://127.0.0.1:8080/user-migration-support/lucy shows user profile but
http://127.0.0.1:8080/user-migration-support/ show:-
There was an unexpected error (type=Not Found, status=404).
Please help
i am facing same issue. how to resolve it
Ok Thanks .. i will recheck.
I know where the error is coming from the code which you highlighted.
currently my app is hosted on localhost//8080 and endpoints localhost/8080/* will be routed to keycloak authentication server like localhost/8180/auth once user is authenticated the requested callback uri is called. These flows all happen perfectly for local keycloak user.
My query is how will the provider classs know my legacy db url to check the username ,email and my hashed password ?
if i request the endpoint by looging in as a local keycloak user then if i check the endpoints are accessible. but if i provide legacy credentials these do not pass.
However i will go through the check points which you have stated.
Don’t take this the wrong way but you might have slightly misunderstood how the migration mechanism works. Please make sure to carefully read the README.md file on Github, especially the
Prerequisites - REST endpoints in the legacy systemsection.Regarding your question, the provider class does not need to know your legacy URL db. Instead, you must provide endpoints custom made for the provider in your legacy system. The first call the provider will make will ask for user data based on the user’s username/e-mail. The second call will send the plaintext password to your legacy system and the burden of hashing and verifying that password against the database is on the legacy system.
Let me know if this helps!
Hi Daniel
Thanks !! for your inputs, I was able to successfully migrate the users from legacy DB to keycloak user store and it is working as i needed it,
Hi Daniel,
Really awesome example!
Could you please tell me how can I use your User Federation Provider without storing the users in the keycloak’s database?
Thank you!
The “meat and potatoes” of storing users is in
LegacyProvider::isValid:https://github.com/daniel-frak/keycloak-user-migration/blob/737ab67a780ea2cd97146682a23ee57705e17ff9/src/main/java/com/danielfrak/code/keycloak/providers/rest/LegacyProvider.java#L64
Here, the
userCredentialManagerstores user information and the federation link is severed.Once you remove that, you might (although not necessarily) also have to modify the
LegacyProvider::getUserModelmethod:https://github.com/daniel-frak/keycloak-user-migration/blob/737ab67a780ea2cd97146682a23ee57705e17ff9/src/main/java/com/danielfrak/code/keycloak/providers/rest/LegacyProvider.java#L44
I’m mentioning this since I’m not sure how
userModelFactory.create()will behave on repeat logins. It might work fine, but you should nonetheless check it.I haven’t tested it, but I think that should work. Let me know if you manage to get it working!
Hello Daniel,
I tried to migrate users on our dev environment and it ran successfully, I tried to give random input in my username/password field to purposefully fail it after providing the details I get 400(bad req)response with a message on the webpage as “We are sorry…
Unexpected error when handling authentication request to identity provider.”
Logs
WARN [org.keycloak.services] (default task-18) KC-SERVICES0013: Failed authentication: javax.ws.rs.ProcessingException: com.fasterxml.jackson.core.JsonParseException: Unrecognized token ‘User’: was expecting (‘true’, ‘false’ or ‘null’)
at [Source: (org.jboss.resteasy.specimpl.AbstractBuiltResponse$InputStreamWrapper); line: 1, column: 6]
at org.jboss.resteasy.resteasy-jaxrs@3.9.1.Final//org.jboss.resteasy.client.jaxrs.internal.ClientResponse.readFrom(ClientResponse.java:245)
at org.jboss.resteasy.resteasy-jaxrs@3.9.1.Final//org.jboss.resteasy.specimpl.BuiltResponse.readEntity(BuiltResponse.java:88)
if I disable user federation for restapi then it invalidates user correctly.
Any clue where the code must handle this exception or i need to write custom code to handle this error.
Thanks!!!
Is this the entire stack trace?
This kind of looks like your legacy REST API is returning an incorrect JSON, but I’m not sure.
The only place that comes to mind where a JsonParseException could happen is RestUserService.
Thanks!!
It was in the legacy Rest API as it needed the fix.
I am not getting the previous error as I stated.
Hi,
Thank you for the example. I’ve been trying it and it works fine for users loggin in through the Keycloack login page when users type their username and password. But I would need that Apps or other web which use secured services would be able to log in and their accounts be migrated. Would this solution work for the described scenario?
Thanks again!
I’m not sure if I understand what you want to achieve, but perhaps this explanation will help:
The best course of action is for you to rewrite the authentication code of your apps to fully use Keycloak as their CAS. Your old authentication system would then be kept running in the background and connected only to Keycloak via this migration plugin.
Basically abandon your old CAS in your apps in favor of Keycloak and then attach it to Keycloak via the plugin.
E.g. clicking “log in” in your app would temporarily redirect to Keycloak, which would then get users from the old authentication system and redirect the user back to the app.
If this doesn’t help, I’m afraid you have to describe the problem in more detail.
Hi Daniel,
Thanks for yout reply. As English is not my natural language I think I didn’t explain the case I had in mind. The scenario I tryied to describe is the following:
If I have an authenticated webservice (a commont wsdl or rest) in the entreprise app with Keycloak configured as their CAS, and other systems (servers) consume it with a basic authenticantion (user/pass in headers on the http call) strategy, would this user (which is registered in the legacy authentication system, and whose information can be retrieved with the rest endpoints I have implemented for the migration plugin) be migrated to keycloak? Or just the user using the keycloak login page (which is propmpted to human user when they access the system) are migrated by the plugin? (this case is already working fine for us).
Thanks,
Salvador.
I hope I have explained my case.
I’m fairly sure that every user who authenticates with Keycloak (be it via a login page or REST endpoint) will be authenticated through the migration plugin. So if you have configured Keycloak to authenticate using Basic Auth, for example, that should work as well 🙂
Thank you! I will try …
Hi Daniel,
Thank you for the article.
I have a spring-boot application that is being used as an API for mobile and web applications that are using oauth tokens (access and refresh). As a consequence the users of the mobile applications may not sign in often if at all.
This means migrating may be an issue as there isn’t the opportunity to capture and store their passwords. Their passwords are currently all stored using bcrypt.
All the examples I read show migration of users by having them sign in. Do you know if there is a way I can migrate the users without having them sign in?
Hi Julius,
While it may be possible, I think the easiest solution would be to just force a re-login for all users, which shouldn’t be too difficult. It’s a minor inconvenience, but could spare a lot of headache.
What you’re attempting might be possible by configuring your app to accept tokens from both sources (old CAS and Keycloak) but then you would somehow have to migrate the users – and while you could do that, the password issue still persists. Either you would still have to keep the old system around for “eventual migration”, or you would have to configure Keycloak to store passwords in exactly the same way as your previous CAS did, which is probably unadvisable (I imagine Keycloak uses more than just bcrypt for security).
Can you please share a sample REST server also?
I’m trying to avoid this, as every language would have a different implementation. Is there anything in the README.md file that is unclear?
If the federated datastore is unavailable, then is there caching at all?
I’m not sure I understand the question. The plugin migrates the user data as soon as the log-in operation is successful – from that moment on Keycloak will not rely on the legacy datastore to authenticate that user.
Can we update the local keycloak db when synchronizing itself? Is there no API to update the local storage? why is this implemented only in isValid()?
thankyou
Again, I’m not sure I uderstand. Could you rephrase the question?
I use both Identity Provider and User Federation. Identity Provider is used to authenticate user through Facebook.
User Federation is used to authenticate user using username and password.
Is it possible to forbid User Federation authenticate, when user authenticates through Identity Provider?
We’ve implemented this plug in and played with it afterwords, so I do not think that User federation could NOT be used when you have external IdPs. However, you can play a little bit with the getUserByUsername(), getUserByEmail(), and isValid() so they could be skipped when exchange token is in fact implemented. However, IMO creation of the user in the Keyclaok’s db is essential because it is good to be linked to the IdP and later on you can do the exchange of the Keycloak token to the external IdP token as well l(if you think that you might need it, in fact)
Hi Daniel,
I just wanted to leave a ‘thank you’ note here. I adapted your your approach for our Keycloak PoC and it worked really well. Thank you for sharing your solution with this neat walkthrough.
It’s always great to hear that I helped someone 🙂
Hi Daniel,
Thanks a lot for the great plugin, it’s working fine for migrating my legacy user database.
Just a quick question that may benefit others as well. In case I want to log in with user_email instead of username, is there anything I need to change in your plugin?
I’m enabling email login on Keycloak and it works fine for Keycloak emails. But when I enter valid email and password from the legacy database, with the due adaptations on the REST service, it does migrate the user to KC database, but shows: Invalid username or password.
Is there a quick fix for this?
Thanks a lot once again, Daniel.
Hi!
It’s probably best to post issues like this on the Github repository’s Issue page so they get more visibility.
Nevertheless, you might want to poke around
RestUserService::findByEmailwhich seems to incorrectly try to find the user by username (instead of by e-mail). If that’s the problem you’re having, don’t hesitate to create a new issue on Github and I encourage you to create a Pull Request for it, as well 🙂Hi Daniel,
Migration works fine. After password validation keycloak tries to create a user in the database at that time I was experiencing an error from hibernate.
Caused by: org.hibernate.exception.GenericJDBCException: could not prepare statement
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:47)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:186)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareStatement(StatementPreparerImpl.java:81)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3173)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3706)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:90)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:604)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:478)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:356)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1472)
keycloak_1 | … 114 more
keycloak_1 | Caused by: java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@3723495e
keycloak_1 | at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.tryLock(BaseWrapperManagedConnection.java:405)
keycloak_1 | at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.WrappedConnection.lock(WrappedConnection.java:168)
keycloak_1 | at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.WrappedConnection.prepareStatement(WrappedConnection.java:471)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.engine.jdbc.internal.StatementPreparerImpl$1.doPrepare(StatementPreparerImpl.java:90)
keycloak_1 | at org.hibernate@5.3.20.Final//org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:176)
keycloak_1 | … 123 more
Can you let me know what may cause the issue? Any solution for it.
Thanks
Hi!
It’s hard to tell why you might be getting JDBC errors. I would suspect there might be something wrong with the input values (username, password etc.).
Could you create an issue on the plugin’s Github page and perhaps provide the full stack trace and some more info on what data is being processed?
we are trying your solution to migrate from a legacy Identity Management and got the below stack trace error if you can help
I encourage you to create a Github issue explaining the problem:
https://github.com/daniel-frak/keycloak-user-migration/issues
I have a vast monolith application that includes the backend and frontend all together, and also that keeps information in a relational database and all the user role dependencies and manages autharizatıon via sprıng security in a role-based approach.
I need to implement authentication via using keycloak to communicate with the legacy system and import the users from there, can I use user storage spı implementation in the monolith?
But in that case, when I build the jar file, it creates the whole monolith jar file in a single piece, and it does not work for me. Do I need to do this as a separate project? and buıld rest apıs for talking wıth the monolıth applıcatıon?
How could I add this jar to the Kubernetes keycloak instance?
Thanks.
Hi! Yes, it should be possible to use the plugin with your application, monolith or not. You just need to expose some endpoints in your monolith (see: https://github.com/daniel-frak/keycloak-user-migration#prerequisites—rest-endpoints-in-the-legacy-system). What happens behind the scenes is of no concern to the plugin, as long as you stick to the required API. For further reference, here is an example implementation of the API as a Spring Boot controller:
https://github.com/daniel-frak/keycloak-user-migration/blob/master/docker/legacy-system-example/src/main/java/dev/codesoapbox/legacysystemexample/authentication/presentation/controllers/UserMigrationController.java
As for installing the plugin, you need to put it in the
providersfolder in your Keycloak’s root folder. See this Dockerfile for an example:https://github.com/daniel-frak/keycloak-user-migration/blob/master/docker/keycloak/Dockerfile
Hello,
There are two issues regarding legacy-system-example ;
Although I am updating the repository users in the file and restarted the docker instance it still keeps seeing the old jar. mvn clean install manually does not works as well. How could I replace the legacy-example jar in the docker?
2) I have extracted my endpoints in my legacy system and when I test them individually it works, then I define the endpoint in keycloak instance as in the REST URI part. But when I try the login with the described user it gets error as “User not found in external repository: %s”, why is it hitting the getUserModel method from LegacyProvider first?
And why couldn’t find the user, also when I add new logs to the spi code it still does not replace the old jar so I can’t find where the problem is. Thanks
Hello in terms of logging, is there a specific reason that you picked this logging method?
I came across different usage methods as well. like as below.
Apache.commons.logging
lombok.extern.slf4j.Slf4j
org.jboss.logging.Logger usage
So is there a specific reason you have proceeded with this project? or what would be the advantages of these approaches each other?
Thanks.
Sorry for the (very) late reply but:
– IIRC I used
org.jboss.logging.Loggerin the plugin code because it’s what I saw used in other plugins and I tried to replicate what is used by Keycloak.– I used Slf4j in the example project because I find it to be a good abstraction over major logging libraries.
I have created custom end point and always getting
keycloak_1 | 2023-02-07 12:32:03,419 WARN [org.keycloak.services] (executor-thread-2) KC-SERVICES0013: Failed authentication: com.danielfrak.code.keycloak.providers.rest.exceptions.RestUserProviderException: com.danielfrak.code.keycloak.providers.rest.rest.http.HttpRequestException: An error occurred while making a HTTP request: GET http://localhost:3030/auth/bob HTTP/1.1
Hi Daniel. Awesome plugin. Can you please me deploy the jar to keycloak 20.3 that I am running. I built the jar and copied it to providers folder but still when I run keycloak I don’t see it as a provider in ‘User Federation’. Am I missing something?
What’s this error I am getting after entering credentials?
Failed authentication: com.danielfrak.code.keycloak.providers.rest.exceptions.RestUserProviderException: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field “password” (class com.danielfrak.code.keycloak.providers.rest.remote.LegacyUser), not marked as ignorable (11 known properties: “lastName”, “groups”, “username”, “attributes”, “id”, “email”, “roles”, “requiredActions”, “enabled”, “emailVerified”, “firstName”])
at [Source: (String)”{“id”:1000,”username”:”johndoe”,”password”:”mypassword”,”email”:”johndoe@example.com”,”firstName”:”John”,”lastName”:”Doe”,”enabled”:true,”emailVerified”:false}”; line: 1, column: 45] (through reference chain: com.danielfrak.code.keycloak.providers.rest.remote.LegacyUser[“password”])
After use is migrated from external ssql db, I get this error during first login:
The given key is not a valid key per specification, future migration might fail: 1000
Hi, if you have found bugs in the code, please create issues on the project’s Github page.
Remember to follow the contributing guidelines:
https://github.com/daniel-frak/keycloak-user-migration/blob/master/CONTRIBUTING.md#write-bug-reports-with-detail-background-and-sample-code
Hi Everyone,
I am not able to get the Option “User migration using a Rest client” in the Keycloak version 21.0.0, instead I only can see Kerberos and Ldap cards as option. Can anyone please help ?
Hi, make sure you use a version compatible with your Keycloak (in this case: 1.0.0). You can see the compatibility matrix on the Github page:
https://github.com/daniel-frak/keycloak-user-migration
There is two main error on this code ;
First and most important one ; At the moment of the fist migration if the user enters his/her password user, then the user still created in the keycloak side because it only triggers LegacyProvider::getUserModel then if you want to enter the right password it does not accept because the password set as null tot he keycloak, you cant delete the user becuase the user created brokenly, in order to overcome this it is needed to define an email and forgot password and resetting the password, but then it is not needed to define an existing user u u can create any user name via this way (forgot password).
So it is crucial to check password before migrating the user
Could you please help with this fiy urgently , how can I call isPasswordValid inside LegacyProvider::getUserModel
Hi, if you have found bugs in the code, please create issues on the project’s Github page.
Remember to follow the contributing guidelines:
https://github.com/daniel-frak/keycloak-user-migration/blob/master/CONTRIBUTING.md#write-bug-reports-with-detail-background-and-sample-code
I want to use your plugin for the legacy user base, I would like to know if during the registration he can also validate the username and email in the legacy base along with the keycloak. So that when the user registers, he will consult the keycloak and legacy base. If he doesn’t do that, do you have any indication for that?
Hi Daniel,
Thanks to this exccelent example …
But you or anyone who reads this article has faced a problem like below :
i coppied jar file and execute
kc.sh -v build
and get below error..
Thanks
Updating the configuration and installing your custom providers, if any. Please wait.
The DelayedHandler was closed before any children handlers were configured. Messages will be written to stderr.
2023-11-30 11:52:53,945 DEBUG [org.jboss.logging] (main) Logging Provider: org.jboss.logging.JBossLogManagerProvider
ERROR: Failed to run ‘build’ command.
Error details:
java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doReaugment(QuarkusEntryPoint.java:84)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:48)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)
at org.keycloak.quarkus.runtime.cli.command.Build.run(Build.java:81)
at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
at picocli.CommandLine.execute(CommandLine.java:2170)
at org.keycloak.quarkus.runtime.cli.Picocli.parseAndRun(Picocli.java:100)
at org.keycloak.quarkus.runtime.KeycloakMain.main(KeycloakMain.java:88)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:61)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)
Caused by: java.nio.file.NoSuchFileException: D:\KeyCloak\keycloak-22.0.5\lib\quarkus\build-system.properties
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:236)
at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:422)
at java.base/java.nio.file.Files.newInputStream(Files.java:160)
at io.quarkus.deployment.mutability.ReaugmentTask.main(ReaugmentTask.java:35)
… 24 more
i just downloaded latest version 23.0.1 and that error fixed…
I had found your code previously, and the idea is good. The problem I’m having tho is that the legacy system doesn’t just use username/password, but also PKI keys to do authentication. How would I go about doing that?