8000 Create the changelog table in the manual migration export for an empty database by Alnyli07 · Pull Request #51730 · keycloak/keycloak · GitHub
[go: up one dir, main page]

Skip to content

Create the changelog table in the manual migration export for an empty database - #51730

Merged
ahus1 merged 1 commit into
keycloak:mainfrom
Alnyli07:fix/manual-export-changelog-table-ddl
Sep 23, 2026
Merged

ahus1 merged 1 commit into
keycloak:mainfrom
Alnyli07:fix/manual-export-changelog-table-ddl

Conversation

@Alnyli07
@Alnyli07 Alnyli07 commented Aug 14, 2026 •
Copy link
Copy Markdown
Contributor

Closes #51729

What

With migration-strategy=manual against an empty database, the generated script offers itself as a creation
script for an empty database but inserts into DATABASECHANGELOG without creating it, so it cannot be applied.

Liquibase writes that DDL into the script only while the table does not exist. Since Liquibase 4.33 parsing a
changelog initializes the change log history service (DatabaseChangeLog), which cre 8000 ates the table on the live
connection, so by the time the export is written Liquibase no longer emits it.

This restores the guarantee added in #32535 / PR #37204, together with the two assertions that were dropped from
BasicDatabaseTest in #41299 / PR #41490.

How

outputChangeLogTableCreationScript emits CreateDatabaseChangeLogTableStatement explicitly, guarded on two
conditions:

  • the table did not exist before this server started — an administrator may have created the bookkeeping
    tables up front so a user without DDL rights can produce the script at all, and in that case the target
    database already has the table and the DDL would break the script with already exists. This is the case
    PR Invalid migration export for empty database #37204 was originally fixing, so the guard is what keeps that fix intact;
  • it is the master changelog table — custom JpaEntityProvider changelog tables are still emitted by Liquibase
    itself (verified: DATABASECHANGELOG_MIGRATION_ appears exactly once on main), so emitting them here too
    would duplicate.

Whether the table pre-existed is recorded in validateChangeSet and in updateChangeSet; the latter is needed
because validation returns as soon as one changelog is out of date, so custom providers may not be seen there.

The same change is applied to the legacy LiquibaseJpaUpdaterProvider, as in fe40730.

Testing

New ManualMigrationEmptyDatabaseDistTest (H2, raw dist, empty database) asserts the script:

  • contains the changelog table DDL — exactly once, since emitting it twice breaks the script just as badly;
  • creates the table before the first insert into it;
  • still marks client-attributes-string-accomodation-fixed-post-create-index as MARK_RAN (asserted as the
    absence of its MySQL-only VALUE(255) index syntax). changeSetExecuted preconditions with
    are resolved against the physical changelog table, so this pins that the fix does not
    remove it.

ManualMigrationCustomProviderDistTest gains the same once/ordering assertions for a custom provider's
bookkeeping table.

The two assertions removed in PR #41490 are restored in BasicDatabaseTest.assertManualDbInitialization, so
PostgreSQLDistTest, MySQLDistTest and MariaDBDistTest guard this again.

Verified end to end on PostgreSQL 17 with an empty schema — the generated script applies with
psql -v ON_ERROR_STOP=1 (exit 0, 99 tables), after which the server starts with migration-strategy=validate.
Applying to a database whose bookkeeping tables were pre-created still works, i.e. no already exists.

Run against PostgreSQL (7 tests), MySQL (3) and MariaDB (3) through the test-database profile, which is where
the restored assertManualDbInitialization assertions apply. The MySQL path matters here because
MySQLCustomChangeLogHistoryService adds the changelog table's primary key to the script; the CREATE TABLE
is emitted before that ALTER TABLE.

The upgrade path is unaffected, which I checked separately on PostgreSQL: with an already migrated database
rolled back by one release, the generated script contains the 23 pending changesets and no CREATE TABLE for
either bookkeeping table — outputChangeLogTableCreationScript is only reached when no changeset has run yet.

Out of scope

DATABASECHANGELOGLOCK remains absent from the script — it is created by CustomLockService on startup, which
is why it was excluded in the first place, and that is unchanged here. So a database populated only by running
the script still lacks the lock table, and the first server start against it needs the privilege to create that
one table (in the run above the server was started with an owner role, which has it). The documentation change
in this PR states that; making manual mode work for a database user with no DDL privileges at all is a larger
question I would rather raise separately.

Documentation

docs/documentation/upgrading/topics/migrate_db.adoc — the "Manual relational database migration" section did
not mention initialize-empty=false, although manual mode is silently skipped without it on an empty database.
Added, together with a note that the two Liquibase bookkeeping tables are still created on startup, so a user
without DDL rights needs them created up front.

Note on AI assistance

AI agents were used while investigating this and while drafting the change, tests and documentation. I have
reviewed and understand every line, ran the verification described above myself, and will handle review feedback
directly.

@Alnyli07
Alnyli07 requested a review from a team as a code owner August 14, 2026 11:44
Copilot AI balanced review requested due to automatic review settings August 14, 2026 11:44
@Alnyli07
Alnyli07 requested a review from a team as a code owner August 14, 2026 11:44
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Restores valid manual migration exports for empty databases by explicitly creating Liquibase’s changelog table when needed.

Changes:

  • Tracks whether changelog tables existed before validation.
  • Emits missing DATABASECHANGELOG DDL while avoiding custom-table duplication.
  • Adds integration coverage and manual migration documentation.

Reviewed changes

Copilot reviewed 4 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
quarkus/runtime/.../QuarkusJpaUpdaterProvider.java Implements changelog-table tracking and DDL export.
model/jpa/.../LiquibaseJpaUpdaterProvider.java Mirrors the fix in the legacy provider.
ManualMigrationEmptyDatabaseDistTest.java Tests empty-database export ordering and uniqueness.
ManualMigrationCustomProviderDistTest.java Verifies custom changelog-table export behavior.
BasicDatabaseTest.java Restores cross-database export assertions.
migrate_db.adoc Documents empty-database manual migration requirements.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +235 to +236
if (MASTER_CHANGELOG_TABLE.equalsIgnoreCase(changelogTableName)
&& Boolean.FALSE.equals(changelogTablePreExisted.get(changelogTableName))) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for spotting this. I don't think this PR changes that behaviour though.

The MySQLCustomChangeLogHistoryService block isn't touched here; a diff against main shows no changed line in it. And when changelogTablePreExisted is true, the generated script is identical to what main produces: main doesn't emit CREATE TABLE in that case either, because Liquibase skips it when the table already exists, and the same ALTER TABLE ... ADD PRIMARY KEY follows it. So the failure you describe already happens on main today.

I'd also be careful with the suggested fix. Liquibase's standard changelog table has no primary key, which is exactly why Keycloak adds one for MySQL. An administrator pre-creating the table for a user without DDL rights will normally create it from that standard DDL, so without the key. Suppressing the statement whenever the table pre-exists would leave those deployments without it. Getting it right means knowing whether the target database already has the key, and in the flow this PR is about, the target isn't the database the script was generated against.

Happy to open a separate issue for it if you think it's worth fixing.

Comment on lines +232 to +233
if (MASTER_CHANGELOG_TABLE.equalsIgnoreCase(changelogTableName)
&& Boolean.FALSE.equals(changelogTablePreExisted.get(changelogTableName))) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for spotting this. I don't think this PR changes that behaviour though.

The MySQLCustomChangeLogHistoryService block isn't touched here; a diff against main shows no changed line in it. And when changelogTablePreExisted is true, the generated script is identical to what main produces: main doesn't emit CREATE TABLE in that case either, because Liquibase skips it when the table already exists, and the same ALTER TABLE ... ADD PRIMARY KEY follows it. So the failure you describe already happens on main today.

I'd also be careful with the suggested fix. Liquibase's standard changelog table has no primary key, which is exactly why Keycloak adds one for MySQL. An administrator pre-creating the table for a user without DDL rights will normally create it from that standard DDL, so without the key. Suppressing the statement whenever the table pre-exists would leave those deployments without it. Getting it right means knowing whether the target database already has the key, and in the flow this PR is about, the target isn't the database the script was generated against.

Happy to open a separate issue for it if you think it's worth fixing.

8000
assertThat(output, containsString("Create Database Change Log Table"));

// The script inserts into the changelog table, so it has to create it - once, or applying it fails
var outputLowerCase = output.toLowerCase();
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, the assertion now uses toLowerCase(Locale.ROOT).

One correction on the impact though. The three database tests wouldn't have returned zero. The count adds two alternatives, public.databasechangelog and keycloak.databasechangelog, and these tests run against PostgreSQL, MySQL and MariaDB, whose exports qualify the table with the schema or database name, so CREATE TABLE keycloak.databasechangelog (. KEYCLOAK lowercases the same way under tr_TR, so that alternative still matches. Only the PUBLIC. one is affected by the dotless ı, and it isn't what those databases produce.

I checked the underlying behaviour at string level ("PUBLIC".toLowerCase(new Locale("tr","TR")) gives publıc and the match fails) and reran PostgreSQLDistTest after the change. The fix is right either way, since the assertion shouldn't depend on the host locale.

Copilot AI review requested due to automatic review settings August 14, 2026 12:03
@Alnyli07
Alnyli07 force-pushed the fix/manual-export-changelog-table-ddl branch from 0bb8298 to 1be1244 Compare August 14, 2026 12:03
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (2)

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/storage/database/liquibase/QuarkusJpaUpdaterProvider.java:239

  • When the table was initially absent, Liquibase has already created it on the live connection before this block—the root cause described by this PR. On databases with auto-committing DDL such as MySQL, this emitted CREATE TABLE therefore fails when the script is applied back to the database it was generated from, which is the documented manual-migration workflow. Prevent or clean up the validation-created table after export, or otherwise make this creation idempotent.
        if (MASTER_CHANGELOG_TABLE.equalsIgnoreCase(changelogTableName)
                && Boolean.FALSE.equals(changelogTablePreExisted.get(changelogTableName))) {
            // Same comment Liquibase emits when it writes this statement itself
            loggingExecutor.comment("Create Database Change Log Table");
            loggingExecutor.execute(new CreateDatabaseChangeLogTableStatement());

model/jpa/src/main/java/org/keycloak/connections/jpa/updater/liquibase/LiquibaseJpaUpdaterProvider.java:236

  • This has the same same-database failure as the Quarkus implementation: when the table was absent at startup, Liquibase creates it on the live connection before export, and auto-committing databases retain it, so the generated CREATE TABLE fails when applied to that database. Prevent or clean up the validation-created table after export, or otherwise make this creation idempotent.
        if (MASTER_CHANGELOG_TABLE.equalsIgnoreCase(changelogTableName)
                && Boolean.FALSE.equals(changelogTablePreExisted.get(changelogTableName))) {
            // Same comment Liquibase emits when it writes this statement itself
            loggingExecutor.comment("Create Database Change Log Table");
            loggingExecutor.execute(new CreateDatabaseChangeLogTableStatement());

@Alnyli07
Copy link
Copy Markdown
Contributor Author

The failing External links check isn't related to this PR. It reports one broken link in the release_notes guide, https://access.redhat.com/products/single-sign/, which redirects to .../single-sign-on/. That link is in docs/documentation/release_notes/topics/19_0_0.adoc, and the only documentation file touched here is upgrading/topics/migrate_db.adoc. The check doesn't run on pull requests that leave documentation untouched, which is why it shows as skipped on other PRs.

It looks like a follow-on from #51602: the URL was updated to /single-sign/ there, and that one now redirects as well. https://access.redhat.com/products/single-sign-on/ returns 200 with no redirect. I've opened #51732 for it.

Copilot AI review requested due to automatic review settings August 17, 2026 18:33
@Alnyli07
Alnyli07 force-pushed the fix/manual-export-changelog-table-ddl branch from 1be1244 to 52fa878 Compare August 17, 2026 18:33
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 6 changed files in this pull request and generated 6 comments.

Suppressed comments (2)

docs/documentation/upgrading/topics/migrate_db.adoc:77

  • The CLI option formatting here (--spi-connections-jpa--quarkus--initialize-empty=false) is inconsistent with the option actually used in tests in this PR (--spi-connections-jpa-quarkus-initialize-empty=false). Align the documentation with the correct CLI option spelling to prevent copy/paste failures.
For an empty database, also set the `initialize-empty` property to "false". Otherwise {project_name} creates the
schema itself and no SQL file is written, regardless of the `migration-strategy` value:

[source,bash]
----
kc.[sh|bat] start --spi-connections-jpa--quarkus--initialize-empty=false
----

quarkus/tests/integration/src/test/java/org/keycloak/it/storage/database/BasicDatabaseTest.java:104

  • This assertion hard-codes schema names (public/keycloak) and an exact formatting of the emitted SQL. If the export changes to omit schema qualification, quote identifiers, or uses a different default schema in some environments, this becomes brittle. Consider matching more flexibly (e.g., schema-optional search, or counting occurrences of databasechangelog ( combined with a stronger nearby marker like the 'Create Database Change Log Table' comment) while still ensuring the statement is emitted exactly once.
        var outputLowerCase = output.toLowerCase(Locale.ROOT);
        var count = countMatches(outputLowerCase, "create table public.databasechangelog (")
                + countMatches(outputLowerCase, "create table keycloak.databasechangelog (");
        assertThat(count, is(1));

@@ -68,11 +69,14 @@ public class QuarkusJpaUpdaterProvider implements JpaUpdaterProvider {
private static final Logger logger = Logger.getLogger(QuarkusJpaUpdaterProvider.class);

public static final String CHANGELOG = "META-INF/jpa-changelog-master.xml";
private static final String MASTER_CHANGELOG_TABLE = "DATABASECHANGELOG";
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The map is gone, replaced by a single flag for the master table.

Comment on lines +78 to +79
// Per changelog table: did it already exist when this server started, before validation created it?
private final Map<String, Boolean> changelogTablePreExisted = new HashMap<>();
Comment on lines +234 to +236
String changelogTableName = database.getDatabaseChangeLogTableName();
if (MASTER_CHANGELOG_TABLE.equalsIgnoreCase(changelogTableName)
&& Boolean.FALSE.equals(changelogTablePreExisted.get(changelogTableName))) {
@@ -68,10 +69,13 @@ public class LiquibaseJpaUpdaterProvider implements JpaUpdaterProvider {
private static final Logger logger = Logger.getLogger(LiquibaseJpaUpdaterProvider.class);

public static final String CHANGELOG = "META-INF/jpa-changelog-master.xml";
private static final String MASTER_CHANGELOG_TABLE = "DATABASECHANGELOG";
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The map is gone, replaced by a single flag for the master table.


public static final String DEPLOYMENT_ID_COLUMN = "DEPLOYMENT_ID";

private final KeycloakSession session;
// Per changelog table: did it already exist when this server started, before validation created it?
private final java.util.Map<String, Boolean> changelogTablePreExisted = new java.util.HashMap<>();
Comment on lines +231 to +233
String changelogTableName = database.getDatabaseChangeLogTableName();
if (MASTER_CHANGELOG_TABLE.equalsIgnoreCase(changelogTableName)
&& Boolean.FALSE.equals(changelogTablePreExisted.get(changelogTableName))) {
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.federation.ldap.LDAPUserLoginTest#loginLDAPUserCredentialVaultAuthenticationSimpleEncryptionStartTLS

Keycloak CI - Java Distribution IT (windows-latest - temurin - 21)

org.openqa.selenium.TimeoutException: 
java.net.SocketTimeoutException: Read timed out
Build info: version: '4.28.1', revision: '73f5ad48a2'
System info: os.name: 'Windows Server 2025', os.arch: 'amd64', os.version: '10.0', java.version: '21.0.12'
Driver info: driver.version: HtmlUnitDriver
...

Report flaky test

Copilot AI review requested due to automatic review settings August 18, 2026 05:40
@Alnyli07
Alnyli07 force-pushed the fix/manual-export-changelog-table-ddl branch from 52fa878 to 8888203 Compare August 18, 2026 05:40
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 6 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

docs/documentation/upgrading/topics/migrate_db.adoc:77

  • The CLI option name in the added example appears to use double separators (--spi-connections-jpa--quarkus--initialize-empty=false). In this PR’s tests the option is passed as --spi-connections-jpa-quarkus-initialize-empty=false (single separators). Update the docs example to the correct option spelling so users can copy/paste it successfully.
For an empty database, also set the `initialize-empty` property to "false". Otherwise {project_name} creates the
schema itself and no SQL file is written, regardless of the `migration-strategy` value:

[source,bash]
----
kc.[sh|bat] start --spi-connections-jpa--quarkus--initialize-empty=false
----

quarkus/tests/integration/src/test/java/org/keycloak/it/storage/database/BasicDatabaseTest.java:104

  • This exact substring match is sensitive to SQL formatting and schema qualification (e.g., no schema prefix, quoted identifiers, multiple spaces/newlines). To make the test more robust, consider matching create table for databasechangelog with a case-insensitive regex and flexible whitespace (and optionally allowing an arbitrary schema prefix), then assert the match count is exactly 1.
        var outputLowerCase = output.toLowerCase(Locale.ROOT);
        var count = countMatches(outputLowerCase, "create table public.databasechangelog (")
                + countMatches(outputLowerCase, "create table keycloak.databasechangelog (");
        assertThat(count, is(1));

Comment on lines +234 to +240
String changelogTableName = database.getDatabaseChangeLogTableName();
if (MASTER_CHANGELOG_TABLE.equalsIgnoreCase(changelogTableName)
&& Boolean.FALSE.equals(changelogTablePreExisted.get(changelogTableName))) {
// Same comment Liquibase emits when it writes this statement itself
loggingExecutor.comment("Create Database Change Log Table");
loggingExecutor.execute(new CreateDatabaseChangeLogTableStatement());
}
Comment on lines +231 to +237
String changelogTableName = database.getDatabaseChangeLogTableName();
if (MASTER_CHANGELOG_TABLE.equalsIgnoreCase(changelogTableName)
&& Boolean.FALSE.equals(changelogTablePreExisted.get(changelogTableName))) {
// Same comment Liquibase emits when it writes this statement itself
loggingExecutor.comment("Create Database Change Log Table");
loggingExecutor.execute(new CreateDatabaseChangeLogTableStatement());
}
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.broker.KcOidcBrokerTokenExchangeTest#

Keycloak CI - Base IT (2)

java.lang.RuntimeException: java.lang.IllegalStateException: Keycloak unexpectedly died :(
	at org.keycloak.testsuite.arquillian.containers.KeycloakQuarkusServerDeployableContainer.start(KeycloakQuarkusServerDeployableContainer.java:72)
	at org.jboss.arquillian.container.impl.ContainerImpl.start(ContainerImpl.java:185)
	at org.jboss.arquillian.container.impl.client.container.ContainerLifecycleController$8.perform(ContainerLifecycleController.java:137)
	at org.jboss.arquillian.container.impl.client.container.ContainerLifecycleController$8.perform(ContainerLifecycleController.java:133)
...

Report flaky test

org.keycloak.testsuite.federation.ldap.LDAPUserLoginTest#loginLDAPUserAuthenticationNoneEncryptionStartTLS

Keycloak CI - Java Distribution IT (windows-latest - temurin - 21)

org.openqa.selenium.TimeoutException: 
java.net.SocketTimeoutException: Read timed out
Build info: version: '4.28.1', revision: '73f5ad48a2'
System info: os.name: 'Windows Server 2025', os.arch: 'amd64', os.version: '10.0', java.version: '21.0.12'
Driver info: driver.version: HtmlUnitDriver
...

Report flaky test

org.keycloak.testsuite.webauthn.passwordless.WebAuthnPwdLessPropertyTest#timeout

Keycloak CI - WebAuthn IT

org.openqa.selenium.WebDriverException: 
unknown error: unhandled inspector error: {"code":-32000,"message":"Node with given id does not belong to the document"}
  (Session info: chrome=151.0.7922.108)
Build info: version: '4.28.1', revision: '73f5ad48a2'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.17.0-1022-azure', java.version: '25.0.4'
...

Report flaky test

@michalvavrik michalvavrik left a comment
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your PR. I think this PR is sound and solves something we need to fix. I only left minor comments.


[source,bash]
< 10BC8 /td> ----
kc.[sh|bat] start --spi-connections-jpa--quarkus--initialize-empty=false
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
kc.[sh|bat] start --spi-connections-jpa--quarkus--initialize-empty=false
<@kc.start parameters="--spi-connections-jpa--quarkus--initialize-empty=false"/>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That block is gone with the paragraph above.

@@ -69,6 +69,24 @@ kc.[sh|bat] start --spi-connections-jpa--quarkus--migration-export=<path>/<file.

After the changes have been written to the file, the server exits.

For an empty database, also set the `initialize-empty` property to "false". Otherwise {project_name} creates the
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My 2 cents (I don't know what maintainers thinks!):

It sounds like you are documenting bug. manual migration strategy must be explicitly as it is not a default strategy. If we ignore explicit configuration for scenario where the database is empty, I think that is validation error rather than expected behavior. They contradict each other.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, createOrUpdateSchema checks initializeEmpty before it looks at the strategy, so an explicit manual or validate is silently ignored on an empty database. Dropped the paragraph.

Do you want that fixed in this PR, or in a separate issue? I'd lean towards separate since it's new behaviour rather than the regression this one restores.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want that fixed in this PR, or in a separate issue? I'd lean towards separate since it's new behaviour rather than the regression this one restores.

I think a separate issue is better, Please create it as at this point, you can describe it better than I do.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #51932 and fixed it in #51933. Good call on that paragraph, documenting it would've just masked the bug!


public static final String DEPLOYMENT_ID_COLUMN = "DEPLOYMENT_ID";

private final KeycloakSession session;
// Per changelog table: did it already exist when this server started, before validation created it?
private final java.util.Map<String, Boolean> changelogTablePreExisted = new java.util.HashMap<>();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You only create the master changelog table, that is a flag, not map. Putting anything else to the map is a dead data.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair, replaced with a single Boolean masterChangelogTablePreExisted. Same in the Quarkus provider.

private static final String DEPLOYMENT_ID_COLUMN = "DEPLOYMENT_ID";
public static final String VERIFY_AND_RUN_MASTER_CHANGELOG = "VERIFY_AND_RUN_MASTER_CHANGELOG";

private final KeycloakSession session;
private Map<String, List<ChangeSet>> changeSets = new HashMap<>();
// Per changelog table: did it already exist when this server started, before validation created it?
private final Map<String, Boolean> changelogTablePreExisted = new HashMap<>();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as in model/jpa/src/main/java/org/keycloak/connections/jpa/updater/liquibase/LiquibaseJpaUpdaterProvider.java

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, same as in model/jpa.

Copilot AI review requested due to automatic review settings August 19, 2026 14:00
@Alnyli07
Alnyli07 force-pushed the fix/manual-export-changelog-table-ddl branch from 8888203 to 384a019 Compare August 19, 2026 14:00
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 6 changed files in this pull request and generated 2 comments.

Suppressed comments (1)

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/storage/database/liquibase/QuarkusJpaUpdaterProvider.java:241

  • The new behavior has two branches: emit DDL when the changelog table was absent at server start, and do not emit it when it pre-existed. Current tests cover the empty-DB path; please add coverage for the pre-created-table path to ensure the script does not include the CREATE TABLE DATABASECHANGELOG statement (and remains applicable to the same DB it was generated from).
        if (isMasterChangelogTable(database) && Boolean.FALSE.equals(masterChangelogTablePreExisted)) {
            // Same comment Liquibase emits when it writes this statement itself
            loggingExecutor.comment("Create Database Change Log Table");
            loggingExecutor.execute(new CreateDatabaseChangeLogTableStatement());
        }

Comment on lines +237 to +241
if (isMasterChangelogTable(database) && Boolean.FALSE.equals(masterChangelogTablePreExisted)) {
// Same comment Liquibase emits when it writes this statement itself
loggingExecutor.comment("Create Database Change Log Table");
loggingExecutor.execute(new CreateDatabaseChangeLogTableStatement());
}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can't be null here: this method's only caller is updateChangeSet, which records the flag for the master table at its start, on the same Database, before reaching this call. ManualMigrationEmptyDatabaseDistTest covers exactly this path.

Comment on lines +234 to +238
if (isMasterChangelogTable(database) && Boolean.FALSE.equals(masterChangelogTablePreExisted)) {
// Same comment Liquibase emits when it writes this statement itself
loggingExecutor.comment("Create Database Change Log Table");
loggingExecutor.execute(new CreateDatabaseChangeLogTableStatement());
}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as in the Quarkus provider: the only caller is updateChangeSet, which sets the flag before this runs.

@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.adapter.servlet.SAMLClockSkewAdapterTest#testTokenTimeIsValid

Keycloak CI - Adapter IT Strict Cookies

org.jboss.arquillian.graphene.enricher.exception.PageObjectInitializationException: 
Can not instantiate Page Object 
class org.keycloak.testsuite.adapter.page.SalesPostClockSkewServlet
 declared in: 
org.keycloak.testsuite.adapter.servlet.SAMLClockSkewAdapterTest
...

Report flaky test

…y database

Liquibase writes the DDL of its changelog table into the generated script only
while that table does not exist. Since Liquibase 4.33 parsing a changelog
initializes the change log history service, which creates the table on the live
connection, so by the time the export is written Liquibase no longer emits it.
The script then inserts into a table it never creates and cannot be applied to
an empty database, even though its header offers it as a creation script.

Emit the statement explicitly, but only when the table did not exist before this
server started - an administrator may have created it up front so that a user
without DDL rights can produce the script at all, and in that case the target
database already has it.

This restores the guarantee added in keycloak#32535 along with the assertions that were
dropped from BasicDatabaseTest in keycloak#41299.

Closes keycloak#51729

Signed-off-by: Ali Tugrul Pinar <ali@keymate.io>
Copilot AI review requested due to automatic review settings August 20, 2026 11:23
@Alnyli07
Alnyli07 force-pushed the fix/manual-export-changelog-table-ddl branch from 384a019 to 39f9d77 Compare August 20, 2026 11:23
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 6 changed files in this pull request and generated 1 comment.


After the changes have been written to the file, the server exits.

For an empty database the script also creates the Liquibase `DATABASECHANGELOG` table, which it records the applied
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.forms.BruteForceTest#testGrantInvalidOtp

Keycloak CI - Forms IT (chrome)

org.opentest4j.AssertionFailedError: Expected error event ==> expected: <true> but was: <false>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
	at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
...

Report flaky test

@michalvavrik michalvavrik left a comment
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the fix.

@shawkins shawkins left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @Alnyli07 for the contribution

@shawkins
shawkins commented Sep 3, 2026
Copy link
Copy Markdown
Contributor

Let's get a review from @Pepo48 or production readiness, then commit this.

@shawkins
shawkins requested a review from a team September 3, 2026 16:02
@Pepo48 Pepo48 left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Alnyli07 looks good from my perspective. Thank you very much for the contribution!

I will try to get someone from SRE team for a final blessing.

@ahus1
ahus1 self-requested a review September 15, 2026 20:59
@ahus1 ahus1 left a comment
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see in the docs the following statement:

It does not create the DATABASECHANGELOGLOCK table

I see that this PR fixes part of the problem, but reading this statement shows that it is not resolving it. If the CN team wants to proceed with it, I'm ok to proceed. Still there should be a follow-up issue for that extra table.

The following statement in the comments is also worrying:

An administrator may have pre-created it so that a user without DDL rights could produce the script at all

This also points to a process that is rather broken and a smooth experience.

Again, I'm ok to create a follow-up issue for that, which might be more docs, or a simplified process.

@michalvavrik
Copy link
Copy Markdown
Member

Thanks @ahus1 for having a look.

It does not create the DATABASECHANGELOGLOCK table
I see that this PR fixes part of the problem, but reading this statement shows that it is not resolving it.

The DATABASECHANGELOGLOCK table is created as this documentation states in the same paragraph. This table is required to handle concurrency during migration, so it does not hold any relevant data you want to keep in the migration script. Can you elaborate bit more where you see a problem, please?

An administrator may have pre-created it so that a user without DDL rights could produce the script at all
This also points to a process that is rather broken and a smooth experience.

Point being is that we create it conditionally when it is missing, hence user without DDL rights can still execute migration script as long as all DDL operations are not necessary, because all tables exist. Please give me a hint, I definitely think we want a smooth experience, I just don't see what you see.

@Alnyli07
Copy link
Copy Markdown
Contributor Author

@ahus1 , @michalvavrik Opened #53088 as the follow-up for the lock table. It turned out you had already proposed exactly this in #10149 back in 2022 and @hmlnarik agreed there, the issue just expired. The fix builds on the pre-existence flag introduced here, so I'll send the PR once this one is merged.

Same rule as for the changelog table: emitted only when the table did not exist before the export run, so applying the script to the generating database keeps working.

@ahus1
ahus1 merged commit e9c2d4f into keycloak:main Sep 23, 2026
98 checks passed
@ahus1
ahus1 commented Sep 23, 2026
Copy link
Copy Markdown
Member

@Alnyli07 - thank you for this PR. I should have added some more context to my previous comment:

A common scenario where people need those SQL scripts: The Keycloak user has (for good security practice) no permissions to run DDL/create tables. So the script is run by a different person / admin manually. This only works when it creates all the tables, including both the databasechangelog and the lock table. Otherwise, you run the script, and then see an error message later when Keycloak tries to create the table, but doesn't have permission.

Thanks for creating the follow-up PR. I hope I'll be faster in providing feedback next time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Manual migration export for an empty database no longer creates the changelog table

7 participants

0