Repository navigation
refactor(inkless:switch): replace the seal and repair commands with rearm and lower-seal - #872
Open
giuseppelillo wants to merge 3 commits into
Open
giuseppelillo wants to merge 3 commits into
giuseppelillo wants to merge 3 commits into
Conversation
giuseppelillo
marked this pull request as draft
November 13, 2026 15:42
giuseppelillo
force-pushed
the
claude/classic-diskless-switch-tooling-f604b8
branch
2 times, most recently
from
November 16, 2026 14:55
eec3dd8 to
2f26931
Compare
Remove the `seal` and `repair` commands of `kafka-topic-switch.sh`, the `RepairDisklessLog` RPC with its admin, broker, and control plane code and metrics, and the `ClearProducerStates` field of `AlterDisklessSwitch`. Migration `V32` drops the `repair_diskless_log_v1` function and types, and the jOOQ code is regenerated. Assisted-by: Claude Opus 5.5
`AlterDisklessSwitch` now accepts two seal offsets and rejects every other one: - `-2` re-arms a pending switch. The controller bumps the leader epoch so the leader seals again. - A seal of at least 0 lowers a committed seal in phase 1, and must be lower than the committed seal. It keeps the recovery after a last-known-leader election picks a leader whose log ends below the seal. `kafka-topic-switch.sh rearm` sends `-2`, and `lower-seal --offset N` sends `N`. `lower-seal` rejects a negative offset, because the controller reads `-2` as a re-arm. The last-known-leader recovery uses `lower-seal` and has no `repair` step. Assisted-by: Claude Opus 5.5
…d repair tests Remove the tests of the `seal` and `repair` commands, `RepairDisklessLog`, and `ClearProducerStates`. Add tests that check: - `AlterDisklessSwitch` accepts `-2` only on a pending switch, and a lower seal only on a committed seal in phase 1. It rejects every other seal offset, and the lower-seal record survives serialization. - `rearm` and `lower-seal` send the expected seal offset, and `lower-seal` rejects a negative offset without sending a request. - Lowering `min.insync.replicas` and running `rearm` lets a pending switch that is stuck below `min.insync.replicas` reach the seal. - A committed switch rejects a re-arm, an abort, and a seal that isn't lower, in a running cluster. `ClassicToDisklessSwitchLastKnownLeaderClusterTest` lowers the seal with `alterDisklessSwitch` and no longer runs `repair`. Assisted-by: Claude Opus 5.5
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The intentional AlterDisklessSwitch v0 wire break and PostgreSQL function removal require owner confirmation of deployed-client compatibility and safe mixed-version upgrades and rollbacks.
No open issues 🎉
What changed in this PR
This PR narrows Inkless’s classic-to-diskless recovery tools while preserving the leader-driven switch flow. Operators can re-arm a pending switch or lower a committed phase 1 seal, but can no longer force a pending seal, cancel a switch, or invoke control-plane repair.
Changes:
- Replace
sealandrepairwithrearmandlower-seal, with matching controller validation and recovery tests. - Remove
RepairDisklessLogacross the client, protocol, broker, and control plane; drop its database function and regenerate jOOQ bindings. - Update recovery documentation and metrics references, and intentionally remove
ClearProducerStatesfrom the wire format.
| File | Description |
|---|---|
tools/src/test/java/org/apache/kafka/tools/TopicSwitchCommandTest.java |
Tests replacement commands, required arguments, and negative-offset rejection. |
tools/src/main/java/org/apache/kafka/tools/TopicSwitchCommand.java |
Implements rearm and lower-seal; preserves state. |
streams/integration-tests/src/test/java/org/apache/kafka/streams/integration/TestingMetricsInterceptingAdminClient.java |
Removes repair delegation. |
storage/inkless/src/test/java/io/aiven/inkless/control_plane/AvailabilityGatedControlPlaneTest.java |
Removes repair availability assertions. |
storage/inkless/src/test/java/io/aiven/inkless/control_plane/AbstractControlPlaneTest.java |
Removes repair behavior tests. |
storage/inkless/src/main/resources/db/migration/V32__Remove_repair_diskless_log.sql |
Drops the repair function and associated types. |
storage/inkless/src/main/jooq/org/jooq/generated/UDTs.java |
Removes repair type registrations. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/RepairDisklessLogResponseV1.java |
Deletes the generated repair response type. | storage/inkless/src/main/jooq/org/jooq/generated/udt/RepairDisklessLogRequestV1.java |
Deletes the generated repair request type. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/RepairDisklessLogResponseV1Record.java |
Deletes the generated repair response record. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/RepairDisklessLogRequestV1Record.java |
Deletes the generated repair request record. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/PurgeDeletedLogsResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/PruneBatchesBelowHighestTieredOffsetResponseV2Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/PruneBatchesBelowHighestTieredOffsetResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/PruneBatchesBelowHighestTieredOffsetRequestV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/ListOffsetsResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/ListOffsetsRequestV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/InitDisklessLogResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/InitDisklessLogRequestV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/InitDisklessLogProducerStateV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/FindBatchesResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/FindBatchesRequestV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/EnforceRetentionResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/EnforceRetentionRequestV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/DeleteRecordsResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/DeleteRecordsRequestV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/CommitBatchResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/CommitBatchRequestV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/BatchMetadataV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/BatchInfoV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/AdvanceCrossTierLogStartResponseV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/records/AdvanceCrossTierLogStartRequestV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/PurgeDeletedLogsResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/PruneBatchesBelowHighestTieredOffsetResponseV2.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/PruneBatchesBelowHighestTieredOffsetResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/PruneBatchesBelowHighestTieredOffsetRequestV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/RepairDisklessLogResponseV1Path.java |
Deletes the generated repair response path. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/RepairDisklessLogRequestV1Path.java |
Deletes the generated repair request path. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/PurgeDeletedLogsResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/PruneBatchesBelowHighestTieredOffsetResponseV2Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/PruneBatchesBelowHighestTieredOffsetResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/PruneBatchesBelowHighestTieredOffsetRequestV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/ListOffsetsResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/ListOffsetsRequestV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/InitDisklessLogResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/InitDisklessLogRequestV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/InitDisklessLogProducerStateV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/FindBatchesResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/FindBatchesRequestV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/EnforceRetentionResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/EnforceRetentionRequestV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/DeleteRecordsResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/DeleteRecordsRequestV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/CommitBatchResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/CommitBatchRequestV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/BatchMetadataV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/BatchInfoV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/AdvanceCrossTierLogStartResponseV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/paths/AdvanceCrossTierLogStartRequestV1Path.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/ListOffsetsResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/ListOffsetsRequestV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/InitDisklessLogResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/InitDisklessLogRequestV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/InitDisklessLogProducerStateV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/FindBatchesResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/FindBatchesRequestV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/EnforceRetentionResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/EnforceRetentionRequestV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/DeleteRecordsResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/DeleteRecordsRequestV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/CommitBatchResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/CommitBatchRequestV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/BatchMetadataV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/BatchInfoV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/AdvanceCrossTierLogStartResponseV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/udt/AdvanceCrossTierLogStartRequestV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/RepairDisklessLogV1.java |
Deletes the generated repair function binding. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/RepairDisklessLogV1Record.java |
Deletes the generated repair function record. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/PurgeDeletedLogsV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/PruneBatchesBelowHighestTieredOffsetV2Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/PruneBatchesBelowHighestTieredOffsetV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/ProducerStateRecord.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/LogsRecord.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/ListOffsetsV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/InitDisklessLogV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/FindBatchesV2Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/FindBatchesV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/FilesRecord.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/EnforceRetentionV2Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/EnforceRetentionV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/DeleteRecordsV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/CommitFileV2Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/CommitFileV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/BatchesRecord.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/records/AdvanceCrossTierLogStartV1Record.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/PurgeDeletedLogsV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/PruneBatchesBelowHighestTieredOffsetV2.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/PruneBatchesBelowHighestTieredOffsetV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/ProducerState.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/Logs.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/ListOffsetsV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/InitDisklessLogV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/FindBatchesV2.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/FindBatchesV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/Files.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/EnforceRetentionV2.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/EnforceRetentionV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/DeleteRecordsV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/CommitFileV2.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/CommitFileV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/Batches.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/tables/AdvanceCrossTierLogStartV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/Tables.java |
Removes repair function accessors. |
storage/inkless/src/main/jooq/org/jooq/generated/routines/MarkFileToDeleteV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/routines/FlushCommitRunV2.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/routines/DeleteTopicV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/routines/DeleteFilesV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/routines/DeleteBatchV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/routines/BatchTimestamp.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/Routines.java |
Removes repair routine accessors. |
storage/inkless/src/main/jooq/org/jooq/generated/Keys.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/Indexes.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/PruneBatchesBelowHighestTieredOffsetErrorV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/ListOffsetsResponseErrorV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/InitDisklessLogErrorV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/FindBatchesResponseErrorV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/FileStateT.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/FileReasonT.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/EnforceRetentionResponseErrorV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/DeleteRecordsResponseErrorV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/CommitBatchResponseErrorV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/enums/AdvanceCrossTierLogStartResponseErrorV1.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/Domains.java |
Updates generated schema metadata to version 32. |
storage/inkless/src/main/jooq/org/jooq/generated/DefaultSchema.java |
Removes repair objects from the generated schema. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/RepairDisklessLogResponse.java |
Deletes the control-plane repair response. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/RepairDisklessLogRequest.java |
Deletes the control-plane repair request. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/postgres/RepairDisklessLogJob.java |
Deletes the Postgres repair job. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/postgres/PostgresControlPlaneMetrics.java |
Removes repair query metrics. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/postgres/PostgresControlPlane.java |
Removes the Postgres repair entry point. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/InMemoryControlPlane.java |
Removes the in-memory repair implementation. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/ControlPlane.java |
Removes repair from the control-plane interface. |
storage/inkless/src/main/java/io/aiven/inkless/control_plane/AvailabilityGatedControlPlane.java |
Removes repair forwarding. |
server/src/main/java/org/apache/kafka/network/RequestConvertToJson.java |
Removes repair request and response conversion. |
metadata/src/test/java/org/apache/kafka/controller/ReplicationControlManagerInklessTest.java |
Tests re-arm, lower-seal restrictions, and record serialization. |
metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java |
Restricts operator changes to pending re-arm and phase 1 seal lowering. |
metadata/src/main/java/org/apache/kafka/controller/Controller.java |
Documents the narrowed controller API. |
docs/inkless/metrics.rst |
Removes repair metric documentation. |
docs/inkless/CLASSIC_TO_DISKLESS_SWITCH.md |
Documents replacement commands and recovery without repair. |
core/src/test/scala/unit/kafka/server/RequestQuotaTest.scala |
Removes repair request construction. |
core/src/test/scala/unit/kafka/server/ReplicaManagerInklessTest.scala |
Replaces repair tests with stalled-switch re-arm coverage. |
core/src/test/java/kafka/server/InklessTopicTypeSwitcherClusterTest.java |
Tests rejection of invalid changes after seal commit. |
core/src/test/java/kafka/server/ClassicToDisklessSwitchLastKnownLeaderClusterTest.java |
Exercises lower-seal recovery without repair. |
core/src/main/scala/kafka/server/ReplicaManager.scala |
Removes broker-side control-plane repair logic. |
core/src/main/scala/kafka/server/KafkaApis.scala |
Removes repair request dispatch and han 8000 dling. |
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java |
Removes repair protocol test factories. |
clients/src/test/java/org/apache/kafka/clients/admin/MockAdminClient.java |
Removes the repair API stub. |
clients/src/main/resources/common/message/RepairDisklessLogResponse.json |
Deletes the repair response schema. |
clients/src/main/resources/common/message/RepairDisklessLogRequest.json |
Deletes the repair request schema. |
clients/src/main/resources/common/message/AlterDisklessSwitchRequest.json |
Removes ClearProducerStates and documents accepted offsets. |
clients/src/main/java/org/apache/kafka/common/requests/RepairDisklessLogResponse.java |
Deletes the repair protocol response. |
clients/src/main/java/org/apache/kafka/common/requests/RepairDisklessLogRequest.java |
Deletes the repair protocol request. |
clients/src/main/java/org/apache/kafka/common/requests/AbstractResponse.java |
Removes repair response parsing. |
clients/src/main/java/org/apache/kafka/common/requests/AbstractRequest.java |
Removes repair request parsing. |
clients/src/main/java/org/apache/kafka/common/protocol/ApiKeys.java |
Removes the repair API key registration. |
clients/src/main/java/org/apache/kafka/clients/admin/RepairDisklessLogResult.java |
Deletes the repair result API. |
clients/src/main/java/org/apache/kafka/clients/admin/RepairDisklessLogOptions.java |
Deletes the repair options API. |
clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java |
Removes repair calls and producer-state clearing. |
clients/src/main/java/org/apache/kafka/clients/admin/ForwardingAdmin.java |
Removes repair delegation. |
clients/src/main/java/org/apache/kafka/clients/admin/AlterDisklessSwitchOptions.java |
Removes the producer-state clearing option. |
clients/src/main/java/org/apache/kafka/clients/admin/Admin.java |
Removes repair APIs and documents the narrowed switch operations. |
Review effort: Balanced
giuseppelillo
force-pushed
the
claude/classic-diskless-switch-tooling-f604b8
branch
from
November 16, 2026 15:08
2f26931 to
9080f4d
Compare
giuseppelillo
marked this pull request as ready for review
November 16, 2026 15:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR reduces
kafka-topic-switch.sh8000 to a read-onlystatecommand and two narrow write commands,rearmandlower-seal. It removes every other operator command that writes a seal offset or changes the control plane.Removed:
sealcommand, with all its options:--offsetwithN ≥ 0,-1, and-2, plus--clear-producer-statesand--dry-run.repaircommand.AlterDisklessSwitchinput except two:-2on a pending switch, and a seal lower than the committed seal in phase 1. TheClearProducerStatesfield is gone, andAlterDisklessSwitchOptionsno longer hasclearProducerStates.RepairDisklessLogbroker RPC, along withAdmin.repairDisklessLog, its broker handler, its control plane code in both implementations, and its metrics.repair_diskless_log_v1Postgres function and its types, which migrationV32__Remove_repair_diskless_log.sqldrops.New:
rearm --topic T --partition Pcommand. It sendsAlterDisklessSwitchwith-2, which the controller accepts only while the switch is pending. The controller bumps the leader epoch, so the leader seals again.lower-seal --topic T --partition P --offset Ncommand. It sendsAlterDisklessSwitchwithN, which the controller accepts only for a committed seal in phase 1 and a seal that is at least 0 and lower than the committed one. The command rejects a negative offset before it sends the request, because the controller reads-2as a re-arm. It keeps the recovery in the "Last-known-leader election" section ofCLASSIC_TO_DISKLESS_SWITCH.mdworking: after a last-known-leader election picks a leader whose log ends below the seal, lowering the seal to that leader's log end offset brings the partition back.Unchanged:
statecommand.Note
After
lower-seal, the seal in KRaft and the diskless log in the control plane disagree, and the records between them are lost.lower-sealwrites the new seal to KRaft only. Every replica truncates its classic log to the new seal. The control plane already has a row for the partition, and the diskless records past the old seal already have their offsets. When the leader sendsInitDisklessLogagain, the control plane answersalready_initializedand changes nothing. So the diskless log still starts at the old seal, and the offsets between the two seals exist in neither log.For example,
ClassicToDisklessSwitchLastKnownLeaderClusterTestcommits a seal at offset 20 and then writes diskless records at offsets 20 to 24. A last-known-leader election picks a leader whose log ends at offset 15, so the partition goes offline. Afterlower-seal --offset 15and a leader restart:No tool reconciles the two, and nothing needs one: the partition serves every remaining record and finishes the switch. The removed
repaircommand didn't change this either, because it only wrote the control plane'sdiskless_start_offsetcolumn, which nothing reads at runtime.Before and after
Each row of the following table describes a problem an operator can face, how the tooling handled it before this PR, and how it's handled after. The last column describes what to do in the cases this PR makes impossible.
seal --offset -2re-arms the switch. The controller bumps the leader epoch, so the leader seals again and registers the partition forInitDisklessLog.rearm --topic T --partition Pdoes the same thing. It sendsAlterDisklessSwitchwith-2.min.insync.replicas, so the high watermark can't reach the sealed log end offset.seal, with or without--offset N, commits a seal without the leader. The seal carries no producer states, so an idempotent producer that was active before the seal fails withOUT_OF_ORDER_SEQUENCE_NUMBERon its next batch. If the seal is above the high watermark, it includes records that aren't fully replicated.min.insync.replicasand runrearm: the leader recomputes its high watermark and proposes the seal with producer states. Adding new replicas doesn't help, because a new replica doesn't fetch until the seal commits.seal --offset Nlowers the seal to the leader's log end offset. The same command also accepts a pending switch,-1, and-2. The documented procedure then runsrepair, which writes a control plane column that nothing reads.lower-seal --offset NsendsAlterDisklessSwitchwithN. The controller accepts it only for a committed seal in phase 1 and a seal lower than the committed one. The procedure has norepairstep. In both cases, every replica truncates its classic log toN, the control plane keeps the diskless log starting at the old seal, and the partition loses the records betweenNand the old seal. A consumer that reads across them gets the diskless records from the old seal on, without an error.seal --clear-producer-statesnever works. The controller builds a record with tags out of order, which can't be serialized. Each attempt makes the active controller resign, and the client getsUNKNOWN_SERVER_ERROR.--offset NwithN ≥ 0), whose seal could disagree with the producer states already committed. The leader commits the seal and the producer states together, from the same log end offset, so they always match.seal --offset -1doesn't revert the partition. The topic keepsdiskless.enable=true, so produce takes the diskless path and fails withUNKNOWN_TOPIC_OR_PARTITION, the classic log stays sealed, and the controller guards for a local-only classic prefix stop applying. Nothing can undo it: the controller rejects any laterAlterDisklessSwitchfor the partition, settingdiskless.enable=trueagain starts no switch, anddiskless.enablecan't be disabled.rearmor a leader election.repairsets the control plane'sdiskless_start_offsetto the new seal. Nothing at runtime reads that column, so it changes nothing for produce, fetch, orListOffsets.Test plan
./gradlew check -x test -x generateJooqClassesandmake docspass, with no uncommitted changes.make valereports no alerts.TopicSwitchCommandTest,ReplicationControlManagerInklessTest,RequestResponseTest,ApiKeysTest,MessageTest,RequestQuotaTest,ControllerApisTest,ReplicaManagerInklessTest,AbstractControlPlaneTestsubclasses, andAvailabilityGatedControlPlaneTest.InklessTopicTypeSwitcherClusterTestpasses. The newtestAlterDisklessSwitchRejectsReArmAndNonLowerSealOnCommittedSwitchswitches a topic, waits for the seal to commit, and checks that the controller rejects-2,-1, the committed seal, and a higher seal, and keeps the seal.ClassicToDisklessSwitchLastKnownLeaderClusterTestpasses in both cases. WithloseTail=true, it lowers the seal withalterDisklessSwitch, restarts the leader without runningrepair, finishes the switch, and reads the expected records from offset 0.ReplicationControlManagerInklessTestcases check thatalterDisklessSwitchlowers a committed seal in phase 1, keeps the leader epoch and the producer states, and writes a record that survives serialization. They also check that it rejects a seal on a pending switch, a lower seal in phase 2, on a classic topic, or on an unknown partition, a seal that isn't lower than the committed one, and every negative seal offset other than-2.TopicSwitchCommandTestchecks thatlower-sealrejects-2and-1without sending a request.testRearmAfterLoweringMinIsrLetsAStuckPendingSwitchReachTheSealputs a leader in a pending switch with its ISR belowmin.insync.replicas. Lowering the config leaves the high watermark in place, and therearmrecord advances it to the log end offset and registers the partition again.testConsolidatingSwitchSealsRegistersThenStartsConsolidationOnClassicPrefixFlagCommitcovers the restarted leader: a first delta from an empty image doesn't register the partition, and the record thatrearmwrites does.testFetchConsolidatingDisklessSwitchPendingReadsFromUnifiedLogWhenConsolidationEnabledcovers consumer reads from a pending partition.🤖 Generated with Claude Code