Repository navigation
Conversation
… expected close warnings
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the DAL cleanup and shutdown-interruption logging changes.
No open issues 🎉
What changed in this PR
This PR avoids unnecessary deletion-log (DAL) initialization during IoTConsensusV2 DataRegion cleanup and suppresses expected interruption warnings during normal buffer shutdown.
Changes:
- Centralize cleanup so it closes existing managers and removes remaining DAL directories without creating a manager.
- Suppress idle-queue interruption warnings during shutdown while retaining warnings for unexpected interruptions.
- Add regression tests for manager cleanup, recreation, orphaned directories, and interruption logging.
| File | Description |
|---|---|
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/consensus/deletion/persist/PageCacheDeletionBufferTest.java |
Tests expected and unexpected interruption logging. |
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/consensus/deletion/DeletionResourceManagerCleanupTest.java |
Tests cleanup without initialization, orphan removal, and manager recreation. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java |
Delegates DAL cleanup to the new manager helper. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/consensus/deletion/persist/PageCacheDeletionBuffer.java |
Suppresses the queue-interruption warning when the buffer is closed. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/consensus/deletion/DeletionResourceManager.java |
Adds cleanup that avoids manager creation and preserves close-before-removal order. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Dropping a DataRegion under IoTConsensusV2 calls
DeletionResourceManager.getInstance()during DAL cleanup. If the region has never used a deletion resource manager, cleanup creates a manager, allocates a buffer, opens a DAL file, and starts a persistence thread just to close them immediately. Closing an idle persistence thread interrupts its blocking queue wait and logs the interruption at WARN unconditionally.For example, with IoTConsensusV2 configured:
Before this change, region cleanup can initialize DAL resources and emit
Interrupted when waiting for taking DeletionResource from blocking queue to serialize.even though no SQL DELETE was issued and cleanup succeeds. After this change, cleanup closes only an existing manager and removes any remaining DAL directory directly. An idle buffer's expected shutdown interruption no longer emits that warning; an interruption while the buffer is open still does.Implementation
DeletionResourceManager.closeAndRemoveDAL(groupId)so callers do not need to create a manager to clean up a region. Preserve the existing non-IoTConsensusV2 guard and close-before-directory-removal order.isClosedbefore logging the interrupted initial queue wait. Keep the interrupt flag and existing persistence/draining behavior.Validation
mvn spotless:apply -pl iotdb-core/datanode(JDK 17): passed.mvn test -pl iotdb-core/datanode -am -Dtest=DeletionResourceManagerCleanupTest,PageCacheDeletionBufferTest,DeletionResourceTest,DeletionRecoverTest -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false: BUILD SUCCESS, all 28 modules succeeded and all 18 DAL tests passed (0 failures/errors).mvn test-compile -pl iotdb-core/datanode -am -P with-zh-locale -DskipTests(JDK 17): BUILD SUCCESS, all 28 modules succeeded. No cluster integration test has been run.This PR has: