Skip to content

[Pipe] Avoid DAL initialization and expected interruption warnings during region cleanup - #19321

Open
Pengzna wants to merge 1 commit into
apache:masterfrom
Pengzna:codex/fix-dal-region-cleanup-warn
Open

Pengzna wants to merge 1 commit into
apache:masterfrom
Pengzna:codex/fix-dal-region-cleanup-warn

Conversation

@Pengzna

@Pengzna Pengzna commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Dropping a DataRegion under IoTConsensusV2 calls DeletionResourceManager.getInstance() during DAL cleanup. If the region has never used a deletion resource manager, cleanup creates a manager, allocates a buffer, opens a DAL file, and starts a persistence thread just to close them immediately. Closing an idle persistence thread interrupts its blocking queue wait and logs the interruption at WARN unconditionally.

For example, with IoTConsensusV2 configured:

CREATE DATABASE root.dal_warn;
INSERT INTO root.dal_warn.d(time, s) VALUES (1, 1);
DROP DATABASE root.dal_warn;

Before this change, region cleanup can initialize DAL resources and emit Interrupted when waiting for taking DeletionResource from blocking queue to serialize. even though no SQL DELETE was issued and cleanup succeeds. After this change, cleanup closes only an existing manager and removes any remaining DAL directory directly. An idle buffer's expected shutdown interruption no longer emits that warning; an interruption while the buffer is open still does.

Implementation

  • Move close-and-remove handling into DeletionResourceManager.closeAndRemoveDAL(groupId) so callers do not need to create a manager to clean up a region. Preserve the existing non-IoTConsensusV2 guard and close-before-directory-removal order.
  • Check isClosed before logging the interrupted initial queue wait. Keep the interrupt flag and existing persistence/draining behavior.
  • Add regression coverage for absent managers, orphaned DAL directories, existing-manager cleanup and recreation, normal idle shutdown, and unexpected interruption.

Validation

  • mvn spotless:apply -pl iotdb-core/datanode (JDK 17): passed.
  • Recompiled the changed manager/buffer sources and the two affected test classes against the locally cached test dependencies, then ran JUnitCore: 10 tests passed.
  • Re-ran the buffer tests against the unmodified master buffer: the new normal-close warning regression test fails as expected.
  • Independently compiled the changed manager/buffer sources with both English and Chinese message classes.
  • JDK 17 Maven reactor: mvn test -pl iotdb-core/datanode -am -Dtest=DeletionResourceManagerCleanupTest,PageCacheDeletionBufferTest,DeletionResourceTest,DeletionRecoverTest -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false: BUILD SUCCESS, all 28 modules succeeded and all 18 DAL tests passed (0 failures/errors).
  • mvn test-compile -pl iotdb-core/datanode -am -P with-zh-locale -DskipTests (JDK 17): BUILD SUCCESS, all 28 modules succeeded. No cluster integration test has been run.

This PR has:

  • been self-reviewed.
  • added comments/Javadoc for the cleanup and shutdown behavior.
  • added unit tests for the affected paths.

Copilot AI balanced review requested due to automatic review settings November 18, 2026 15:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the DAL cleanup and shutdown-interruption logging changes.

No open issues 🎉

What changed in this PR

This PR avoids unnecessary deletion-log (DAL) initialization during IoTConsensusV2 DataRegion cleanup and suppresses expected interruption warnings during normal buffer shutdown.

Changes:

  • Centralize cleanup so it closes existing managers and removes remaining DAL directories without creating a manager.
  • Suppress idle-queue interruption warnings during shutdown while retaining warnings for unexpected interruptions.
  • Add regression tests for manager cleanup, recreation, orphaned directories, and interruption logging.
File Description
iotdb-core/​datanode/​src/​test/​java/​org/​apache/​iotdb/​db/​pipe/​consensus/​deletion/​persist/​PageCacheDeletionBufferTest.java Tests expected and unexpected interruption logging.
iotdb-core/​datanode/​src/​test/​java/​org/​apache/​iotdb/​db/​pipe/​consensus/​deletion/​DeletionResourceManagerCleanupTest.java Tests cleanup without initialization, orphan removal, and manager recreation.
iotdb-core/​datanode/​src/​main/​java/​org/​apache/​iotdb/​db/​storageengine/​dataregion/​DataRegion.java Delegates DAL cleanup to the new manager helper.
iotdb-core/​datanode/​src/​main/​java/​org/​apache/​iotdb/​db/​pipe/​consensus/​deletion/​persist/​PageCacheDeletionBuffer.java Suppresses the queue-interruption warning when the buffer is closed.
iotdb-core/​datanode/​src/​main/​java/​org/​apache/​iotdb/​db/​pipe/​consensus/​deletion/​DeletionResourceManager.java Adds cleanup that avoids manager creation and preserves close-before-removal order.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants