Skip to content

Initialize linked modules within the current JS value scope - #512

Open
René Larch (r-Larch) wants to merge 5 commits into
microsoft:mainfrom
r-Larch:fix/embedding-module-scope
Open

René Larch (r-Larch) wants to merge 5 commits into
microsoft:mainfrom
r-Larch:fix/embedding-module-scope

Conversation

@r-Larch

@r-Larch René Larch (r-Larch) commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #464.

Calling process._linkedBinding() for a module registered through NodeEmbeddingRuntimeSettings.Modules failed or crashed the process:

Called from Before
a script in NodeEmbeddingThreadRuntime.Run InvalidOperationException: A Root scope cannot be created within another scope.
MainScript thread left without a current scope → JSInvalidThreadAccessException: There is no active JS value scope (the stack overflow in the issue)
an ES module via ImportAsync native assertion (v8_scope_data_->nest_level()) == (nest_level) in CloseV8Scope, exit code 134

Cause: ModuleInitializeCallbackAdapter always opened a new root scope for the napi_env that Node.js creates for each linked module (node_napi_env__::New in EmbeddedRuntime::RegisterModule). Linked modules are initialized lazily, while the runtime's scope is normally current on the JS thread. A nested root scope either throws, or drops its parent and leaves JSValueScope.Current null when it is disposed.

Fix: if a scope is current, open a callback scope within it (the same as for JS→.NET callbacks); otherwise fall back to a root scope as before. The module's exports then share the runtime's context (sync context, require, marshaller), and later calls from JS into the exported .NET functions find a matching parent scope. This is close to the workaround in the issue, which removes the scope entirely, but keeps a scope.

Note: draft PR #500 also rewrites this adapter (TryEnterRuntimeScope(env)) as part of a larger scope refactor. This PR is deliberately minimal so it can land on current main on its own. Happy to close it if #500 lands first and covers #464.

Tests

NodejsEmbeddingTests.LinkedModuleFromScript, LinkedModuleFromMainScript and LinkedModuleFromESModule (plus test/test-linked-module.mjs). Each one registers a linked module with a dirty array pool, calls the module, and then checks that the JS thread is still usable. All three fail on main; two of them crash the test host.

From review: LinkedModuleInitializationError checks that an error thrown by the initializer reaches the caller of _linkedBinding(). The adapter moves a pending exception from the runtime's env to the module's env. LinkedModuleFromHandleScope initializes a module while a handle scope is current.

Local run (Windows x64, .NET SDK 10.0.402): net10.0 160/162, net9.0 129/130, net8.0 129/130. The only failure, napi-dotnet/overloads, fails the same way on main. I could not run net472 locally.

🤖 Generated with Claude Code

Copilot AI balanced review requested due to automatic review settings October 21, 2026 20:14
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Handle-scope initialization and error forwarding to the module environment remain unresolved.

2 open issues
What changed in this PR

Fixes linked-module initialization by sharing the current JavaScript runtime context. Includes the UTF-8 buffer fixes stacked from #510.

Changes:

  • Uses callback scopes for linked-module initialization when a scope is active.
  • Explicitly terminates UTF-8 strings and disposes module-name buffers.
  • Adds buffer, error-propagation, and linked-module regression tests.
File Description
test/​Utf8BufferTests.cs Tests UTF-8 termination and null handling.
test/​TestUtils.cs Adds a helper to dirty pooled buffers.
test/​test-linked-module.mjs Exercises linked binding from an ES module.
test/​NodejsEmbeddingTests.cs Adds error and linked-module integration tests.
src/​NodeApi/​Runtime/​Utf8StringArray.cs Writes string terminators explicitly.
src/​NodeApi/​Runtime/​PooledBuffer.cs Terminates UTF-8 buffers and distinguishes empty from null.
src/​NodeApi/​Runtime/​NodejsRuntime.Embedding.cs Disposes the module-name buffer.
src/​NodeApi/​Runtime/​NodeEmbedding.cs Initializes linked modules within the current context.

Review effort: Balanced

Comment thread src/NodeApi/Runtime/NodeEmbedding.cs Resolved
Comment thread src/NodeApi/Runtime/NodeEmbedding.cs Resolved
René Larch (r-Larch) added a commit to r-Larch/node-api-dotnet that referenced this pull request Oct 21, 2026
Addresses review feedback on microsoft#512:

- Node-API tracks pending exceptions per napi_env, and Node.js only reports
  those of the module's env to the caller of process._linkedBinding(). Errors
  thrown by the initializer within the runtime's scope were only logged as
  "Uncaught N-API callback exception" (DEP0168). Move a pending exception from
  the scope's env to the module's env before returning.
- A callback scope cannot be nested in a handle or escapable scope, so
  initializing a module while one is current threw outside the adapter's
  try/catch. Use the current scope as is in that case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 21, 2026 20:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Native scope lifetimes and cross-environment exception handling warrant human validation across supported runtimes.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

René Larch (r-Larch) added a commit to r-Larch/node-api-dotnet that referenced this pull request Oct 29, 2026
Addresses review feedback on microsoft#512:

- Node-API tracks pending exceptions per napi_env, and Node.js only reports
  those of the module's env to the caller of process._linkedBinding(). Errors
  thrown by the initializer within the runtime's scope were only logged as
  "Uncaught N-API callback exception" (DEP0168). Move a pending exception from
  the scope's env to the module's env before returning.
- A callback scope cannot be nested in a handle or escapable scope, so
  initializing a module while one is current threw outside the adapter's
  try/catch. Use the current scope as is in that case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 02:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the linked-module initialization and UTF-8 buffer changes.

No open issues 🎉

Review effort: Balanced

René Larch (r-Larch) added a commit to r-Larch/node-api-dotnet that referenced this pull request Nov 12, 2026
Addresses review feedback on microsoft#512:

- Node-API tracks pending exceptions per napi_env, and Node.js only reports
  those of the module's env to the caller of process._linkedBinding(). Errors
  thrown by the initializer within the runtime's scope were only logged as
  "Uncaught N-API callback exception" (DEP0168). Move a pending exception from
  the scope's env to the module's env before returning.
- A callback scope cannot be nested in a handle or escapable scope, so
  initializing a module while one is current threw outside the adapter's
  try/catch. Use the current scope as is in that case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 12, 2026 18:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the linked-module initialization scope and exception-propagation changes.

No open issues 🎉

Review effort: Balanced

@vmoroz

Copy link
Copy Markdown
Member

René Larch (@r-Larch) , could you please rebase this PR to fix the merge conflicts.

René Larch (r-Larch) added a commit to r-Larch/node-api-dotnet that referenced this pull request Nov 18, 2026
Addresses review feedback on microsoft#512:

- Node-API tracks pending exceptions per napi_env, and Node.js only reports
  those of the module's env to the caller of process._linkedBinding(). Errors
  thrown by the initializer within the runtime's scope were only logged as
  "Uncaught N-API callback exception" (DEP0168). Move a pending exception from
  the scope's env to the module's env before returning.
- A callback scope cannot be nested in a handle or escapable scope, so
  initializing a module while one is current threw outside the adapter's
  try/catch. Use the current scope as is in that case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@r-Larch

Copy link
Copy Markdown
Contributor Author

Vladimir Morozov (@vmoroz) rebased onto main, thanks for the review.

One change came with the move to LibNode 26 (#517): the error thrown by a linked module's init function no longer reaches the caller of process._linkedBinding(). That is a Node.js bug, not in this PR. GetLinkedBinding() reads module.exports after calling the register function, and that V8 API call clears the pending exception. A plain napi_throw_error on the module's env is lost the same way, and C++ linked bindings are affected too. I opened nodejs/node#67571 with a fix and cctests (they fail on main without it).

So LinkedModuleInitializationError now checks what still holds with LibNode 26: the error does not stay pending on the runtime's env, and the next JS call works. Without moving the exception to the module's env (napi_get_and_clear_last_exception + napi_throw), that test fails. Once LibNode includes the Node.js fix, the test can assert the error message again (8c2f65a).

Copilot AI balanced review requested due to automatic review settings November 18, 2026 11:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the targeted linked-module initialization scope fix.

No open issues 🎉

Review effort: Balanced

@vmoroz

Copy link
Copy Markdown
Member

René Larch (@r-Larch) , while reviewing this PR and its relationship to #500, I also saw this issue with exceptions.
I started to work on it in LibNode - the fork that I use to compile it: microsoft/v8-jsi#289
My plan to update Node.js code after the LibNode code proves to be right.

Copilot AI balanced review requested due to automatic review settings November 18, 2026 22:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the linked-module initialization scope and exception-handling changes.

No open issues 🎉

Review effort: Balanced

@r-Larch

Copy link
Copy Markdown
Contributor Author

Vladimir Morozov (@vmoroz) thanks, that's the same root cause. I tried LibNode 26.1.2 locally: with it, LinkedModuleInitializationError passes again when it asserts that init failed reaches the caller of _linkedBinding(), and it fails on 26.0.18.

Should I bump LibNode to 26.1.2 in this PR and restore that assertion, or keep this PR as it is and do it in a follow-up?

(I also pushed a115ca4 to fix the import order that broke the format check.)

On the Node.js side, nodejs/node#67571 covers only GetLinkedBinding() and has an approval from jasnell. Your change also covers napi_module_register_by_symbol, which is not in it. I can add that case and a test to #67571, or close it in favor of your upstream PR, whichever you prefer.

@vmoroz

Copy link
Copy Markdown
Member

René Larch (@r-Larch) , I am currently working on the PR for this repo to use 26.1.2 plus a few more changes to pipelines. It may take a few hours.
Note that the Release pipeline is currently paused. So, we will have one combined release in the end.

If you have time now, then please add the use of 26.1.2 into your PR. It should help us to merge this PR sooner.

As for the Node.js, I will create a PR as soon as the two PRs mentioned above are done.

@vmoroz

Copy link
Copy Markdown
Member

Actually, if you can adapt my changes to LibNode to your Node.js PR it would be the best.
Note that they are much simpler and do not use the TryCatch.
(I had to use the AI Max thinking mode to make it simpler and do more detailed fix.)

I can sign off your PR in Node.js and hopefully get it merged sooner.

Otherwise, as I just realized, if I create the Node.js PR, it may not look good because I would duplicate your effort and many Node.js team members may be unhappy with such doing.

@vmoroz

Copy link
Copy Markdown
Member

René Larch (@r-Larch) , the PR #520 that updates the LibNode version is merged.
Your PR passed all the validation and I can merge it now.
Let me know if you want me to do it, or you will add a new commit that restores the assertion.

Calling process._linkedBinding() for a module registered through
NodeEmbeddingRuntimeSettings.Modules failed or crashed the process (microsoft#464):

- From a script run via NodeEmbeddingThreadRuntime.Run: "A Root scope cannot
  be created within another scope."
- From MainScript: the nested root scope dropped its parent on dispose, so the
  thread was left without a current scope ("There is no active JS value
  scope", reported as a stack overflow in the issue).
- From an ES module imported via ImportAsync: native assertion
  "(v8_scope_data_->nest_level()) == (nest_level)" in CloseV8Scope.

ModuleInitializeCallbackAdapter always opened a new root scope for the
napi_env that Node.js creates for each linked module. Modules are initialized
lazily, while the runtime's scope is normally current on the JS thread, so
open a callback scope within it instead and only fall back to a root scope
when no scope is current. The module's exports then share the runtime's
context, and later calls from JS into the exported .NET callbacks find a
matching parent scope.

Fixes microsoft#464.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Addresses review feedback on microsoft#512:

- Node-API tracks pending exceptions per napi_env, and Node.js only reports
  those of the module's env to the caller of process._linkedBinding(). Errors
  thrown by the initializer within the runtime's scope were only logged as
  "Uncaught N-API callback exception" (DEP0168). Move a pending exception from
  the scope's env to the module's env before returning.
- A callback scope cannot be nested in a handle or escapable scope, so
  initializing a module while one is current threw outside the adapter's
  try/catch. Use the current scope as is in that case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With LibNode 26 the error thrown by a linked module's init function no
longer reaches the caller of process._linkedBinding(): Node.js
GetLinkedBinding() reads module.exports after the init function returns,
and that V8 API call clears the pending exception. This affects every
linked binding and is fixed in nodejs/node#67571.

The test now checks that the error does not stay pending on the
runtime's env, which fails without moving it to the module's env.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LibNode 26.1.2 includes the fix for the init error being cleared in
GetLinkedBinding() (microsoft/v8-jsi#289), so the error reaches the caller
of process._linkedBinding() again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 19, 2026 10:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the linked-module initialization scope and error-propagation changes.

No open issues 🎉

Review effort: Balanced

@r-Larch

Copy link
Copy Markdown
Contributor Author

Vladimir Morozov (@vmoroz) rebased onto main with LibNode 26.1.2 and restored the assertion in 1a01c64: LinkedModuleInitializationError checks again that init failed reaches the caller of process._linkedBinding(). From my side it's ready to merge.

For Node.js, I adapted your change into nodejs/node#67571: the HasPendingException() checks in GetLinkedBinding() and napi_module_register_by_symbol, plus a cctest for a linked Node-API binding and an addon test for require(), both for an init function that throws after returning new exports. Thanks for offering to review it there.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Process Crash trying: .NET -> load ESM module -> loading .NET module

3 participants