Skip to content

Let StopIteration be caught in try/except - #920

Open
HoneyBearTech wants to merge 1 commit into
custom-components:masterfrom
HoneyBearTech:fix-stopiteration
Open

HoneyBearTech wants to merge 1 commit into
custom-components:masterfrom
HoneyBearTech:fix-stopiteration

Conversation

@HoneyBearTech

@HoneyBearTech HoneyBearTech commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

What was broken
A StopIteration could never be caught in pyscript, even in the simplest script:

it = iter([])
try:
    next(it)
except StopIteration:              # never matches
    log.info("empty")
# RuntimeError: coroutine raised StopIteration

def first(items):
    for item in items:
        return item
    raise StopIteration("no items")

try:
    first([])
except StopIteration as exc:       # never matches either, and exc.value is lost
    pass
# RuntimeError: coroutine raised StopIteration

with contextlib.suppress(StopIteration):
    next(it)                       # not suppressed: RuntimeError

An uncaught one was reported as RuntimeError: coroutine raised StopIteration, which doesn't say what really happened.

Why

  • pyscript evaluates every statement and expression in an async method of AstEval (aeval() and the ast_* methods).
  • Since PEP 479, Python turns a StopIteration that leaves a coroutine into RuntimeError("coroutine raised StopIteration").
  • So a StopIteration from next(), an iterator's __next__, any native function, or a script's own raise StopIteration left the coroutine it was raised in as a RuntimeError. except StopIteration never matched, with blocks passed the RuntimeError to __exit__, and the original exception and its value were only reachable through __cause__.

What I changed

  • A private exception, EvalStopIteration (a RuntimeError subclass), carries the original StopIteration out of pyscript's coroutines. Only eval.py creates it, at the point where Python's conversion would happen:
    • aeval(), and the operator nodes (binary and unary operators and comparisons) around their per-operator methods, catch Python's RuntimeError as it arrives from the coroutine they awaited. They recognise Python's own conversion by its exact type, its StopIteration cause and a traceback that ends at the awaiting frame, never by message or arguments. So a script's own raise RuntimeError("coroutine raised StopIteration") stays a RuntimeError, and the exception's arguments are never compared or touched.
    • Methods that run native code catch the StopIteration itself and raise the carrier instead:
      • function calls (call_func), including creating an instance (a native __new__ or __init__, a metaclass __call__) and a zero-argument super();
      • assignment (__setitem__, unpacking) and * unpacking;
      • setting and deleting an attribute (obj.x = v, obj.x += 1, del obj.x): a native property setter or deleter, a native __setattr__/__delattr__ or a metaclass's, and the property lookup before it. The catch is inside EvalAttrSet.setattr_async and delattr_async, the helpers Support @property and @classmethod on pyscript class methods #918 added, because by the time the exception reaches the assignment code Python has already turned it into a RuntimeError;
      • name lookups in a class namespace from a metaclass's __prepare__;
      • the comprehension loops and their scope save and restore;
      • matching an except clause (a metaclass's __instancecheck__);
      • class annotations and nonlocal resolution;
      • eval(), exec() and locals().
    • async for, async def and async with now go straight to the same methods as for, def and with, so aeval() checks them directly.
  • try/except matches and binds the original StopIteration. Tuple handlers, except Exception as exc, .value and .args work as in Python.
    • The except type expressions, the handler bodies and the finally clause run with the original StopIteration as the exception being handled. So a bare raise, raise ... from, __context__ and sys.exception() behave as in Python.
    • ast_try re-raises explicitly, and no longer returns from inside a finally block.
  • with and async with pass the original exception, its type and its traceback to __exit__/__aexit__, which can suppress it. contextlib.suppress(StopIteration), @contextlib.contextmanager and nullcontext() work.
  • A bare raise re-raises sys.exception() explicitly, and raises RuntimeError("No active exception to reraise") when there is none, as in Python.
  • for loops, comprehensions and generator expressions still end normally when their iterator runs out. An element of a generator expression that raises StopIteration lets it out of the generator, so Python itself gives RuntimeError("generator raised StopIteration") with the StopIteration as its cause, as in Python.
  • What stays a RuntimeError:
    • a genuine one that Python raises, for example from a native generator or coroutine that lets a StopIteration escape;
    • what native code that calls back into pyscript receives, which is the same RuntimeError Python would give it.
  • An uncaught StopIteration is reported as StopIteration, with the script line that raised it. EvalExceptionFormatter builds that traceback, and the Jupyter kernel's error reply now uses the same exception for its name and value.
  • The extra handling only runs when a StopIteration is involved, so try statements are as fast as before. A loop of 10,000 try statements runs within a few percent of master, both without an exception and with a caught ValueError.

Testing

  • New test_stop_iteration_matches_python in tests/test_unit_eval.py. It runs 57 scripts in CPython and in pyscript and compares every result. They cover:
    • next() with and without a default, raise StopIteration, raise StopIteration(5) and several arguments;
    • except StopIteration, tuple handlers, except Exception as exc, .value and .args, handler expressions, and an except expression that fails;
    • a bare raise in a handler, raise ... from, __context__, sys.exception(), and a handler that clears the traceback before re-raising;
    • finally, including one that clears the traceback of the exception passing through it;
    • with and async with whose __exit__/__aexit__ sees and suppresses it, contextlib.suppress, @contextlib.contextmanager and nullcontext();
    • a function that raises it and a caller that catches it, at top level and in functions;
    • native callbacks (a sorted key, functools.partial, a property, map), native async functions and tasks (where a genuine RuntimeError stays one);
    • for loops, list, set and dict comprehensions, and generator expressions, including an element that raises it;
    • native special methods that raise it: +, ==, not, unary -, in, __setitem__, a property setter, __iter__ and __hash__;
    • properties and classmethods (from Support @property and @classmethod on pyscript class methods #918): a native or pyscript getter, setter and deleter through obj.x, obj.x = v, obj.x += 1 and del obj.x, a native __setattr__/__delattr__, a metaclass's __setattr__/__delattr__ and property, a classmethod called on the class and on an instance, and a getter's AttributeError falling back to a pyscript or compiled __getattr__ that raises it, each checking the exception's type and value, including a StopIteration subclass;
    • a native setter or deleter that runs a native generator or coroutine which lets a StopIteration escape, where the genuine RuntimeError stays one;
    • a class namespace from __prepare__, eval() and exec(), and decorators that raise it;
    • creating instances: raised in __init__ and caught by the caller of Cls(), in a base __init__ reached through super().__init__() (pyscript or compiled) or Base.__init__(), and in __new__ (pyscript or compiled);
    • a script's own RuntimeError("coroutine raised StopIteration"), and a RuntimeError whose argument has an __eq__ that must never be called.
  • New test_stop_iteration_uncaught_report checks the error report:
    • 3 uncaught scripts are reported as StopIteration with the script line, and without RuntimeError;
    • an exception raised while handling one shows the StopIteration as its context;
    • in a generator expression, the report shows the StopIteration as the direct cause of RuntimeError: generator raised StopIteration.
  • test_jupyter_kernel_msgs checks that raise StopIteration(5) gives the error name StopIteration and value 5.
  • The full test suite, ruff, pylint, mypy, codespell and sphinx-build -W pass on Python 3.14.

Docs
A new item in Language Limitations says that StopIteration can be caught as in Python, and lists the two small differences that remain:

  • a pyscript async def that raises it and is awaited through a native task gives StopIteration, where Python gives RuntimeError("coroutine raised StopIteration");
  • a with statement whose __exit__/__aexit__ returns an object whose __bool__ or __len__ raises StopIteration raises RuntimeError instead.

Note
Remaining differences from Python that this PR doesn't fix:

  • except* (exception groups) isn't supported in pyscript, as before.
  • except BaseException only catches Exception subclasses in pyscript, as before.
  • A pyscript async def that raises StopIteration, awaited through a native task (for example a @pyscript_compile coroutine doing await asyncio.create_task(f())), gives StopIteration where Python gives RuntimeError("coroutine raised StopIteration").
  • A with block whose __exit__/__aexit__ returns an object whose __bool__ or __len__ raises StopIteration gives RuntimeError instead of StopIteration.
  • An object whose compiled __getattribute__ raises StopIteration for every name, including __class__, can still give RuntimeError: pyscript's own isinstance() check on a value, before the attribute is looked up, runs that method outside the catch. This was already the case before this rebase.

My lambda PR (#919) and this PR both change eval.py. Whichever merges second needs a rebase of about 6 hunks, in ast_lambda, try_handlers, loopvar_scope_save and the list, dict and set comprehension loops, plus the same StopIteration catch in the two comprehension-scope helpers the lambda PR adds (loopvar_scope_enter and loopvar_scope_exit). I'm happy to do it. Both PRs also add tests at the end of tests/test_unit_eval.py, which is a small "keep both" conflict. This PR is rebased onto master after #918. My super().prop PR (#923) merges cleanly with this PR's eval.py changes. Both add tests at the end of the test file, so whichever merges later needs a small test-file update.

Python turns a StopIteration that leaves a coroutine into
RuntimeError("coroutine raised StopIteration"), so next() on an empty
iterator, a native __next__ or a raise StopIteration in a script could
never be caught with except StopIteration.

Where that happens to one of the interpreter's own coroutines, the
StopIteration is now put in a private carrier exception right away:
aeval() and the operator nodes catch Python's RuntimeError as it
arrives from the coroutine they await, and the helpers that run native
code catch the StopIteration itself. try/except matches and binds the original
StopIteration, and with blocks pass it to __exit__. The handler
expressions and bodies, the finally clause and __exit__ run with it as
the exception being handled, so a bare raise, exception context and
sys.exception() behave as in Python. An element of a generator
expression that raises it gives RuntimeError("generator raised
StopIteration"), as in Python. An uncaught one is reported as
StopIteration with the script line, also in Jupyter. A RuntimeError
from a native generator or coroutine, or one a script raises, is
unchanged.

The two small differences that remain are listed in the language
limitations in the docs.

Tests compare the results with CPython.

Signed-off-by: Sean Stoube <31805425+HoneyBearTech@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant