Skip to content

[mypyc] Raise IndexError for indexes that don't fit in Py_ssize_t - #22477

Open
Jahongir-Qurbonov wants to merge 1 commit into
python:masterfrom
Jahongir-Qurbonov:mypyc-large-index-error
Open

Jahongir-Qurbonov wants to merge 1 commit into
python:masterfrom
Jahongir-Qurbonov:mypyc-large-index-error

Conversation

@Jahongir-Qurbonov

Copy link
Copy Markdown
Contributor

CPython raises IndexError: cannot fit 'int' into an index-sized integer when a list, tuple, str or bytes object is indexed with an int that doesn't fit in Py_ssize_t. Compiled code raised other exceptions:

  • OverflowError: Python int too large to convert to C ssize_t from the list, tuple and str item access primitives (get, set and borrowed get).
  • ValueError: int too large to convert to i64 from the specialized b[i], ord(s[i]) and s[i] == "x" code (and BytesWriter/StringWriter indexing). A literal index such as b[2**63] was even a compile error.
  • The lib-rt primitives also raised OverflowError for ints that fit in Py_ssize_t but aren't short ints (e.g. 2**62), which CPython treats as just out of range ("list index out of range" etc.). list.pop() and list.insert() had the same problem: CPython raises IndexError: pop index out of range and clamps the index, respectively.

The slow paths for non-short int indexes now use PyNumber_AsSsize_t(index, PyExc_IndexError), like CPython. Values that fit fall through to the usual "index out of range" error. The specialized code uses a new slow-path primitive when coercing a non-short int index to i64; the fast path for short ints doesn't change. list.pop() and list.insert() only raise OverflowError when the index doesn't fit in Py_ssize_t, as in CPython.

Slicing and bytearray indexing already matched CPython.

This touches the same run-u64.test lines as #22471 (the OverflowError assertions for tuple/str indexes with large u64 values); I'll resolve the conflict in whichever PR lands second.

CPython raises IndexError("cannot fit 'int' into an index-sized
integer") when indexing a list, tuple, str or bytes object with an int
that doesn't fit in Py_ssize_t. Compiled code raised OverflowError
instead from the lib-rt item access primitives, and ValueError ("int
too large to convert to i64") from the specialized bytes[i], ord(s[i])
and s[i] == "x" code. The lib-rt primitives also raised OverflowError
for ints that fit in Py_ssize_t but aren't short ints (e.g. 2**62),
which CPython treats as just out of range.

Set the right IndexError in the non-short int slow paths of the list,
tuple, str and bytes item access primitives. In the specialized code,
use a slow path primitive that raises IndexError when coercing a
non-short int index to i64. The fast path for short ints is unchanged.
This also fixes a compile error for an index literal that doesn't fit
in i64 (e.g. b[2**63]).

list.pop() and list.insert() now only raise OverflowError if the index
doesn't fit in Py_ssize_t, like CPython: other non-short indexes are
out of range for pop() and clamped for insert().
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant