You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
GH-44949: [C++][JSON] Reject duplicate field names in explicit schemas - #55641
When an explicit JSON schema contains two fields named foo, the parser silently reuses the first field's index. The parsed fields then no longer match the explicit schema, and the reader can report an unrelated conversion error. Reject duplicate sibling field names with an Invalid status that identifies the duplicate.
What changes are included in this PR?
Check the existing field-name index while constructing JSON object builders. This also covers nested structs and structs inside lists.
Document the uniqueness requirement for JSON explicit schemas.
Add parameterized regression tests for non-adjacent duplicate fields, identical and different field types, empty names, and all unexpected-field modes. Verify that the same name in separate objects remains valid.
Exercise the issue's reproducer through the table reader with threading enabled and disabled.
Are these changes tested?
Built from main commit 55803660f015f9945fe1b9bc84f0c138ae70ce98 with Clang 22.1.8, Debug assertions, and BUILD_WARNING_LEVEL=CHECKIN (-Werror).
Before the fix: the new regression selection ran 8 tests; 5 failed as expected and the 3 valid-schema cases passed. The table-reader cases reproduced the conversion error reported in the issue.
After the fix: the full arrow-json-test executable passed all 254 tests across 60 suites, including the 8 added parameterized test instances.
clang-format 18.1.8 --dry-run --Werror, cpplint 1.6.1 --quiet --verbose=2, and git diff --check passed for all changed files.
The complete Arrow CI matrix and Python bindings were not built or run locally.
Are there any user-facing changes?
JSON reading now reports Invalid: JSON parse error: duplicate field 'foo' in explicit schema for duplicate sibling names. General Arrow schemas and CSV handling are unchanged. There are no public API signature changes.
The reason will be displayed to describe this comment to others. Learn more.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.
FindFieldIndex uses the existing std::unordered_map<std::string_view, int> (name_to_index_), rather than scanning the accumulated fields. Its expected lookup cost is constant in the number of fields, so this adds one hash-table lookup per field and preserves expected linear initialization complexity (with string hashing also depending on field-name length). The hash table already exists and is populated by AddField.
This does add a constant-factor cost: AddField performs its own lookup afterward. I have not benchmarked the percentage overhead, so I cannot quantify it yet. Pathological hash collisions can still produce worst-case quadratic behavior, as with the existing map-based implementation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Fixes #44949.
When an explicit JSON schema contains two fields named
foo, the parser silently reuses the first field's index. The parsed fields then no longer match the explicit schema, and the reader can report an unrelated conversion error. Reject duplicate sibling field names with anInvalidstatus that identifies the duplicate.What changes are included in this PR?
Are these changes tested?
Built from main commit
55803660f015f9945fe1b9bc84f0c138ae70ce98with Clang 22.1.8, Debug assertions, andBUILD_WARNING_LEVEL=CHECKIN(-Werror).arrow-json-testexecutable passed all 254 tests across 60 suites, including the 8 added parameterized test instances.clang-format 18.1.8 --dry-run --Werror,cpplint 1.6.1 --quiet --verbose=2, andgit diff --checkpassed for all changed files.The complete Arrow CI matrix and Python bindings were not built or run locally.
Are there any user-facing changes?
JSON reading now reports
Invalid: JSON parse error: duplicate field 'foo' in explicit schemafor duplicate sibling names. General Arrow schemas and CSV handling are unchanged. There are no public API signature changes.