Skip to content

GH-44949: [C++][JSON] Reject duplicate field names in explicit schemas - #55641

Open
AmanBoronbaev wants to merge 1 commit into
apache:mainfrom
AmanBoronbaev:fix/gh44949-json-duplicate-schema-fields
Open

AmanBoronbaev wants to merge 1 commit into
apache:mainfrom
AmanBoronbaev:fix/gh44949-json-duplicate-schema-fields

Conversation

@AmanBoronbaev

@AmanBoronbaev AmanBoronbaev commented Nov 16, 2026 •

Copy link
Copy Markdown

Rationale for this change

Fixes #44949.

When an explicit JSON schema contains two fields named foo, the parser silently reuses the first field's index. The parsed fields then no longer match the explicit schema, and the reader can report an unrelated conversion error. Reject duplicate sibling field names with an Invalid status that identifies the duplicate.

What changes are included in this PR?

  • Check the existing field-name index while constructing JSON object builders. This also covers nested structs and structs inside lists.
  • Document the uniqueness requirement for JSON explicit schemas.
  • Add parameterized regression tests for non-adjacent duplicate fields, identical and different field types, empty names, and all unexpected-field modes. Verify that the same name in separate objects remains valid.
  • Exercise the issue's reproducer through the table reader with threading enabled and disabled.

Are these changes tested?

Built from main commit 55803660f015f9945fe1b9bc84f0c138ae70ce98 with Clang 22.1.8, Debug assertions, and BUILD_WARNING_LEVEL=CHECKIN (-Werror).

  • Before the fix: the new regression selection ran 8 tests; 5 failed as expected and the 3 valid-schema cases passed. The table-reader cases reproduced the conversion error reported in the issue.
  • After the fix: the full arrow-json-test executable passed all 254 tests across 60 suites, including the 8 added parameterized test instances.
  • clang-format 18.1.8 --dry-run --Werror, cpplint 1.6.1 --quiet --verbose=2, and git diff --check passed for all changed files.

The complete Arrow CI matrix and Python bindings were not built or run locally.

Are there any user-facing changes?

JSON reading now reports Invalid: JSON parse error: duplicate field 'foo' in explicit schema for duplicate sibling names. General Arrow schemas and CSV handling are unchanged. There are no public API signature changes.

Copilot AI balanced review requested due to automatic review settings November 16, 2026 22:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #44949 has been automatically assigned in GitHub to PR creator.

@AmanBoronbaev

Copy link
Copy Markdown
Author

Regarding the initialization-complexity question:

FindFieldIndex uses the existing std::unordered_map<std::string_view, int> (name_to_index_), rather than scanning the accumulated fields. Its expected lookup cost is constant in the number of fields, so this adds one hash-table lookup per field and preserves expected linear initialization complexity (with string hashing also depending on field-name length). The hash table already exists and is populated by AddField.

This does add a constant-factor cost: AddField performs its own lookup afterward. I have not benchmarked the percentage overhead, so I cannot quantify it yet. Pathological hash collisions can still produce worst-case quadratic behavior, as with the existing map-based implementation.

Code at the submitted commit:

  • Lookup:
    int FindFieldIndex(std::string_view name) const {
    auto it = name_to_index_.find(name);
    return it != name_to_index_.end() ? it->second : -1;
    }
  • Existing AddField lookup:
    int AddField(std::string_view name, BuilderPtr builder) {
    auto index = FindFieldIndex(name);
    if (ARROW_PREDICT_TRUE(index == -1)) {
    name = context_->InternString(name);
    index = num_fields();
    field_infos_.push_back(FieldInfo{name, builder});
    name_to_index_.emplace(name, index);
    }
  • Map declaration:
    std::unordered_map<std::string_view, int> name_to_index_;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python] read_json error message not clear when duplicate field in schema

2 participants