Skip to content

Do not count ignore_nan_inequality pairs against max_diffs - #672

Open
tony1233200 wants to merge 1 commit into
qlustered:devfrom
tony1233200:fix/433-max-diffs-nan
Open

tony1233200 wants to merge 1 commit into
qlustered:devfrom
tony1233200:fix/433-max-diffs-nan

Conversation

@tony1233200

Copy link
Copy Markdown

Summary

Fixes #433.

With ignore_nan_inequality=True, NaN-vs-NaN pairs are forgiven when the values are compared — but they still consumed the max_diffs budget on the way there:

  • _diff() calls _count_diff() on entry for every level, before the NaN forgiveness check further down;
  • _diff_by_forming_pairs_and_comparing_one_by_one() and the _diff_dict() intersect loop count every pair/key before descending.

So with enough NaN pairs (the reporter's case: long numpy arrays inside large dicts), the traversal hit the limit, logged DeepDiff has reached the max number of diffs of 2... and stopped before reaching the real diff, returning {}:

a = np.array([np.nan, np.nan, np.nan, np.nan, 1])
b = np.array([np.nan, np.nan, np.nan, np.nan, 2])
DeepDiff(a, b, ignore_nan_inequality=True, max_diffs=2)
# before: {}  (+ max-diffs warning)
# after:  {'values_changed': {'root[4]': {'new_value': 2.0, 'old_value': 1.0}}}

Change

deepdiff/diff.py only:

  • New _is_ignored_nan_inequality(t1, t2) helper holding the existing forgiveness condition; the check inside _diff() now uses it too (no behaviour change there).
  • The three counting sites above skip _count_diff() for pairs the helper identifies as forgiven NaN pairs. Everything else still counts exactly as before, so max_diffs keeps capping traversals that contain real diffs (covered by a sanity test).

Tests

  • tests/test_diff_numpy.py: test_numpy_ignore_nan_inequality_does_not_consume_max_diffs — the issue's exact repro.
  • tests/test_diff_text.py: test_ignore_nan_inequality_does_not_consume_max_diffs (list and dict forms) and test_max_diffs_still_applies_to_real_diffs.

tests/test_diff_text.py + tests/test_diff_numpy.py: 201 passed, 3 skipped. Full tests/ run: the failure set is byte-identical to unmodified master in the same environment (27 pre-existing environment failures, e.g. optional-dependency related; none in the touched paths).

Note for the maintainer: per AGENTS.md this targets dev; the patch was prepared against master, whose deepdiff/diff.py differs from dev's only by this change, so it should apply cleanly.

Disclosure: this change was prepared with the help of an AI coding assistant; all code and tests were run and verified locally as described above.

With ignore_nan_inequality=True, NaN-vs-NaN pairs are forgiven when
the values are compared, but they still consumed the max_diffs budget:
_diff counted every level on entry and the iterable/dict loops counted
every pair before descending. With enough NaN pairs (e.g. long numpy
arrays), the traversal hit the limit, logged the max-diffs warning and
stopped before reaching the real diffs, returning an empty result.

Skip the counting for pairs that ignore_nan_inequality forgives, via a
shared _is_ignored_nan_inequality helper (also used for the existing
forgiveness check in _diff). Real diffs keep counting as before.

Fixes qlustered#433
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants