Repository navigation
Do not count ignore_nan_inequality pairs against max_diffs - #672
Open
tony1233200 wants to merge 1 commit into
Open
tony1233200 wants to merge 1 commit into
tony1233200 wants to merge 1 commit into
Conversation
With ignore_nan_inequality=True, NaN-vs-NaN pairs are forgiven when the values are compared, but they still consumed the max_diffs budget: _diff counted every level on entry and the iterable/dict loops counted every pair before descending. With enough NaN pairs (e.g. long numpy arrays), the traversal hit the limit, logged the max-diffs warning and stopped before reaching the real diffs, returning an empty result. Skip the counting for pairs that ignore_nan_inequality forgives, via a shared _is_ignored_nan_inequality helper (also used for the existing forgiveness check in _diff). Real diffs keep counting as before. Fixes qlustered#433
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #433.
With
ignore_nan_inequality=True, NaN-vs-NaN pairs are forgiven when the values are compared — but they still consumed themax_diffsbudget on the way there:_diff()calls_count_diff()on entry for every level, before the NaN forgiveness check further down;_diff_by_forming_pairs_and_comparing_one_by_one()and the_diff_dict()intersect loop count every pair/key before descending.So with enough NaN pairs (the reporter's case: long numpy arrays inside large dicts), the traversal hit the limit, logged
DeepDiff has reached the max number of diffs of 2...and stopped before reaching the real diff, returning{}:Change
deepdiff/diff.pyonly:_is_ignored_nan_inequality(t1, t2)helper holding the existing forgiveness condition; the check inside_diff()now uses it too (no behaviour change there)._count_diff()for pairs the helper identifies as forgiven NaN pairs. Everything else still counts exactly as before, somax_diffskeeps capping traversals that contain real diffs (covered by a sanity test).Tests
tests/test_diff_numpy.py:test_numpy_ignore_nan_inequality_does_not_consume_max_diffs— the issue's exact repro.tests/test_diff_text.py:test_ignore_nan_inequality_does_not_consume_max_diffs(list and dict forms) andtest_max_diffs_still_applies_to_real_diffs.tests/test_diff_text.py+tests/test_diff_numpy.py: 201 passed, 3 skipped. Fulltests/run: the failure set is byte-identical to unmodified master in the same environment (27 pre-existing environment failures, e.g. optional-dependency related; none in the touched paths).Note for the maintainer: per AGENTS.md this targets
dev; the patch was prepared against master, whosedeepdiff/diff.pydiffers fromdev's only by this change, so it should apply cleanly.