Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve AssertionError error message for sequences inside dictionaries #12717

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Improve AssertionError error message for sequences inside dictionaries #12717

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e60e7ee
changes
reaganjlee Aug 16, 2024
1d3d99e
test
reaganjlee Aug 16, 2024
5210674
changelog
reaganjlee Aug 16, 2024
6c5a720
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 16, 2024
770d8ba
mypy
reaganjlee Aug 16, 2024
a1b0239
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 16, 2024
41c48fb
mypy + comment clean
reaganjlee Aug 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/11980.improvement.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Improves `AssertionError` error message for sequences inside dictionaries when not running verbose mode.
18 changes: 15 additions & 3 deletions src/_pytest/assertion/util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from typing import Mapping
from typing import Protocol
from typing import Sequence
from typing import Union
from unicodedata import normalize

from _pytest import outcomes
Expand Down Expand Up @@ -354,7 +355,7 @@ def _compare_eq_iterable(
def _compare_eq_sequence(
left: Sequence[Any],
right: Sequence[Any],
highlighter: _HighlightFunc,
highlighter: Union[_HighlightFunc, Callable[[str], str]],
reaganjlee marked this conversation as resolved.
Show resolved Hide resolved
verbose: int = 0,
) -> list[str]:
comparing_bytes = isinstance(left, bytes) and isinstance(right, bytes)
Expand Down Expand Up @@ -501,10 +502,21 @@ def _compare_eq_dict(
if diff:
explanation += ["Differing items:"]
for k in diff:
left_val = left[k]
right_val = right[k]
if issequence(left_val) and issequence(right_val):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I like this approach, given this is really only specific for sequences inside dictionaries... this is not general because really we could have anything in there, for example other dicts or dataclasses, each with their own custom diff code... 🤔

The core of the original issue was the fact that it was truncating the diff. One idea is to not truncate anything when computing the diff, however truncation is there for a reason, to avoid dumping MBs of text in case of large data structures...

I'm a bit -1 on this solution, given it only fixes a very specific use case and makes the code slightly worse, but I don't have a definitive suggestion either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I initially thought we could take all the cases in _compare_eq_any and use those to create the new diffs but they seemed too long for cases outside of sequences due to their multi-line diffs, e.g.:

E       assert {1, 2} == {1, 3}
E         
E         Extra items in the left set:
E         2
E         Extra items in the right set:
E         3

Besides this though, I figured sequence comparisons would be the most likely use case, and it'd be worth having it there despite the bit of complexity. I could be wrong though!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative method might be to figure out a truncated diff for the other data types such as a simple Differing items for each instead of their specific diffs, but this seems overkill for just comparisons within dictionaries

# explanation.append(saferepr("BLAHBLAH: string"))
expl = _compare_eq_sequence(
left_val, right_val, lambda item: item, verbose
)
expl = "".join(expl)
explanation += [highlighter(saferepr({k: expl}))]
continue

explanation += [
highlighter(saferepr({k: left[k]}))
highlighter(saferepr({k: left_val}))
+ " != "
+ highlighter(saferepr({k: right[k]}))
+ highlighter(saferepr({k: right_val}))
]
extra_left = set_left - set_right
len_extra_left = len(extra_left)
Expand Down
6 changes: 6 additions & 0 deletions testing/test_assertion.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -741,6 +741,12 @@ def test_dict_different_items(self) -> None:
" }",
]

def test_dict_sequence_items(self) -> None:
value1 = {"asdf": [1, 1, 1, 1, 1, 1, 1, 1]}
value2 = {"asdf": [1, 1, 1, 1, 1, 1, 1, 2]}
lines = callequal(value1, value2, verbose=0)
assert "{'asdf': 'At index 7 diff: 1 != 2'}" in lines

def test_sequence_different_items(self) -> None:
lines = callequal((1, 2), (3, 4, 5), verbose=2)
assert lines == [
Expand Down