diff --git a/AUTHORS.md b/AUTHORS.md index 842256b..cd3db13 100644 --- a/AUTHORS.md +++ b/AUTHORS.md @@ -61,3 +61,5 @@ Authors in order of the timeline of their contributions: - [William Jamieson](https://github.com/WilliamJamieson) for [NumPy 2.0 compatibility](https://github.com/seperman/deepdiff/pull/422) - [Leo Sin](https://github.com/leoslf) for Supporting Python 3.12 in the build process - [sf-tcalhoun](https://github.com/sf-tcalhoun) for fixing "Instantiating a Delta with a flat_dict_list unexpectedly mutates the flat_dict_list" +- [dtorres-sf](https://github.com/dtorres-sf) for fixing iterable moved items when iterable_compare_func is used. +- [Florian Finkernagel](https://github.com/TyberiusPrime) for pandas and polars support. diff --git a/CHANGELOG.md b/CHANGELOG.md index d5629e1..c700657 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,18 @@ # DeepDiff Change log +- v8-0-0 + + - With the introduction of `threshold_to_diff_deeper`, the values returned are different than in previous versions of DeepDiff. You can still get the older values by setting `threshold_to_diff_deeper=0`. However to signify that enough has changed in this release that the users need to update the parameters passed to DeepDiff, we will be doing a major version update. + - [x] `use_enum_value=True` makes it so when diffing enum, we use the enum's value. It makes it so comparing an enum to a string or any other value is not reported as a type change. + - [x] `threshold_to_diff_deeper=float` is a number between 0 and 1. When comparing dictionaries that have a small intersection of keys, we will report the dictionary as a `new_value` instead of reporting individual keys changed. If you set it to zero, you get the same results as DeepDiff 7.0.1 and earlier, which means this feature is disabled. The new default is 0.33 which means if less that one third of keys between dictionaries intersect, report it as a new object. + - [x] Deprecated `ordered-set` and switched to `orderly-set`. The `ordered-set` package was not being maintained anymore and starting Python 3.6, there were better options for sets that ordered. I forked one of the new implementations, modified it, and published it as `orderly-set`. + - [x] Added `use_log_scale:bool` and `log_scale_similarity_threshold:float`. They can be used to ignore small changes in numbers by comparing their differences in logarithmic space. This is different than ignoring the difference based on significant digits. + - [x] json serialization of reversed lists. + - [x] Fix for iterable moved items when `iterable_compare_func` is used. + - [x] Pandas and Polars support + + - v7-0-1 - Fixes the translation between Difflib opcodes and Delta flat rows. - v7-0-0 diff --git a/README.md b/README.md index 15a05d2..3b99a24 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,18 @@ Tested on Python 3.8+ and PyPy3. Please check the [ChangeLog](CHANGELOG.md) file for the detailed information. +DeepDiff 8-0-0 + +With the introduction of `threshold_to_diff_deeper`, the values returned are different than in previous versions of DeepDiff. You can still get the older values by setting `threshold_to_diff_deeper=0`. However to signify that enough has changed in this release that the users need to update the parameters passed to DeepDiff, we will be doing a major version update. + +- [x] `use_enum_value=True` makes it so when diffing enum, we use the enum's value. It makes it so comparing an enum to a string or any other value is not reported as a type change. +- [x] `threshold_to_diff_deeper=float` is a number between 0 and 1. When comparing dictionaries that have a small intersection of keys, we will report the dictionary as a `new_value` instead of reporting individual keys changed. If you set it to zero, you get the same results as DeepDiff 7.0.1 and earlier, which means this feature is disabled. The new default is 0.33 which means if less that one third of keys between dictionaries intersect, report it as a new object. +- [x] Deprecated `ordered-set` and switched to `orderly-set`. The `ordered-set` package was not being maintained anymore and starting Python 3.6, there were better options for sets that ordered. I forked one of the new implementations, modified it, and published it as `orderly-set`. +- [x] Added `use_log_scale:bool` and `log_scale_similarity_threshold:float`. They can be used to ignore small changes in numbers by comparing their differences in logarithmic space. This is different than ignoring the difference based on significant digits. +- [x] json serialization of reversed lists. +- [x] Fix for iterable moved items when `iterable_compare_func` is used. +- [x] Pandas and Polars support + DeepDiff 7-0-1 - Fixes the translation between Difflib opcodes and Delta flat rows. diff --git a/docs/authors.rst b/docs/authors.rst index 5d18e02..1ca60ae 100644 --- a/docs/authors.rst +++ b/docs/authors.rst @@ -89,6 +89,10 @@ Authors in order of the timeline of their contributions: - `sf-tcalhoun `__ for fixing “Instantiating a Delta with a flat_dict_list unexpectedly mutates the flat_dict_list” +- `dtorres-sf `__ for fixing iterable + moved items when iterable_compare_func is used. +- `Florian Finkernagel `__ for pandas +and polars support. .. _Sep Dehpour (Seperman): http://www.zepworks.com .. _Victor Hahn Castell: http://hahncastell.de diff --git a/docs/buildme.py b/docs/buildme.py index 1334a8a..7aef5eb 100755 --- a/docs/buildme.py +++ b/docs/buildme.py @@ -5,12 +5,9 @@ It will remove the contents of the BUILD_PATH folder and recreate it. """ import os -import time import datetime import shutil from dotenv import load_dotenv -from watchdog.observers import Observer -from watchdog.events import FileSystemEventHandler from sphinx.cmd.build import main as sphinx_main CACHE_PATH = '/tmp/sphinx_doctree' @@ -27,45 +24,15 @@ def delete_dir_contents(directory): shutil.rmtree(directory) -class MyHandler(FileSystemEventHandler): - - def __init__(self): - self.last_modified = datetime.datetime.now() - - def on_any_event(self, event): - load_dotenv(override=True) - build_path = os.environ.get('BUILD_PATH', '_build') - doc_version = os.environ.get('DOC_VERSION', '') - if not build_path.endswith('/'): - build_path = build_path + '/' - build_path += doc_version - if event is None: - print('initial build') - else: - print(f'event type: {event.event_type} path : {event.src_path}') - if event is not None and ( - datetime.datetime.now() - self.last_modified < datetime.timedelta(seconds=2)): - return - else: - self.last_modified = datetime.datetime.now() - argv = ['-b', 'html', '-d', CACHE_PATH, '.', build_path] - ensure_dir(build_path) - delete_dir_contents(build_path) - delete_dir_contents('/tmp/sphinx_doctree') # Disable this for faster build time but it might not properly invalidate the cache - sphinx_main(argv) - print('waiting for file changes. Press Ctrl+c to cancel.') - - if __name__ == "__main__": - event_handler = MyHandler() - event_handler.on_any_event(event=None) - observer = Observer() - observer.schedule(event_handler, path='.', recursive=True) - observer.start() - - try: - while True: - time.sleep(1) - except KeyboardInterrupt: - observer.stop() - observer.join() + load_dotenv(override=True) + build_path = os.environ.get('BUILD_PATH', '_build') + doc_version = os.environ.get('DOC_VERSION', '') + if not build_path.endswith('/'): + build_path = build_path + '/' + build_path += doc_version + argv = ['-b', 'html', '-d', CACHE_PATH, '.', build_path] + ensure_dir(build_path) + delete_dir_contents(build_path) + delete_dir_contents('/tmp/sphinx_doctree') # Disable this for faster build time but it might not properly invalidate the cache + sphinx_main(argv) diff --git a/docs/changelog.rst b/docs/changelog.rst index 085f2f3..b72e579 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -5,6 +5,14 @@ Changelog DeepDiff Changelog + - With the introduction of `threshold_to_diff_deeper`, the values returned are different than in previous versions of DeepDiff. You can still get the older values by setting `threshold_to_diff_deeper=0`. However to signify that enough has changed in this release that the users need to update the parameters passed to DeepDiff, we will be doing a major version update. + - `use_enum_value=True` makes it so when diffing enum, we use the enum's value. It makes it so comparing an enum to a string or any other value is not reported as a type change. + - `threshold_to_diff_deeper=float` is a number between 0 and 1. When comparing dictionaries that have a small intersection of keys, we will report the dictionary as a `new_value` instead of reporting individual keys changed. If you set it to zero, you get the same results as DeepDiff 7.0.1 and earlier, which means this feature is disabled. The new default is 0.33 which means if less that one third of keys between dictionaries intersect, report it as a new object. + - Deprecated `ordered-set` and switched to `orderly-set`. The `ordered-set` package was not being maintained anymore and starting Python 3.6, there were better options for sets that ordered. I forked one of the new implementations, modified it, and published it as `orderly-set`. + - Added `use_log_scale:bool` and `log_scale_similarity_threshold:float`. They can be used to ignore small changes in numbers by comparing their differences in logarithmic space. This is different than ignoring the difference based on significant digits. + - json serialization of reversed lists. + - Fix for iterable moved items when `iterable_compare_func` is used. + - Pandas and Polars support - v7-0-1 diff --git a/docs/ignore_types_or_values.rst b/docs/ignore_types_or_values.rst index 85b3855..fc19109 100644 --- a/docs/ignore_types_or_values.rst +++ b/docs/ignore_types_or_values.rst @@ -108,6 +108,8 @@ ignore_type_in_groups: Tuple or List of Tuples, default = None 2. or ignore_type_in_groups=[(str, bytes), (typeA, typeB)] +Note: The example below shows you have to use this feature. For enum types, however, you can just use :ref:`use_enum_value_label` + Example: Ignore Enum to string comparison >>> from deepdiff import DeepDiff >>> from enum import Enum diff --git a/docs/index.rst b/docs/index.rst index cf623a8..466b9e3 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -31,23 +31,35 @@ The DeepDiff library includes the following modules: What Is New *********** +DeepDiff 8-0-0 +-------------- + + - With the introduction of `threshold_to_diff_deeper`, the values returned are different than in previous versions of DeepDiff. You can still get the older values by setting `threshold_to_diff_deeper=0`. However to signify that enough has changed in this release that the users need to update the parameters passed to DeepDiff, we will be doing a major version update. + - `use_enum_value=True` makes it so when diffing enum, we use the enum's value. It makes it so comparing an enum to a string or any other value is not reported as a type change. + - `threshold_to_diff_deeper=float` is a number between 0 and 1. When comparing dictionaries that have a small intersection of keys, we will report the dictionary as a `new_value` instead of reporting individual keys changed. If you set it to zero, you get the same results as DeepDiff 7.0.1 and earlier, which means this feature is disabled. The new default is 0.33 which means if less that one third of keys between dictionaries intersect, report it as a new object. + - Deprecated `ordered-set` and switched to `orderly-set`. The `ordered-set` package was not being maintained anymore and starting Python 3.6, there were better options for sets that ordered. I forked one of the new implementations, modified it, and published it as `orderly-set`. + - Added `use_log_scale:bool` and `log_scale_similarity_threshold:float`. They can be used to ignore small changes in numbers by comparing their differences in logarithmic space. This is different than ignoring the difference based on significant digits. + - json serialization of reversed lists. + - Fix for iterable moved items when `iterable_compare_func` is used. + - Pandas and Polars support + + DeepDiff 7-0-0 -------------- -- DeepDiff 7 comes with an improved delta object. `Delta to flat - dictionaries `__ - have undergone a major change. We have also introduced `Delta - serialize to flat - rows `__. -- Subtracting delta objects have dramatically improved at the cost of - holding more metadata about the original objects. -- When ``verbose=2``, and the “path” of an item has changed in a report - between t1 and t2, we include it as ``new_path``. -- ``path(use_t2=True)`` returns the correct path to t2 in any reported - change in the - ```tree view`` `__ -- Python 3.7 support is dropped and Python 3.12 is officially - supported. + - DeepDiff 7 comes with an improved delta object. `Delta to flat + dictionaries `__ + have undergone a major change. We have also introduced `Delta + serialize to flat + rows `__. + - Subtracting delta objects have dramatically improved at the cost of + holding more metadata about the original objects. + - When ``verbose=2``, and the “path” of an item has changed in a report + between t1 and t2, we include it as ``new_path``. + - ``path(use_t2=True)`` returns the correct path to t2 in any reported + change in the `tree view `__ + - Python 3.7 support is dropped and Python 3.12 is officially + supported. DeepDiff 6-7-1 @@ -57,7 +69,7 @@ DeepDiff 6-7-1 is used. - Better handling of force adding a delta to an object. - Fix for - ```Can't compare dicts with both single and double quotes in keys`` `__ + `Can't compare dicts with both single and double quotes in keys `__ - Updated docs for Inconsistent Behavior with math_epsilon and ignore_order = True diff --git a/docs/numbers.rst b/docs/numbers.rst index e82bed4..0c616ad 100644 --- a/docs/numbers.rst +++ b/docs/numbers.rst @@ -158,12 +158,15 @@ use_log_scale: Boolean, default=False >>> >>> DeepDiff(t1, t2) {'values_changed': {"root['foo']": {'new_value': 140, 'old_value': 110}, "root['bar']": {'new_value': 298, 'old_value': 306}}} + >>> DeepDiff(t1, t2, use_log_scale=True, log_scale_similarity_threshold=0.01) {'values_changed': {"root['foo']": {'new_value': 140, 'old_value': 110}, "root['bar']": {'new_value': 298, 'old_value': 306}}} + >>> DeepDiff(t1, t2, use_log_scale=True, log_scale_similarity_threshold=0.1) {'values_changed': {"root['foo']": {'new_value': 140, 'old_value': 110}}} + >>> DeepDiff(t1, t2, use_log_scale=True, log_scale_similarity_threshold=0.3) - { + {} .. _log_scale_similarity_threshold_label: @@ -172,7 +175,7 @@ Log Scale Similarity Threshold ------------ log_scale_similarity_threshold: float, default = 0.1 - :ref:`use_log_scale_label` along with log_scale_similarity_threshold can be used to ignore small changes in numbers by comparing their differences in logarithmic space. This is different than ignoring the difference based on significant digits. See above example. + :ref:`use_log_scale_label` along with log_scale_similarity_threshold can be used to ignore small changes in numbers by comparing their differences in logarithmic space. This is different than ignoring the difference based on significant digits. See the example above. Performance Improvement of Numbers diffing diff --git a/requirements-dev.txt b/requirements-dev.txt index eb68923..5241e2b 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -8,7 +8,6 @@ numpy==2.0.0 pytest==8.2.2 pytest-cov==5.0.0 python-dotenv==1.0.1 -watchdog>=4.0.1 Sphinx==6.2.1 # We use the html style that is not supported in Sphinx 7 anymore. sphinx-sitemap==2.6.0 sphinxemoji==0.2.0