Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Qwen2-VL image processor fast #35733

Merged

Conversation

yonigozlan
Copy link
Member

What does this PR do?

Adds a fast image processor for Qwen2-VL. Follows issue #33810.
This image processor is a result of this work on comparing different image processing method.

More on how to use fast image processors here.

This PR doesn't include optimization for batch processing as this will be added to the base image processor fast in this PR #35069.
Once #35069 is merged, this fast processor will need to be refactored and the batch processing optimizations will be added.

Fixes #34272

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@yonigozlan yonigozlan changed the title add qwen2_vl image processor fast add Qwen2-VL image processor fast Jan 16, 2025
@yonigozlan yonigozlan requested a review from qubvel January 16, 2025 16:52
@yonigozlan
Copy link
Member Author

yonigozlan commented Jan 16, 2025

Hey @qubvel !
I'm having an issue here with make fix-copies trying to add

class TimmWrapperImageProcessor(metaclass=DummyObject):
    _backends = ["torchvision"]

    def __init__(self, *args, **kwargs):
        requires_backends(self, ["torchvision"])

to dummy_torchvision_objects.py, but once it's added, I'm getting another error with python utils/check_inits.py:

Traceback (most recent call last):
  File "/home/ubuntu/models_implem/transformers/utils/check_inits.py", line 373, in <module>
    check_all_inits()
  File "/home/ubuntu/models_implem/transformers/utils/check_inits.py", line 298, in check_all_inits
    raise ValueError("\n\n".join(failures))
ValueError: Problem in src/transformers/__init__.py, both halves do not define the same objects.
Differences for torchvision backend:
  TimmWrapperImageProcessor in TYPE_HINT but not in _import_structure.

Weird that it is not picked up on the main branch though!

Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Good timing with the qwen 2.5 release

Copy link
Member

@qubvel qubvel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding more fast image processors!

src/transformers/__init__.py Outdated Show resolved Hide resolved
@yonigozlan yonigozlan force-pushed the add-qwen2vl-image-processor-fast branch from 44b01d1 to 90d0945 Compare January 21, 2025 16:35
@yonigozlan yonigozlan merged commit 107f9f5 into huggingface:main Jan 21, 2025
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

image_transforms preprocess quite slow when run large image with qwen2vl
3 participants