Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New version of the CSVSource #602

Merged
merged 3 commits into from
Nov 6, 2024
Merged

New version of the CSVSource #602

merged 3 commits into from
Nov 6, 2024

Conversation

daniil-quix
Copy link
Collaborator

@daniil-quix daniil-quix commented Nov 5, 2024

A new version of CSVSource to make it more flexible:

  • No need to pack the message value to a field. Every column is produced to Kafka now
  • Added optional "key_extractor" and "timestamp_extractor" callables to specify Kafka keys and timestamps based on the row
  • Added optional delay param to sleep between the rows for simulation

Breaking change:

  • The "name" parameter is required now (previously the filename was used as a name)
  • key_serializer and value_serializer params have been removed.

quixstreams/sources/core/csv.py Outdated Show resolved Hide resolved
quixstreams/sources/core/csv.py Show resolved Hide resolved
quixstreams/sources/core/csv.py Show resolved Hide resolved
quixstreams/sources/core/csv.py Show resolved Hide resolved
@daniil-quix daniil-quix merged commit 92ff62e into main Nov 6, 2024
3 checks passed
@daniil-quix daniil-quix deleted the feature/new-csv-source branch November 6, 2024 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants