Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cu): handle hashing wasm memory >2GB #1010

Merged
merged 1 commit into from
Sep 10, 2024

Conversation

TillaTheHun0
Copy link
Member

@TillaTheHun0 TillaTheHun0 commented Sep 10, 2024

/**
 * The memoryStream, if derived from an iterable like a Buffer,
 * may emit the entire data stream as a single chunk.
 *
 * This can break things like Crypto hash, which can only handle
 * data chunks less than 2GB.
 *
 * So we use this Transform stream to receive chunks,
 * then re-emit "sub-chunks" of the given size -- in this case
 * the default highWaterMark of 64kb
 *
 * This allows for hashing to work for any size memory, derived
 * from any iterable, while respecting backpressure
 */

@TillaTheHun0 TillaTheHun0 merged commit 390416a into main Sep 10, 2024
3 checks passed
@TillaTheHun0 TillaTheHun0 deleted the tillathehun0/cu-handle-large-wasm-hash branch September 10, 2024 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant