Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it an option to write to sandbox and then to pack to avoid duplication content in the final pack #183

Open
unkcpz opened this issue Feb 6, 2025 · 0 comments

Comments

@unkcpz
Copy link
Member

unkcpz commented Feb 6, 2025

When directly use API to write to pack, if the content is already there, the content will duplicate in the pack. It was to avoid to read and write twice for computing the hash.

          > Maybe the way would be to stream anyways to sandbox first, and then copy to the pack. But then you are going to write most data twice.

For the loose to pack, it is not a problem since hash is there already. For directly write to pack, I think it will be a good feature to have a option key to support write to sandbox first. So users can choose to have a more disk space optimized solution or a more performance solution.
Maybe there is also a way that the file handler move the pointer in file after it finds the content already exist, not sure but would be interesting to check. But anyway it is another issue out the scope of this PR. I'll open an issue here and not contaminate with the discussion.

Originally posted by @unkcpz in #179 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant