Last 12 weeks · 2 commits
2 of 6 standards met
What would you like to have changed? I think this mostly applied to , but may apply to others as well. There's currently a way to skip preserving UID and GID during the archive creation process. I'd like to request the same for extraction as well. Why is this feature a useful, necessary, and/or important addition to this project? I'm using it inside a project where I don't really care about the UID and GID because they're modified later anyway. My terminal is flooded with a lot of warnings which I'd rather not see. What alternatives are there, or what are you doing in the meantime to work around the lack of this feature? I'm thinking about processing the output, and potentially filtering these warnings out. Please link to any relevant issues, pull requests, or other discussions. N/A
Hello! I would like to propose an optimization for XZ decompression. Currently, XZ decompression is performed sequentially in a single thread. We have implemented a concurrent parallel block decompressor in a fork of the underlying dependency at (which functions as a drop-in replacement). By leveraging the XZ format's native index block boundaries, the parallel decompressor parses the index backwards in O(1) time and decompresses independent blocks concurrently using a worker pool. To keep memory utilization bounded and prevent Garbage Collector overhead, we also introduced pooling for both decompressed block buffers and the large LZMA decoder dictionary slices. Benchmarks (on a 2-core / 4-thread CPU with a 20MB payload): Vanilla decompressor: ~14 MB/s Optimized sequential decompressor: ~30 MB/s Optimized parallel decompressor:** ~70 MB/s On systems with 4, 8, or more physical cores, the throughput scales near-linearly, easily exceeding 120+ MB/s. We can utilize this optimization when the input source implements and the total compressed stream size is known. Integration Instruction To utilize the parallel reader within your Go code when dealing with random-access inputs, you can check if the underlying stream supports seeking and pass it to the parallel decompressor.
Repository: mholt/archives. Description: Cross-platform library to create & extract archives, compress & decompress files, and walk virtual file systems across various formats Stars: 441, Forks: 46. Primary language: Go. Languages: Go (100%). License: MIT. Homepage: https://pkg.go.dev/github.com/mholt/archives Topics: 7zip, archives, brotli, bzip2, compression, extract, fs, go, golang, gzip, lz4, lzip, rar, snappy, streams, tar, xz, zip, zlib, zstandard. Latest release: v0.1.5 (10mo ago). Open PRs: 2, open issues: 7. Last activity: 2w ago. Community health: 57%. Top contributors: mholt, M0Rf30, sephriot, darkliquid, dpgarrick, dirkmueller, Gusted, Hashim1999164, solvingj, joonas and others.