Commit activity
Last 12 weeks · 10 commits
Community health
3 of 6 standards met
57
✓README✓License✓Contributing○Code of Conduct○Issue Template○PR Template
Recent PRs & issues
Active · Last activity 1mo ago
See all on GitHub →
In my benchmarks, memchr-n tends to nearly always beat memchr, I wrote about why in the readme All Benchmarks Below from my M1 Pro MBP Or On runs, there's some trades, memchr mostly wins (which makes sense, memchr-n does more work to build a searcher) onlycount results (memchr-n wins across the board) only count for 1 needle (also specialized in memchr) For 2 needles For 3 needles For larger bytesets (memchr-n only) fallback SWAR implementation Full disclosure: I did use AI to modify the rebar benchmarks here, but they seem to correlate well with my more manual criterion benchmarks in my own repo.
Replaces the per-vector + round trip with accumulation that stays in the vector domain, folded to a scalar once at the end. SSE2/AVX2: / against zero + lane accumulation; final fold is a lane sum / 255. aarch64: (UADDLV) per vector into a scalar accumulator. wasm32: bitmask+popcnt semantics unchanged, behind the new abstraction. Throughput on 7950X, count of one byte over 256 MiB, flat across densities 0–100%: NEON tested with qemu-user: roughly 0.98 GB/s -> 4.37 GB/s
Implemented AVX-512. Also naturally had to adapt and change vector byte limit to 64. Some a bit off-topic but rather small and simple changes, made a change to which, in my opinion, could be made a bit simpler, albeit functionally equal; Changed "Used for haystacks bigger than N bytes." to "Used for haystacks of N bytes or more." (as N bytes is enough to use the respective strategy but the former implies that it must be bigger than N, which is incorrect); Also made a small change on the safety comments of and inside of . These mention calling sse2 routines where we're calling avx2 routines, so I believe these were overlooked mistakes. Solves https://github.com/BurntSushi/memchr/issues/236. Let me know your thoughts. Thank you so much for this lib!
Recent fixes
The new jetscii aarch64 algorithm supports an arbitrary set of bytes (though currently limited to 16 to match the existing limitation of the x86 implementation). It seems to be pretty competitive with memchr3, being a bit faster for smaller haystacks, or when iterating over more common bytes. I think this is probably largely because iterating uses a 64bit bitset of already identified as matching positions, rather than restarting the search every time, and it's able to process 64 bytes at a time without having to do any fixups in case of matches. I'd like to improve it by using aligned loads like memchr does (using a possibly unaligned load at the start + end)
In AVX-512, we don't want an explicit operation since returns a scalar mask itself already. The current code was setup in a way that assumes that always returns a vector matching the input type. But AVX-512 breaks that assumption. We fix this by putting the result of vector comparisons into its own trait. This way, we can change its return type to be a scalar in AVX-512, but keep it as a vector for all other targets.
Structured data for AI agents
Repository: BurntSushi/memchr. Description: Optimized string search routines for Rust. Stars: 1563, Forks: 152. Primary language: Rust. Languages: Rust (99.9%), Python (0.1%). License: Unlicense. Topics: bytes, memchr, rabin-karp, rust, simd, string, string-searching, twoway. Open PRs: 25, open issues: 14. Last activity: 1mo ago. Community health: 57%. Top contributors: BurntSushi, bluss, atouchet, waywardmonkeys, cholcombe973, allan2, nicokoch, mkroening, tgross35, timotree3 and others.