Fast and powerful CSV (delimited text) parser that gracefully handles large files and malformed input
by mholtJavaScript
Last 12 weeks · 19 commits
2 of 6 standards met
The UTF-8 byte order mark is stripped from string input only. Every other input reaches the parser through , which never sees , so the mark stays inside the first field of the first row: Same for a , a , and a Node . Nothing renders the character, so the first field simply never compares equal to what it looks like — is false, and a number in that position stays a string. strips it at https://github.com/mholt/PapaParse/blob/master/papaparse.js#L148 but that branch is guarded by . Why it has not shown up hides it. 08265f1 added a separate for column keys, so the header row is cleaned wherever it comes from, and only is left exposed. The existing coverage misses it for the same reason the bug is easy to miss. Both BOM cases in pass a string literal, and — a fixture that exists precisely for this — is only ever read into a string before being handed to . The streaming path has no BOM coverage at all. The change Strip it once at the front of the stream, in the first-chunk block of , which is where every input path converges. Doing it per streamer would mean four copies. A mark anywhere other than the very start is data and is left alone. Tests Three added to , using the existing fixture through the pipe idiom already used there. Worth being explicit about which one does the work: only "strips the utf-8 BOM from a piped stream" fails without the change. The variant passes either way — it is there to document that the header path was already handled by 08265f1 — and "keeps a BOM character that is not at the start of the input" is a guard against over-stripping. gives 253 passing. is left untouched, since it does not appear in contributor commits.
and start with but assigns every method to and leaves its prototype empty, so both throw on their first line. Nothing calls them, which is why this has never surfaced. The consequence is that never throttles the source: stops the parser but the stream keeps emitting, and keeps decoding and queueing, so the whole rest of the input piles up in memory for as long as the caller stays paused. That is the case exists to avoid. Measured on an 8 MB source with the parser paused after the first row, sampled 250 ms later: The fix. Give real no-op / hooks that a streamer with a throttleable source overrides, and have / call them. This PR also corrects to name . That self-assignment is inert while is empty, but it is load-bearing the moment anything is put on it: without that one-word change every existing string pause/resume test fails with . It is included here rather than split out because this change is what makes it matter. The change is deliberately narrow. It does not touch , the polling, or ; those are separate problems. The test. , "pause() applies backpressure to the source stream". A counts the bytes it has pushed; the parser pauses at the first row and 250 ms later the test asserts no more than two 13 KB blocks were pushed in the meantime. Bounded and fast: the source is capped at 2 MB and destroyed at the end, so nothing large is allocated. Reported in #1132. Red-green evidence:** Confirming the line is load-bearing rather than cosmetic, with the rest of the branch applied and only that line reverted: Suite and lint on the branch:
Fixes #786. Adds a config option (milliseconds) for requests. When set, it configures on the request and surfaces a clear timeout error through the callback if the remote file doesn't finish downloading in time. The value is validated with , following the same validation pattern used elsewhere in (throws a descriptive if not parsable). There was a previous attempt at this in #789, which got a positive review but was abandoned by the author. This re-implements it against current master (the codebase has changed since 2020) and adds test coverage: Config validation: throws for non-numeric , coerces numeric strings. A request that legitimately exceeds and produces a timeout error. A request that finishes well within a generous , to make sure normal downloads aren't affected. The timeout test needed a way to reliably produce a slow response without depending on real network flakiness, so I added a small artificial delay in the local test server () specifically for . Also documented the new option in .
Repository: mholt/PapaParse. Description: Fast and powerful CSV (delimited text) parser that gracefully handles large files and malformed input Stars: 13552, Forks: 1186. Primary language: JavaScript. Languages: JavaScript (90.7%), CSS (7.3%), HTML (2.1%). License: MIT. Homepage: http://PapaParse.com Topics: csv, csv-parser, javascript. Latest release: 5.7.0 (1d ago). Open PRs: 31, open issues: 184. Last activity: 1d ago. Community health: 42%. Top contributors: mholt, pokoli, bluej100, gabegorelick, Turbo87, janisdd, MonkeyDZeke, robd, jsg2021, dboskovic and others.