kev

Problem PyPI already has an unrelated package named . Anyone who runs silently installs the wrong thing, then gets mysterious s when they try to use this project. Reported in #159. Fix Add a runtime guard in (previously empty). On every it checks the installed distribution's metadata. If the / fields don't contain , it fires an with a clear message pointing to the correct install command: This uses only (stdlib, Python 3.8+), adds no dependencies, and is a no-op when the right package is installed. The makes it safe if metadata is missing for any reason. What was not changed is left as-is — renaming the distribution is a separate decision that requires coordination around a PyPI publish under a new slug (e.g. ). That can be a follow-up if desired. Closes #159

Kev-27B needs one H100 80 GB or an H200 today: its frozen Qwen3.8-27B backbone is 55 GB resident in bf16, and nothing smaller can load it. The part Kev trained is small (the LoRA adapter is 0.5 GB, the pointer head 10 MB); the memory is the base. This PR lets the base load quantized, int8 or 4-bit NF4, so Kev-27B runs on a 40 GB A100 (int8, 25.3 GiB loaded) or, by the numbers below, a 24 GB card (nf4, 14.3 GiB). Mechanism () passes a quantization config to the base's , built in one place, : int8 is torchao weight-only: int8 weights with one bf16 scale per output channel, activations stay bf16 and weights are dequantized inside each matmul. nf4 is bitsandbytes 4-bit NF4 (blocks of 64, double-quantized scales, bf16 compute). The adapter stays unmerged: LoRA adds its low-rank update next to the quantized instead of being folded into . That is already how a bf16-backbone checkpoint loads when is off; quantization now forces it (), because folding would requantize . An int8 row's step is , and most elements of a rank-16 delta are smaller than half a step, so they would round away. Since fused kernels need merged weights, is off for quantized loads too. Two kinds of weight stay bf16 (): , which Kev does not use. The Gated DeltaNet decay and beta projections, / . They are tiny (hidden x 48), and their output multiplies the recurrent state at every token, so an error there compounds over the whole state. The skip list is a regex, , because the two libraries match different names: torchao matches parameter names (), bitsandbytes module names (). A bare works for bitsandbytes and silently does nothing for torchao. My first int8 run had exactly that bug and quantized both; the int8 row below is the corrected run. () swaps in a base saved pre-quantized by . sees in its and loads it as is instead of quantizing again. That is a 27.5 GiB (int8) or 16.5 GiB (nf4) download instead of 52 GiB, with nothing to quantize at load. The saved base carries the original's tokenizer (identical encodings, delimiter ids and vocab), and loads both from it, so an offline run does not need the original base cached. A full-weight checkpoint has no base to replace and refuses . requires , because the precision decides the loader's device map and the no-merge rule. checks, on the tiny hybrid base: the saved and the at-load base give bit-identical probabilities; stays a bf16 while is quantized; the adapter is still a even with . Why not bitsandbytes LLM.int8 I tried it first (8-bit bitsandbytes). It keeps outlier columns in fp16, but it picks those columns from the whole batch. So a question's answer depends on which other requests share its batch. That breaks the isolation Kev serves under: a question's answer must not depend on what else is asked with it. On the served path, CUDA-graph batches against eager moved answers by up to 0.040, against bf16's 0.009. CUDA-graph capture also crashed inside it (a assert), and it was 2.6x slower than bf16. torchao weight-only has none of these problems: served graphs vs eager is 0.008. Results Checkpoint and splits: on the transfer-v4 and decision-v7 development partitions (764 and 1,468 questions; accuracy and Brier over the 656 and 1,264 clean ones), scored by 's path (). Reference: bf16 on an RTX PRO 6000 (Colab G4). It reproduces the model card's transfer-v4 dev 0.848 / 0.229. int8 and nf4: the pre-quantized bases loaded back through on an A100 40GB with . : a question's largest probability change against bf16. A flip is a changed argmax. Other GPU noise:** the A100 rows also carry the arithmetic differences between the two GPUs. No decision rule was registered before these runs, so I read them against the nearest bar the repo has: the release tolerance for bf16 serving against fp32 (max =3.7.1modal_app.pypyprojectuv.locktorch.compileskills/kev-deployscripts/save_quantized.pypythontest_quantized_base_keeps_the_adapter_unmergedtests/test_conventions.pyKEV_QUANTKEV_BASELoadOptions.from_envTorchAoConfigBitsAndBytesConfigkev.modelscripts/quant_eval.pyKEV_BASEscripts/quant_compare.pytests/test_model.pyquantbase`).

Release candidate, not a release. Do not merge before Jared decides. Nothing in this PR or the steps behind it is public. There is no public Hub repo or revision, no public card, and no README, Space, collection or GitHub-release change. Stacked on #176 (round 23 result and confirmation, branch ), whose committed verdicts this card cites. Round 23 confirmed one checkpoint, : round 22's final full-weight SFT of Qwen3.8-27B, blended 0.85 / 0.15 toward the released Kev-27B, with the SFT's pointer head. It passed every registered stage against Kev-27B: breadth-v1 test +1.2 pp [+0.3, +2.2]; tasksource-heldout-v1 test +5.3 [+3.7, +6.8]; pooled hard / devtools / documents-v1 test +8.9 [+7.5, +10.3]; locked transfer-v4 0.8887 (583 of 656) against the 0.886 bar, with served Brier 0.154; the bf16 serving check. Until now the checkpoint existed only as a round arm on the Modal volume. Its still carried the raw temperature 1.0. The release temperature had been fitted only on a local copy of , and there was no card. This PR turns the arm into a reviewable private candidate, Kev-27B v2. It keeps every step reproducible and leaves the round's own files untouched. What was done, in order (record: and ) 1. Copy on the volume, never over anything. (new, ) runs in a CPU container under app . It copied to , and it refuses a destination that exists. It hashes both copies the way does (shards hashed in parallel, same value). Both are , the round's hash. The source is untouched (, T 1.0). Naming: the internal id is , because already names the released Kev-27B (B1 v2) in , in and on the volume. Only the display name is "Kev-27B v2". 2. Release temperature. ran on the copy's with the round's registered pool rows: , limited to its eight held-out sources; , limited to ; minus the records; 648 questions in all. It found no overlap with training and fitted T = 1.3195, the round's pool fit exactly. Out-of-fold ECE goes from 0.048 to 0.038 and the intervals overlap; the 90 % interval of T is [1.20, 1.45]. The resulting () is byte-identical to the monitor's local fit, and it went back into the copy only. 3. Private Hub repo. (new) runs in a CPU container, so the 51 GB never touch a laptop. It created as a private repo and uploaded commit . The card README was updated later, at ; the weights have not changed since . now refuses an existing public repo (). Before, it passed to , which does nothing to a repo that already exists, so a candidate could have landed in a public repo. It links full-weight shards into its staging directory instead of copying them. It uploads with the trial files. The Hub's LFS hashes give the same weights hash () and hash (). 4. The Hub copy loads and scores exactly. ran on an H200, loading with a token. All 252 rows were served at T 1.3195. Their logits equal round 23's committed raw logits times fp32(1/T) bit for bit (PyTorch divides by a scalar as a multiply by its reciprocal), so the backbone and head outputs are identical. Argmax matches on 252 of 252. 5. Card and numbers. follows the formal card style, and its 219 numbers are in . Sources: the round's verdicts and read-out, the test breadth report, the serving reports, the staging record and . comes from . The script now accepts an arm with a registered in place of a trial, and prints where a pool arm has no held-out-pairs figure. For recorded releases whose rows are missing, the Jev documents figure and the locked decision read are now optional. All four recorded releases (, , , ) reproduce byte for byte from the research checkout's rows. What the card says plainly. These points are in "Read this first", not buried: Post-hoc selection. Round 23 was designed after round 24's confirmation and confirmed on the same test and locked partitions. Short states are not better than Kev-27B. Locked −0.8 [−2.0, +0.5]. On transfer-r3 test with included, −2.1 [−3.5, −0.8]. Long contracts are worse. CUAD in longdoc-v1 test scores −1.6 pp [−3.0, −0.3], with ECE 0.053 against 0.007. That read landed while this was staged and is in the card. No scienthoon read. The suite was removed before round 23; the SFT parent was −5.5 pp there. In distribution. hard-v1, devtools-v1 and documents-v1 are in the training data. The tasksource family list stays private ("119 commercially licensed task families; list private"). , not . Every weight derives from fine-tunes of the one base, and the Hub's is for merges of several listed base models. Kev-27B is itself an adapter of the same base. Not in this PR Releasing. If Jared approves: 1. tag Kev-27B's current weights; 2. publish the staged copy to from a container; 3. move the card to ; 4. then update the README, the Space, the collection and kev-deploy. Spend. Modal metered $3,971.71 before and about $3,977 after (workspace-wide, other apps running). App : a CPU copy and upload plus one ~10-minute H200 read, about $1-2. Test plan [x] Unit suites (the CI job) on this branch: 452 passed, 22 skipped, 1 failed. The failure was a without in the new tests (); after the fix, passes (19) and so do the two new tests, and . [x] : 978 claims verified. [x] : the four recorded releases are byte-identical to their committed JSON, run from the research checkout's rows. [x] Volume copy: weights sha256 equal (). Source untouched. The copy's is at T 1.3195. [x] Hub: the repo is private; LFS-derived weights and hashes are equal; semif-v1 rows from the Hub load match round 23's read, 252 of 252 bit for bit before T.