"Matches ours" is how much of each validator's weight row coincides with our validator's current row. Chain consensus pays only what validators holding at least half the stake agree on, so while a majority holds other weights, incentive does not follow our scores.
DEFLATE Pareto: time vs size
Current frontierMiner on itFrontier at the end of an earlier day (lighter = older)Baseline (never paid)IncumbentOther minersOutside scoring bounds
Loading…
x: mean per-file time ÷ incumbent, log scale (lower is faster). y: mean per-file compressed size, % of original (lower is smaller). The faint step lines are where the frontier stood at the end of each earlier day (UTC), so you can see how far it moved. Dashed lines are the scoring bounds; points past them are not paid.
DEFLATE leaderboard
Loading…
page 1
DEFLATE submissions
Loading…
page 1
FloatLib kernels
Loading…
King of the hill: one Kernel.lean with binary32 add and mul proved equal to FloatLib's IEEE spec, and TorchLean's sum and batched matrix multiply bmm built from them, proved equal to TorchLean's. The gate checks the proofs, then times all four against the champion's; an entry at or below the promotion bar, and significantly faster, takes the board. Pay goes to the newest champions by rank.
FloatLib full entry: the board
Loading…
Every hotkey's best accepted entry, timed against the champion; click a row for its timings per input type. Kernel columns are ns per operation summed over that kernel's input types, green where faster than the champion's and red where slower, by more than 1%. The champion's own ratio is against the champion it took over from. Admission is the latest published scoring's verdict.
FloatLib submission
FloatLib champions
Loading…
Every promotion, newest first. A newer promotion supersedes a champion without demoting it; a demotion (by an operator) hands the board back to the newest earlier champion. Paid ranks follow the published rank shares; missing ranks are not redistributed.
FloatLib practice boards (unpaid)
Loading…
One kernel per board, for trying one idea at a time; nothing here is paid. "Shipped" means no miner has taken the board from the incumbent the competition shipped with.
FloatLib overfitting: PROD entries on held-out inputs
Loading…
Every accepted PROD full entry, pushed from PROD to this host (DEV holds no PROD credentials) and re-timed here against the shipped baseline on three workloads: legacy (the fixed classes PROD scores on), refined (stratified-v1) and held out (ML-shaped data, NaN/Inf and subnormal streams, sizes around the legacy shapes, none of it ever shown to miners). Always PROD's entries, whichever deployment is selected above. Scores are time ÷ baseline, the geometric mean over kernel × input family; lower is faster. Gap = held-out ÷ legacy score: 1.00 means the speed-up carries over, above it the entry was tuned to the benchmark. Fallback = held-out ratio on NaN/Inf/subnormal streams ÷ on clean data. Cliff = ratio at the legacy fixed shape ÷ its neighbours'; well below 1 means shape-tuned. Once the monitor times the suite-v2 suites, public v2 (published), scoring v2 (the private suite PROD admits on) and held out v2 (sealed) appear too; Gap v2 = held-out v2 ÷ scoring v2. When PROD switches to scoring v2 it keeps its usual formula, total time ÷ total time, with every case sized to take about the same time on the baseline; Gate v2 is that ratio here, against the shipped baseline.
One Kernel.lean of exact tensor kernels, proved equal to TorchLean's binary32 specs, then timed on every model-family board (overall and one per model) against that board's reference: its current champion, or the kernel the competition shipped with. An entry at or below the promotion bar, significantly faster (the bootstrap's lower bound on the gain is above zero) and within the per-cell guard takes the board. Each board pays its two newest champions by rank, out of its share of the competition. Scores, admissions and ranks are the coordinator's stored decisions; nothing here is re-scored.
TorchLean boards
Loading…
Click a board for its challengers and champions. Scores are time ÷ the board's reference, the geometric mean over its cells (model × batch × input kind) and environments; lower is faster. A champion's score is against the reference it took over from. "Pay" is each paid rank's fraction of TorchLean's allocation: rank share × the board's reward share (zero while the board is paused under the treasury policy).
TorchLean board
Challengers against the current reference
Loading…
Champions
Loading…
Challengers are evaluations against this reference under the active benchmark configuration; a superseded evaluation (re-planned) is left out, a pending one is still being timed. Champions are every promotion on the board, newest first; an operator disqualification hands the board back to the newest eligible earlier champion. A former champion's source is published once it holds no board.
TorchLean submissions
Loading…
Newest first; click an entry for its report. Processing: queued and proving before the proof check, waiting proved but not yet planned on every board, benchmarking while an evaluation is timed, completed once every active board has a decision (which does not mean promoted). Board pills carry the latest evaluation's score, coloured by its admission.
TorchLean submission
DEFLATE reruns on corpus 1, 2 and 3
Loading…
Every submission whose source the API publishes is rebuilt and measured again with the gate's own engine, paired with the incumbent, on each corpus. Corpus 1 is the public scoring corpus, corpus 2 the held-out one, corpus 3 an out-of-distribution corpus (different sources, file types and file sizes) that is not scored. Gains are the mean per-file reduction in compressed size against the incumbent. Sizes are exact; times are relative to the incumbent on the rerun host. Frontier submissions keep their source private; they appear here once dethroned, or straight away when PROD pushes a copy for rerunning (marked "PROD copy": measured, then the copy is deleted).
Trends: is the progress real?
with the same span before it
Loading…
Miners only, by the day they submitted; days with fewer than 5 rerun miners are left out, and every point's tooltip gives its count. Reading it: progress is real when corpus 2 (held out, same kind of data) rises with corpus 1. A widening gap between them, a falling share of the gain kept, or new corpus 1 records that corpus 2 does not follow, mean submissions are fitting the public files rather than compressing better. Corpus 3 is different data and unscored: there, a rising share of collapses means submissions are tuning to corpus 1's kind of data. "Recent" ends at the newest rerun submission, not today.
Pareto on each corpus (reruns)
Frontier on this corpusOn this corpus's frontierOff itOn corpus 1's frontier, off this oneBaseline
Loading…
Frontiers among the rerun submissions only; hover a point for its numbers, click it for per-file gains. A hollow point is on corpus 1's frontier but not this corpus's: a trade-off that holds only for corpus 1's data.
Does the gain carry over?
MinerMiner that keeps less than half its corpus 1 gainBaselineSame gain on both
Loading…
x: gain on corpus 1; y: gain on the chosen corpus, both against the incumbent. Below the diagonal means the gain shrinks away from corpus 1. Corpus 2 shares corpus 1's distribution, so it tests memorisation; corpus 3 does not, so it tests tuning to that distribution.
Files: where gains come from, where parsers break
Loading…
Across every rerun miner: the median, best and worst gain on each file, and how many miners came out at least 10% bigger than the incumbent on it. Held-out corpora appear in the operator view only.
PROD hardware vs DEV hardware: does the speed ranking hold?
On the frontier with either machine's timesOn it with one machine's times onlyBaselineOtherSame on both
Loading…
Where the machines disagree
Loading…
Every PROD submission rerun on this host (DEV, s66), compared with its official numbers from PROD's validator, on the two corpora the competition scores, combined the scorer's way. Sizes are deterministic and must match exactly; only time can differ. Times are relative to the incumbent measured on the same machine in the same run, so a uniformly faster server cancels out: what remains is how differently parsers respond to the hardware (caches, vector units, memory bandwidth) plus measurement noise. The control row measures that noise: DEV's own gate also runs on this host, so its official numbers against the reruns differ only by run. Reruns use 3 repetitions; the gate uses 11.
Every rerun submission
Loading…
"Retention" is the gain on a corpus divided by the gain on corpus 1 (shown when the corpus 1 gain is at least 0.5%). "Verified" compares the rerun's compressed bytes with the gate's published report: per file on public corpora, totals on held-out ones.