The same question, conditioned on our constraints
A technique is not good or bad, it is applicable or not, and the difference is a
written document. 13 techniques in this corpus are excellent work
and inapplicable here — each for a stated reason a reviewer can argue with. That
distinction is the whole job.
Index built 2026-08-10T07:23:47. 1822 passages from 90 arXiv papers and 12 synthetic internal notes. Embeddings: Snowflake/snowflake-arctic-embed-s (ONNX int8, Apache-2.0), loaded from Demo 1. One embedding stack across both demonstrations. Demo 1 harness live model configured
Retrieval, extraction, every statistical test and the backlog are model-free. Only the synthesis paragraph on screen 2 uses a model, and it degrades to extraction.
Retrieval, extraction, every statistical test and the backlog are model-free. Only the synthesis paragraph on screen 2 uses a model, and it degrades to extraction.
The written constraint set
on-premises · no GPU at inference · <3 s end-to-end · explainability required · 16 vCPU / 64 GB
| Constraint | How a reviewer checks it | Techniques it rejects |
|---|---|---|
| Deployment location: on-premises No component may require a call to an external API at query time, including as part of an evaluation metric. Data residency is contractual. | Does the method's definition include calling a hosted model you do not run? | 2 |
| No GPU at inference The GPU pool is reserved for vision workloads. Retrieval and generation run on CPU only. Methods that require fine-tuning are also excluded, because training is a GPU job. | Does the method need a GPU to train, or to serve within the budget? | 8 |
| Latency budget: 3 s end-to-end at p95 Measured from request to complete answer, including retrieval, reranking and generation. The budget is shared across all stages. | What does one query cost in wall-clock on 16 vCPU, and what is left for generation? | 5 |
| Capacity: 16 vCPU / 64 GB RAM per service Index build as well as serving. A method whose index construction is a multi-day CPU job fails this even if its query path is cheap. | What is the peak resident memory, and what does a full re-index cost? | 3 |
| Explainability required Every claim in an answer must resolve to a passage an engineer can open. Any score used in a release decision must be inspectable and auditable. | Can a maintenance engineer see why this answer, and audit the score? | 3 |
Correctly rejected as inapplicable 13
Each of these is a technique a literature search returns near the top, and each one fails a specific constraint for a specific reason. None of these is a criticism of the paper.
RankZephyr — 7B listwise reranker arXiv:2312.02724
rejected
No GPU at inference Latency budget
A 7B decoder must generate a permutation over a sliding window of candidates for every query. On 16 vCPU that is tens of seconds, so the 3 s budget is missed by an order of magnitude.
Would apply if: A GPU serving path, or distillation into a model two orders of magnitude smaller.
RankGPT — permutation-generation reranking arXiv:2304.09542
rejected
Deployment location Latency budget
The method is defined by calling GPT-3.5 or GPT-4 inside the query path. On-premises deployment and data residency rule that out, and it adds a network round trip inside the latency budget.
Would apply if: An on-premises model of comparable capability, which changes the method.
G-Eval — GPT-4 as the metric arXiv:2303.16634
rejected
Deployment location Explainability required
The metric itself is a proprietary model. An evaluation harness whose ground truth requires an external API call per sample cannot ship here, and a score produced that way cannot be audited by the engineer whose work it gates.
Would apply if: A calibrated local judge with a measured false-positive rate, reported with an interval.
Self-RAG — learned critic with reflection tokens arXiv:2310.11511
rejected
No GPU at inference
Requires training a critic language model that emits reflection tokens. Training is a GPU job, and the released 7B and 13B critics also miss the CPU inference budget.
Would apply if: A released critic small enough to serve on CPU, trained by someone else.
GraphRAG — entity graph plus community summaries arXiv:2404.16130
rejected
Capacity Latency budget
Index construction runs a language model over every chunk to extract entities and relations, then summarises every community. On an enterprise corpus that is a very large model bill and a multi-day CPU job, and query-time map-reduce over community summaries also breaks the 3 s budget.
Would apply if: GPU capacity for indexing plus a caching layer for query time.
Cache-Augmented Generation — preload the knowledge base arXiv:2412.15605
rejected
Capacity Explainability required
The method is preloading the whole knowledge base into the context window and reusing the KV cache. That needs a very-long-context model and GPU-scale KV memory, and it removes per-claim provenance because everything is in context at once.
Would apply if: Nothing available on this platform; the method is defined by the thing we cannot do.
QLoRA — 4-bit fine-tuning arXiv:2305.14314
rejected
No GPU at inference
It is a fine-tuning technique. 4-bit NF4 weights and paged optimisers still assume CUDA. There is nothing to run on 16 vCPU.
Would apply if: GPU capacity for training. The served model could then be CPU-quantised separately.
BitNet b1.58 — 1.58-bit LLMs arXiv:2402.17764
rejected
No GPU at inference
The 1.58-bit property comes from pre-training from scratch. An existing checkpoint cannot be converted, so this is out of scope for any organisation that is not pre-training its own models.
Would apply if: Someone else releasing a suitable pre-trained 1.58-bit model.
vLLM / PagedAttention arXiv:2309.06180
rejected
No GPU at inference
The entire contribution is GPU KV-cache memory management. With no GPU there is nothing for it to manage.
Would apply if: A GPU serving path.
D2F — diffusion LLM parallel decoding arXiv:2508.09192
rejected
No GPU at inference
The reported speedups are GPU parallel-decoding results, measured against GPU autoregressive baselines. Nothing transfers to a CPU serving path today.
Would apply if: A CPU implementation, which does not currently exist.
FD-LLM — fine-tuned fault diagnosis arXiv:2412.01218
rejected
No GPU at inference Explainability required
LoRA fine-tuning of a large language model on tokenised vibration signals. Training needs a GPU, and a fine-tuned model's fault call carries no traceable feature attribution, so a maintenance engineer cannot audit the diagnosis.
Would apply if: A probabilistic model with attribution alongside the language model.
LongRAG — long retrieval units with a long-context reader arXiv:2406.15319
rejected
Latency budget Capacity
Uses 4K-token retrieval units and long-context readers. Token counts per query are far above what a CPU-served model can process inside 3 s.
Would apply if: A GPU serving path, or a much smaller latency-per-token.
PatchCore — memory-bank visual anomaly detection arXiv:2106.08265
rejected
No GPU at inference Latency budget
An ImageNet-backbone CNN with a memory bank that grows with the training set. GPU-trained, and out of scope for a text retrieval service in any case.
Would apply if: GPU capacity, and a vision problem to apply it to.
Survives the same test 8
Blended RAG — BM25 plus sparse and dense hybrid arXiv:2404.07220
applicable
All three retrievers run on CPU, the fusion is rank-based arithmetic, and every returned passage is inspectable.
MiniLM cross-encoder reranking arXiv:2002.10957
partial
Latency budget
A 22M-parameter cross-encoder is CPU-feasible, but cost scales with the candidate count. Applicable at a small, fixed number of candidates and not as a way to widen the pool.
Would apply if: Fix the candidate count and measure it; do not rerank the corpus.
Drowning in Documents — when not to rerank arXiv:2411.11767
applicable
A negative result that constrains how the technique above may be used. Costs nothing to apply and prevents an expensive mistake.
Late chunking arXiv:2409.04701
applicable
Encoder-only, one forward pass over a long input, no generation. CPU-feasible.
Lost in the Middle — positional effects arXiv:2307.03172
applicable
A finding about prompt composition. Free to apply and it argues for better ranking rather than more context.
DLinear — linear baselines for time series arXiv:2205.13504
applicable
Linear weights are directly inspectable, so it satisfies explainability by construction, and it is trivially cheap.
Probabilistic models plus LLM for fault diagnosis arXiv:2510.03815
applicable
Accepted precisely because it adds temperature scaling and risk-coverage curves, which is what the explainability constraint asks for.
Semantic chunking is not worth the cost arXiv:2410.13070
applicable
A negative result that saves index-build time. Applies directly under the capacity constraint.
From a finding to a hypothesis card
The card's fields are borrowed with provenance rather than invented: section structure from Model Cards for Model Reporting (arXiv:1810.03993), reporting discipline from REFORMS (arXiv:2308.07832), the mandatory leakage checks from arXiv:2207.07048, and Status, Consequences and Confirmation from Nygard's ADR and MADR. The Confirmation field is the bridge to screen 4.
Source claims
2411.11767 — Drowning in Documents: Consequences of Scaling Reranker Inference
“Rerankers give diminishing and then negative returns as the number of reranked candidates grows; when the full corpus is reranked they are less accurate than the retriever alone.”
Sets the condition: the candidate count must be fixed and small, or the intervention is not the one the literature supports.
2002.10957 — MiniLM: Deep Self-Attention Distillation
“A distilled cross-encoder retains most of the teacher's ranking quality at a fraction of the cost.”
Why a cross-encoder is affordable on CPU at all.
ENG-2411-017 — Cross-encoder reranking: latency measurements (synthetic internal note)
“Reranking 20 candidates costs a median 310 ms on 16 vCPU; reranking 50 costs a median 780 ms and 1.9 s at p95.”
Fixes the operating point at 20 candidates so the arm is affordable inside the 3 s budget. Synthetic.
Claims that cut the other way
ENG-2502-004 — Reranking did not help on short factoid queries (synthetic internal note)
“Context recall@5 moved 0.71 to 0.73; paired bootstrap difference +0.02, 95% CI [-0.03, +0.07]. Not significant.”
The reason this is a hypothesis and not a change request. Synthetic.
2503.08046 — MultiConIR: Towards multi-condition Information Retrieval
“Reranker advantage inverts as query condition-count grows.”
Predicts the effect may be negative on complex queries.
HYP-2026-014 — Cross-encoder reranking over a fixed candidate pool improves context recall@5
Claim
Adding a cross-encoder reranking stage over a fixed pool of 20 first-stage candidates increases context recall@5 on the technical-manual question set by at least 10 percentage points relative to hybrid retrieval alone.
Intervention
Hybrid first stage, then ms-marco-MiniLM-L-6-v2 cross-encoder reranking, k=5.
Control
The identical hybrid first stage, k=5, no reranking. Same index, same chunks, same query set.
Population
The fixed technical-manual question set, every question scored by both arms.
Metric
context_recall@5 from atlas_shared.evals.context_recallExpected effect
0.70 → 0.80 (10 percentage points), from the published reranking literature.
Decision threshold
Adopt only if the paired difference is significant at alpha=0.05 AND the lower bound of the 95% interval on the effect is above zero AND the measured reranking latency leaves at least 2 s of the budget for generation.
Test
stats.mcnemar on per-question hit@5, and stats.paired_bootstrap on per-question recall@5Variation approach
10,000 paired bootstrap resamples, seed fixed at 20260809. Retrieval is deterministic, so there is no seed variance to average over.
Effort
About one hour: the intervention is a flag on an existing retrieval call. No training, no new index.
Compute budget
One CPU-minute per arm at this question-set size. No GPU.
Status
proposed
Why this test
Both arms are scored on the same questions, so the comparison is paired and an unpaired two-proportion test would be wrong: it would inflate the variance and hide a real difference. McNemar uses only the discordant pairs, which is exactly the information a paired design carries. The bootstrap is there because recall@5 is a continuous per-question score with no closed-form variance.
Sample size, computed before the run
Baseline
0.7
Target
0.8
alpha
0.05
Power
0.8
Cohen's h
0.232
Items needed per arm
146
To detect 70% -> 80% at alpha=0.05 with 80% power you need about 146 items per arm. On a paired design you may need fewer, but only if the systems disagree often; McNemar's power depends on discordant pairs, not n.
Leakage checks — mandatory, not optional
- Gold passage IDs must be resolved independently of either arm's output.
- Both arms must run against the same index build, so a chunk ID means the same object.
- The reranker must not have been tuned on this question set.
- The candidate pool is fixed before scoring, not chosen to favour an arm.
This field exists because of a real failure mode: an evaluation whose labels were derived from one arm's output will flatter that arm, and it will do so in the direction you were hoping for. See the decision log, ADR-007.