The same fine-tune, now with a hybrid retriever (MiniLM + BM25, top-5 over 42,412 chunks) feeding it the actual passages before it answers. This is where the facts arrive — correctness jumps and it becomes the decisive winner of the experiment.
The winner (+2.80 over the closed fine-tune, p < 0.001). Retrieval supplies the facts the weights never held: correctness climbs 2.35 → 3.85 and groundedness 0.87 → 1.65 — but only once real passages are in the prompt.
The thesis in one screen: fine-tuning taught the shape, retrieval supplied the facts. Every answer below shows the passages the retriever fed in.
| Class | Gemma2ForCausalLM |
| Layers | 26 |
| Hidden size | 2,304 |
| Attention | 8 heads / 4 KV · dim 256 · GQA |
| Feed-forward | GeGLU · inner 9,216 |
| Attention window | sliding 4,096 · logit soft-cap |
| Norm | RMSNorm |
| Context | 8,192 tokens |
| Vocabulary | 256,128 |
| Embeddings | tied input/output |
| Init from | gemma-2-2b-it · 4-bit NF4 |
| Method | QLoRA · rank-16 / α-32 |
| Targets | all linear modules |
| Trainable params | 20.8M LoRA (~0.8%) |
| Schedule | LR 2e-4 cosine · seq 512 · bf16 |
| Data | 2,683 QA pairs · early-stopped ~1 epoch |
| Val perplexity | 3.87 (ref 4.26) |
| Hardware | Modal L4 |
| Retriever | MiniLM-L6 + BM25 · RRF |
| Index | FAISS flat · 42,412 chunks |
| Top-k | 5 · recall@5 0.93 |
The fine-tune's supervised set is 2,683 grounded QA pairs, distilled by a teacher LLM from a curated Yu-Gi-Oh corpus and filtered through a blind-judge gauntlet. That corpus is two free sources: Yugipedia editorial prose (CC BY-SA 4.0, attributed) and YGOPRODeck card facts. Card effect text is © Konami, used only as labelled fair-use context — never a verbatim-recall target.
| Source | Role | License | Size | Share |
|---|---|---|---|---|
| Yugipedia · tips | rulings & interactions | CC BY-SA 4.0 | 8.47 MB | 31% |
| Yugipedia · rulings | rulings | CC BY-SA 4.0 | 5.90 MB | 22% |
| YGOPRODeck · card facts | card identity (stats + fair-use text) | free / © Konami | 6.04 MB | 22% |
| Yugipedia · archetype | mechanics | CC BY-SA 4.0 | 2.74 MB | 10% |
| Yugipedia · lore (ep + char) | lore | CC BY-SA 4.0 | 3.92 MB | 14% |
| Yugipedia · mechanics | glossary | CC BY-SA 4.0 | 0.40 MB | 1% |
78% free-licensed Yugipedia prose (20.7 MB cleaned, clears the 20 MB floor) + 22% card facts. Of the prose, ~82% is on-target rulings/interactions and ~18% is lore. Card knowledge is a YGOPRODeck snapshot fetched 2026-08-01.