Hallucination detection and correction in LLMs
Which model checks which turns out to matter more than how good either model is.
- Detection
- 77.6%
- Correction
- 91.0%
- F1 gain
- 4.6×
Overview
Open-source LLMs hallucinate, and the usual responses — fine-tuning, better decoding, a second model as checker — are normally evaluated one at a time. This project layered all three and measured factual consistency across question answering, dialogue, and summarization on the HaluEval benchmark.
Three layers: LoRA adapters fine-tuned on TruthfulQA and injected into query and value projections; Self-Consistency decoding that aggregates multiple reasoning paths by majority vote; and a zero-shot Generator-Checker setup where one model answers and a second audits the output and returns corrective feedback. Qwen2.5-3B-Instruct and Gemma-3-4B-IT throughout.
Challenges
LoRA fine-tuning on TruthfulQA improved Qwen's detection but exposed a transferability limit: gains on TruthfulQA did not carry cleanly to HaluEval, which is a different distribution of the same nominal task.
The verification pipeline was run bidirectionally on the assumption that two comparable models would check each other comparably. They did not.
Decisions
The calls that shaped this project, the alternatives they beat, and what each one cost.
Run the verification pipeline in both directions
over fix one model as generator and the other as checker
WhyA single direction would have reported 77.6% detection and 91.0% correction and called the method a success. Running it both ways is the only reason the asymmetry surfaced at all — and the asymmetry is the actual finding.
Trade-offDoubles the evaluation cost to test a symmetry most people assume holds.
Layer LoRA, Self-Consistency, and multi-agent verification together
over evaluate each mitigation on its own
WhyEach is usually benchmarked in isolation, which leaves open whether they compose or overlap. Stacking them showed that Self-Consistency recovers most of what LoRA alone leaves on the table.
Trade-offHarder to attribute a given improvement to a single layer without further ablation.
LoRA adapters in query and value projections
over full fine-tuning
WhyThe goal was to improve factual grounding without touching model architecture, so the mitigation stays portable across models. Query and value projections are where the adaptation has the most leverage per parameter.
Trade-offLimited capacity to correct behaviour, and it exposed a transferability gap — TruthfulQA gains did not carry cleanly to HaluEval.
Outcomes
Self-Consistency was the strongest single recovery mechanism, lifting summarization F1 from 0.03 to 0.37 for Qwen and 0.00 to 0.38 for Gemma — up to 4.6× over the LoRA-only models.
The Generator-Checker asymmetry was stark. Gemma→Qwen reached 77.6% hallucination detection and 91.0% correction; Qwen→Gemma collapsed to near-zero under an identical framework. Qwen audits well, and Gemma does not reliably catch what Qwen produces.
The conclusion is a design rule rather than a score: in multi-agent reliability systems, model pairing is a first-class variable. Correction effectiveness depends on generator-checker compatibility, not on either model's capability alone.