Hybrid scorecard: what each detector contributes on the same benchmark
This page keeps the three subsystems on one board instead of letting them hide behind different datasets. The classifier, RuntimeGate, and trichromatic layer all see the same benchmark set here, so the gap between them is evidence, not storytelling.
The gate is still carrying most of the lift, but the hybrid matters because it preserves rare catches and keeps miss buckets explicit.
The point of the overlap harness is monotonic evidence. A later council outcome can raise severity, but it cannot silently downgrade a real quarantine that fired earlier in the chain.
Same prompts, different detectors
| Lane | Detected | Rate | What it tells us |
|---|---|---|---|
| classifier_raw | 69 / 200 | 34.5% | Good on public-style attack patterns, weak on SCBE-specific structural families |
| classifier_overlay | 14 / 200 | 7% | Very conservative live thresholding when used as an overlay instead of a raw label surface |
| gate | 98 / 200 | 49% | Current backbone for structural and policy-shaped detection |
| trichromatic | 12 / 200 | 6% | Too weak as a broad detector, but still useful as a structural escalation layer |
| hybrid | 109 / 200 | 54.5% | Best combined result on this harness after the monotonic-veto fix |
Overlap, not intuition
The overlap table is where the real story lives. Most caught attacks are coming from the gate, but there are still rare union-only catches worth preserving.
| Component overlap | Count | Meaning |
|---|---|---|
| gate | 86 | The gate alone found the attack and still forms the core of the hybrid stack |
| classifier | 9 | Raw classifier-only catches still exist and justify keeping the overlay path |
| gate + trichromatic | 8 | Structural agreement between the gate and hidden-band diagnostics |
| trichromatic | 1 | Rare trichromatic-only catch preserved by the monotonic combiner |
| none | 91 | Misses still visible, still useful, still shaping the next dataset pull |
Where the hybrid still fails
The harness is useful because it leaves the misses visible. The highest miss buckets are the categories that need either more data or better structural checks next.
| Category | Misses | Why it matters |
|---|---|---|
| prompt_extraction | 9 | Polite extraction requests still slip past too often |
| multilingual | 9 | The model and rule stack still need more multilingual examples |
| role_confusion | 8 | Persona and role hijack are still under-covered |
| tongue_manipulation | 7 | SCBE-specific language abuse still needs direct coverage |
| model_extraction | 7 | Weights and architecture theft patterns still under-trigger |
| autonomous_escalation | 7 | Scope creep chains still need stronger escalation handling |
What changed in the combiner
The latest run reflects a monotonic severity fix. In practical terms, a real trichromatic quarantine can no longer be neutralized by a later all-pass council outcome. That change preserved one union-only attack and removed the old benign numeric false positive at the same time.
Evidence packet
artifacts/benchmark/hybrid_overlap_latest.json timestamp: 2026-03-30T02:08:57Z benchmark: 200 attacks, 10 benign prompts monotonic combiner active