Scanner QA needs known targets, pinned identity, reproducible execution and an independently reviewed truth set. A large number of findings alone says little about accuracy.
Next step: Review fixture identity, truth ownership and negative controls before accepting a benchmark result.
Governed fixtures
The QFinch QA Lab uses intentionally vulnerable public applications and repositories with pinned source revisions or container digests. Authorization and isolation are established before active testing.
Repeatable execution
Automation records target identity, scanner configuration, engine provenance, timestamps, logs, reports and checksums. Capacity failures and warning states remain part of the result.
Candidate truth and independent approval
Known application documentation, source behaviour and reproducible requests create candidate mappings. A separate reviewer must approve whether each case belongs in the scored truth set. Until then, results are described as engineering coverage, not formal recall.
Measure misses and noise
Detected cases, missed cases and unverified alerts are retained. Fixing a scanner means rerunning the same comparison and preserving before/after evidence—not changing the expected set to fit the result.
Current boundary
Current evidence supports selected engineering comparisons and production-path scanner execution. It does not establish independent market parity across every scanner family.