Benchmarks and methodology
This page explains how we measure TrueSecurix, so that any number we publish can be checked. Deepfake-detection results below are measured on a held-out split of the current production model. Liveness and presentation-attack numbers (APCER, BPCER, ACER) require the gated public PAD datasets and are measured as we obtain access; the liveness system is engineered to the ISO/IEC 30107-3 methodology.
How we evaluate
We measure on a held-out split: a set of samples the model is never trained on. Training data and evaluation data are kept separate, so the numbers reflect performance on data the model has not seen.
Deepfake detection
Deepfake detection is evaluated across multiple generator families, not a single one: face-swap, diffusion, and GAN. This checks that detection holds across the different ways a fake face can be produced, rather than overfitting to one generator.
Liveness and presentation-attack detection
Liveness and presentation-attack detection is engineered to the ISO/IEC 30107-3 methodology. We report the standard metrics from that framework:
- APCER (attack presentation classification error rate): the share of attack presentations wrongly accepted as genuine.
- BPCER (bona fide presentation classification error rate): the share of genuine presentations wrongly rejected.
- ACER (average classification error rate): the average of APCER and BPCER.
The metric we optimise for
The headline metric we optimise for is the lowest false-positive and false-reject rate on genuine users. We tune first to avoid rejecting real customers. In these results, "caught" means a sample was flagged or routed to human review, not silently passed.
Datasets
Our evaluation harness targets recognised public datasets so that results can be compared against known benchmarks:
- Deepfake: FaceForensics++, DFDC, Celeb-DF.
- Liveness and presentation attack: OULU-NPU, SiW-M, CelebA-Spoof.
When we publish numbers, each result will state the exact dataset and protocol used, so it can be reproduced.
Deepfake detection results (measured)
Two evaluations. The first is the KYC decision layer on a held-out set of 90 genuine selfies and 120 deepfakes spanning 10 generator families (face-swap, diffusion, and GAN, including compressed and blurred variants). The second is the trained detector's raw discrimination (ROC-AUC) on a larger held-out sample. In both, "caught" means flagged or routed to human review, never silently passed.
| Metric | Value | Set |
|---|---|---|
| Deepfakes caught (flagged or routed to review) | 100% | 90 real / 120 fake, 10 generators |
| Genuine users auto-rejected | 0.0% | same held-out set |
| Genuine selfies routed to human review | 32% | same held-out set |
| Detector ROC-AUC (trained CLIP probe) | 0.976 | 600 held-out images (300 real / 300 fake) |
| Per-generator catch rate (dalle3, faceswap, face2face, faceshifter, imagen, and more) | 100% | 12 samples per generator |
| Detector latency (trained probe) | ~13 ms/image (GPU) | full multi-signal check ~1s on CPU |
The 0.0% auto-reject rate is a property of the consensus decision layer: uncertain genuine users are routed to review, never auto-rejected. The detector's raw ROC-AUC is 0.976; the decision layer converts that into zero false rejects by design.
Liveness and presentation-attack results (in progress)
APCER, BPCER, and ACER are measured on the recognised public PAD datasets (OULU-NPU, SiW-M, CelebA-Spoof). These datasets are access-gated; we publish each number here with its exact dataset and protocol as we complete measurement. The liveness signal is live in the API today, engineered to the ISO/IEC 30107-3 methodology, and formal certification by an accredited lab is in progress.