Blog
How physics-based synthetic RAW data reveals the failure zones real datasets can't reach.
Every perception model has a failure zone.
A region defined by combinations of light level, object size, and resolution where detection performance degrades and eventually collapses. The problem isn't that the zone exists, it's that with real data alone, you often can't see where it is.
Real ADAS datasets tend to cluster around conditions that are easy to collect: decent lighting, clear weather, normal traffic. The hard stuff - night driving, rain, small targets near the resolution limit - is thin, because it's expensive and logistically difficult to capture at scale. And some scenarios, like a pedestrian crossing a dark road at 120 km/h, simply can't be safely recreated.
The side effect of this is easy to miss: if your test set doesn't cover those conditions, your benchmark doesn't either. You end up measuring performance on the cases that were convenient, not necessarily the ones that matter.
New research from Jetraw team from Dotphoton and University of Glasgow through the Centre for Doctoral Training in Applied Photonics, presented at the AUTOPILOT Workshop at CVPR 2026, quantified exactly this problem.
The team evaluated AODRaw, a state-of-the-art RAW object detection model, on real data across a range of light levels and a variety of scenes, including traffic-related scenarios. The result looked reassuring: performance metrics appeared roughly stable across illumination conditions. The model looked robust.
It wasn't. There just weren't enough dark examples in the real dataset to show otherwise.
Using a physics-based low-light RAW augmentation pipeline built on the Poisson-Gaussian noise model, the team generated synthetic data that filled the sparse, dark regions of the real dataset. With a balanced evaluation set covering the full light spectrum, the picture changed completely: the mean average precision dropped around 60% in extreme low-light conditions with no gain adjustment. Below a certain illumination threshold, the model made no detections at all.
The same model. A different dataset. A completely different conclusion.
This is the core finding: the failure mode wasn't new. It was always there. Real data was just too sparse in the critical range to expose it.
A benchmark that can't reach your model's failure modes is giving you an incomplete picture and this is especially dangerous in safety-critical systems, because the conditions most likely to cause accidents are exactly the ones least likely to be well-represented in your test set.
Dark scenes at distance, pedestrians near the resolution limit, low-SNR conditions in rain - these aren't exotic edge cases. They are the scenarios your system needs to handle reliably.
If your evaluation set underrepresents them, your reported metrics are optimistic by construction.
What certification actually requires isn't a single mAP score, but a characterization of how the model behaves across the full range of operating conditions: what's the minimum light level for reliable detection? At what distance does performance start degrading? Which combinations of illumination and resolution push the model into unreliable territory?
You can only answer those questions if your evaluation set actually covers those conditions. That's where physics-grounded synthetic data becomes necessary. It transforms sparse real-world datasets into a continuous evaluation space. It gives you precise control over the conditions you test and the failure zones that real data alone cannot reliably cover.
Jetraw AI is a synthetic data engine for vision AI. Built on the same sensor modelling technology as the one behind the Jetraw compressor, it generates sensor-accurate RAW data that reflects how scenes would truly be captured under specific operating conditions.
Rather than optimising for visual realism alone, Jetraw AI evaluates statistical consistency at the pixel distribution level, allowing synthetic data to closely match the characteristics of real sensor output. This significantly narrows the gap between synthetic and real data, enabling reliable scaling of rare, safety-critical scenarios without the exponential cost and logistical constraints of real-world data collection.
In practice, it lets you answer the questions that matter
If your ADAS evaluation program can't answer these questions today, the data probably isn't there yet.
Interested in what physics-based RAW augmentation can do for your perception stack? Get on touch: get@dotphoton.com
The full paper presented at the Autopilot Workshop at CVPR 2026, June 3–7, Denver, Colorado. Link to read the full paper: https://arxiv.org/html/2605.22455v1
Watch the 4-minute research summary: Valeria Pais walks through the methodology and findings — CVPR Autopilot Workshop 2026, Denver.
We use cookies to enhance your browsing experience and analyse site traffic. By clicking "Accept all" you consent to our use of cookies. Manage preferences