What Actually Happens When the AI Counts a Car
Every round runs on four systems working in parallel — detection, tracking, streaming, and fairness. Here's a frame-by-frame look at those 55 seconds. Wondering whether the numbers can be trusted? Our Rush Hour fairness breakdown digs into that.
1. The Vision Pipeline
Detection — identifying what's on camera
Each CCTV frame — and there are 30 of them every second — passes through a Convolutional Neural Network. We think of it as a pattern-recognition engine trained on millions of traffic images shot in every condition imaginable: bright afternoons, rainy nights, foggy dawns, sunsets that blow out the lens with glare.
It sorts everything into categories — car, truck, bus, motorbike, bicycle, pedestrian — and draws a bounding box around each one. The whole process takes under 100 milliseconds per frame running on GPU clusters, which is fast enough to keep pace with actual traffic in real time.
30 fps
Frame rate
Nothing skipped
<100ms
Inference time
Per-frame GPU processing
6+
Object classes
Cars, trucks, buses, bikes, walkers
2. Tracking Each Vehicle
Counting — tallying what crosses the line
Detection identifies what's on screen. Tracking follows where it heads next. Rush Hour relies on Kalman filtering — a predictive math model that projects where each vehicle will land in the following frame, based on its current speed and heading.
This matters because a car traveling at 40 km/h shows up in roughly 15 consecutive frames. Without tracking, the system would tally that same car 15 separate times. Instead, Kalman filters tag each object with a unique ID and follow it frame to frame. Only one thing triggers the count: the moment the object crosses a virtual line drawn across the road.
Overlapping vehicles — say a bus hiding a car behind it — trigger deep learning Re-Identification (ReID). It matches the car's visual signature before and after the occlusion, so the same vehicle that disappeared behind the bus and reappeared gets logged once, not twice.
3. When Conditions Aren't Ideal
How the system copes with imperfect footage
Night Time
Headlight glare and long shadows complicate things after dark. We trained the model on night footage from all 6 cities, and infrared-capable cameras at some locations add extra data. Even so, accuracy dips by about 3% at night.
Rain & Fog
Droplets on the lens, lower visibility, headlights bouncing off wet asphalt — rain throws a lot at the model. It leans on temporal consistency here: if an object had already been tracked for 10 frames before the blur set in, it keeps its ID.
Heavy Traffic (Gridlock)
Bumper-to-bumper traffic means near-constant occlusion. The system leans on ReID plus partial detections, and once a vehicle drops below 30% visibility, it defers to the Kalman prediction instead of the raw detection.
Road Works / Accidents
Stopped traffic, emergency vehicles, anything outside the norm triggers the outlier detection system. Flagged rounds get reviewed, and depending on what happened, the threshold gets adjusted or the round gets voided.
4. Getting Video to Your Screen
Delivery — from camera to browser
Real-time counting doesn't mean much if the footage itself lags 5 seconds behind. That's why Rush Hour runs on Dolby Millicast, a WebRTC-based platform built for sub-second latency anywhere in the world. The car crossing the zone on your screen is crossing it right now — not a few seconds ago.
Adaptive bitrate scales video quality to match your connection. On slow mobile data, you'll get a lower-resolution stream that still runs smoothly; on fast WiFi, full HD. The count overlay itself is synced server-side, so even a brief buffering hiccup on your end won't throw off the number you're seeing — it's always the real, current count.
5. Keeping It Fair
Normalization — balancing wildly different traffic volumes
Rush hour in Tokyo can push 40+ vehicles past the camera every minute. Sydney at 3 AM might see 2. Without adjusting for that gap, an "over 5" bet would be a lock in Tokyo and nearly unwinnable in late-night Sydney. That's exactly what the fairness engine corrects for.
Dynamic thresholds are set from historical density data — millions of data points collected per camera, per hour. That's why the Over/Under line for Tokyo at 8:00 AM looks nothing like the line for Sydney at 2:00 AM. The house edge doesn't come from touching the count; it comes from calibrating that threshold so Over and Under land close to 50/50, with a small mathematical tilt toward the house.
The takeaway:
Rush Hour's house edge lives in the threshold math, not in the outcome. The AI reports exactly what it observes. The fairness engine sets the number you're betting against. Nobody's steering the traffic — the math is what shapes the game.
6. How Accurate Is It, Really?
Putting numbers on the precision
Under controlled testing, the AI lands at 97-99% accuracy in clear daytime conditions. That drops to roughly 94-96% at night, and 92-95% in heavy rain or fog. We arrive at these figures by comparing the AI's counts against manual human counts on identical footage.
Each round leaves behind a complete audit trail — camera ID, city, timestamp, threshold, AI count, every bet outcome. These records are kept and available for review. Any round flagged by the outlier detection system, whether it's a road closure or an accident throwing off the pattern, gets marked and can be voided if the count can't be trusted.
97-99%
Daytime, clear
94-96%
Night
92-95%
Rain / fog