Traffic monitoring and violation detection is a classic computer vision problem that looks deceptively simple but becomes complex very quickly in real-world conditions. Variations in lighting, camera angles, occlusions, vehicle density, and inconsistent road markings make rule-based approaches unreliable at scale.

In this article, I’ll walk through how I designed and implemented an end-to-end traffic violation detection system using modern computer vision techniques. The goal was not just to detect vehicles, but to track them across frames, understand movement patterns, and identify violations in a way that could realistically work in production.

Problem Overview

The core problem was to automatically detect and analyse vehicle behaviour from video streams to identify traffic violations such as illegal turns, lane violations, or restricted-area movement.

Key challenges included:

• Detecting vehicles accurately in crowded scenes
• Maintaining consistent tracking across frames
• Handling partial occlusion and fast movement
• Designing a pipeline that could scale beyond a single video

This required more than just object detection, it needed tracking, spatial reasoning, and system-level thinking.

System Architecture

At a high level, the system consists of five stages:

Video ingestion 1.

Vehicle detection 1.

Multi-object tracking 1.

Violation logic & analytics 1.

Visualisation & reporting

Each stage was designed to be modular so components could be improved independently.

Vehicle Detection

For detection, I used YOLO-based models due to their balance of speed and accuracy in real-time scenarios.

Key decisions:

• Fine-tuned YOLO models for vehicle classes
• Used SAHI (Slicing Aided Hyper Inference) to improve detection accuracy on high-resolution frames
• Balanced inference speed with recall to avoid missing fast-moving vehicles

YOLO provided reliable bounding boxes even in moderately dense traffic, while SAHI helped with smaller vehicles at distance.

Multi-Object Tracking

Detection alone isn’t enough, violations require understanding movement over time.

For tracking, I used DeepSORT, which combines:

• Kalman filtering for motion prediction
• Appearance embeddings for identity consistency

This allowed the system to:

• Assign unique IDs to vehicles
• Track them across frames
• Handle temporary occlusions reasonably well

Tracking stability was critical, as even small ID switches can invalidate violation logic.

Violation Detection Logic

Once vehicles were reliably tracked, the next challenge was defining violation rules.

Rather than hardcoding pixel-based rules, I implemented:

• Region-based logic (entry/exit zones)
• Directional flow analysis
• Temporal thresholds to reduce false positives

For example:

• Vehicles entering restricted zones were flagged only after consistent tracking
• Sudden detection spikes were ignored unless sustained across frames

This approach made the system more robust to noise and camera artefacts.

Data Pipeline & Performance Considerations

To keep the system production-ready:

• Frames were processed in batches where possible
• Inference and tracking were decoupled
• Intermediate metadata (IDs, coordinates, timestamps) was stored for later analysis

Performance trade-offs were carefully managed:

• Higher resolution improved detection but increased latency
• Tracking stability was prioritised over raw FPS for accuracy

These decisions are often overlooked but matter significantly in real deployments.

Visualisation & Outputs

To make the system usable:

• Violations were overlaid directly on video output
• Vehicle IDs and paths were visualised
• Structured outputs were generated for downstream analytics

This made it easier to validate system behaviour and explain results to non-technical stakeholders.

Key Challenges & Learnings

Some of the most important lessons from this project:

• Detection accuracy is only half the problem, tracking quality matters just as much
• Real-world video is noisy, and systems must tolerate imperfect data
• Simple rules often outperform complex models when designed carefully
• Modular design makes iteration significantly easier
• These learnings influenced how I approach applied AI systems beyond computer vision.

Final Thoughts

This project reinforced the importance of engineering judgement when building applied AI systems. Models alone don’t solve problems, thoughtful system design, realistic assumptions, and careful trade-offs are what make solutions viable in production.

The full implementation, including model setup and pipeline code, is available on GitHub:

👉 GitHub Repository: CLICK HERE