AutoVision-ADAS: 77 GHz Camera-Radar Spatial Fusion & Monocular Time-to-Collision (TTC)

1. Why Single-Modality Perception Fails

Vision-only systems suffer in heavy rain, direct headlight glare, and dense fog, leading to missed detections or phantom braking. Radar sensors provide Doppler velocity and metric range but lack semantic classification and spatial resolution.

2. Extrinsic Coordinate Frame Fusion

We project 3D radar track positions $P_{\text{radar}} = [X_r, Y_r, Z_r, 1]^T$ into normalized camera pixel coordinates $p = [u, v, 1]^T$ via extrinsic translation $t$, rotation $R$, and intrinsic matrix $K$: $$s \cdot \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = K \begin{bmatrix} R & t \end{bmatrix} \begin{bmatrix} X_r \\ Y_r \\ Z_r \\ 1 \end{bmatrix}$$ Validated radar targets are spatially gated with YOLOv8 camera bounding boxes via Intersection-over-Union (IoU), combining optical semantic classification with Doppler velocity precision for failsafe AEB decisions.