Video-Based Vehicle Signature Analysis and Tracking Phase 1: Verification of Concept Preliminary Testing

This report describes the results of the PATH/Caltrans-funded project Video-Base Signature Analysis and Tracking (VSAT) System, Phase 1: Verification of Concept and Preliminary Testing. The VSAT System was conceived in 1995 by Loragen Systems, of Glendale, California, as a means for non-intrusively tracking individual vehicles on freeways for data collection purposes. The concept involves the use of a computer vision methods to make simple measurements of external dimensions, points of optical demarcation, and predominant colors of each vehicle. A conventional color video camera serves as the primary sensor in a self-contained detection module including a dedicated image processing computer and wireless communications components. Detection modules are placed directly above traffic lanes on an overcrossing or similar support structure, with one detector for each lane. For each passing vehicle, as Video Signature Vector (VSV) would be measured and transmitted by the detection module to a central correlation computer, via a local site repeater or a low-power local commercial digital service. The correlation computer continuously receives VSV's asynchronously transmitted by all detection modules, and attempts to match VSV's to re-identify each vehicle at each detectorized site, in order to determine the progress of the vehicle through the freeway network. If proven accurate and cost-effective, VSAT is potentially useful as a means for tracking the progress of individual vehicles in freeway traffic for such purposes as traffic flow model validation, generation of origin-destination data, travel time estimation, validation of local modal-based emission models, and possible applications in law enforcement. Potential advantages are low cost in widespread deployment, simplicity and reliability of detection, minimal bandwidth and storage requirements for transmission of the signature vector, and reasonable identification ability without violation of privacy rights. Phase 1 of this four-phase study involves field data collection and laboratory data reduction for the purpose of validating the operational concept of the method, Phase 1 was restricted to an assessment of the detectability and uniqueness of the video-based Vehicle Signature Vector (VSV). Two identical portable field data acquisition systems were designed to permit the synchronized recording of video images of vehicles flowing beneath two successive freeway overcrossings. These were used at three pairs of test sites along US Highway 101 in the Central Coast area of California. Each pair of sites consisted of two accessible overcrossings separated by approximately 0.5 miles. Field tests were conducted over a range of traffic conditions and times of day. Time-synchronized video-tapes from both overcrossings and each test site were studied in the laboratory on a frame-by-frame basis. The S-VHS video tapes from each pair of sites were post-processed and analyzed in the laboratory on a frame-by-frame basis using video editing equipment and a reference video monitor. For each vehicle recorded by each camera, manual screen measurements were made of dimensions between points of optical demarcation (such as the windshield-to-hood transition) along a virtual centerline through each vehicle. Extremal length and width were also measured from the images of each vehicle. A PC-based computer vision system was programmed to provide an objective characterization of the predominant color for each vehicle. From this collection of measurements for each observed vehicle, a VSV was manually generated. Time-indexed lists of the VSVs for each vehicle, and all possible pair-wise comparisions of VSVs for each of four test conditions were created in Microsoft EXCEL spreadsheets. Data sets were segregated by four test conditions, corresponding to four ambient lighting conditions: overhead sun (mid-day), 45 degree sun (afternoon), reduced light (dusk), and low light (night). For each test condition, VSV's were compared for each vehicle at the first site with every vehicle at the second site. A correlation error factor was developed based upon a normalized sum of the absolute difference between the vector components from each site. Used for comparison purposes, a "match" is detected if the correlation error for the pairing is below some fixed threshold, which was generally set to be inversely proportional to the intensity of the ambient illumination for the test condition. Results were accumulated on the accuracy of matching the same vehicle at consecutive sites (auto-correlation) and the possibility of falsely matching dissimilar vehicles at consecutive sites (cross-correlation). Auto-correlation was assessed by comparing the VSV of each vehicle observed at the first overcrossing with its VSV at the second overcrossing. Cross-correlation was assessed by comparing the VSV for each vehicle at the first overcrossing with the VSV of all other vehicles observed within the data collection period at the second overcrossing. Correct (auto-correlation) matches were observed for 97.27% of all vehicles at mid-day, 98.89% in the afternoon, and 95.15% at dusk. False (cross-correlation) matches occurred for 0.22% of all possible vehicle pairings at mid-day, 1.66% in the afternoon, and 2.02% at dusk. For daylight conditions, we also assessed the relative value of color as a VSV component, and the relative value of restricting vehicle comparisons at successive sites to a "reasonable time of arrival window". The additional color information was found to increase correct matches from 98.3% to 99.0% and reduce false matches from 5.4% to 0.3%. The restriction to "reasonable time of arrival window" was found to add almost no additional accuracy beyond the addition of color information for either metric, although we do not consider the sample size in this test large enough to be statistically sound. Informally, it appears that the use of an adaptive correlation threshold may improve accuracy with respect to both metrics, but the design and evaluation of adaptive algorithms were not within the scope of this study. Accuracy during data reduction was limited primarily by the ability to make manual measurements of VSV components - vehicle dimensions from the video CRT display, and color intensity and hue via computer image processing. The VSV was found to be difficult and sometimes impossible to measure at night (low light), with 75.49% correct matches and 27.05% false matches (without arrival window). General conclusions are that the VSV is a reliable and repeatable means for the characterization and successive re-identification of vehicles under daylight and transitional illumination conditions. The VSV is unusable if the illumination level is inadequate to produce an acceptable video image. Overall, we conclude that the VSAT method is valid for the tracking of individual vehicles through a highway network, but only during conditions of adequate ambient lighting, or with either supplemental illumination or the use of improved dynamic range video cameras. Keywords Video, detection, sensing, sensor, computer vision, image processing, traffic monitoring, vehicle tracking, transportation electronics, video signature vector, video signature analysis, advanced traffic management, surveillance, monitoring, correlation, ensemble averaging, network tracking, object identification.