Mobile and Stationary Computer Vision based Traffic Surveillance Techniques for Advanced ITS Applications

During the past decade, new sensing technologies, such as inductive loops, laser range scanners, radar detectors and computer vision sensors have been greatly enhanced and applied to the Intelligent Transportation System (ITS) area. Among all these sensor systems, computer vision-based approaches are one of the most popular and promising techniques used in ITS for traffic evaluation and management, driver assistance, as well as other safety related research. This is primarily due to the advantages of easy maintenance, high flexibility, and low cost for traffic scene monitoring and analysis. Many stationary vision sensors have been already installed near the roadway, particularly at intersections. In addition, more and more vision sensors are now being installed on mobile vehicles, in order to have real time surrounding traffic information. This dissertation focuses on both mobile and stationary computer vision based traffic surveillance techniques, including the development of a new vision sensor, a survey anddevelopment of vision algorithms, as well as their applications in three different aspects of ITS areas with high quantitative requirements. These areas are outlined in further detail below. Portable Loop Fault Detection For many years, it has been difficult to quantitatively measure real-time freeway traffic conditions. Numerous research projects have been carried out in traffic surveillance; for example, the Freeway Performance Measurement System (PeMS) operated by Caltrans and UC Berkeley collects, processes, aggregates, and examines traffic data primarily through loop detectors buried beneath the freeway. This type of stationary embedded loop sensor system provides a point measurement for the traffic flow, roadway occupancy and average speed. By aggregating these directly measured traffic data, they can be used to estimate and provide a larger picture of the traffic conditions in certain area. However, the results obtained from embedded loop sensors are not entirely reliable. The embedded loop data delivered to a Traffic Management Center (TMC) may contain errors at one or more sensors, and between the loop detector and the TMC database. As a result, loop fault detection is important. In this dissertation, a stationary-vision based technique has been developed as part of a Portable Loop Fault Detection Tool (PLFDT). This work is complementary to recent research focusing on aggregated faulty loop data at a macroscopic level (the macroscopic level generally considers a large roadway network as consisting of links (roadways) and nodes (e.g., intersections)). The objectives of the PLFDT is to develop a real time, multi-lane, multi-vehicle tracking system for freeways using video cameras as the baseline measurement technique to compare the loop detection signal for direct fault detection for inductive loop system. Localized Traffic Density Measurement The embedded loop sensor system provides a direct measurement to traffic flow, roadway occupancy and average speed (only for double-loop detector). This type of sensor network does not directly measure traffic density; instead it can only be estimated. In this dissertation, we have developed systematic techniques to measure traffic conditions by utilizing both on- and off-board computer vision systems. A unique development technique is a combined computer vision and Global Positioning System (GPS) equipped mobile traffic surveillance system to measure localized traffic density. In addition, we correlate the localized density measurement with estimates from embedded loop sensor system using a space-time diagram. Experiments have shown the complementary nature of these sensing techniques. Further, most traffic surveillance computer vision algorithms and techniques are typically based on observing vehicles from stationary rectilinear cameras mounted near roadways. For many applications, some of the key tasks include extracting traffic information such as average traffic speed, flow,and density. However, less research has been carried out in observing and generating the localized traffic map around specific vehicles in the traffic stream. In this dissertation, we have developed a vision-equipped vehicle test bed for traffic surveillance purposes and have experimentally demonstrated the generation of localized traffic density from video processing and synthesizing. In contrast to the off-board surveillance systems (e.g. embedded loop sensor networks and stationary vision monitoring system), this type of on-board surveillance system provides a temporal- and spatial- continuous measurement of the localized traffic density. One of the key components developed is an Orthogonal Omni-directional Vision (OODV) System that has been developed to observe lane-level activity surrounding a vehicle, as well as the ability to observe the surrounding roadway geometry. This vision system uses a special catadioptric mirror providing a 360 degree orthogonal view of the environment. It is different from other catadioptric mirror-based Omni-directional vision systems in that it directly provides an orthogonal image without the need of warping a polar-coordinate based image to a perspective view. Based on this unique OODV, a roadway traffic surveillance system was designed and implemented. It consists of three major components: • A GPS stamped roadway traffic data collection technique; • Post video processing that includes automatic vehicle detection/tracking with theability to correct using interactive tools; • A traffic parameter (localized density) estimation process. Combined with a GPS receiver that provides approximately 2 - 3 meters spatial resolution, this traffic surveillance system can be applied not only in several traffic applications which require localized traffic density/flow/average speed measurements, but also in some other applications that require detailed roadway geometry acquisition, and vehicle activity analysis. Based on experimentation, it has been shown that the designed mobile surveillance system reports a high detection rate under the dynamic freeway environment in the experiments, with assistance from a human interactive detection module. In order to have a better understanding of dynamic traffic conditions, we have incorporated this localized traffic density measurements into a Dynamic Roadway Network Database (DRND), which has been developed to fuse the roadway traffic data and the probe vehicle data. We believe that with the increasing use of on-board vision sensors, more and more localized traffic information samples can be reported to this type of database. The combinational analysis of temporal-spatial variable density and the embedded loop sensor data will provide a better and more reliable method for traffic condition estimation and prediction. Bicycle Safety Support System In addition to these new traffic data collection/analysis techniques and verification process, computer vision techniques are being applied in safety studies as well. In the third part of this dissertation, stationary vision based observations have been made of the timing of bicyclists’ intersection crossing maneuvers, to support of efforts in improving traffic signal timing to accommodate the needs of bicyclists. Video recordings were made of bicyclists’ crossings and the video images were processed to extract the bicyclists’ trajectories. These were synchronized with video images of the traffic signals so that the timing of the bicyclists’ maneuvers could be determined relative to the signal phases. The processed data have yielded cumulative distributions of the crossing speeds of bicyclists who did not have to stop at the intersection and the start-up times and final crossing speeds of the bicyclists who had to cross from a standing start. This study provides a foundation in recommendation of minimal green signal time in terms of safety purpose. The key contributions of this dissertation are: • A unique OODV System has been designed and developed; • A vision based systematic approach for developing a mobile traffic surveillance system has been proposed and implemented; • A time-space diagram based flow calculation method from localized density has been proposed and experimentally verified; • A stationary-vision based technique has been developed as part of a Portable Loop Fault Detection Tool (PLFDT) to provide baseline data; and • A stationary vision-based intersection monitoring system has been developed for the quantitative study of bicyclist crossings at signalized intersections.