REAL-TIME EMBEDDED SYSTEM OF MULTI-TASK CNN FOR ADVANCED DRIVING ASSISTANCE

In this research, we've engineered a real-time embedded system for advanced driving assistance. Our approach involves employing a multi-task Convolutional Neural Network (CNN) capable of simultaneously executing three tasks: object detection, semantic segmentation, and disparity estimation. Confronted with the limitations of edge computing, we've streamlined resource usage by sharing a common encoder and decoder among these tasks. To enhance computational efficiency, we've opted for a blend of depth-wise separable convolution and bilinear interpolation, departing from the conventional transposed convolution. This strategic change reduced the multiply-accumulate operations to 23.3% and the convolution parameters to 16.7%.Our experimental findings demonstrate that the decoder's complexity reduction not only avoids compromising recognition accuracy but, in fact, enhances it. Furthermore, we've embraced a semi-supervised learning approach to heighten network accuracy when deployed in a target domain divergent from the source domain used during training. Specifically, we've employed manually crafted correct answers only for object detection to train the whole network for optimal performance in the target domain. For the foreground object categories, we generate pseudo-correct responses for semantic segmentation by employing bounding boxes from object detection and iteratively refining them. Conversely, for the background categories, we rely on the initial inference outcomes as pseudo-correct responses, abstaining from further adjustments. Semantic segmentation of object classes with widely different appearances can be achieved thanks to this method, which tells the rough position, size, and shape of each object to the task. Our experimental results substantiate that the incorporation of this semi-supervised learning technique leads to enhancements in both object detection and semantic segmentation accuracy. We implemented this multi-task CNN on an embedded Graphics Processing Unit (GPU) board, added multi-object tracking functionality, and achieved a throughput of 18 fps with 26 Watt power consumption

Integrated face detection, tracking, and pose estimation

This paper presents a proposal of an integrated method for face detection, tracking, and head pose estimation. We use the de-facto Viola-Jones method for face and face part detection. We adopt affine motion model estimation as a tracking method. The combination enables efficient detection around the search area limited by tracking. Moreover, it reduces false detection because of the consistent processing with earlier results. In addition, the method re-initializes the position and size of the face and face parts in every frame. That initialization immediately corrects tracking jitter. The head pose is estimated using coordinates of both eyes and a mouth relative to the nose as the origin in the coordinate system. The computational cost is low because it uses only those three points. Experimental results show accurate estimation of the head pose. The average error is 6.50 deg in yaw angle, and 7.65 deg in pitch angle. © 2012 IEEE

Vehicle detection and tracking with affine motion segmentation in stereo video

This paper proposes a novel method for vehicle detection and tracking based on stereo vision, motion analysis, and road detection. Combining stereo vision with motion analysis makes object detection more appropriate than each individual method. Object tracking with motion estimation, which follows the next object detection starting with an initial solution given by the tracking, improves detection accuracy. Road detection with motion segmentation, considering a parallax in stereo vision as a motion, enables on-road vehicle and obstacle detection. All elements constructing the proposed method are founded commonly on affine motion segmentation. Software and hardware implementations become efficient because their parts share the common principal procedure. Simulation results show the proposed method successfully detects and tracks vehicles on moving background in a complicated scene of downtown. © 2011 IEEE

動画像符号化用動き検出VLSIプロセッサの高性能化に関する研究

取得学位：博士(工学)，学位授与番号：博乙第279号，学位授与年月日：平成16年9月30

A 8ms delay image transmission system and its applications

金沢大学理工研究域電子情報学系Based on the real-time prototype hardware proposed on SID2007, We\u27ve finished developing a real-time image transmission system. This system achieved low system delay(8msec) and high image quality for both desktop images and motion pictures. In this paper, we explained the whole system and CPU control flow in details. System delay was analyzed in encode and decode side. In the end, some possible application fields were introduced

A VLSI architecture for VGA 30 fps video segmentation with affine motion model estimation

金沢大学理工研究域電子情報学系This paper proposes a VLSI architecture for VGA 30 fps video segmentation with affine motion model estimation. The adopted algorithm is formulated as a contextual statistical labeling problem exploiting multiscale Markov random field (MRF) models. The algorithm optimization for VLSI implementation is characterized by image division method, ICM labeling limited to region boundary, and omission of motion models estimation for new regions. The optimization reduces the computational costs by 82 %, the amount of memory by 95 %, and the amount of data traffic by 99 % without accuracy degradation. The VLSI architecture is characterized by pipeline processing of the divided images, concurrent motion models estimation for multiple regions, and a common processing element of update and detection labeling. The architecture enables VGA 30 fps video segmentation with 167 MHz frequency. The estimated core area using 0.18μm technology is 30 mm2. This processor is applicable to the video recognition applications such as vehicle safety, robot, and surveillance systems under the restriction of energy consumption

Adiabatic SRAM with a large margin of VT variation by controlling the cell-power-line and word-line voltage

金沢大学理工研究域電子情報学系An adiabatic 1-kb SRAM circuit was designed, which enables gradual charging during writing and reading while maintaining a large VDD so that the problems of VT variation and electromigration in the nanocircuit can be resolved. In the writing mode, the voltage of the memory cell power line is reduced to ground gradually using a high-resistivity nMOSFET, and we turn off the nMOSFET so that the memory cell power line is set in a high-impedance state. Then, we can write data easily by inputting adiabatic signal from one bit line, while the other bit line is set to ground. For reading, a verifying operation is proposed for resolving the electromigration problem. The word line voltage is changed stepwise while the voltages of the bit lines are verified. The reading method enables a gradual current flow in the memory cell. We designed the cell layout and found that there is no area penalty. In addition, a new charge recycle circuit with tank capacitors is proposed. ©2009 IEEE