Real-time Image Signal Processor Stats Management to Save Power and CPU Cycles

This publication describes techniques and apparatuses, implemented on a digital image capture device, directed at minimizing power consumption and central processing unit (CPU) cycles during image capture and processing events. An image sensor on the device captures a scene as a frame and generates raw image data. An on-device image signal processor (ISP) receives the raw image data and generates a statistics output (“stats output”) that includes image statistics for the frame. The stats output further includes a descriptive tag for the image statistics, saved in a header of the stats output. Software implemented on the device (e.g., a Statistics Manager) receives the stats output, parses the descriptive tag from the header, compares the descriptive tag to one or more previous descriptive tags, and determines if a change in the stats output is greater than a threshold. Upon determining that the change in the stats output is less than the threshold, the Statistics Manager determines that processing of the stats output by an Image Processing Module (e.g., 3A algorithms, other ISP software algorithms) is not necessary. Upon determining that the change in the stats output is greater than the threshold, the Statistics Manager determines processing of the stats output by the Image Processing Module is necessary. Through the use of such techniques and apparatuses, an image capture device can avoid unnecessary processing of stats outputs

Multimodal segmentation of lifelog data

A personal lifelog of visual and audio information can be very helpful as a human memory augmentation tool. The SenseCam, a passive wearable camera, used in conjunction with an iRiver MP3 audio recorder, will capture over 20,000 images and 100 hours of audio per week. If used constantly, very soon this would build up to a substantial collection of personal data. To gain real value from this collection it is important to automatically segment the data into meaningful units or activities. This paper investigates the optimal combination of data sources to segment personal data into such activities. 5 data sources were logged and processed to segment a collection of personal data, namely: image processing on captured SenseCam images; audio processing on captured iRiver audio data; and processing of the temperature, white light level, and accelerometer sensors onboard the SenseCam device. The results indicate that a combination of the image, light and accelerometer sensor data segments our collection of personal data better than a combination of all 5 data sources. The accelerometer sensor is good for detecting when the user moves to a new location, while the image and light sensors are good for detecting changes in wearer activity within the same location, as well as detecting when the wearer socially interacts with others

Clementine Sensor Processing System

The design of the DSPSE Satellite Controller (DSC) is baselined as a single-string satellite controller. The DSC performs two main functions: health and maintenance of the spacecraft; and image capture, storage, and playback. The DSC contains two processors: a radiation-hardened Mil-Std-1750, and a commercial R3000. The Mil-Std-1750 processor performs all housekeeping operations, while the R3000 is mainly used to perform the image processing functions associated with the navigation functions, as well as performing various experiments. The DSC also contains a data handling unit (DHU) used to interface to various spacecraft imaging sensors and to capture, compress, and store selected images onto the solid-state data recorder. The development of the DSC evolved from several key requirements; the DSPSE satellite was to do the following: (1) have a radiation-hardened spacecraft control system and be immune to single-event upsets (SEU's); (2) use an R3000-based processor to run the star tracker software that was developed by SDIO (due to schedule and cost constraints, there was no time to port the software to a radiation-hardened processor); and (3) fly a commercial processor to verify its suitability for use in a space environment. In order to enhance the DSC reliability, the system was designed with multiple processing paths. These multiple processing paths provide for greater tolerance to various component failures. The DSC was designed so that all housekeeping processing functions are performed by either the Mil-Std-1750 processor or the R3000 processor. The image capture and storage is performed either by the DHU or the R3000 processor

Analysis of image noise in multispectral color acquisition

The design of a system for multispectral image capture will be influenced by the imaging application, such as image archiving, vision research, illuminant modification or improved (trichromatic) color reproduction. A key aspect of the system performance is the effect of noise, or error, when acquiring multiple color image records and processing of the data. This research provides an analysis that allows the prediction of the image-noise characteristics of systems for the capture of multispectral images. The effects of both detector noise and image processing quantization on the color information are considered, as is the correlation between the errors in the component signals. The above multivariate error-propagation analysis is then applied to an actual prototype system. Sources of image noise in both digital camera and image processing are related to colorimetric errors. Recommendations for detector characteristics and image processing for future systems are then discussed

Estimating Epipolar Geometry With The Use of a Camera Mounted Orientation Sensor

Context: Image processing and computer vision are rapidly becoming more and more commonplace, and the amount of information about a scene, such as 3D geometry, that can be obtained from an image, or multiple images of the scene is steadily increasing due to increasing resolutions and availability of imaging sensors, and an active research community. In parallel, advances in hardware design and manufacturing are allowing for devices such as gyroscopes, accelerometers and magnetometers and GPS receivers to be included alongside imaging devices at a consumer level. Aims: This work aims to investigate the use of orientation sensors in the field of computer vision as sources of data to aid with image processing and the determination of a scene’s geometry, in particular, the epipolar geometry of a pair of images - and devises a hybrid methodology from two sets of previous works in order to exploit the information available from orientation sensors alongside data gathered from image processing techniques. Method: A readily available consumer-level orientation sensor was used alongside a digital camera to capture images of a set of scenes and record the orientation of the camera. The fundamental matrix of these pairs of images was calculated using a variety of techniques - both incorporating data from the orientation sensor and excluding its use Results: Some methodologies could not produce an acceptable result for the Fundamental Matrix on certain image pairs, however, a method described in the literature that used an orientation sensor always produced a result - however in cases where the hybrid or purely computer vision methods also produced a result - this was found to be the least accurate. Conclusion: Results from this work show that the use of an orientation sensor to capture information alongside an imaging device can be used to improve both the accuracy and reliability of calculations of the scene’s geometry - however noise from the orientation sensor can limit this accuracy and further research would be needed to determine the magnitude of this problem and methods of mitigation

FMX (EEPIS FACIAL EXPRESSION MECHANISM EXPERIMENT): PENGENALAN EKSPRESI WAJAH MENGGUNAKAN NEURAL NETWORK BACKPROPAGATION

In the near future, it is expected that the robot can interact with humans. Communication itself has many varieties. Not only from word to word, but body language also be the medium. One of them is using facial expressions. Facial expression in human communication is always used to show human emotions. Whether it is happy, sad, angry, shocked, disappointed, or even relaxed? This final project focused on how to make robots that only consist of head, so it could make a variety facial expression like human beings. This Face Humanoid Robot divided into several subsystems. There are image processing subsystem, hardware subsystem and subsystem of controllers. In image processing subsystem, webcam is used for image data acquisition processed by a computer. This process needs Microsoft Visual C compiler for programming that has been installed with the functions of the Open Source Computer Vision Library (OpenCV). Image processing subsystem is used for recognizing human facial expressions. With image processing, it can be seen the pattern of an object. Backpropagation Neural Network is useful to recognize the object pattern. Subsystem hardware is a Humanoid Robot Face. Subsystem controller is a single microcontroller ATMega128 and a camera that can capture images at a distance of 50 to 120 cm. The process of running the robot is as follows. Images captured by a camera webcam. From the images that have been processed with image processing by a computer, human facial expression is obtained. Data results are sent to the subsystem controller via serial communications. Microcontroller subsystem hardware then ordered to make that facial expression. Result of this final project is all of the subsystems can be integrated to make the robot that can respond the form of human expression. The method used is simple but looks quite capable of recognizing human facial expression. Keyword: OpenCV, Neural Network BackPropagation, Humanoid Robo

An Innovative Method for Measuring Instrument Data Acquisition using Image Processing Techniques

Measuring instruments are essential for obtaining accurate data, but data acquisition can be challenging. We propose a novel method for measuring instrument data acquisition using a camera to capture the instrument display and image processing techniques to extract the measured values. We demonstrate the effectiveness and accuracy of this method by applying it to capture the magnetic field of a permanent magnet using a gauss meter and webcam. Our image processing process involves Python libraries for video processing, including the OpenCV library for contour detection and thresholding. The processed data is then saved to a text file for further analysis. Our results show that our proposed method is effective and accurate, and offers a practical solution for cases where a direct cable connection is not possible or is difficult to establish. This method has potential applications in scientific research, engineering, and manufacturing

Image Denoising with Graph-Convolutional Neural Networks

Recovering an image from a noisy observation is a key problem in signal processing. Recently, it has been shown that data-driven approaches employing convolutional neural networks can outperform classical model-based techniques, because they can capture more powerful and discriminative features. However, since these methods are based on convolutional operations, they are only capable of exploiting local similarities without taking into account non-local self-similarities. In this paper we propose a convolutional neural network that employs graph-convolutional layers in order to exploit both local and non-local similarities. The graph-convolutional layers dynamically construct neighborhoods in the feature space to detect latent correlations in the feature maps produced by the hidden layers. The experimental results show that the proposed architecture outperforms classical convolutional neural networks for the denoising task.Comment: IEEE International Conference on Image Processing (ICIP) 201

A stereoscopic ranging system using standard PC technology

A stereoscopic ranging system is currently being developed as a key source of positional information for an underwater ROV station keeping system. Advancements in PC technology make it possible to use a relatively simple image capture card and a PC as a platform for the fast capture and processing of video images. We make use of the extensive capabilities of fast data buses and the high processing power of fast PCs with Pentium II or III processors. Using this approach we are developing an image processing system that is largely manufacturer independent and promises a good path for both hardware and software upgrading in the future

Navigating the roadblocks to spectral color reproduction: data-efficient multi-channel imaging and spectral color management

Commercialization of spectral imaging for color reproduction will require the identification and traversal of roadblocks to its success. Among the drawbacks associated with spectral reproduction is a tremendous increase in data capture bandwidth and processing throughput. Methods are proposed for attenuating these increases with data-efficient methods based on adaptive multi-channel visible-spectrum capture and with low-dimensional approaches to spectral color management. First, concepts of adaptive spectral capture are explored. Current spectral imaging approaches require tens of camera channels although previous research has shown that five to nine channels can be sufficient for scenes limited to pre-characterized spectra. New camera systems are proposed and evaluated that incorporate adaptive features reducing capture demands to a similar few channels with the advantage that a priori information about expected scenes is not needed at the time of system design. Second, proposals are made to address problems arising from the significant increase in dimensionality within the image processing stage of a spectral image workflow. An Interim Connection Space (ICS) is proposed as a reduced dimensionality bottleneck in the processing workflow allowing support of spectral color management. In combination these investigations into data-efficient approaches improve two critical points in the spectral reproduction workflow: capture and processing. The progress reported here should help the color reproduction community appreciate that the route to data-efficient multi-channel visible spectrum imaging is passable and can be considered for many imaging modalities