Optimal Radiometric Calibration for Camera-Display Communication

We present a novel method for communicating between a camera and display by embedding and recovering hidden and dynamic information within a displayed image. A handheld camera pointed at the display can receive not only the display image, but also the underlying message. These active scenes are fundamentally different from traditional passive scenes like QR codes because image formation is based on display emittance, not surface reflectance. Detecting and decoding the message requires careful photometric modeling for computational message recovery. Unlike standard watermarking and steganography methods that lie outside the domain of computer vision, our message recovery algorithm uses illumination to optically communicate hidden messages in real world scenes. The key innovation of our approach is an algorithm that performs simultaneous radiometric calibration and message recovery in one convex optimization problem. By modeling the photometry of the system using a camera-display transfer function (CDTF), we derive a physics-based kernel function for support vector machine classification. We demonstrate that our method of optimal online radiometric calibration (OORC) leads to an efficient and robust algorithm for computational messaging between nine commercial cameras and displays.Comment: 10 pages, Submitted to CVPR 201

Optical Camera Communications: Principles, Modulations, Potential and Challenges

Optical wireless communications (OWC) are emerging as cost-effective and practical solutions to the congested radio frequency-based wireless technologies. As part of OWC, optical camera communications (OCC) have become very attractive, considering recent developments in cameras and the use of fitted cameras in smart devices. OCC together with visible light communications (VLC) is considered within the framework of the IEEE 802.15.7m standardization. OCCs based on both organic and inorganic light sources as well as cameras are being considered for low-rate transmissions and localization in indoor as well as outdoor short-range applications and within the framework of the IEEE 802.15.7m standardization together with VLC. This paper introduces the underlying principles of OCC and gives a comprehensive overview of this emerging technology with recent standardization activities in OCC. It also outlines the key technical issues such as mobility, coverage, interference, performance enhancement, etc. Future research directions and open issues are also presented

Improved Visible Light Communication Receiver Performance by Leveraging the Spatial Dimension

In wireless communications systems, signals can be transmitted as time (temporal) or spatial variants across 3D space, and in both ways. However, using temporal variant communication channels in high-speed data transmission introduces inter-symbol interference (ISI) which makes the systems unreliable. On the other hand, spatial diversity in signal processing reduces the ISI and improves the system throughput or performance by allowing more signals from different spatial locations at the same time. Therefore, the spatial features or properties of visible light signals can be very useful in designing a reliable visible light communication (VLC) system with higher system throughput and making it more robust against ambient noise and interference. By allowing only the signals of interest, spatial separability in VLC can minimize the noise to a greater extent to improve signal-to-noise ratio (SNR) which can ensure higher data rates (in the order of Gbps-Tbps) in VLC. So, designing a VLC system with spatial diversity is an exciting area to explore and might set the foundation for future VLC system architectures and enable different VLC based applications such as vehicular VLC, multi-VLC, localization, and detection using VLC, etc. This thesis work is motivated by the fundamental challenges in reusing spatial information in VLC systems to increase the system throughput or gain through novel system designing and their prototype implementations

A Study of 5G Cellular Connectivity to Unmanned Aerial Vehicles

The market of unmanned aerial vehicles (UAVs) has seen significant growth in the past ten years on both the commercial and military sides. The applications for UAVs are endless and options by manufacturers allow users to modify their drones for their specific goals. This industry has opened up the excitement of piloting vehicles in the air, photography, videography, exploration of nature from a different point of view and many other hobbies assisted by the emergence of UAVs. The growth of this industry coincides with the roll out of new 5G cellular network technology. This upgrade in cellular network infrastructure allows users higher bandwidth, lower latency, more devices per cell and higher reliability. This has created the question, is 5G suited to support UAV activity? Potentially allowing for two-way transmission of images, videos and data between ground users and the unmanned aerial vehicle. There are many challenges that are presented in flying under these communication conditions which need to be explored such as signal reliability, especially in rural areas, the effects of rapidly changing altitudes or velocity of the drone and the effects of antennas that are tuned for terrestrial users. The first of its kind work provided in this thesis, will show results for different UAV experiments on a commercial 5G cellular network in the Clemson, South Carolina area. This is a comprehensive study of both low-band and mid-band 5G cellular coverage relating to UAVs as well as a baseline to existing LTE coverage when available. Featuring first of its kind permission to conduct research on a fully commercial cellular network. This research area is largely new, limited information is currently public on the research into commercial 5G cellular networks supporting UAVs. Other researchers are also starting to collect different key performance indicators (KPIs) for flight signals. Most of their works differ in setup, often using private base stations to give connection, but many of these works will be discussed further in the thesis. LTE and 5G enabled flight allows for a wide variety of applications to use UAVs such as natural disaster assessment, animal poaching surveillance, wild fire detection and prevention, assessing the scene of an accident before police arrive and other more hobby or recreational uses. The end goal is to assure that 5G connection is strong enough to transmit the UAVs real time data, which necessary to help first responders on the ground. When many of the potential uses of a cellular connected UAV are potentially life saving, every second counts and signal needs to be fast, reliable and low latency. Therefore, reliable and high bandwidth communication is necessary for unmanned aerial vehicles to take the next step in real life use cases and to begin to explore the option of beyond visual line of sight (BVLoS) flight and 5G might be the network tools which can get it there

Optical Camera Communications for IoT–Rolling-Shutter Based MIMO Scheme with Grouped LED Array Transmitter

In optical camera communications (OCC), the provision of both flicker-free illumination and high data rates are challenging issues, which can be addressed by utilizing the rolling-shutter (RS) property of the image sensors as the receiver (Rx). In this paper, we propose an RS-based multiple-input multiple-output OCC scheme for the Internet of things (IoT) application. A simplified design of multi-channel transmitter (Tx) using a 7.2 × 7.2 cm2 small 8 × 8 distributed light emitting diode (LED) array, based on grouping of LEDs, is proposed for flicker-free transmission. We carry out an experimental investigation of the indoor OCC system by employing a Raspberry Pi camera as the Rx, with RS capturing mode. Despite the small area of the display, flicker-free communication links within the range of 20–100 cm are established with data throughput of 960 to 120 bps sufficient for IoT. A method to extend link spans up to 1.8 m and the data throughput to 13.44 kbps using different configurations of multi-channel Tx is provided. The peak signal-to-noise ratio of ~14 and 16 dB and the rate of successfully received bits of 99.4 and 81% are measured for the shutter speeds of 200 and 800 µs for a link span of 1 m, respectively

VRCodes : embedding unobtrusive data for new devices in visible light

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 97-101).This thesis envisions a public space populated with active visible surfaces which appear different to a camera than to the human eye. Thus, they can act as general digital interfaces that transmit machine-compatible data as well as provide relative orientation without being obtrusive. We introduce a personal transceiver peripheral, and demonstrate this visual environment enables human participants to hear sound only from the location they are looking in, authenticate with proximal surfaces, and gather otherwise imperceptible data from an object in sight. We present a design methodology that assumes the availability of many independent and controllable light transmitters where each individual transmitter produces light at different color wavelengths. Today, controllable light transmitters take the form of digital billboards, signage and overhead lighting built for human use; light-capturing receivers take the form of mobile cameras and personal video camcorders. Following the software-defined approach, we leverage screens and cameras as parameterized hardware peripherals thus allowing flexibility and development of the proposed framework on general-purpose computers in a manner that is unobtrusive to humans. We develop VRCodes which display spatio-temporally modulated metamers on active screens thus conveying digital and positional information to a rolling-shutter camera; and physically-modified optical setups which encode data in a point-spread function thus exploiting the camera's wide-aperture. These techniques exploit how the camera sees something different from the human. We quantify the full potential of the system by characterizing basic bounds of a parameterized transceiver hardware along with the medium in which it operates. Evaluating performance highlights the underutilized temporal, spatial and frequency dimensions available to the interaction designer concerned with human perception. Results suggest that the one-way point-to-point transmission is good enough for extending the techniques toward a two-way bidrectional model with realizable hardware devices. The new visual environment contains a second data layer for machines that is synthetic and quantifiable; human interactions serve as the context.by Grace Woo.Ph.D