An Audification and Visualization System (AVS) of an Autonomous Vehicle for Blind and Deaf People Based on Deep Learning

When blind and deaf people are passengers in fully autonomous vehicles, an intuitive and accurate visualization screen should be provided for the deaf, and an audification system with speech-to-text (STT) and text-to-speech (TTS) functions should be provided for the blind. However, these systems cannot know the fault self-diagnosis information and the instrument cluster information that indicates the current state of the vehicle when driving. This paper proposes an audification and visualization system (AVS) of an autonomous vehicle for blind and deaf people based on deep learning to solve this problem. The AVS consists of three modules. The data collection and management module (DCMM) stores and manages the data collected from the vehicle. The audification conversion module (ACM) has a speech-to-text submodule (STS) that recognizes a user’s speech and converts it to text data, and a text-to-wave submodule (TWS) that converts text data to voice. The data visualization module (DVM) visualizes the collected sensor data, fault self-diagnosis data, etc., and places the visualized data according to the size of the vehicle’s display. The experiment shows that the time taken to adjust visualization graphic components in on-board diagnostics (OBD) was approximately 2.5 times faster than the time taken in a cloud server. In addition, the overall computational time of the AVS system was approximately 2 ms faster than the existing instrument cluster. Therefore, because the AVS proposed in this paper can enable blind and deaf people to select only what they want to hear and see, it reduces the overload of transmission and greatly increases the safety of the vehicle. If the AVS is introduced in a real vehicle, it can prevent accidents for disabled and other passengers in advance

The Braking-Pressure and Driving-Direction Determination System (BDDS) Using Road Roughness and Passenger Conditions of Surrounding Vehicles

A fully autonomous vehicle must ensure not only fully autonomous driving but also the safety and comfort of its passengers. However, the self-driving technology that is currently completed focuses only on perfect driving and does not guarantee the safety and comfort of passengers. This paper proposes a braking-pressure and driving-direction determination system (BDDS), which computes the brake pressure and steering angle optimized for passenger safety by utilizing more diverse information than existing autonomous vehicles. The BDDS proposed in this paper consists of two modules. The road roughness classification module (RRCM) classifies the roughness of the road by using the pressure data applied to the suspension and the K-NN algorithm and computes the optimal brake pressure. The passenger recognition and sharing module (PRSM) identifies the current occupant status of the vehicle by using a body pressure sensor and CNN, shares the information with surrounding vehicles, and computes the optimal steering angle using passenger information and road information. As a result of the simulations described in this paper, the parameters of AI models were optimized. In addition, the RRCS was about 7% more accurate than the K-means clustering algorithm, and PRS was about 9% more accurate than the existing seat recognition system

The Lightweight Autonomous Vehicle Self-Diagnosis (LAVS) Using Machine Learning Based on Sensors and Multi-Protocol IoT Gateway

This paper proposes the lightweight autonomous vehicle self-diagnosis (LAVS) using machine learning based on sensors and the internet of things (IoT) gateway. It collects sensor data from in-vehicle sensors and changes the sensor data to sensor messages as it passes through protocol buses. The changed messages are divided into header information, sensor messages, and payloads and they are stored in an address table, a message queue, and a data collection table separately. In sequence, the sensor messages are converted to the message type of the other protocol and the payloads are transferred to an in-vehicle diagnosis module (In-VDM). The LAVS informs the diagnosis result of Cloud or road side unit(RSU) by the internet of vehicles (IoV) and of drivers by Bluetooth. To design the LAVS, the following two modules are needed. First, a multi-protocol integrated gateway module (MIGM) converts sensor messages for communication between two different protocols, transfers the extracted payloads to the In-VDM, and performs IoV to transfer the diagnosis result and payloads to the Cloud through wireless access in vehicular environment(WAVE). Second, the In-VDM uses random forest to diagnose parts of the vehicle, and delivers the results of the random forest as an input to the neural network to diagnose the total condition of the vehicle. Since the In-VDM uses them for self-diagnosis, it can diagnose a vehicle with efficiency. In addition, because the LAVS converts payloads to a WAVE message and uses IoV to transfer the WAVE messages to RSU or the Cloud, it prevents accidents in advance by informing the vehicle condition of drivers rapidly

A Total Crop-Diagnosis Platform Based on Deep Learning Models in a Natural Nutrient Environment

This paper proposes a total crop-diagnosis platform (TCP) based on deep learning models in a natural nutrient environment, which collects the weather information based on a farm’s location information, diagnoses the collected weather information and the crop soil sensor data with a deep learning technique, and notifies a farm manager of the diagnosed result. The proposed TCP is composed of 1 gateway and 2 modules as follows. First, the optimized farm sensor gateway (OFSG) collects data by internetworking sensor nodes which use Zigbee, Wi-Fi and Bluetooth protocol and reduces the number of sensor data fragmentation times through the compression of a fragment header. Second, the data storage module (DSM) stores the collected farm data and weather data in a farm central server. Third, the crop self-diagnosis module (CSM) works in the cloud server and diagnoses by deep learning whether or not the status of a farm is in good condition for growing crops according to current weather and soil information. The TCP performance shows that the data processing rate of the OFSG is increased by about 7% compared with existing sensor gateways. The learning time of the CSM is shorter than that of the long short-term memory models (LSTM) by 0.43 s, and the success rate of the CSM is higher than that of the LSTM by about 7%. Therefore, the TCP based on deep learning interconnects the communication protocols of various sensors, solves the maximum data size that sensor can transfer, predicts in advance crop disease occurrence in an external environment, and helps to make an optimized environment in which to grow crops

A Design of a Lightweight In-Vehicle Edge Gateway for the Self-Diagnosis of an Autonomous Vehicle

This paper proposes a Lightweight In-Vehicle Edge Gateway (LI-VEG) for the self-diagnosis of an autonomous vehicle, which supports a rapid and accurate communication between in-vehicle sensors and a self-diagnosis module and between in-vehicle protocols. A paper on the self-diagnosis module has been published previously, thus this paper only covers the LI-VEG, not the self-diagnosis. The LI-VEG consists of an In-Vehicle Sending and Receiving Layer (InV-SRL), an InV-Management Layer (InV-ML) and an InV-Data Translator Layer (InV-DTL). First, the InV-SRL receives the messages from FlexRay, Control Area Network (CAN), Media Oriented Systems Transport (MOST), and Ethernet and transfers the received messages to the InV-ML. Second, the InV-ML manages the message transmission and reception of FlexRay, CAN, MOST, and Ethernet and an Address Mapping Table. Third, the InV-DTL decomposes the message of FlexRay, CAN, MOST, and Ethernet and recomposes the decomposed messages to the frame suitable for a destination protocol. The performance analysis of the LI-VEG shows that the transmission delay time about message translation and transmission is reduced by an average of 10.83% and the transmission delay time caused by traffic overhead is improved by an average of 0.95%. Therefore, the LI-VEG has higher compatibility and is more cost effective because it applies a software gateway to the OBD, compared to a hardware gateway. In addition, it can reduce the transmission error and overhead caused by message decomposition because of a lightweight message header

Improvement of Growth and Morphology of Vegetable Seedlings with Supplemental Far-Red Enriched LED Lights in a Plant Factory

Although light-emitting diode (LED) lamps have been broadly applied in horticultural production to improve plant yield and quality, compared to natural light there is a disadvantage in the lack of far-red light in the LED spectrum. Far-red light has been studied widely to control plant growth and development. Therefore, this study aimed to find the effect of supplemental far-red-enriched LED lights to control the growth of tomato, red pepper, cucumber, gourd, watermelon and bottle gourd seedlings. The treatments were cool white LED:far-red LED at ratios of 5:0, 5:1, 5:2 and 5:3. The growth of tomato and red pepper seedlings, including hypocotyl length, was correlated to far-red light and light intensity. The phytochrome photostationary state (PSS) value of maximum hypocotyl length by supplemental far-red-enriched light ranged from 0.69 to 0.77 in tomato and red pepper seedlings. Although hypocotyl lengths of cucumber and watermelon were greatly affected by PSS, the PSS value for maximum hypocotyl length was lower than for tomato and red pepper. These results show that manipulating supplemental far-red enrichment can be used to control vegetable seedling growth with some variation among plant species.Y

An Integrated Self-Diagnosis System for an Autonomous Vehicle Based on an IoT Gateway and Deep Learning

This paper proposes “An Integrated Self-diagnosis System (ISS) for an Autonomous Vehicle based on an Internet of Things (IoT) Gateway and Deep Learning” that collects information from the sensors of an autonomous vehicle, diagnoses itself, and the influence between its parts by using Deep Learning and informs the driver of the result. The ISS consists of three modules. The first In-Vehicle Gateway Module (In-VGM) collects the data from the in-vehicle sensors, consisting of media data like a black box, driving radar, and the control messages of the vehicle, and transfers each of the data collected through each Controller Area Network (CAN), FlexRay, and Media Oriented Systems Transport (MOST) protocols to the on-board diagnostics (OBD) or the actuators. The data collected from the in-vehicle sensors is transferred to the CAN or FlexRay protocol and the media data collected while driving is transferred to the MOST protocol. Various types of messages transferred are transformed into a destination protocol message type. The second Optimized Deep Learning Module (ODLM) creates the Training Dataset on the basis of the data collected from the in-vehicle sensors and reasons the risk of the vehicle parts and consumables and the risk of the other parts influenced by a defective part. It diagnoses the vehicle’s total condition risk. The third Data Processing Module (DPM) is based on Edge Computing and has an Edge Computing based Self-diagnosis Service (ECSS) to improve the self-diagnosis speed and reduce the system overhead, while a V2X based Accident Notification Service (VANS) informs the adjacent vehicles and infrastructures of the self-diagnosis result analyzed by the OBD. This paper improves upon the simultaneous message transmission efficiency through the In-VGM by 15.25% and diminishes the learning error rate of a Neural Network algorithm through the ODLM by about 5.5%. Therefore, in addition, by transferring the self-diagnosis information and by managing the time to replace the car parts of an autonomous driving vehicle safely, this reduces loss of life and overall cost