Neural network-based method for visual recognition of driver’s voice commands using attention mechanism

Visual speech recognition or automated lip-reading systems actively apply to speech-to-text translation. Video data proves to be useful in multimodal speech recognition systems, particularly when using acoustic data is difficult or not available at all. The main purpose of this study is to improve driver command recognition by analyzing visual information to reduce touch interaction with various vehicle systems (multimedia and navigation systems, phone calls, etc.) while driving. We propose a method of automated lip-reading the driver’s speech while driving based on a deep neural network of 3DResNet18 architecture. Using neural network architecture with bi-directional LSTM model and attention mechanism allows achieving higher recognition accuracy with a slight decrease in performance. Two different variants of neural network architectures for visual speech recognition are proposed and investigated. When using the first neural network architecture, the result of voice recognition of the driver was 77.68 %, which was lower by 5.78 % than when using the second one the accuracy of which was 83.46 %. Performance of the system which is determined by a real-time indicator RTF in the case of the first neural network architecture is equal to 0.076, and the second — RTF is 0.183 which is more than two times higher. The proposed method was tested on the data of multimodal corpus RUSAVIC recorded in the car. Results of the study can be used in systems of audio-visual speech recognition which is recommended in high noise conditions, for example, when driving a vehicle. In addition, the analysis performed allows us to choose the optimal neural network model of visual speech recognition for subsequent incorporation into the assistive system based on a mobile device

EBS in Children with De Novo Pathogenic Variants Disturbing <i>Krt14</i>

Epidermolysis bullosa simplex (EBS) is a dermatological condition marked by skin fragility and blister formation resulting from separation within the basal layer of the epidermis, which can be attributed to various genetic etiologies. This study presents three pathogenic de novo variants in young children, with clinical manifestations appearing as early as the neonatal period. The variants contribute to the EBS phenotype through two distinct mechanisms: direct keratin abnormalities due to pathogenic variants in the Krt14 gene, and indirect effects via pathogenic mutation in the KLHL24 gene, which interfere with the natural proteasome-mediated degradation pathway of KRT14. We report one severe case of EBS with mottled pigmentation arising from the Met119Thr pathogenic variant in KRT14, another case involving a pathogenic KLHL24 Met1Val variant, and a third case featuring the hot spot mutation Arg125His in KRT14, all manifesting within the first few weeks of life. This research underscores the complexity of genetic influences in EBS and highlights the importance of early genetic screening for accurate diagnosis and management