AUTOMATIC LIP-READING OF HEARING IMPAIRED PEOPLE

Inability to use speech interfaces greatly limits the deaf and hearing impaired people in the possibility of human-machine interaction. To solve this problem and to increase the accuracy and reliability of the automatic Russian sign language recognition system it is proposed to use lip-reading in addition to hand gestures recognition. Deaf and hearing impaired people use sign language as the main way of communication in everyday life. Sign language is a structured form of hand gestures and lips movements involving visual motions and signs, which is used as a communication system. Since sign language includes not only hand gestures, but also lip movements that mimic vocalized pronunciation, it is of interest to investigate how accurately such a visual speech can be recognized by a lip-reading system, especially considering the fact that the visual speech of hearing impaired people is often characterized with hyper-articulation, which should potentially facilitate its recognition. For this purpose, thesaurus of Russian sign language (TheRusLan) collected in SPIIRAS in 2018–19 was used. The database consists of color optical FullHD video recordings of 13 native Russian sign language signers (11 females and 2 males) from “Pavlovsk boarding school for the hearing impaired”. Each of the signers demonstrated 164 phrases for 5 times. This work covers the initial stages of this research, including data collection, data labeling, region-of-interest detection and methods for informative features extraction. The results of this study can later be used to create assistive technologies for deaf or hearing impaired people

AUTOMATIC DETECTION AND RECOGNITION OF 3D MANUAL GESTURES FOR HUMAN-MACHINE INTERACTION

In this paper, we propose an approach to detect and recognize 3D one-handed gestures for human-machine interaction. The logical structure of the modules of the system for recording a gestural database is described. The logical structure of the database of 3D gestures is presented. Examples of frames showing gestures in the format of Full High Definition, in the map depth mode and in the infrared illustrated. Models of a deep convolutional network for detecting faces and hand shapes are described. The results of automatic detection of the area with the face and the shape of the hand are given. Identified the distinctive features of the gesture at a certain point in time. The process of recognizing 3D one-handed gestures is described. Due to its versatility, this method can be used in tasks of biometrics, computer vision, machine learning, automatic systems of face recognition, sign languages

Neural network-based method for visual recognition of driver’s voice commands using attention mechanism

Visual speech recognition or automated lip-reading systems actively apply to speech-to-text translation. Video data proves to be useful in multimodal speech recognition systems, particularly when using acoustic data is difficult or not available at all. The main purpose of this study is to improve driver command recognition by analyzing visual information to reduce touch interaction with various vehicle systems (multimedia and navigation systems, phone calls, etc.) while driving. We propose a method of automated lip-reading the driver’s speech while driving based on a deep neural network of 3DResNet18 architecture. Using neural network architecture with bi-directional LSTM model and attention mechanism allows achieving higher recognition accuracy with a slight decrease in performance. Two different variants of neural network architectures for visual speech recognition are proposed and investigated. When using the first neural network architecture, the result of voice recognition of the driver was 77.68 %, which was lower by 5.78 % than when using the second one the accuracy of which was 83.46 %. Performance of the system which is determined by a real-time indicator RTF in the case of the first neural network architecture is equal to 0.076, and the second — RTF is 0.183 which is more than two times higher. The proposed method was tested on the data of multimodal corpus RUSAVIC recorded in the car. Results of the study can be used in systems of audio-visual speech recognition which is recommended in high noise conditions, for example, when driving a vehicle. In addition, the analysis performed allows us to choose the optimal neural network model of visual speech recognition for subsequent incorporation into the assistive system based on a mobile device

Неврологические проявления реактивации вируса опоясывающего герпеса

Herpes zoster remains an actual problem for elderly patients and those with immunosuppression. Usually the clinic of Herpes zoster is considered as a combination of neuralgia and typical vesicular rash (sometimes with intoxication). However, the rash does not appear in all patients, and the spectrum of neurological manifestations of the virus reactivation is extremely wide. Most of them reduce the quality of life, and some may result in disability or even death. Due to polymorphic symptoms and difficulties in laboratory diagnosis, many cases of the disease may remain unrecognized, the patient will not receive the necessary etiotropic therapy, so the course of the disease will become more protracted, and the prognosis will worsen. This review is based on 62 domestic and foreign publications without limiting the depth of the search. Here we describe relatively rare forms of the disease with injury of the sensory and motor peripheral nerves, brain and cranial vessels. We offer You a detailed description of pain: herpetic and postherpetic neuralgia. Also in the article we describe examples of damage of motor neurons and give some basic information about damage to the central nervous system. A separate section is devoted to diagnostic methods. This section describes the advantages and disadvantages of diagnostic methods used in real clinical practice, indicates the materials for the study, and the markers of subclinical virus reactivation. The “treatment” section is mainly devoted to the peculiarities of etiotropic therapy for various forms of the disease (with an indication of the recommended drugs, their doses and duration of the course). At the end of the article a list of clinical indications for examination for reactivation of the herpes zoster virus is defined.Опоясывающий герпес остается актуальной проблемой для пожилых пациентов и лиц с иммуносупрессией различного генеза. Традиционно клинику этого заболевания рассматривают как сочетание невралгий и типичных везикулезных высыпаний (иногда с синдромом общей интоксикации). Однако высыпания появляются не у всех больных, а спектр неврологических проявлений реактивации вируса опоясывающего герпеса чрезвычайно широк, причем большинство из них снижают качество жизни, а некоторые могут закончиться инвалидизацией или даже летальным исходом. Из-за полиморфной клиники и трудностей лабораторной диагностики многие случаи болезни могут остаться нераспознанными, пациент не получит этиотропную терапию, вследствие чего течение болезни станет более затяжным, а прогноз ухудшится. Данный обзор выполнен по материалам 62 отечественных и зарубежных публикаций без ограничения глубины поиска. В нём описаны варианты течения болезни с поражением чувствительных и двигательных периферических нервов, центральной нервной системы и сосудов. Дана подробная характеристика болей: герпетических и постгерпетических невралгий. Также в статье приведены примеры поражения мотонейронов и даны основные сведения о поражении центральной нервной системы. Отдельный раздел посвящен актуальным методам диагностики различных форм опоясывающего герпеса. В этом разделе описаны достоинства и недостатки используемых в реальной клинической практике методов диагностики, указаны материалы для исследования, а также маркеры субклинической реактивации вируса. Раздел «Лечение», в основном, посвящен особенностям этиотропной терапии при разных формах болезни (с указанием рекомендуемых препаратов, их доз и длительности курса). Наконец, определен перечень клинических показаний для обследования на предмет реактивации вируса опоясывающего герпеса

Протективные технологии современных методов респираторной поддержки в неонатальной практике

The article  presents  an analysis of literature data  on modern  protective regimens  for invasive respiratory support  in premature newborns  with respiratory distress  syndrome. We have considered positive and negative  aspects of the used methods  of invasive ventilation of the lungs, which are currently widely used as a method  of respiratory therapy  in obstetric hospitals  at any level, even in the category  of children  with extremely and very low birth weight. Modern  protective mechanical ventilation provides for 2 main directions for reducing ventilator-induced lung damage: a decrease in tidal volume (Vt) and the principle  of tolerable  (permissive) hypercapnia. The use of the technique of permissive hypercapnia and regimens with a target volume can reduce the likelihood of ventilator-induced lung injury in newborns. Despite the limited indications for mechanical ventilation in modern neonatology and the widespread use of non-invasive ventilation, for patients who really need mechanical ventilation, the use of volume-targeted regimens offers the best chance of reducing  ventilation complications.Представлен анализ литературы по применению современных протективных режимов инвазивной респираторной поддержки недоношенных новорожденных с респираторным дистресс-синдромом. Рассмотрены положительные и отрицательные стороны используемых методов инвазивной вентиляции легких, которые в настоящее время нашли широкое применение в качестве метода респираторной терапии в акушерских стационарах любого уровня, в том числе у категории детей с экстремально и очень низкой массой тела при рождении. Современная протективная ИВЛ предусматривает два основных направления снижения вентилятор-индуцированного повреждения легких: уменьшение дыхательного  объема (Vt) и принцип  допустимой  (пермиссивной) гиперкапнии. Применение методики  пермиссивной гиперкапнии и режимов с целевым объемом позволяет  снизить вероятность вентилятор-индуцированного повреждения легких у новорожденных детей. Несмотря на ограничение показаний к ИВЛ в современной неонатологии и широкому применению неинвазивной вентиляции, для пациентов, действительно нуждающихся в ИВЛ, применение режимов с целевым объемом дает лучшие шансы на уменьшение осложнений вентиляции

Норманская теория и становление российской государственности

The authors analyze the question of the origin of Russian statehood in ancient times. The main goal of the research is to study economic and social state of Russia at the beginning of the VIth century. The authors conduct a detailed analysis of the Norman and anti-Norman theories. They conclude assessment of the degree of influence of each of the theories on the formation of Russia as a full-fledged state.В статье рассматривается вопрос зарождения Российской государственности. Изучается экономическое и социальное состояние Руси в начале VI в. Авторами проводится подробный анализ норманской и антинорманской теорий. Дается оценка степени влияния каждой из теорий на дальнейшую концептуализацию формирования Руси как полноценного государства