AUTOMATIC LIP-READING OF HEARING IMPAIRED PEOPLE

Inability to use speech interfaces greatly limits the deaf and hearing impaired people in the possibility of human-machine interaction. To solve this problem and to increase the accuracy and reliability of the automatic Russian sign language recognition system it is proposed to use lip-reading in addition to hand gestures recognition. Deaf and hearing impaired people use sign language as the main way of communication in everyday life. Sign language is a structured form of hand gestures and lips movements involving visual motions and signs, which is used as a communication system. Since sign language includes not only hand gestures, but also lip movements that mimic vocalized pronunciation, it is of interest to investigate how accurately such a visual speech can be recognized by a lip-reading system, especially considering the fact that the visual speech of hearing impaired people is often characterized with hyper-articulation, which should potentially facilitate its recognition. For this purpose, thesaurus of Russian sign language (TheRusLan) collected in SPIIRAS in 2018–19 was used. The database consists of color optical FullHD video recordings of 13 native Russian sign language signers (11 females and 2 males) from “Pavlovsk boarding school for the hearing impaired”. Each of the signers demonstrated 164 phrases for 5 times. This work covers the initial stages of this research, including data collection, data labeling, region-of-interest detection and methods for informative features extraction. The results of this study can later be used to create assistive technologies for deaf or hearing impaired people

Neural network-based method for visual recognition of driver’s voice commands using attention mechanism

Visual speech recognition or automated lip-reading systems actively apply to speech-to-text translation. Video data proves to be useful in multimodal speech recognition systems, particularly when using acoustic data is difficult or not available at all. The main purpose of this study is to improve driver command recognition by analyzing visual information to reduce touch interaction with various vehicle systems (multimedia and navigation systems, phone calls, etc.) while driving. We propose a method of automated lip-reading the driver’s speech while driving based on a deep neural network of 3DResNet18 architecture. Using neural network architecture with bi-directional LSTM model and attention mechanism allows achieving higher recognition accuracy with a slight decrease in performance. Two different variants of neural network architectures for visual speech recognition are proposed and investigated. When using the first neural network architecture, the result of voice recognition of the driver was 77.68 %, which was lower by 5.78 % than when using the second one the accuracy of which was 83.46 %. Performance of the system which is determined by a real-time indicator RTF in the case of the first neural network architecture is equal to 0.076, and the second — RTF is 0.183 which is more than two times higher. The proposed method was tested on the data of multimodal corpus RUSAVIC recorded in the car. Results of the study can be used in systems of audio-visual speech recognition which is recommended in high noise conditions, for example, when driving a vehicle. In addition, the analysis performed allows us to choose the optimal neural network model of visual speech recognition for subsequent incorporation into the assistive system based on a mobile device

Неврологические проявления реактивации вируса опоясывающего герпеса

Herpes zoster remains an actual problem for elderly patients and those with immunosuppression. Usually the clinic of Herpes zoster is considered as a combination of neuralgia and typical vesicular rash (sometimes with intoxication). However, the rash does not appear in all patients, and the spectrum of neurological manifestations of the virus reactivation is extremely wide. Most of them reduce the quality of life, and some may result in disability or even death. Due to polymorphic symptoms and difficulties in laboratory diagnosis, many cases of the disease may remain unrecognized, the patient will not receive the necessary etiotropic therapy, so the course of the disease will become more protracted, and the prognosis will worsen. This review is based on 62 domestic and foreign publications without limiting the depth of the search. Here we describe relatively rare forms of the disease with injury of the sensory and motor peripheral nerves, brain and cranial vessels. We offer You a detailed description of pain: herpetic and postherpetic neuralgia. Also in the article we describe examples of damage of motor neurons and give some basic information about damage to the central nervous system. A separate section is devoted to diagnostic methods. This section describes the advantages and disadvantages of diagnostic methods used in real clinical practice, indicates the materials for the study, and the markers of subclinical virus reactivation. The “treatment” section is mainly devoted to the peculiarities of etiotropic therapy for various forms of the disease (with an indication of the recommended drugs, their doses and duration of the course). At the end of the article a list of clinical indications for examination for reactivation of the herpes zoster virus is defined.Опоясывающий герпес остается актуальной проблемой для пожилых пациентов и лиц с иммуносупрессией различного генеза. Традиционно клинику этого заболевания рассматривают как сочетание невралгий и типичных везикулезных высыпаний (иногда с синдромом общей интоксикации). Однако высыпания появляются не у всех больных, а спектр неврологических проявлений реактивации вируса опоясывающего герпеса чрезвычайно широк, причем большинство из них снижают качество жизни, а некоторые могут закончиться инвалидизацией или даже летальным исходом. Из-за полиморфной клиники и трудностей лабораторной диагностики многие случаи болезни могут остаться нераспознанными, пациент не получит этиотропную терапию, вследствие чего течение болезни станет более затяжным, а прогноз ухудшится. Данный обзор выполнен по материалам 62 отечественных и зарубежных публикаций без ограничения глубины поиска. В нём описаны варианты течения болезни с поражением чувствительных и двигательных периферических нервов, центральной нервной системы и сосудов. Дана подробная характеристика болей: герпетических и постгерпетических невралгий. Также в статье приведены примеры поражения мотонейронов и даны основные сведения о поражении центральной нервной системы. Отдельный раздел посвящен актуальным методам диагностики различных форм опоясывающего герпеса. В этом разделе описаны достоинства и недостатки используемых в реальной клинической практике методов диагностики, указаны материалы для исследования, а также маркеры субклинической реактивации вируса. Раздел «Лечение», в основном, посвящен особенностям этиотропной терапии при разных формах болезни (с указанием рекомендуемых препаратов, их доз и длительности курса). Наконец, определен перечень клинических показаний для обследования на предмет реактивации вируса опоясывающего герпеса

Норманская теория и становление российской государственности

The authors analyze the question of the origin of Russian statehood in ancient times. The main goal of the research is to study economic and social state of Russia at the beginning of the VIth century. The authors conduct a detailed analysis of the Norman and anti-Norman theories. They conclude assessment of the degree of influence of each of the theories on the formation of Russia as a full-fledged state.В статье рассматривается вопрос зарождения Российской государственности. Изучается экономическое и социальное состояние Руси в начале VI в. Авторами проводится подробный анализ норманской и антинорманской теорий. Дается оценка степени влияния каждой из теорий на дальнейшую концептуализацию формирования Руси как полноценного государства