FingerReader: A Wearable Device to Support Text Reading on the Go

Visually impaired people report numerous difficulties with accessing printed text using existing technology, including problems with alignment, focus, accuracy, mobility and efficiency. We present a finger worn device that assists the visually impaired with effectively and efficiently reading paper-printed text. We introduce a novel, local-sequential manner for scanning text which enables reading single lines, blocks of text or skimming the text for important sections while providing real-time auditory and tactile feedback. The design is motivated by preliminary studies with visually impaired people, and it is small-scale and mobile, which enables a more manageable operation with little setup

Towards a unified framework for identity documents analysis and recognition

Identity documents recognition is far beyond classical optical character recognition problems. Automated ID document recognition systems are tasked not only with the extraction of editable and transferable data but with performing identity validation and preventing fraud, with an increasingly high cost of error. A significant amount of research is directed to the creation of ID analysis systems with a specific focus for a subset of document types, or a particular mode of image acquisition, however, one of the challenges of the modern world is an increasing demand for identity document recognition from a wide variety of image sources, such as scans, photos, or video frames, as well as in a variety of virtually uncontrolled capturing conditions. In this paper, we describe the scope and context of identity document analysis and recognition problem and its challenges; analyze the existing works on implementing ID document recognition systems; and set a task to construct a unified framework for identity document recognition, which would be applicable for different types of image sources and capturing conditions, as well as scalable enough to support large number of identity document types. The aim of the presented framework is to serve as a basis for developing new methods and algorithms for ID document recognition, as well as for far more heavy challenges of identity document forensics, fully automated personal authentication and fraud prevention.This work was partially supported by the Russian Foundation for Basic Research (Project No. 18-29-03085 and 19-29-09055)

MIDV-2020: A Comprehensive Benchmark Dataset for Identity Document Analysis

Identity documents recognition is an important sub-field of document analysis, which deals with tasks of robust document detection, type identification, text fields recognition, as well as identity fraud prevention and document authenticity validation given photos, scans, or video frames of an identity document capture. Significant amount of research has been published on this topic in recent years, however a chief difficulty for such research is scarcity of datasets, due to the subject matter being protected by security requirements. A few datasets of identity documents which are available lack diversity of document types, capturing conditions, or variability of document field values. In addition, the published datasets were typically designed only for a subset of document recognition problems, not for a complex identity document analysis. In this paper, we present a dataset MIDV-2020 which consists of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock identity documents, each with unique text field values and unique artificially generated faces, with rich annotation. For the presented benchmark dataset baselines are provided for such tasks as document location and identification, text fields recognition, and face detection. With 72409 annotated images in total, to the date of publication the proposed dataset is the largest publicly available identity documents dataset with variable artificially generated data, and we believe that it will prove invaluable for advancement of the field of document analysis and recognition. The dataset is available for download at ftp://smartengines.com/midv-2020 and http://l3i-share.univ-lr.fr

Isothermal-based DNA biosensors for application in pharmacogenetics

Tesis por compendio[EN] The determination of genetic biomarkers is progressively becoming more extended and popular, being commercialized even in kits for personalized medicine. Establishing specific genotype variations for each patient, such as single nucleotide polymorphisms (SNPs), could be a fundamental tool in the field of diagnosis, prognosis and therapy selection. However, the use of DNA testing is not fully implemented in general healthcare, mainly due to technical and economic barriers associated to the current technologies, which are limited only to specialized centers and large hospitals. In this thesis, the main goal was to overcome these obstacles by developing simpler, faster and more affordable point-of-care (POC) genotyping systems. Allele discrimination was achieved by employing isothermal enzymatic reactions, like recombinase polymerase amplification (RPA), ligation of oligonucleotides and loop-mediated isothermal amplification (LAMP). These processes were integrated to colorimetric indicators and immunoenzymatic assays, in a microarray format. Using compact discs and polycarbonate chips as platforms, the detection was achieved through widespread electronics, like disc-reader, flatbed scanner and smartphone. To demonstrate their capacities, the resulting systems were applied for identifying SNPs in human samples, associated to therapies for tobacco smoking cessation, major depression disorder and blood clotting-related diseases. After selecting the proper conditions, all studied strategies discriminated SNPs in samples containing as low as 100 copies of genomic DNA, with an error rate below 15%. Most importantly, the developed methods have reduced assays times varying between 70 and 140 minutes, at a cost similar to a conventional PCR-based analog, but maintaining or raising amplification efficiency and eliminating the need of specialized temperature cyclers and fluorescence scanners. In conclusion, the biosensors based in isothermal reactions and consumer electronics devices greatly improve the competitivity of POC DNA analysis. It was demonstrated that the technologies developed in this thesis could support genotyping assays in low-resource areas, such as primary healthcare centers and emerging countries. Through this democratization of genetic testing and by performing adequate association studies, molecular diagnostics and personalized medicine practices could have their application extended to the clinical routine.[ES] La determinación de biomarcadores genéticos es cada vez más extensa y popular, estando incluso comercializándose kits para medicina personalizada. Establecer las variaciones específicas en el genotipo de cada paciente, como los polimorfismos de un solo nucleótido (SNP) podría ser una herramienta fundamental en el campo del diagnóstico, pronóstico y selección de la terapia. Sin embargo, el uso de pruebas de ADN no se encuentra completamente implementado en la atención médica general, principalmente debido a las barreras técnicas y económicas asociadas a las tecnologías actuales, limitadas solamente a centros especializados y grandes hospitales. En esta tesis, el objetivo principal fue superar estos obstáculos mediante el desarrollo de sistemas de genotipado point-of-care (POC), más simples, rápidos y asequibles. La discriminación alélica se logró mediante el uso de reacciones enzimáticas isotermas, como la amplificación de la recombinasa polimerasa (RPA), la ligación de oligonucleótidos y la amplificación isotérmica mediada por bucle (LAMP). Estos procesos se integraron a indicadores colorimétricos y ensayos inmunoenzimáticos en formato de micromatriz. Utilizando discos compactos y chips de policarbonato como plataforma de ensayo, se ha logrado la detección mediante dispositivos electrónicos de consumo, como un lector de discos, escáner documental y teléfono móvil. Para demostrar sus capacidades, los sistemas resultantes se aplicaron a la identificación de SNPs en muestras humanas, asociados a terapias antitabaquismo, para depresión y enfermedades relacionadas con la coagulación de la sangre. Tras seleccionar las condiciones adecuadas, todas las estrategias estudiadas discriminaron SNPs en muestras conteniendo tan solo 100 copias de ADN genómico, con una tasa de error inferior al 15%. Más importante, los métodos desarrollados han reducido los tiempos de ensayo a valores entre 70 y 140 minutos, a un coste similar a un análogo convencional basado en la reacción en cadena de la polimerasa (PCR), pero manteniendo o aumentando la eficiencia de amplificación y eliminando la necesidad de termocicladores y escáneres de fluorescencia. En conclusión, los biosensores basados en reacciones isotérmicas y dispositivos de electrónica de consumo mejoran en gran medida la competitividad del análisis POC de ADN. Se ha demostrado que las tecnologías desarrolladas en esta tesis podrían apoyar los ensayos de genotipado en áreas de recursos escasos, como centros de atención primaria y países emergentes. A través de esta democratización de las pruebas genéticas y realización estudios de asociación adecuados, el diagnóstico molecular y las prácticas en medicina personalizada podrían extender su aplicación a la rutina clínica.[CA] La determinació de biomarcadors genètics és cada vegada més extensa i popular, estant fins i tot comercialitzant-se kits per a medicina personalitzada. Establir les variacions específiques en el genotip de cada pacient, com els polimorfismes d'un sol nucleòtid (SNP) podria ser una eina fonamental en el camp del diagnòstic, pronòstic i selecció de la teràpia. No obstant això, l'ús de proves d'ADN no es troba completament implementat en l'atenció mèdica general, principalment a causa de les barreres tècniques i econòmiques associades a les tecnologies actuals, limitades solament a centres especialitzats i grans hospitals. En aquesta tesi, l'objectiu principal va ser superar aquests obstacles mitjançant el desenvolupament de sistemes de genotipat point-of-care (POC), més simples, ràpids i assequibles. La discriminació al·lèlica es va aconseguir mitjançant l'ús de reaccions enzimàtiques isotermes, com l'amplificació de la recombinasa polimerasa (RPA), la lligació de oligonucleòtids i l'amplificació isotèrmica mediada per bucle (LAMP). Aquests processos es van integrar a indicadors colorimètrics i assajos inmunoenzimàtics en format de micromatriu. Utilitzant discos compactes i xips de policarbonat com a plataforma d'assaig, s'ha conseguit la detecció mitjançant dispositius electrònics de consum, com un lector de discos, escàner documental i telèfon mòbil. Per a demostrar les seues capacitats, els sistemes resultants es van aplicar a la identificació de polimorfismes en mostres humanes, associats a teràpies antitabaquisme, per a depressió i malalties relacionades amb la coagulació de la sang. Després de seleccionar les condicions adequades, totes les estratègies estudiades van ser capaces de discriminar SNPs en mostres contenint tan sols 100 còpies d'ADN genòmic, amb una taxa d'error inferior al 15%. Més important, els mètodes desenvolupats han reduït els temps d'assaig a valors entre 70 i 140 minuts, a un cost similar a un anàleg convencional basat en la reacció en cadena de la polimerasa (PCR), però mantenint o augmentant l'eficiència d'amplificació i eliminant la necessitat de termocicladors i escàners de fluorescència. En conclusió, els biosensors basats en reaccions isotèrmiques i dispositius d'electrònica de consum milloren en gran manera la competitivitat de l'anàlisi POC del ADN. S'ha demostrat que les tecnologies desenvolupades en aquesta tesi podrien donar suport als assajos de genotipat en àrees de recursos escassos, com a centres d'atenció primària i països emergents. A través d'aquesta democratització de les proves genètiques i realització estudis d'associació adequats, el diagnòstic molecular i les pràctiques en medicina personalitzada podrien estendre la seua aplicació a la rutina clínica.[PT] A determinação de biomarcadores genéticos está tornando-se cada vez mais extensa e popular, sendo comercializada até em kits para medicina personalizada. O estabelecimento de variações específicas de genotipo para cada paciente, tais como os polimorfismo de nucleotídeo único, pode ser uma ferramenta fundamental no campo do diagnóstico, prognóstico e seleção de terapias. No entanto, o uso de testes de DNA ainda não encontra-se totalmente implementado na área de saúde geral, principalmente devido às barreiras técnicas e econômicas associadas às tecnologias atuais, limitadas apenas a centros especializados e grandes hospitais. Nesta tese, o principal objetivo foi superar esses obstáculos desenvolvendo sistemas de genotipagem point-of-care (POC) de DNA, mais simples, rápidos e acessíveis. A discriminação de alelos foi alcançada empregando reações enzimáticas isotérmicas, como amplificação por recombinase polimerase (RPA), ligação de oligonucleotídeos e amplificação isotérmica mediada por loop (LAMP). Tais processos foram integrados a indicadores colorimétricos e ensaios imunoenzimáticos, em formato micromatriz. Usando discos compactos e chips de policarbonato como plataforma de ensaio, os analitos foram detectados através de dispositivos eletrônicos de consumo, como leitor de disco, scanner de mesa e smartphone. Para demonstrar suas capacidades, os sistemas resultantes foram aplicados para identificação de polimorfismos em amostras de DNA humano, associados a terapias antitabagismo, para depressão e doenças relacionadas à coagulação do sangue. Após a seleção das condições adequadas, todas as estratégias estudadas foram capazes de discriminar SNPs em amostras contendo até 100 cópias de DNA genômico, com uma taxa de erro inferior a 15%. Mais importante, os métodos desenvolvidos reduziram o tempo de ensaio a valores entre 70 e 140 minutos, com um custo similar a um método análogo baseado em reação em cadeia da polimerase (PCR), mas mantendo ou aumentando a eficiência da amplificação e eliminando a necessidade de cicladores de temperatura e scanners de fluorescência especializados. Em conclusão, os biosensores baseados em reações enzimáticas isotérmicas e dispositivos eletrônicos de consumo incrementam grandemente a competitividade da análise POC de DNA. Foi demonstrado que as tecnologias desenvolvidas nesta tese poderiam dar suporte a ensaios de genotipagem em lugares com poucos recursos, como centros de atenção primária e países emergentes. Através desta democratização dos testes genéticos e com a realização de estudos de associação adequados, o diagnóstico molecular e as práticas de medicina personalizada poderiam ter sua aplicação extendida à rotina clínica.The authors acknowledge the financial support received from the Generalitat Valenciana (GVA-PROMETEOII/2014/040 Project and GRISOLIA/2014/024 PhD grant) and the Spanish Ministry of Economy and Competitiveness (MINECO CTQ2013-45875-R project)Yamanaka, ES. (2020). Isothermal-based DNA biosensors for application in pharmacogenetics [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/148366TESISCompendi

DocScanner: Robust Document Image Rectification with Progressive Learning

Compared with flatbed scanners, portable smartphones are much more convenient for physical documents digitizing. However, such digitized documents are often distorted due to uncontrolled physical deformations, camera positions, and illumination variations. To this end, we present DocScanner, a novel framework for document image rectification. Different from existing methods, DocScanner addresses this issue by introducing a progressive learning mechanism. Specifically, DocScanner maintains a single estimate of the rectified image, which is progressively corrected with a recurrent architecture. The iterative refinements make DocScanner converge to a robust and superior performance, while the lightweight recurrent architecture ensures the running efficiency. In addition, before the above rectification process, observing the corrupted rectified boundaries existing in prior works, DocScanner exploits a document localization module to explicitly segment the foreground document from the cluttered background environments. To further improve the rectification quality, based on the geometric priori between the distorted and the rectified images, a geometric regularization is introduced during training to further improve the performance. Extensive experiments are conducted on the Doc3D dataset and the DocUNet Benchmark dataset, and the quantitative and qualitative evaluation results verify the effectiveness of DocScanner, which outperforms previous methods on OCR accuracy, image similarity, and our proposed distortion metric by a considerable margin. Furthermore, our DocScanner shows the highest efficiency in runtime latency and model size

A Book Reader Design for Persons with Visual Impairment and Blindness

The objective of this dissertation is to provide a new design approach to a fully automated book reader for individuals with visual impairment and blindness that is portable and cost effective. This approach relies on the geometry of the design setup and provides the mathematical foundation for integrating, in a unique way, a 3-D space surface map from a low-resolution time of flight (ToF) device with a high-resolution image as means to enhance the reading accuracy of warped images due to the page curvature of bound books and other magazines. The merits of this low cost, but effective automated book reader design include: (1) a seamless registration process of the two imaging modalities so that the low resolution (160 x 120 pixels) height map, acquired by an Argos3D-P100 camera, accurately covers the entire book spread as captured by the high resolution image (3072 x 2304 pixels) of a Canon G6 Camera; (2) a mathematical framework for overcoming the difficulties associated with the curvature of open bound books, a process referred to as the dewarping of the book spread images, and (3) image correction performance comparison between uniform and full height map to determine which map provides the highest Optical Character Recognition (OCR) reading accuracy possible. The design concept could also be applied to address the challenging process of book digitization. This method is dependent on the geometry of the book reader setup for acquiring a 3-D map that yields high reading accuracy once appropriately fused with the high-resolution image. The experiments were performed on a dataset consisting of 200 pages with their corresponding computed and co-registered height maps, which are made available to the research community (cate-book3dmaps.fiu.edu). Improvements to the characters reading accuracy, due to the correction steps, were quantified and measured by introducing the corrected images to an OCR engine and tabulating the number of miss-recognized characters. Furthermore, the resilience of the book reader was tested by introducing a rotational misalignment to the book spreads and comparing the OCR accuracy to those obtained with the standard alignment. The standard alignment yielded an average reading accuracy of 95.55% with the uniform height map (i.e., the height values of the central row of the 3-D map are replicated to approximate all other rows), and 96.11% with the full height maps (i.e., each row has its own height values as obtained from the 3D camera). When the rotational misalignments were taken into account, the results obtained produced average accuracies of 90.63% and 94.75% for the same respective height maps, proving added resilience of the full height map method to potential misalignments

Developing a Living Archive of Aboriginal Languages

National Foreign Language Resource CenterThe fluctuating fortunes of Northern Territory bilingual education programs in Australian languages and English have put at risk thousands of books developed for these programs in remote schools. In an effort to preserve such a rich cultural and linguistic heritage, the Living Archive of Aboriginal Languages project is establishing an open access, online repository comprising digital versions of these materials. Using web technologies to store and access the resources makes them accessible to the communities of origin, the wider academic community, and the general public. The process of creating, populating, and implementing such an archive has posed many interesting technical, cultural and linguistic challenges, some of which are explored in this pape

MIDV-2020: a comprehensive benchmark dataset for identity document analysis

Identity documents recognition is an important sub-field of document analysis, which deals with tasks of robust document detection, type identification, text fields recognition, as well as identity fraud prevention and document authenticity validation given photos, scans, or video frames of an identity document capture. Significant amount of research has been published on this topic in recent years, however a chief difficulty for such research is scarcity of datasets, due to the subject matter being protected by security requirements. A few datasets of identity documents which are available lack diversity of document types, capturing conditions, or variability of document field values. In this paper, we present a dataset MIDV-2020 which consists of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock identity documents, each with unique text field values and unique artificially generated faces, with rich annotation. The dataset contains 72409 annotated images in total, making it the largest publicly available identity document dataset to the date of publication. We describe the structure of the dataset, its content and annotations, and present baseline experimental results to serve as a basis for future research. For the task of document location and identification content-independent, feature-based, and semantic segmentation-based methods were evaluated. For the task of document text field recognition, the Tesseract system was evaluated on field and character levels with grouping by field alphabets and document types. For the task of face detection, the performance of Multi Task Cascaded Convolutional Neural Networks-based method was evaluated separately for different types of image input modes. The baseline evaluations show that the existing methods of identity document analysis have a lot of room for improvement given modern challenges. We believe that the proposed dataset will prove invaluable for advancement of the field of document analysis and recognition.This work is partially supported by Russian Foundation for Basic Research (projects 19-29-09066 and 19-29-09092). All source images for MIDV-2020 dataset were obtained from Wikimedia Commons. Author attributions for each source images are listed in the original MIDV-500 description table (ftp://smartengines.com/midv-500/documents.pdf). Face images by Generated Photos (https://generated.photos)

Algorithm for choosing the best frame in a video stream in the task of identity document recognition

During the process of document recognition in a video stream using a mobile device camera, the image quality of the document varies greatly from frame to frame. Sometimes recognition system is required not only to recognize all the specified attributes of the document, but also to select final document image of the best quality. This is necessary, for example, for archiving or providing various services; in some countries it can be required by law. In this case, recognition system needs to assess the quality of frames in the video stream and choose the "best" frame. In this paper we considered the solution to such a problem where the "best" frame means the presence of all specified attributes in a readable form in the document image. The method was set up on a private dataset, and then tested on documents from the open MIDV-2019 dataset. A practically applicable result was obtained for use in recognition systems.This work was partially supported by the Russian Foundation for Basic Research (projects ## 17-29-03161, 18-07-01387)

Cybersecurity & Ethics for Lawyers in Plain English

Meeting proceedings of a seminar by the same name, held April 26, 2022