cellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM

Expensive scientific camera hardware is amongst the main cost factors in modern, high-performance microscopes. Recent technological advantages have, however, yielded consumer-grade camera devices that can provide surprisingly good performance. The camera sensors of smartphones in particular have benefited of this development. Combined with computing power and due to their ubiquity, smartphones provide a fantastic opportunity for "imaging on a budget". Here we show that a consumer cellphone is capable even of optical super-resolution imaging by (direct) Stochastic Optical Reconstruction Microscopy (dSTORM), achieving optical resolution better than 80 nm. In addition to the use of standard reconstruction algorithms, we investigated an approach by a trained image-to-image generative adversarial network (GAN). This not only serves as a versatile technique to reconstruct video sequences under conditions where traditional algorithms provide sub-optimal localization performance, but also allows processing directly on the smartphone. We believe that "cellSTORM" paves the way for affordable super-resolution microscopy suitable for research and education, expanding access to cutting edge research to a large community

The design and development of multi-agent based RFID middleware system for data and devices management

Thesis (D. Tech. (Electrical Engineering)) - Central University of technology, Free State, 2012Radio frequency identification technology (RFID) has emerged as a key technology for automatic identification and promises to revolutionize business processes. While RFID technology adoption is improving rapidly, reliable and widespread deployment of this technology still faces many significant challenges. The key deployment challenges include how to use the simple, unreliable raw data generated by RFID deployments to make business decisions; and how to manage a large number of deployed RFID devices. In this thesis, a multi-agent based RFID middleware which addresses some of the RFID data and device management challenges was developed. The middleware developed abstracts the auto-identification applications from physical RFID device specific details and provides necessary services such as device management, data cleaning, event generation, query capabilities and event persistence. The use of software agent technology offers a more scalable and distributed system architecture for the proposed middleware. As part of a multi-agent system, application-independent domain ontology for RFID devices was developed. This ontology can be used or extended in any application interested with RFID domain ontology. In order to address the event processing tasks within the proposed middleware system, a temporal-based RFID data model which considers both applications’ temporal and spatial granules in the data model itself for efficient event processing was developed. The developed data model extends the conventional Entity-Relationship constructs by adding a time attribute to the model. By maintaining the history of events and state changes, the data model captures the fundamental RFID application logic within the data model. Hence, this new data model supports efficient generation of application level events, updating, querying and analysis of both recent and historical events. As part of the RFID middleware, an adaptive sliding-window based data cleaning scheme for reducing missed readings from RFID data streams (called WSTD) was also developed. The WSTD scheme models the unreliability of the RFID readings by viewing RFID streams as a statistical sample of tags in the physical world, and exploits techniques grounded in sampling theory to drive its cleaning processes. The WSTD scheme is capable of efficiently coping with both environmental variations and tag dynamics by automatically and continuously adapting its cleaning window size, based on observed readings

DESIGN FRAMEWORK FOR INTERNET OF THINGS BASED NEXT GENERATION VIDEO SURVEILLANCE

Modern artificial intelligence and machine learning opens up new era towards video surveillance system. Next generation video surveillance in Internet of Things (IoT) environment is an emerging research area because of high bandwidth, big-data generation, resource constraint video surveillance node, high energy consumption for real time applications. In this thesis, various opportunities and functional requirements that next generation video surveillance system should achieve with the power of video analytics, artificial intelligence and machine learning are discussed. This thesis also proposes a new video surveillance system architecture introducing fog computing towards IoT based system and contributes the facilities and benefits of proposed system which can meet the forthcoming requirements of surveillance. Different challenges and issues faced for video surveillance in IoT environment and evaluate fog-cloud integrated architecture to penetrate and eliminate those issues. The focus of this thesis is to evaluate the IoT based video surveillance system. To this end, two case studies were performed to penetrate values towards energy and bandwidth efficient video surveillance system. In one case study, an IoT-based power efficient color frame transmission and generation algorithm for video surveillance application is presented. The conventional way is to transmit all R, G and B components of all frames. Using proposed technique, instead of sending all components, first one color frame is sent followed by a series of gray-scale frames. After a certain number of gray-scale frames, another color frame is sent followed by the same number of gray-scale frames. This process is repeated for video surveillance system. In the decoder, color information is formulated from the color frame and then used to colorize the gray-scale frames. In another case study, a bandwidth efficient and low complexity frame reproduction technique that is also applicable in IoT based video surveillance application is presented. Using the second technique, only the pixel intensity that differs heavily comparing to previous frame’s corresponding pixel is sent. If the pixel intensity is similar or near similar comparing to the previous frame, the information is not transferred. With this objective, the bit stream is created for every frame with a predefined protocol. In cloud side, the frame information can be reproduced by implementing the reverse protocol from the bit stream. Experimental results of the two case studies show that the IoT-based proposed approach gives better results than traditional techniques in terms of both energy efficiency and quality of the video, and therefore, can enable sensor nodes in IoT to perform more operations with energy constraints

Radio Communications

In the last decades the restless evolution of information and communication technologies (ICT) brought to a deep transformation of our habits. The growth of the Internet and the advances in hardware and software implementations modified our way to communicate and to share information. In this book, an overview of the major issues faced today by researchers in the field of radio communications is given through 35 high quality chapters written by specialists working in universities and research centers all over the world. Various aspects will be deeply discussed: channel modeling, beamforming, multiple antennas, cooperative networks, opportunistic scheduling, advanced admission control, handover management, systems performance assessment, routing issues in mobility conditions, localization, web security. Advanced techniques for the radio resource management will be discussed both in single and multiple radio technologies; either in infrastructure, mesh or ad hoc networks

Human factors in instructional augmented reality for intravehicular spaceflight activities and How gravity influences the setup of interfaces operated by direct object selection

In human spaceflight, advanced user interfaces are becoming an interesting mean to facilitate human-machine interaction, enhancing and guaranteeing the sequences of intravehicular space operations. The efforts made to ease such operations have shown strong interests in novel human-computer interaction like Augmented Reality (AR). The work presented in this thesis is directed towards a user-driven design for AR-assisted space operations, iteratively solving issues arisen from the problem space, which also includes the consideration of the effect of altered gravity on handling such interfaces.Auch in der bemannten Raumfahrt steigt das Interesse an neuartigen Benutzerschnittstellen, um nicht nur die Mensch-Maschine-Interaktion effektiver zu gestalten, sondern auch um einen korrekten Arbeitsablauf sicherzustellen. In der Vergangenheit wurden wiederholt Anstrengungen unternommen, Innenbordarbeiten mit Hilfe von Augmented Reality (AR) zu erleichtern. Diese Arbeit konzentriert sich auf einen nutzerorientierten AR-Ansatz, welcher zum Ziel hat, die Probleme schrittweise in einem iterativen Designprozess zu lösen. Dies erfordert auch die Berücksichtigung veränderter Schwerkraftbedingungen

Enabling technologies and cyber-physical systems for mission-critical scenarios

Programa Oficial de Doutoramento en Tecnoloxías da Información e Comunicacións en Redes Móbiles . 5029P01[Abstract] Reliable transport systems, defense, public safety and quality assurance in the Industry 4.0 are essential in a modern society. In a mission-critical scenario, a mission failure would jeopardize human lives and put at risk some other assets whose impairment or loss would significantly harm society or business results. Even small degradations of the communications supporting the mission could have large and possibly dire consequences. On the one hand, mission-critical organizations wish to utilize the most modern, disruptive and innovative communication systems and technologies, and yet, on the other hand, need to comply with strict requirements, which are very different to those of non critical scenarios. The aim of this thesis is to assess the feasibility of applying emerging technologies like Internet of Things (IoT), Cyber-Physical Systems (CPS) and 4G broadband communications in mission-critical scenarios along three key critical infrastructure sectors: transportation, defense and public safety, and shipbuilding. Regarding the transport sector, this thesis provides an understanding of the progress of communications technologies used for railways since the implantation of Global System for Mobile communications-Railways (GSM-R). The aim of this work is to envision the potential contribution of Long Term Evolution (LTE) to provide additional features that GSM-R would never support. Furthermore, the ability of Industrial IoT for revolutionizing the railway industry and confront today's challenges is presented. Moreover, a detailed review of the most common flaws found in Radio Frequency IDentification (RFID) based IoT systems is presented, including the latest attacks described in the literature. As a result, a novel methodology for auditing security and reverse engineering RFID communications in transport applications is introduced. The second sector selected is driven by new operational needs and the challenges that arise from modern military deployments. The strategic advantages of 4G broadband technologies massively deployed in civil scenarios are examined. Furthermore, this thesis analyzes the great potential for applying IoT technologies to revolutionize modern warfare and provide benefits similar to those in industry. It identifies scenarios where defense and public safety could leverage better commercial IoT capabilities to deliver greater survivability to the warfighter or first responders, while reducing costs and increasing operation efficiency and effectiveness. The last part is devoted to the shipbuilding industry. After defining the novel concept of Shipyard 4.0, how a shipyard pipe workshop works and what are the requirements for building a smart pipe system are described in detail. Furthermore, the foundations for enabling an affordable CPS for Shipyards 4.0 are presented. The CPS proposed consists of a network of beacons that continuously collect information about the location of the pipes. Its design allows shipyards to obtain more information on the pipes and to make better use of it. Moreover, it is indicated how to build a positioning system from scratch in an environment as harsh in terms of communications as a shipyard, showing an example of its architecture and implementation.[Resumen] En la sociedad moderna, los sistemas de transporte fiables, la defensa, la seguridad pública y el control de la calidad en la Industria 4.0 son esenciales. En un escenario de misión crítica, el fracaso de una misión pone en peligro vidas humanas y en riesgo otros activos cuyo deterioro o pérdida perjudicaría significativamente a la sociedad o a los resultados de una empresa. Incluso pequeñas degradaciones en las comunicaciones que apoyan la misión podrían tener importantes y posiblemente terribles consecuencias. Por un lado, las organizaciones de misión crítica desean utilizar los sistemas y tecnologías de comunicación más modernos, disruptivos e innovadores y, sin embargo, deben cumplir requisitos estrictos que son muy diferentes a los relativos a escenarios no críticos. El objetivo principal de esta tesis es evaluar la viabilidad de aplicar tecnologías emergentes como Internet of Things (IoT), Cyber-Physical Systems (CPS) y comunicaciones de banda ancha 4G en escenarios de misión crítica en tres sectores clave de infraestructura crítica: transporte, defensa y seguridad pública, y construcción naval. Respecto al sector del transporte, esta tesis permite comprender el progreso de las tecnologías de comunicación en el ámbito ferroviario desde la implantación de Global System for Mobile communications-Railway (GSM-R). El objetivo de este trabajo es analizar la contribución potencial de Long Term Evolution (LTE) para proporcionar características adicionales que GSM-R nunca podría soportar. Además, se presenta la capacidad de la IoT industrial para revolucionar la industria ferroviaria y afrontar los retos actuales. Asimismo, se estudian con detalle las vulnerabilidades más comunes de los sistemas IoT basados en Radio Frequency IDentification (RFID), incluyendo los últimos ataques descritos en la literatura. Como resultado, se presenta una metodología innovadora para realizar auditorías de seguridad e ingeniería inversa de las comunicaciones RFID en aplicaciones de transporte. El segundo sector elegido viene impulsado por las nuevas necesidades operacionales y los desafíos que surgen de los despliegues militares modernos. Para afrontarlos, se analizan las ventajas estratégicas de las tecnologías de banda ancha 4G masivamente desplegadas en escenarios civiles. Asimismo, esta tesis analiza el gran potencial de aplicación de las tecnologías IoT para revolucionar la guerra moderna y proporcionar beneficios similares a los alcanzados por la industria. Se identifican escenarios en los que la defensa y la seguridad pública podrían aprovechar mejor las capacidades comerciales de IoT para ofrecer una mayor capacidad de supervivencia al combatiente o a los servicios de emergencias, a la vez que reduce los costes y aumenta la eficiencia y efectividad de las operaciones. La última parte se dedica a la industria de construcción naval. Después de definir el novedoso concepto de Astillero 4.0, se describe en detalle cómo funciona el taller de tubería de astillero y cuáles son los requisitos para construir un sistema de tuberías inteligentes. Además, se presentan los fundamentos para posibilitar un CPS asequible para Astilleros 4.0. El CPS propuesto consiste en una red de balizas que continuamente recogen información sobre la ubicación de las tuberías. Su diseño permite a los astilleros obtener más información sobre las tuberías y hacer un mejor uso de las mismas. Asimismo, se indica cómo construir un sistema de posicionamiento desde cero en un entorno tan hostil en términos de comunicaciones, mostrando un ejemplo de su arquitectura e implementación

Cruiser and PhoTable: Exploring Tabletop User Interface Software for Digital Photograph Sharing and Story Capture

Digital photography has not only changed the nature of photography and the photographic process, but also the manner in which we share photographs and tell stories about them. Some traditional methods, such as the family photo album or passing around piles of recently developed snapshots, are lost to us without requiring the digital photos to be printed. The current, purely digital, methods of sharing do not provide the same experience as printed photographs, and they do not provide effective face-to-face social interaction around photographs, as experienced during storytelling. Research has found that people are often dissatisfied with sharing photographs in digital form. The recent emergence of the tabletop interface as a viable multi-user direct-touch interactive large horizontal display has provided the hardware that has the potential to improve our collocated activities such as digital photograph sharing. However, while some software to communicate with various tabletop hardware technologies exists, software aspects of tabletop user interfaces are still at an early stage and require careful consideration in order to provide an effective, multi-user immersive interface that arbitrates the social interaction between users, without the necessary computer-human interaction interfering with the social dialogue. This thesis presents PhoTable, a social interface allowing people to effectively share, and tell stories about, recently taken, unsorted digital photographs around an interactive tabletop. In addition, the computer-arbitrated digital interaction allows PhoTable to capture the stories told, and associate them as audio metadata to the appropriate photographs. By leveraging the tabletop interface and providing a highly usable and natural interaction we can enable users to become immersed in their social interaction, telling stories about their photographs, and allow the computer interaction to occur as a side-effect of the social interaction. Correlating the computer interaction with the corresponding audio allows PhoTable to annotate an automatically created digital photo album with audible stories, which may then be archived. These stories remain useful for future sharing -- both collocated sharing and remote (e.g. via the Internet) -- and also provide a personal memento both of the event depicted in the photograph (e.g. as a reminder) and of the enjoyable photo sharing experience at the tabletop. To provide the necessary software to realise an interface such as PhoTable, this thesis explored the development of Cruiser: an efficient, extensible and reusable software framework for developing tabletop applications. Cruiser contributes a set of programming libraries and the necessary application framework to facilitate the rapid and highly flexible development of new tabletop applications. It uses a plugin architecture that encourages code reuse, stability and easy experimentation, and leverages the dedicated computer graphics hardware and multi-core processors of modern consumer-level systems to provide a responsive and immersive interactive tabletop user interface that is agnostic to the tabletop hardware and operating platform, using efficient, native cross-platform code. Cruiser's flexibility has allowed a variety of novel interactive tabletop applications to be explored by other researchers using the framework, in addition to PhoTable. To evaluate Cruiser and PhoTable, this thesis follows recommended practices for systems evaluation. The design rationale is framed within the above scenario and vision which we explore further, and the resulting design is critically analysed based on user studies, heuristic evaluation and a reflection on how it evolved over time. The effectiveness of Cruiser was evaluated in terms of its ability to realise PhoTable, use of it by others to explore many new tabletop applications, and an analysis of performance and resource usage. Usability, learnability and effectiveness of PhoTable was assessed on three levels: careful usability evaluations of elements of the interface; informal observations of usability when Cruiser was available to the public in several exhibitions and demonstrations; and a final evaluation of PhoTable in use for storytelling, where this had the side effect of creating a digital photo album, consisting of the photographs users interacted with on the table and associated audio annotations which PhoTable automatically extracted from the interaction. We conclude that our approach to design has resulted in an effective framework for creating new tabletop interfaces. The parallel goal of exploring the potential for tabletop interaction as a new way to share digital photographs was realised in PhoTable. It is able to support the envisaged goal of an effective interface for telling stories about one's photos. As a serendipitous side-effect, PhoTable was effective in the automatic capture of the stories about individual photographs for future reminiscence and sharing. This work provides foundations for future work in creating new ways to interact at a tabletop and to the ways to capture personal stories around digital photographs for sharing and long-term preservation