Modular Instrumentation for Controlling and Monitoring In-Vitro Cultivation Environment and Image-based Functionality Measurements of Human Stem Cells

Artificial animal cell culture was successfully developed by Ross Harrison in 1907. But it was not until the 1940’s and 1950’s that several developments occurred, which expedited the cell culturing in-vitro (C-Vitro) to be a consistent and reproducible technique to study isolated living-cells in a controlled environment. Currently, CVitro is one of the major tools in cellular and molecular biology both in the academia and industry. They are extensively utilised to study the cellular physiology/biochemistry, to screen drugs/therapeutic compounds, to understand the effects of drugs/toxic compounds and also to identify the pathways of carcinogenesis/mutagenesis. It is also used in large scale manufacturing of vaccines and therapeutic proteins. In any experimental setup, it is important that the C-Vitro model should represent the physiological phenomena of interest with reasonable accuracy so that all experimental results are statistically consistent and reproducible. In this direction, sensors and measurement systems play important roles in in-situ detection and/or control/manipulation of cells/tissues/environment. This thesis aimed to develop new technology for tailored cell culturing and integrated measurements. Firstly, design and assembly of a portable Invert-upright microscope interchangeable modular cell culturing platform (iuCMP) was envisioned. In contrast to conventional methods, micro-scaled systems mimic the cells' natural microenvironment more precisely, facilitating accurate and tractable models. The iuCMP integrates modular measurement schemes with a mini culture chamber using biocompatible cell-friendly materials, automated environment-control (temperature and gas concentrations), oxygen sensing and simultaneous functional measurements (electrophysiological and image-based). Time lapse microscopy is very useful in cell biology, but integration of advanced >i>in-vitro/device based biological systems (e.g. lab/organ/body-on-chips, or mini-bioreactors/microfluidic systems) into conventional microscopes can be challenging in several circumstances due to multiple reasons. But in iuCMP the main advantage is, the microscope can be switched either as an inverted or as an upright system and therefore can accommodate virtually any in-vitro device. It can capture images from regions that are otherwise inaccessible by conventional microscopes, for example, cells cultured on physical or biochemical sensor systems. The modular design also allows accommodating more sensor or measurement systems quite freely. We have demonstrated the system for video-based beating analysis of cardiomyocytes, cell orientation analysis on nanocellulose, and simultaneous long-term in-situ microscopy with oxygen and temperature sensing in hypoxia. In an example application, the system was utilised for long-term temperature stressing and simultaneous mechanobiological analysis of human induced pluripotent stem cell-derived cardiomyocytes (hiPSC-CMs). For this the iuCMP together with a temperature sensor plate (TSP) and a novel non-invasive beating analysis software (CMaN—cardiomyocyte function analysis tool, scripted as a subpart of this thesis), was applied for automated temperature response studies in hiPSC-CM cultures. In-situ temperature sensing is usually challenging with bulky external sensors, but TSPs solved this issue. In the temperature response study, we showed that the relationship between hiPSC-CM beating frequency and temperature is non-linear and measured the Q10 temperature coefficients. Moreover, we observed the hiPSC-CM contractile networking, including propagation of the action potential signal between dissociated clusters and their non-invasive measurements. It was the first case where these events were reported in hiPSC-CM clusters and their noninvasive measurements by image processing. The software CMaN comes with a user-friendly interface and, is equipped with features for batch processing, movement centre detection and cluster finding. It can extract six different signals of the contractile motion of cardiomyocytes (clusters or single cells) per processing. This ensures a minimum of one useful beating signal even in the cases of complex beating videos. On the processing end, compared to similar tools, CMaN is faster, more sensitive, and computationally less expensive and allows ROI based processing. In the case of healthy cells, the waveform of the signal from the CMaN resembles an ECG signal with positive and negative segments, allowing the computation of contraction and relaxation features separately. In addition to iuCMP, a Modular optical pH measurement system (MO-pH) for 24/7 non-contact cell culture measurements was also developed. The MO-pH incorporates modular sterilisable optical parts and is used in phenol-red medium cell cultures. The modular assembly of MO-pH cassettes is unique and reusable. Measurements are carried out in a closed flow system without wasting any culture medium and requires no special manual attention or recalibrations during culture. Furthermore, a new absorption correction model was put forward that minimised errors caused e.g. by biolayers in spectrometric pH measurement, which improved the pH measurement accuracy. MO-pH has been applied in long-term human adipose stem cells (hASC) expansion cultures in CO2 dependent and independent media. Additionally, the MO-pH was also utilised to comprehend the behaviour of pH, temperature and humidity in water jacked incubators as well as to record the pH response as a function of temperature in the presence and absence of CO2 in the context of stem cell cultures. The resulting plots clearly showed the interplay between measured parameters indicating a few stress sources present all through the culture. Additionally, it provided an overall picture of behaviour of critical control parameters in an incubator and pointed out the need for bioprocess systems with automatic process monitoring and smart control for maximum yield, optimal growth and maintenance of the cells. Besides, we also integrated MO-pH into flasks with reclosable lids (RL-F) and tested its applicability in stem cell cultures. A standalone system around an RL-F flask was built by combining the cell culture, medium perfusion and optical measurements. The developed RL-F system has been successfully tested in ASC-differentiation cultures. Finally, a few trial experiments for image-based pH estimation aimed for iuCMP have also been carried out. This includes tests with LCD illumination, optical projection tomography, and webcam systems. In reality, the pH is not distributed uniformly in tissues, and has shown a gradient of up to 1.0 pH unit within 1 cm distance. Therefore, producing reliable pH maps also in in-vitro can be important in understanding various common pathologies and location of lesions. A reliable and adequately developed long-term pH mapping method will be an important addition into the iuCMP

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Human robot interaction in a crowded environment

Human Robot Interaction (HRI) is the primary means of establishing natural and affective communication between humans and robots. HRI enables robots to act in a way similar to humans in order to assist in activities that are considered to be laborious, unsafe, or repetitive. Vision based human robot interaction is a major component of HRI, with which visual information is used to interpret how human interaction takes place. Common tasks of HRI include finding pre-trained static or dynamic gestures in an image, which involves localising different key parts of the human body such as the face and hands. This information is subsequently used to extract different gestures. After the initial detection process, the robot is required to comprehend the underlying meaning of these gestures [3]. Thus far, most gesture recognition systems can only detect gestures and identify a person in relatively static environments. This is not realistic for practical applications as difficulties may arise from people‟s movements and changing illumination conditions. Another issue to consider is that of identifying the commanding person in a crowded scene, which is important for interpreting the navigation commands. To this end, it is necessary to associate the gesture to the correct person and automatic reasoning is required to extract the most probable location of the person who has initiated the gesture. In this thesis, we have proposed a practical framework for addressing the above issues. It attempts to achieve a coarse level understanding about a given environment before engaging in active communication. This includes recognizing human robot interaction, where a person has the intention to communicate with the robot. In this regard, it is necessary to differentiate if people present are engaged with each other or their surrounding environment. The basic task is to detect and reason about the environmental context and different interactions so as to respond accordingly. For example, if individuals are engaged in conversation, the robot should realize it is best not to disturb or, if an individual is receptive to the robot‟s interaction, it may approach the person. Finally, if the user is moving in the environment, it can analyse further to understand if any help can be offered in assisting this user. The method proposed in this thesis combines multiple visual cues in a Bayesian framework to identify people in a scene and determine potential intentions. For improving system performance, contextual feedback is used, which allows the Bayesian network to evolve and adjust itself according to the surrounding environment. The results achieved demonstrate the effectiveness of the technique in dealing with human-robot interaction in a relatively crowded environment [7]

Intelligent image cropping and scaling

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University, 2011.Nowadays, there exist a huge number of end devices with different screen properties for watching television content, which is either broadcasted or transmitted over the internet. To allow best viewing conditions on each of these devices, different image formats have to be provided by the broadcaster. Producing content for every single format is, however, not applicable by the broadcaster as it is much too laborious and costly. The most obvious solution for providing multiple image formats is to produce one high resolution format and prepare formats of lower resolution from this. One possibility to do this is to simply scale video images to the resolution of the target image format. Two significant drawbacks are the loss of image details through ownscaling and possibly unused image areas due to letter- or pillarboxes. A preferable solution is to find the contextual most important region in the high-resolution format at first and crop this area with an aspect ratio of the target image format afterwards. On the other hand, defining the contextual most important region manually is very time consuming. Trying to apply that to live productions would be nearly impossible. Therefore, some approaches exist that automatically define cropping areas. To do so, they extract visual features, like moving reas in a video, and define regions of interest (ROIs) based on those. ROIs are finally used to define an enclosing cropping area. The extraction of features is done without any knowledge about the type of content. Hence, these approaches are not able to distinguish between features that might be important in a given context and those that are not. The work presented within this thesis tackles the problem of extracting visual features based on prior knowledge about the content. Such knowledge is fed into the system in form of metadata that is available from TV production environments. Based on the extracted features, ROIs are then defined and filtered dependent on the analysed content. As proof-of-concept, this application finally adapts SDTV (Standard Definition Television) sports productions automatically to image formats with lower resolution through intelligent cropping and scaling. If no content information is available, the system can still be applied on any type of content through a default mode. The presented approach is based on the principle of a plug-in system. Each plug-in represents a method for analysing video content information, either on a low level by extracting image features or on a higher level by processing extracted ROIs. The combination of plug-ins is determined by the incoming descriptive production metadata and hence can be adapted to each type of sport individually. The application has been comprehensively evaluated by comparing the results of the system against alternative cropping methods. This evaluation utilised videos which were manually cropped by a professional video editor, statically cropped videos and simply scaled, non-cropped videos. In addition to and apart from purely subjective evaluations, the gaze positions of subjects watching sports videos have been measured and compared to the regions of interest positions extracted by the system