Doctor of Philosophy

dissertationThe optimization of novel stretchable fingernail sensors for detecting fingertip touch force direction is introduced. The fingernail sensor uses optical reflectance photoplethysmography to measure the change in blood perfusion in the fingernail bed when the finger pad touches a surface with various forces. This "fingernail sensing" technique involves mounting an array of LEDs (Light Emitting Diodes) and photodetectors on the fingernail surface to detect changes in the reflection intensity as a function of applied force. The intensity changes correspond to changes in blood volume underneath the fingernail and allow for fingertip force detection without haptic obstruction, which has several applications in the area of human-machine interaction. This dissertation experimentally determines the optimal optical parameters for the transmittance of light through the human fingernail bed. Specifically, the effect of varying the wavelength and optical path length on light transmittance through the nail bed are thoroughly investigated. Light transmittance through the human fingernail is optimized when using green light (525nm) and when placing optoelectronic pairs as close together as possible. The optimal locations of the optoelectronic devices are predicted by introducing an optical model that describes light transmittance between an LED and a photodiode in the fingernail area based on optical experimentation. A reduced configuration is derived from the optimal optoelectronic locations in order to facilitate iv the fabrication of the optimized fingernail sensor without significantly compromising the recognition accuracy. This results in an overall force direction recognition accuracy of 95%. Using novel fabrication techniques, we successfully build a stretchable fingernail sensor prototype, which fully conforms to the two-dimensional fingernail surface and is independent of its geometry. Namely, we overcome the challenges of patterning conductive lines on a stretchable substrate, and embedding rigid optical components in a stretchable platform while maintaining electrical conductivity. A finite element analysis is conducted to optimize the electrical contact resistance between the optoelectronic components and underlying stretchable conductors, as a function of the bending curvature and substrate thickness. The functionality of the stretchable sensor is tested in relation to the design parameters. Finally, applications and potential impacts of this work are discussed

Gesture to Voice Conversion for Speech and Hearing Impaired Disabilities

Glove-based systems stand for one of the most significant efforts meant at obtain hand movement data. It also analyzes the kind of the devices, offers a road map of the development of the technology, and converse precincts of current technology and drift at the frontiers of investigate.  The progress of the most admired devices for hand society achievement, glove-based systems, started about 30 years ago and keeps on to appoint a growing number of researchers. It is then not surprising that an extensive amount of research effort has been staunch to developing technologies for cram contact and management and for enhances our facility to act upon such tasks

Stretchable capacitive tactile skin on humanoid robot fingers - first experiments and results

A stretchable tactile sensor skin has been demonstrated on the dorsal side of a robotic hand for the first time. The sensors can detect normal pressures on the same scale as human skin but also in excess of 250 kPa and withstand strains in excess of 15%. Using tactile information from the sensors mounted on a glove worn by a humanoid robot's hand, obstacle detection and surface reconstruction tasks were successfully completed in order to demonstrate the performance of the sensors under applied strains and pressure

Doctor of Philosophy

dissertationFingernail imaging is a method of sensing finger force using the color patterns on the nail and surrounding skin. These patterns form as the underlying tissue is compressed and blood pools in the surrounding vessels. Photos of the finger and surrounding skin may be correlated to the magnitude and direction of force on the fingerpad. An automated calibration routine is developed to improve the data-collection process. This includes a novel hybrid force/position controller that manages the interaction between the fingerpad and a flat surface, implemented on a Magnetic Levitation Haptic Device. The kinematic and dynamics parameters of the system are characterized in order to appropriately design a nonlinear compensator. The controller settles within 0.13 s with less than 30% overshoot. A new registration A new registration technique, based on Active Appearance Models, is presented. Since this method accounts for the variation inherent in the finger, it reduces registration and force prediction errors while removing the need to tune registration parameters or reject unregistered images. Modifications to the standard model are also investigated. The number of landmark points is reduced to 25 points with no loss of accuracy, while the use of the green channel is found to have no significant effect on either registration or force prediction accuracy. Several force prediction models are characterized, and the EigenNail Magnitude Model, a Principal Component Regression model on the gray-level intensity, is shown to fit the data most accurately. The mean force prediction error using this prediction and modeling method is 0.55 N. White LEDs and green LEDs are shown to have no statistically significant effect on registration or force prediction. Finally, two different calibration grid designs are compared and found to have no significant effect. Together, these improvements prepare the way for fingernail imaging to be used in less controlled situations. With a wider range of calibration data and a more robust registration method, a larger range of force data may be predicted. Potential applications for this technology include human-computer interaction and measuring finger interaction forces during grasping experiments

소아에서 호흡 주기에 따른 pulse oximeter plethysmographic amplitude의 변화에 대한 연구 : 접촉 압력에 따른 차이

학위논문 (박사)-- 서울대학교 대학원 : 의과대학 의학과, 2019. 2. 김희수 .Backgrounds: Predicting fluid responsiveness is crucial for adequate fluid management. Respiratory variations in photoplethysmographic waveform amplitude (ΔPPG) enable volume status assessment. However, previous studies have been shown inconsistent results regarding the ability to predict fluid responsiveness of ΔPPG when used for children. The contact force between the measurement site and sensor is one of the factors affecting the amplitude of photoplethysmography (PPG). Changes in the contact force can affect the ΔPPG value, thereby hindering its indicator in children. We aimed to evaluate contact force effects on ΔPPG in children under general anesthesia. Methods: A two-stage study was conducted. In the first observational study, children aged 3–5 years and scheduled for simple elective surgery were enrolled. After anesthetic induction, mechanical ventilation commenced at a tidal volume of 10 mL/kg. PPG signals were obtained in the supine position from the index finger using a force-sensor–integrated clip-type PPG sensor that increased the contact force from 0–1.4 N for 20 respiratory cycles at each force. The AC amplitude (pulsatile component), DC amplitude (non-pulsatile component), AC/DC ratio, and ΔPPG were calculated. In the second study, the changes in the ability of ΔPPG as indicator for fluid responsiveness according to the contact force changes were evaluated. Children aged 1 month-5 years and scheduled for major surgery including neurosurgery and cardiac surgery were enrolled. After anesthetic induction, mechanical ventilation commenced with a tidal volume of 10 ml/kg. ΔPPG was calculated at five different contacting force level (0–0.3N, 0.3–0.6N, 0.6–0.9N, 0.9–1.2N, and 1.2–1.5N) and individually adjusted contacting force, before volume expansion. Subjects were considered as fluid responders if volume expansion increased the stroke volume index (SVI) by >15%. Results: In the first study, data from 34 children were analyzed. Seven contact forces at 0.2-N increments were evaluated for each patient. The normalized AC amplitude increased maximally at a contact force of 0.4–0.6 N and decreased with increasing contact force. However, the normalized DC amplitude increased with a contact force exceeding 0.4 N. ΔPPG decreased slightly and increased from the point when the AC amplitude started to decrease as contact force increased. In a 0.2–1.2 N contact force range, significant changes in the normalized AC amplitude, normalized DC amplitude, AC/DC ratio, and respiratory variations in photoplethysmography amplitude were observed (all P < 0.005). In second study, data from 38 children were analyzed. There was significant difference in ΔPPG between fluid responders and non-responders only at contacting force level of 0.9-1.2N (P = 0.002) and individually adjusted contacting force (P < 0.001). Additionally, ΔPPG at those contacting force level could predict fluid responsiveness in mechanically ventilated children. Conclusions: PPG amplitude and ΔPPG changed according to variable contact forces. When contacting force is controlled to an adequate degree, the ability of ΔPPG to predict fluid responsiveness can be improved.배경: 적절한 수액 요법을 위해 수액 반응성을 예측하는 것은 매우 중요하다. 호흡에 따른 photoplethysmography (PPG) 파형 높이의 변화(ΔPPG)는 체내 혈류 상태를 반영한다. 그러나 소아에서 수액 반응성 예측을 위한 ΔPPG의 유용성에 대한 과거 연구들은 서로 상충되는 결과를 보여주고 있다. PPG 측정 센서와 측정 부위의 접촉 압력은 PPG 파형 높이와 ΔPPG 값에 영향을 미칠 수 있다. 따라서 본 연구에서는 접촉 압력이 PPG 파형과 ΔPPG의 신뢰성에 미치는 영향을 알아보고자 한다. 방법: 2단계 연구가 수행되었다. 첫 번째 관찰연구에는 3-5세의 간단한 수술을 받는 소아 환자들을 대상으로 연구가 진행되었다. 마취 유도 후 일회 호흡량 10 mL/kg로 기계 환기를 시작하였다. 앙와위 자세에서, 클립식 PPG 센서를 검지 손가락에 부착하고 접촉 압력을 0에서 1.4N까지 0.2N씩 증가시키며 각 압력에서 20번의 호흡 주기 동안 PPG 파형을 얻었다. AC, DC 진폭, AC/DC 비 및 ΔPPG를 계산하였다. 두 번째 연구에서는 접촉 압력 변화에 따른 수액 반응성의 지표로서 ΔPPG의 예측력의 신뢰성을 평가하였다. 신경외과 및 흉부외과 수술 등 대 수술을 받는 1개월-5세 사이의 소아를 대상으로, 다섯 단계의 접촉 압력 (0-0.3N, 0.3-0.6N, 0.6-0.9N, 0.9-1.2N 및 1.2-1.5N) 및 개별 조정된 접촉 압력에서 수액 투여 전 ΔPPG를 측정하였다. 수액 투여 후 일회 박출 지수가 15% 이상 증가한 경우 반응군으로 정의하였다. 결과: 총 34명의 데이터가 첫 번째 연구에서 분석되었다. Normalized AC 진폭은 접촉 압력 0.4-0.6N에서 최대였으며 접촉 압력이 증가함에 따라 감소하였다. 반면 normalized DC 진폭은 접촉 압력 0.4N 이상에서 증가하였다. ΔPPG는 약간 감소하다가 AC 진폭이 감소하기 시작한 지점에서 꾸준히 증가하였다. 전체적으로 0.2-1.2N 사이에서 각 접촉 압력 단계의 AC, DC 진폭, AC/DC 비 및 ΔPPG는 유의하게 변화하였다. 두 번째 연구에서는 총 38명의 데이터가 분석되었다. 접촉 압력 0.9-1.2N과, PPG 파형을 최대화시키는 접촉 압력 (0.9±0.3N)에서 수액 반응군과 비반응군 사이의 ΔPPG에 유의한 차이가 있었다. 또한 이 두 접촉 압력 범위에서의 ΔPPG가 수액 반응성을 예측할 수 있었다. 결론: 기계 환기를 받는 소아에서 PPG 센서와 측정 부위의 접촉 압력에 따라 PPG 파형 진폭 및 ΔPPG가 유의하게 변화하였다. 접촉 압력이 적절하게 조절된다면, 소아에서 ΔPPG의 수액 반응성에 대한 예측력을 개선시킬 수 있을 것이다.Abstract i List of Tables v List of Figures vi 1. General Introduction 1 1.1 Photoplethysmography 1 1.2 PPG waveform analysis and factors affecting PPG waveform 1 1.3 Respiratory variations in photoplethysmographic waveform amplitude (ΔPPG) 2 1.4 Necessity and importance of this research 5 2. First study protocol and results 6 2.1 Methods 6 2.2 Results 12 3. Second study protocol and results 18 3.1 Methods 18 3.2 Results 24 4. Discussion 32 4.1 Discussion of the first study 32 4.2 Discussion of the second study 36 5. References 40 국문 초록 45Docto

Influence of contact conditions on thermal responses of the hand

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2009.Includes bibliographical references (leaves 82-87).The objective of the research conducted for this thesis was to evaluate the influence of contact conditions on the thermal responses of the finger pad and their perceptual effects. A series of experiments investigated the thermal and perceptual effects of different contact conditions including contact force, contact duration, the object's surface temperature, and its surface roughness. The thermal response of the finger pad was measured using an infrared camera as the contact force varied from 0.1 to 6 N. It was determined that the decrease in skin temperature was highly dependent on the magnitude of contact force as well as contact duration. A second set of experiments investigated the effect of surface texture on the thermal response of the finger pad, and demonstrated, contrary to predictions, that a greater change in skin temperature occurs when the finger is in contact with rougher surfaces. The effect of varying surface texture on the perception of temperature was also investigated. The changes in temperature due to varying surface texture are perceptible, and demonstrate that the perception of surface roughness is not only influenced by changes in temperature, but in turn affects the perception of temperature. The final set of experiments examined the effect of varying the surface temperature of the thermal display on the perceived magnitude of finger force. Over the range of 20 to 38 'C, the surface temperature of the display did not have a significant effect on the perceived magnitude of force. The results of these experiments can be incorporated into thermal models that are used to create more realistic displays for virtual environments and teleoperated systems.by Jessica Anne Galie.S.M

Analysis and validation of an artifact resistant design for oxygen saturation measurement using photo pletyhsmographic ring sensors

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering; and, (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2004.Includes bibliographical references (leaves 99-103).Recent advances in continuous noninvasive health monitoring technologies provide clinicians and researchers with a previously unrealistic opportunity for closely tracking the developments and treatments of various pathologies both within and outside of a clinical setting. At the same time, miniaturized, wireless communication technologies have greatly enhanced the transmission of sensor data while reducing the size requirements for traditional, wearable sensors. The synergism of these innovations has led to the development of the Ring Sensor, a miniaturized, telemetric, photo plethysmograph sensor for continuous health monitoring. Previous work on the Ring Sensor has led to significant power savings in regards to data acquisition and transmission. Additionally, early long-term monitoring tests have indicated that the Ring Sensor is capable of acquiring a reliable waveform nearly 30% of the time. However, the utility of the Ring Sensor has remained somewhat limited. This thesis addresses several of the remaining issues associated with the Ring Sensor. The main design consideration associated with the Ring Sensor is achieving minimal power consumption while maintaining high signal quality. To this end, significant effort has been channeled to the development of an appropriate motion artifact model, representing the complex interplay between internal hemodynamics and external influences. Additionally, an artifact resistant, power-efficient, high-speed modulation scheme has been incorporated into the design of the Ring Sensor. It has been shown that this design significantly reduces the amount of data corrupted by motion while also minimizing the power consumed by the LEDs (one of the single largest power consuming elements).(cont.) This thesis also details the refinement of both the analog signal processing circuit and the redesigning of the sensor band for a more secure device interface. In particular, the order and type of filtering utilized by the Ring Sensor have been optimized for signal quality and stability. An improved sensor unit assembly provides a secure, pressurized contact with the patient's skin while protecting the optical components and wires from the external environment, while additional sensors, incorporated into both the sensor band and the ring unit, provide temperature and light feedback for signal quality assurance. In addition to these advancements, preliminary work towards sensor calibration for oxygen saturation measurements is provided. The thesis concludes with promising results obtained from field testing work conducted in the Massachusetts General Hospital's Pulmonary Function Testing Lab.by Phillip Andrew Shaltis.S.M

Planning Framework for Robotic Pizza Dough Stretching with a Rolling Pin

Stretching a pizza dough with a rolling pin is a nonprehensile manipulation. Since the object is deformable, force closure cannot be established, and the manipulation is carried out in a nonprehensile way. The framework of this pizza dough stretching application that is explained in this chapter consists of four sub-procedures: (i) recognition of the pizza dough on a plate, (ii) planning the necessary steps to shape the pizza dough to the desired form, (iii) path generation for a rolling pin to execute the output of the pizza dough planner, and (iv) inverse kinematics for the bi-manual robot to grasp and control the rolling pin properly. Using the deformable object model described in Chap. 3, each sub-procedure of the proposed framework is explained sequentially

GRAB: A Dataset of Whole-Body Human Grasping of Objects

Training computers to understand, model, and synthesize human grasping requires a rich dataset containing complex 3D object shapes, detailed contact information, hand pose and shape, and the 3D body motion over time. While "grasping" is commonly thought of as a single hand stably lifting an object, we capture the motion of the entire body and adopt the generalized notion of "whole-body grasps". Thus, we collect a new dataset, called GRAB (GRasping Actions with Bodies), of whole-body grasps, containing full 3D shape and pose sequences of 10 subjects interacting with 51 everyday objects of varying shape and size. Given MoCap markers, we fit the full 3D body shape and pose, including the articulated face and hands, as well as the 3D object pose. This gives detailed 3D meshes over time, from which we compute contact between the body and object. This is a unique dataset, that goes well beyond existing ones for modeling and understanding how humans grasp and manipulate objects, how their full body is involved, and how interaction varies with the task. We illustrate the practical value of GRAB with an example application; we train GrabNet, a conditional generative network, to predict 3D hand grasps for unseen 3D object shapes. The dataset and code are available for research purposes at https://grab.is.tue.mpg.de.Comment: ECCV 202