Robustness of Object Recognition under Extreme Occlusion in Humans and Computational Models

Most objects in the visual world are partially occluded, but humans can recognize them without difficulty. However, it remains unknown whether object recognition models like convolutional neural networks (CNNs) can handle real-world occlusion. It is also a question whether efforts to make these models robust to constant mask occlusion are effective for real-world occlusion. We test both humans and the above-mentioned computational models in a challenging task of object recognition under extreme occlusion, where target objects are heavily occluded by irrelevant real objects in real backgrounds. Our results show that human vision is very robust to extreme occlusion while CNNs are not, even with modifications to handle constant mask occlusion. This implies that the ability to handle constant mask occlusion does not entail robustness to real-world occlusion. As a comparison, we propose another computational model that utilizes object parts/subparts in a compositional manner to build robustness to occlusion. This performs significantly better than CNN-based models on our task with error patterns similar to humans. These findings suggest that testing under extreme occlusion can better reveal the robustness of visual recognition, and that the principle of composition can encourage such robustness.Comment: To be presented at the 41st Annual Meeting of the Cognitive Science Societ

Facial Feature Tracking and Occlusion Recovery in American Sign Language

Facial features play an important role in expressing grammatical information in signed languages, including American Sign Language(ASL). Gestures such as raising or furrowing the eyebrows are key indicators of constructions such as yes-no questions. Periodic head movements (nods and shakes) are also an essential part of the expression of syntactic information, such as negation (associated with a side-to-side headshake). Therefore, identification of these facial gestures is essential to sign language recognition. One problem with detection of such grammatical indicators is occlusion recovery. If the signer's hand blocks his/her eyebrows during production of a sign, it becomes difficult to track the eyebrows. We have developed a system to detect such grammatical markers in ASL that recovers promptly from occlusion. Our system detects and tracks evolving templates of facial features, which are based on an anthropometric face model, and interprets the geometric relationships of these templates to identify grammatical markers. It was tested on a variety of ASL sentences signed by various Deaf native signers and detected facial gestures used to express grammatical information, such as raised and furrowed eyebrows as well as headshakes.National Science Foundation (IIS-0329009, IIS-0093367, IIS-9912573, EIA-0202067, EIA-9809340

Feasibility and performance of a device for automatic self-detection of symptomatic acute coronary artery occlusion in outpatients with coronary artery disease : a multicentre observational study

Background Time delay between onset of symptoms and seeking medical attention is a major determinant of mortality and morbidity in patients with acute coronary artery occlusion. Response time might be reduced by reliable self-detection. We aimed to formally assess the proof-of-concept and accuracy of self-detection of acute coronary artery occlusion by patients during daily life situations and during the very early stages of acute coronary artery occlusion. Methods In this multicentre, observational study, we tested the operational feasibility, specificity, and sensitivity of our RELF method, a three-lead detection system with an automatic algorithm built into a mobile handheld device, for detection of acute coronary artery occlusion. Patients were recruited continuously by physician referrals from three Belgian hospitals until the desired sample size was achieved, had been discharged with planned elective percutaneous coronary intervention, and were able to use a smartphone; they were asked to perform random ambulatory selfrecordings for at least 1 week. A similar self-recording was made before percutaneous coronary intervention and at 60 s of balloon occlusion. Patients were clinically followed up until 1 month after discharge. We quantitatively assessed the operational feasibility with an automated dichotomous quality check of self-recordings. Performance was assessed by analysing the receiver operator characteristics of the ST difference vector magnitude. This trial is registered with ClinicalTrials.gov, number NCT02983396. Findings From Nov 18, 2016, to April 25, 2018, we enrolled 64 patients into the study, of whom 59 (92%) were eligible for self-applications. 58 (91%) of 64 (95% CI 81.0-95.6) patients were able to perform ambulatory self-recordings. Of all 5011 self-recordings, 4567 (91%) were automatically classified as successful within 1 min. In 65 balloon occlusions, 63 index tests at 60 s of occlusion in 55 patients were available. The mean specificity of daily life recordings was 0.96 (0.95-0.97). The mean false positive rate during daily life conditions was 4.19% (95% CI 3.29-5.10). The sensitivity for the target conditions was 0.87 (55 of 63; 95% CI 0.77-0.93) for acute coronary artery occlusion, 0.95 (54 of 57; 0.86-0.98) for acute coronary artery occlusion with electrocardiogram (ECG) changes, and 1.00 (35 of 35) for acute coronary artery occlusion with ECG changes and ST-segment elevation myocardial infarction criteria (STEMI). The index test was more sensitive to detect a 60 s balloon occlusion than the STEMI criteria on 12-lead ECG (87% vs 56%; p<0.0001). The proportion of total variation in study estimates due to heterogeneity between patients (I-2) was low (12.6%). The area under the receiver operator characteristics curve was 0.973 (95% CI 0.956-0.990) for acute coronary artery occlusion at different cutoff values of the magnitude of the ST difference vector. No patients died during the study. Interpretation Self-recording with our RELF device is feasible for most patients with coronary artery disease. The sensitivity and specificity for automatic detection of the earliest phase of acute coronary artery occlusion support the concept of our RELF device for patient empowerment to reduce delay and increase Survival without overloading emergency services. Copyright (C) 2019 The Author(s). Published by Elsevier Ltd

On Shape-Mediated Enrolment in Ear Biometrics

Ears are a new biometric with major advantage in that they appear to maintain their shape with increased age. Any automatic biometric system needs enrolment to extract the target area from the background. In ear biometrics the inputs are often human head profile images. Furthermore ear biometrics is concerned with the effects of partial occlusion mostly caused by hair and earrings. We propose an ear enrolment algorithm based on finding the elliptical shape of the ear using a Hough Transform (HT) accruing tolerance to noise and occlusion. Robustness is improved further by enforcing some prior knowledge. We assess our enrolment on two face profile datasets; as well as synthetic occlusion

Corrective treatment and anatomic considerations for laparoscopic cholecystectomy injuries

BACKGROUND: Complete reports of biliary and vascular injuries after laparoscopic cholecystectomy are rare. STUDY DESIGN: Fifteen patients with complex laparoscopic cholecystectomy injuries underwent corrective operations. The injuries consisted of 14 bile duct injuries and one large laceration of a cirrhotic liver. Five of the bile duct injuries were accompanied by inadvertent occlusion of the right hepatic artery, and one was further complicated by portal vein occlusion. One hepatic artery occlusion and one portal vein occlusion were successfully reconstructed. Two patients with arterial occlusion required right hepatic lobectomy. Corrective biliary operations consisted of common hepaticojejunostomy (seven cases), right and left hepaticojejunostomies (one case), right anterior and left hepaticojejunostomies (two cases), right hepaticojejunostomy (one case), right posterior hepaticojejunostomy (one case), and left hepaticojejunostomy after right lobectomy (two cases). RESULTS: Except for a patient with a severe laceration of a cirrhotic liver who died as a result of hepatic failure, the remaining 14 patients are alive and well with normal hepatic function tests at six and 37 months after corrective operations. CONCLUSIONS: A knowledge of anatomy is critical to the prevention of injuries to the hepatobiliary tree and related structures during laparoscopic cholecystectomy

Minor stroke due to large artery occlusion. When is intravenous thrombolysis not enough? Results from the SITS International Stroke Thrombolysis Register

Purpose: Beyond intravenous thrombolysis, evidence is lacking on acute treatment of minor stroke caused by large artery occlusion. To identify candidates for additional endovascular therapy, we aimed to determine the frequency of non-haemorrhagic early neurological deterioration in patients with intravenous thrombolysis-treated minor stroke caused by occlusion of large proximal and distal cerebral arteries. Secondary aims were to establish risk factors for non-haemorrhagic early neurological deterioration and report three-month outcomes in patients with and without non-haemorrhagic early neurological deterioration. Method: We analysed data from the SITS International Stroke Thrombolysis Register on 2553 patients with intravenous thrombolysis-treated minor stroke (NIH Stroke Scale scores 0–5) and available arterial occlusion data. Non-haemorrhagic early neurological deterioration was defined as an increase in NIH Stroke Scale score ≥4 at 24 h, without parenchymal hematoma on follow-up imaging within 22–36 h. Findings: The highest frequency of non-haemorrhagic early neurological deterioration was seen in 30% of patients with terminal internal carotid artery or tandem occlusions (internal carotid artery + middle cerebral artery) (adjusted odds ratio: 10.3 (95% CI 4.3–24.9), p &#60; 0.001) and 17% in extracranial carotid occlusions (adjusted odds ratio 4.3 (2.5–7.7), p &#60; 0.001) versus 3.1% in those with no occlusion. Proximal middle cerebral artery-M1 occlusions had non-haemorrhagic early neurological deterioration in 9% (adjusted odds ratio 2.1 (0.97–4.4), p = 0.06). Among patients with any occlusion and non-haemorrhagic early neurological deterioration, 77% were dead or dependent at three months. Conclusions: Patients with minor stroke caused by internal carotid artery occlusion, with or without tandem middle cerebral artery involvement, are at high risk of disabling deterioration, despite intravenous thrombolysis treatment. Acute vessel imaging contributes usefully even in minor stroke to identify and consider endovascular treatment, or intensive monitoring at a comprehensive stroke centre, for patients at high risk of neurological deterioration

A dynamic model for real-time tracking of hands in bimanual movements

The problem of hand tracking in the presence of occlusion is addressed. In bimanual movements the hands tend to be synchronised effortlessly. Different aspects of this synchronisation are the basis of our research to track the hands. The spatial synchronisation in bimanual movements is modelled by the position and the temporal synchronisation by the velocity and acceleration of each hand. Based on a dynamic model, we introduce algorithms for occlusion detection and hand tracking

Parsing Occluded People by Flexible Compositions

This paper presents an approach to parsing humans when there is significant occlusion. We model humans using a graphical model which has a tree structure building on recent work [32, 6] and exploit the connectivity prior that, even in presence of occlusion, the visible nodes form a connected subtree of the graphical model. We call each connected subtree a flexible composition of object parts. This involves a novel method for learning occlusion cues. During inference we need to search over a mixture of different flexible models. By exploiting part sharing, we show that this inference can be done extremely efficiently requiring only twice as many computations as searching for the entire object (i.e., not modeling occlusion). We evaluate our model on the standard benchmarked "We Are Family" Stickmen dataset and obtain significant performance improvements over the best alternative algorithms.Comment: CVPR 15 Camera Read