Segmental Spatiotemporal CNNs for Fine-grained Action Segmentation

Joint segmentation and classification of fine-grained actions is important for applications of human-robot interaction, video surveillance, and human skill evaluation. However, despite substantial recent progress in large-scale action classification, the performance of state-of-the-art fine-grained action recognition approaches remains low. We propose a model for action segmentation which combines low-level spatiotemporal features with a high-level segmental classifier. Our spatiotemporal CNN is comprised of a spatial component that uses convolutional filters to capture information about objects and their relationships, and a temporal component that uses large 1D convolutional filters to capture information about how object relationships change across time. These features are used in tandem with a semi-Markov model that models transitions from one action to another. We introduce an efficient constrained segmental inference algorithm for this model that is orders of magnitude faster than the current approach. We highlight the effectiveness of our Segmental Spatiotemporal CNN on cooking and surgical action datasets for which we observe substantially improved performance relative to recent baseline methods.Comment: Updated from the ECCV 2016 version. We fixed an important mathematical error and made the section on segmental inference cleare

Supporting the Delivery of Total Knee Replacements Care for Both Patients and Their Clinicians With a Mobile App and Web-Based Tool: Randomized Controlled Trial Protocol

Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 01.03.2017. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.Background: Total knee replacement (TKR) surgeries have increased in recent years. Exercise programs and other interventions following surgery can facilitate the recovery process. With limited clinician contact time, patients with TKR have a substantial burden of self-management and limited communication with their care team, thus often fail to implement an effective rehabilitation plan. Objective: We have developed a digital orthopedic rehabilitation platform that comprises a mobile phone app, wearable activity tracker, and clinical Web portal in order to engage patients with self-management tasks for surgical preparation and recovery, thus addressing the challenges of adherence to and completion of TKR rehabilitation. The study will determine the efficacy of the TKR platform in delivering information and assistance to patients in their preparation and recovery from TKR surgery and a Web portal for clinician care teams (ie, surgeons and physiotherapists) to remotely support and monitor patient progress. Methods: The study will evaluate the TKR platform through a randomized controlled trial conducted at multiple sites (N=5) in a number of states in Australia with 320 patients undergoing TKR surgery; the trial will run for 13 months for each patient. Participants will be randomized to either a control group or an intervention group, both receiving usual care as provided by their hospital. The intervention group will receive the app and wearable activity tracker. Participants will be assessed at 4 different time points: 4 weeks before surgery, immediately before surgery, 12 weeks after surgery, and 52 weeks after surgery. The primary outcome measure is the Oxford Knee Score. Secondary outcome measures include quality of life (Short-Form Health Survey); depression, anxiety, and stress (Depression, Anxiety, and Stress Scales); self-motivation; self-determination; self-efficacy; and the level of satisfaction with the knee surgery and care delivery. The study will also collect quantitative usage data related to all components (app, activity tracker, and Web portal) of the TKR platform and qualitative data on the perceptions of the platform as a tool for patients, carers, and clinicians. Finally, an economic evaluation of the impact of the platform will be conducted. Results: Development of the TKR platform has been completed and deployed for trial. The research protocol is approved by 2 human research ethics committees in Australia. A total of 5 hospitals in Australia (2 in New South Wales, 2 in Queensland, and 1 in South Australia) are expected to participate in the trial. Conclusions: The TKR platform is designed to provide flexibility in care delivery and increased engagement with rehabilitation services. This trial will investigate the clinical and behavioral efficacy of the app and impact of the TKR platform in terms of service satisfaction, acceptance, and economic benefits of the provision of digital services

An Intelligent Tool for Activity Data Collection

Activity recognition systems using simple and ubiquitous sensors require a large variety of real-world sensor data for not only evaluating their performance but also training the systems for better functioning. However, a tremendous amount of effort is required to setup an environment for collecting such data. For example, expertise and resources are needed to design and install the sensors, controllers, network components, and middleware just to perform basic data collections. It is therefore desirable to have a data collection method that is inexpensive, flexible, user-friendly, and capable of providing large and diverse activity datasets. In this paper, we propose an intelligent activity data collection tool which has the ability to provide such datasets inexpensively without physically deploying the testbeds. It can be used as an inexpensive and alternative technique to collect human activity data. The tool provides a set of web interfaces to create a web-based activity data collection environment. It also provides a web-based experience sampling tool to take the user’s activity input. The tool generates an activity log using its activity knowledge and the user-given inputs. The activity knowledge is mined from the web. We have performed two experiments to validate the tool’s performance in producing reliable datasets

Application of a highly selective Cathepsin S two-step activity-based probe in multicolor bio-orthogonal correlative light-electron microscopy

Cathepsin S is a lysosomal cysteine protease highly expressed in immune cells such as dendritic cells, B cells and macrophages. Its functions include extracellular matrix breakdown and cleavage of cell adhesion molecules to facilitate immune cell motility, as well as cleavage of the invariant chain during maturation of major histocompatibility complex II. The identification of these diverse specific functions has brought the challenge of delineating cathepsin S activity with great spatial precision, relative to related enzymes and substrates. Here, the development of a potent and highly selective two-step activity-based probe for cathepsin S and the application in multicolor bio-orthogonal correlative light-electron microscopy is presented. LHVS, which has been reported as a selective inhibitor of cathepsin S with nanomolar potency, formed the basis for our probe design. However, in competitive activity-based protein profiling experiments LHVS showed significant cross-reactivity toward Cat L. Introduction of an azide group in the P2 position expanded the selectivity window for cathepsin S, but rendered the probe undetectable, as demonstrated in bio-orthogonal competitive activity-based protein profiling. Incorporation of an additional azide handle for click chemistry on the solvent-exposed P1 position allowed for selective labeling of cathepsin S. This highlights the influence of click handle positioning on probe efficacy. This probe was utilized in multicolor bio-orthogonal confocal and correlative light-electron microscopy to investigate the localization of cathepsin S activity at an ultrastructural level in bone marrow-derived dendritic cells. The tools developed in this study will aid the characterization of the variety of functions of cathepsin S throughout biology.Microscopic imaging and technolog

Limited effect of patient and disease characteristics on compliance with hospital antimicrobial guidelines

Objective: Physicians frequently deviate from guidelines that promote prudent use of antimicrobials. We explored to what extent patient and disease characteristics were associated with compliance with guideline recommendations for three common infections. Methods: In a 1-year prospective observational study, 1,125 antimicrobial prescriptions were analysed for compliance with university hospital guidelines. Results: Compliance varied significantly between and within the groups of infections studied. Compliance was much higher for lower respiratory tract infections (LRTIs; 79%) than for sepsis (53%) and urinary tract infections (UTIs; 40%). Only predisposing illnesses and active malignancies were associated with more compliant prescribing, whereas alcohol/ intravenous drug abuse and serum creatinine levels > 130 mu mol/l were associated with less compliant prescribing. Availability of culture results had no impact on compliance with guidelines for sepsis but was associated with more compliance in UTIs and less in LRTIs. Narrowing initial broad-spectrum antimicrobial therapy to cultured pathogens was seldom practised. Most noncompliant prescribing concerned a too broad spectrum of activity when compared with guideline-recommended therapy. Conclusion: Patient characteristics had only a limited impact on compliant prescribing for a variety of reasons. Physicians seemed to practise defensive prescribing behaviour, favouring treatment success in current patients over loss of effectiveness due to resistance in future patients

Application of a highly selective cathepsin S two-step activity-based probe in multicolor bio-orthogonal correlative light-electron microscopy

Cathepsin S is a lysosomal cysteine protease highly expressed in immune cells such as dendritic cells, B cells and macrophages. Its functions include extracellular matrix breakdown and cleavage of cell adhesion molecules to facilitate immune cell motility, as well as cleavage of the invariant chain during maturation of major histocompatibility complex II. The identification of these diverse specific functions has brought the challenge of delineating cathepsin S activity with great spatial precision, relative to related enzymes and substrates. Here, the development of a potent and highly selective two-step activity-based probe for cathepsin S and the application in multicolor bio-orthogonal correlative light-electron microscopy is presented. LHVS, which has been reported as a selective inhibitor of cathepsin S with nanomolar potency, formed the basis for our probe design. However, in competitive activity-based protein profiling experiments LHVS showed significant cross-reactivity toward Cat L. Introduction of an azide group in the P2 position expanded the selectivity window for cathepsin S, but rendered the probe undetectable, as demonstrated in bio-orthogonal competitive activity-based protein profiling. Incorporation of an additional azide handle for click chemistry on the solvent-exposed P1 position allowed for selective labeling of cathepsin S. This highlights the influence of click handle positioning on probe efficacy. This probe was utilized in multicolor bio-orthogonal confocal and correlative light-electron microscopy to investigate the localization of cathepsin S activity at an ultrastructural level in bone marrow-derived dendritic cells. The tools developed in this study will aid the characterization of the variety of functions of cathepsin S throughout biology.Bio-organic Synthesi

Live-cell imaging of sterculic acid - a naturally occurring 1,2-cyclopropene fatty acid - by bioorthogonal reaction with turn-on tetrazine-fluorophore conjugates

In the field of lipid research, bioorthogonal chemistry has made the study of lipid uptake and processing in living systems possible, whilst minimising biological properties arising from detectable pendant groups. To allow the study of unsaturated free fatty acids in live cells, we here report the use of sterculic acid, a 1,2-cyclopropene-containing oleic acid analogue, as a bioorthogonal probe. We show that this lipid can be readily taken up by dendritic cells without toxic side effects, and that it can subsequently be visualised using an inverse electron-demand Diels-Alder reaction with quenched tetrazine-fluorophore conjugates. In addition, the lipid can be used to identify changes in protein oleoylation after immune cell activation. Finally, this reaction can be integrated into a multiplexed bioorthogonal reaction workflow by combining it with two sequential copper-catalysed Huisgen ligation reactions. This allows for the study of multiple biomolecules in the cell simultaneously by multimodal confocal imaging.NWOERC-CoG 865175Molecular PhysiologyBio-organic Synthesi

In Vitro Enhancement of Respiratory Syncytial Virus Infection by Maternal Antibodies Does Not Explain Disease Severity in Infants.

Respiratory syncytial virus (RSV) is the leading cause of severe respiratory illness in infants. At this young age, infants typically depend on maternally transferred antibodies (matAbs) and their innate immune system for protection against infections. RSV-specific matAbs are thought to protect from severe illness, yet severe RSV disease occurs mainly below 6 months of age, when neutralizing matAb levels are present. To investigate this discrepancy, we asked if disease severity is related to antibody properties other than neutralization. Some antibody effector functions are mediated via their Fc binding region. However, it has been shown that this binding may lead to antibody-dependent enhancement (ADE) of infection or reduction of neutralization, both possibly leading to more disease. In this study, we first showed that high levels of ADE of RSV infection occur in monocytic THP-1 cells in the presence of RSV antibodies and that neutralization by these antibodies was reduced in Vero cells when they were transduced with Fc gamma receptors. We then demonstrated that antibodies from cotton rats with formalin-inactivated (FI)-RSV-induced pulmonary pathology were capable of causing ADE. Human matAbs also caused ADE and were less neutralizing in vitro in cells that carry Fc receptors. However, these effects were unrelated to disease severity because they were seen both in uninfected controls and in infants hospitalized with different levels of RSV disease severity. We conclude that ADE and reduction of neutralization are unlikely to be involved in RSV disease in infants with neutralizing matAbs.IMPORTANCE It is unclear why severity of RSV disease peaks at the age when infants have neutralizing levels of maternal antibodies. Additionally, the exact reason for FI-RSV-induced enhanced disease, as seen in the 1960s vaccine trials, is still unclear. We hypothesized that antibodies present under either of these conditions could contribute to disease severity. Antibodies can have effects that may lead to more disease instead of protection. We investigated two of those effects: antibody-dependent enhancement of infection (ADE) and neutralization reduction. We show that ADE occurs in vitro with antibodies from FI-RSV-immunized RSV-infected cotton rats. Moreover, passively acquired maternal antibodies from infants had the capacity to induce ADE and reduction of neutralization. However, no clear association with disease severity was seen, ruling out that these properties explain disease in the presence of maternal antibodies. Our data contribute to a better understanding of the impact of antibodies on RSV disease in infants