213 research outputs found
Detecting Worker Attention Lapses in Human-Robot Interaction: An Eye Tracking and Multimodal Sensing Study
The advent of industrial robotics and autonomous systems endow human-robot
collaboration in a massive scale. However, current industrial robots are
restrained in co-working with human in close proximity due to inability of
interpreting human agents' attention. Human attention study is non-trivial
since it involves multiple aspects of the mind: perception, memory, problem
solving, and consciousness. Human attention lapses are particularly problematic
and potentially catastrophic in industrial workplace, from assembling
electronics to operating machines. Attention is indeed complex and cannot be
easily measured with single-modality sensors. Eye state, head pose, posture,
and manifold environment stimulus could all play a part in attention lapses. To
this end, we propose a pipeline to annotate multimodal dataset of human
attention tracking, including eye tracking, fixation detection, third-person
surveillance camera, and sound. We produce a pilot dataset containing two fully
annotated phone assembly sequences in a realistic manufacturing environment. We
evaluate existing fatigue and drowsiness prediction methods for attention lapse
detection. Experimental results show that human attention lapses in production
scenarios are more subtle and imperceptible than well-studied fatigue and
drowsiness.Comment: 6 page
Recommended from our members
The autophagic degradation of cytosolic pools of peroxisomal proteins by a new selective pathway.
Damaged or redundant peroxisomes and their luminal cargoes are removed by pexophagy, a selective autophagy pathway. In yeasts, pexophagy depends mostly on the pexophagy receptors, such as Atg30 for Pichia pastoris and Atg36 for Saccharomyces cerevisiae, the autophagy scaffold proteins, Atg11 and Atg17, and the core autophagy machinery. In P. pastoris, the receptors for peroxisomal matrix proteins containing peroxisomal targeting signals (PTSs) include the PTS1 receptor, Pex5, and the PTS2 receptor and co-receptor, Pex7 and Pex20, respectively. These shuttling receptors are predominantly cytosolic and only partially peroxisomal. It remains unresolved as to whether, when and how the cytosolic pools of peroxisomal receptors, as well as the peroxisomal matrix proteins, are degraded under pexophagy conditions. These cytosolic pools exist both in normal and mutant cells impaired in peroxisome biogenesis. We report here that Pex5 and Pex7, but not Pex20, are degraded by an Atg30-independent, selective autophagy pathway. To enter this selective autophagy pathway, Pex7 required its major PTS2 cargo, Pot1. Similarly, the degradation of Pex5 was inhibited in cells missing abundant PTS1 cargoes, such as alcohol oxidases and Fox2 (hydratase-dehydrogenase-epimerase). Furthermore, in cells deficient in PTS receptors, the cytosolic pools of peroxisomal matrix proteins, such as Pot1 and Fox2, were also removed by Atg30-independent, selective autophagy, under pexophagy conditions. In summary, the cytosolic pools of PTS receptors and their cargoes are degraded via a pexophagy-independent, selective autophagy pathway under pexophagy conditions. These autophagy pathways likely protect cells from futile enzymatic reactions that could potentially cause the accumulation of toxic cytosolic products.Abbreviations: ATG: autophagy related; Cvt: cytoplasm to vacuole targeting; Fox2: hydratase-dehydrogenase-epimerase; PAGE: polyacrylamide gel electrophoresis; Pot1: thiolase; PMP: peroxisomal membrane protein; Pgk1: 3-phosphoglycerate kinase; PTS: peroxisomal targeting signal; RADAR: receptor accumulation and degradation in the absence of recycling; RING: really interesting new gene; SDS: sodium dodecyl sulphate; TCA, trichloroacetic acid; Ub: ubiquitin; UPS: ubiquitin-proteasome system Vid: vacuole import and degradation
Automated cropping intensity extraction from isolines of wavelet spectra
Timely and accurate monitoring of cropping intensity (CI) is essential to help us understand changes in food production. This paper aims to develop an automatic Cropping Intensity extraction method based on the Isolines of Wavelet Spectra (CIIWS) with consideration of intra- class variability. The CIIWS method involves the following procedures: (1) characterizing vegetation dynamics from time–frequency dimensions through a continuous wavelet transform performed on vegetation index temporal profiles; (2) deriving three main features, the skeleton width, maximum number of strong brightness centers and the intersection of their scale intervals, through computing a series of wavelet isolines from the wavelet spectra; and (3) developing an automatic cropping intensity classifier based on these three features. The proposed CIIWS method improves the understanding in the spectral–temporal properties of vegetation dynamic processes. To test its efficiency, the CIIWS method is applied to China’s Henan province using 250 m 8 days composite Moderate Resolution Imaging Spectroradiometer (MODIS) Enhanced Vegetation Index (EVI) time series datasets. An overall accuracy of 88.9% is achieved when compared with in-situ observation data. The mapping result is also evaluated with 30 m Chinese Environmental Disaster Reduction Satellite (HJ-1)-derived data and an overall accuracy of 86.7% is obtained. At county level, the MODIS-derived sown areas and agricultural statistical data are well correlated (r2 = 0.85). The merit and uniqueness of the CIIWS method is the ability to cope with the complex intra-class variability through continuous wavelet transform and efficient feature extraction based on wavelet isolines. As an objective and meaningful algorithm, it guarantees easy applications and greatly contributes to satellite observations of vegetation dynamics and food security efforts
Multimodal Fish Feeding Intensity Assessment in Aquaculture
Fish feeding intensity assessment (FFIA) aims to evaluate the intensity
change of fish appetite during the feeding process, which is vital in
industrial aquaculture applications. The main challenges surrounding FFIA are
two-fold. 1) robustness: existing work has mainly leveraged single-modality
(e.g., vision, audio) methods, which have a high sensitivity to input noise. 2)
efficiency: FFIA models are generally expected to be employed on devices. This
presents a challenge in terms of computational efficiency. In this work, we
first introduce an audio-visual dataset, called AV-FFIA. AV-FFIA consists of
27,000 labeled audio and video clips that capture different levels of fish
feeding intensity. To our knowledge, AV-FFIA is the first large-scale
multimodal dataset for FFIA research. Then, we introduce a multi-modal approach
for FFIA by leveraging single-modality pre-trained models and modality-fusion
methods, with benchmark studies on AV-FFIA. Our experimental results indicate
that the multi-modal approach substantially outperforms the single-modality
based approach, especially in noisy environments. While multimodal approaches
provide a performance gain for FFIA, it inherently increase the computational
cost. To overcome this issue, we further present a novel unified model, termed
as U-FFIA. U-FFIA is a single model capable of processing audio, visual, or
audio-visual modalities, by leveraging modality dropout during training and
knowledge distillation from single-modality pre-trained models. We demonstrate
that U-FFIA can achieve performance better than or on par with the
state-of-the-art modality-specific FFIA models, with significantly lower
computational overhead. Our proposed U-FFIA approach enables a more robust and
efficient method for FFIA, with the potential to contribute to improved
management practices and sustainability in aquaculture
Optimal energy portfolio method for regulable hydropower plants under the spot market
The energy allocation method for regulable hydropower plants under the spot market significantly impacts their income. The available studies generally draw on the Conditional Value-at-Risk (CVaR) approach, which typically assumes a fixed risk aversion coefficient for generators. This assumption is based on the assumption that the total energy the power plant can allocate is constant during the decision period. However, the amount of energy that the regulable hydropower plant can generate will be affected by inflow and water level during the decision period, and the assumption of the fixed risk aversion coefficient is only partially consistent with the actual decision behavior of the hydropower plant. In this regard, the time-varying relative risk aversion (TVRRA) based method is proposed for the energy allocation of regulable hydropower plants. That method takes the change value of the hydropower plant’s energy generation as the basis for adjusting the time-varying relative risk aversion coefficient to make the energy allocation results more consistent with the actual decision-making needs of the hydropower plant. A two-layer optimal method is proposed to obtain the income-maximizing energy portfolio based on regulable hydropower plants’ time-varying relative risk aversion coefficient. The inner point method solves the optimal energy portfolio of income and risk in the upper layer. The time-varying relative risk aversion coefficient in the lower layer accurately describes the dynamic risk preference of hydropower plants for each period. The results and comparison show that the proposed method increases the income of the energy portfolio by 31%, and water disposal of regulated hydropower plants is reduced by 2%. The energy portfolio optimization method for regulable hydropower plants proposed in this paper not only improves the economic income of hydropower plants but also improves the utilization rate of hydro energy resources and enhances the market competitiveness of regulable hydropower plants
Imitation with Spatial-Temporal Heatmap: 2nd Place Solution for NuPlan Challenge
This paper presents our 2nd place solution for the NuPlan Challenge 2023.
Autonomous driving in real-world scenarios is highly complex and uncertain.
Achieving safe planning in the complex multimodal scenarios is a highly
challenging task. Our approach, Imitation with Spatial-Temporal Heatmap, adopts
the learning form of behavior cloning, innovatively predicts the future
multimodal states with a heatmap representation, and uses trajectory refinement
techniques to ensure final safety. The experiment shows that our method
effectively balances the vehicle's progress and safety, generating safe and
comfortable trajectories. In the NuPlan competition, we achieved the second
highest overall score, while obtained the best scores in the ego progress and
comfort metrics
- …