Context De-confounded Emotion Recognition

Context-Aware Emotion Recognition (CAER) is a crucial and challenging task that aims to perceive the emotional states of the target person with contextual information. Recent approaches invariably focus on designing sophisticated architectures or mechanisms to extract seemingly meaningful representations from subjects and contexts. However, a long-overlooked issue is that a context bias in existing datasets leads to a significantly unbalanced distribution of emotional states among different context scenarios. Concretely, the harmful bias is a confounder that misleads existing models to learn spurious correlations based on conventional likelihood estimation, significantly limiting the models' performance. To tackle the issue, this paper provides a causality-based perspective to disentangle the models from the impact of such bias, and formulate the causalities among variables in the CAER task via a tailored causal graph. Then, we propose a Contextual Causal Intervention Module (CCIM) based on the backdoor adjustment to de-confound the confounder and exploit the true causal effect for model training. CCIM is plug-in and model-agnostic, which improves diverse state-of-the-art approaches by considerable margins. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our CCIM and the significance of causal insight.Comment: Accepted by CVPR 2023. CCIM is available at https://github.com/ydk122024/CCI

The Jiao Tong University Spectroscopic Telescope Project

The Jiao Tong University Spectroscopic Telescope (JUST) is a 4.4-meter f/6.0 segmentedmirror telescope dedicated to spectroscopic observations. The JUST primary mirror is composed of 18 hexagonal segments, each with a diameter of 1.1 m. JUST provides two Nasmyth platforms for placing science instruments. One Nasmyth focus fits a field of view of 10 arcmin and the other has an extended field of view of 1.2 deg with correction optics. A tertiary mirror is used to switch between the two Nasmyth foci. JUST will be installed at a site at Lenghu in Qinghai Province, China, and will conduct spectroscopic observations with three types of instruments to explore the dark universe, trace the dynamic universe, and search for exoplanets: (1) a multi-fiber (2000 fibers) medium-resolution spectrometer (R=4000-5000) to spectroscopically map galaxies and large-scale structure; (2) an integral field unit (IFU) array of 500 optical fibers and/or a long-slit spectrograph dedicated to fast follow-ups of transient sources for multimessenger astronomy; (3) a high-resolution spectrometer (R~100000) designed to identify Jupiter analogs and Earth-like planets, with the capability to characterize the atmospheres of hot exoplanets.Comment: 28 pages, 6 figure

Contributions to Case-Based Reasoning Enabled Decision Support System for Smart Agriculture

Nowadays, high demands for food from the world-wide growing population are impacting the environment and putting many pressures on agricultural productivity. As a farming management concept, smart agriculture tries to integrate advanced technologies like Internet of Things, Artificial Intelligence, and Remote Sensing into current farming practices for the purpose of boosting productivity and improving the quality of agricultural products. The core of smart agriculture emphasizes on the use of information systems and communication technologies in the cyber-physical farm management cycle. However, farmers can hardly take advantage of collected information to make proper decisions because it is difficult to transfer the explosive amount of raw data from sensors, actuators, and networks into practical knowledge for managing farming operations. Therefore, delivering an agricultural decision support system to farmers to assist them in making evidence-based decisions is needed. The ultimate objective of this thesis is to design and implement a decision support system within the Aggregate Farming in the Cloud (AFarCloud) platform. Meanwhile, the proposed decision support system tries to overcome the current challenging problems in this topic. To achieve this objective, this thesis follows the below three research areas. The first area aims at providing a general solution for delivering an agricultural decision support system for the AFarCloud platform. An architectural proposal of the decision support system framework for managing farming operations is presented in this thesis. The proposed framework defines an algorithm manager and an algorithm toolbox. The former component is responsible to configure registered decision support algorithms, while the latter component is capable of selecting a certain algorithm to generate decision supports. The proposed framework demonstrates how smart agriculture can benefit from the support of a decision support system, and therefore assist farmers in making evidence-based decisions. The second area focuses on designing a case-based reasoning (CBR) algorithm to generate decision supports for farmers. This CBR algorithm is implemented within the framework proposed in the first research area, in particular, within the algorithm toolbox component. According to the nature of the CBR algorithm, it can be divided into five steps, including representation, retrieval, reuse, revision, and retention. In this thesis, an improved CBR algorithm is proposed to overcome the detected shortcomings of the current research work. Firstly, an associated case representation formalism is presented for enhancing the typical feature vector representation. The proposed representation formalism contains the similar and dissimilar associations between past cases, enabling to compare potential similar cases preferentially. Secondly, a triangular similarity measure is designed by taking advantage of cosine and Euclidean distance measures. For providing a precise measurement, the magnitude differences between two compared N-dimensional vectors are taken into consideration. Thirdly, a fast case retrieval algorithm is developed, enabling to determine a list of similar past cases by comparing a fewer number of cases. As a consequence, the retrieval efficiency is improved while the retrieval accuracy can be guaranteed as well. Fourthly, a learning-based approach for solution reuse and revision is studied. This reuse and revision approach tries to identify the difference between the problem part of compared cases, and then update the retrieved solution based on previous experiences. Lastly, an associated case retention approach is put forward. Apart from the typical addition and deletion strategies, the proposed retention approach also concerns to update the existed associations and generate new associations for the learned cases. By enhancing each step of the CBR loop, the proposed CBR algorithm is able to generate promising decision supports with great efficiency and accuracy. The third area considers a hybrid decision support mechanism for the AFarCloud platform. It is noted that though the improved CBR algorithm can generate a satisfied result for the most queries, it may be unable to generate the decision supports when the CBR algorithm fails to retrieve a list of similar past cases. Under this circumstance, the decision support system should start other registered algorithms to carry on the task. Therefore, for coordinating the interaction between various decision support algorithms, a mediator design pattern is adopted in this hybrid decision support mechanism. Owing to the design of the mediator component, different decision support algorithms do no need to interact with each other directly. Instead, the communication work between the algorithm manager and decision support algorithms is handled by this mediator component. This hybrid decision support mechanism is verified through a preliminary proof, considering the CBR algorithm and an artificial neural network algorithm. The result suggests that the hybrid decision support mechanism can enhance the robustness of the overall decision support system. Lastly, the proposed decision support system, along with the improved CBR algorithm, are all verified by simulation. The simulation results demonstrate that the proposal in this thesis is effective and achieves better performance than previous works. ----------RESUMEN---------- Hoy en día, la alta demanda de alimentos a consecuencia del crecimiento de la población mundial está afectando al medio ambiente y ejerciendo muchas presiones sobre la producción agrícola. La agricultura inteligente, entendida como un concepto de la gestión agrícola, intenta la integración de tecnologías avanzadas, tales como Internet de las cosas, la inteligencia artificial y la teledetección, en las prácticas agrícolas actuales, con el fin de aumentar la productividad y mejorar la calidad de los productos agrícolas. El núcleo de la agricultura inteligente hace hincapié en el uso de los sistemas de información y las tecnologías de comunicaciones en el ciclo de gestión de la granja ciberfísica. Sin embargo es difícil que los granjeros puedan aprovechar la información recopilada para tomar las decisiones adecuadas. La explosiva cantidad de datos procedentes de los sensores, actuadores y redes cuesta transformarla en un conocimiento práctico que resulte útil para la administración de las operaciones agrícolas. Por lo tanto, es necesario proporcionar a los agricultores un sistema de apoyo a las decisiones agrícolas que les ayude a tomar decisiones basadas en la evidencia. El objetivo final de esta tesis es diseñar e implementar un sistema de soporte a decisiones dentro de la plataforma Aggregate Farming in the Cloud (AFarCloud). En particular, el sistema de apoyo a decisiones propuesto trata de superar los desafíos actuales en este tema. Para lograr este objetivo, esta tesis sigue las siguientes tres áreas de investigación. La primera área tiene como objetivo proporcionar una solución general para entregar un sistema de soporte a decisiones agrícolas para la plataforma AFarCloud. En esta tesis se presenta la propuesta de un marco de referencia para la arquitectura de un sistema de soporte a decisiones para la administración de las operaciones agrícolas. Este marco define dos componentes: un administrador de algoritmos, y un grupo de herramientas de algoritmos. El primero es responsable de configurar los algoritmos de soporte a decisiones que estén registrados, mientras que el segundo es capaz de seleccionar un cierto algoritmo de entre los registrados para generar soportes a decisiones. El marco propuesto demuestra cómo la agricultura inteligente puede beneficiarse del apoyo de un sistema de soporte a decisiones, y por lo tanto, cómo ayuda a los agricultores a tomar decisiones basadas en la evidencia. La segunda área se centra en el diseño de un algoritmo de razonamiento basado en casos (CBR) para generar soporte a decisiones para los agricultores. Este algoritmo CBR se implementa dentro del marco propuesto en la primera área de investigación, en particular, en el grupo de herramientas de algoritmos. De acuerdo a su naturaleza el algoritmo CBR se puede dividir en cinco pasos, que incluyen: representación, recuperación, reutilización, revisión y retención. En esta tesis, se propone un algoritmo CBR mejorado para superar las deficiencias detectadas en el trabajo de investigación. En primer lugar, se presenta la representación formal de los casos asociados para mejorar la representación típica del vector de características. La representación formal propuesta contiene las asociaciones de similitud y diferencias entre casos pasados, permitiendo comparar preferentemente los casos similares posibles. En segundo lugar, se diseña una medida de similitud triangular aprovechando las medidas de distancia del coseno y euclidiana. Para proporcionar una medición precisa se tienen en cuenta las diferencias al comparar la magnitud entre dos vectores N-dimensionales. En tercer lugar, se desarrolla un algoritmo de recuperación rápida de casos, que permite obtener una lista de casos pasados similares comparando un número menor de casos. En consecuencia, la eficiencia de la recuperación mejora a la vez que se garantiza su precisión. En cuarto lugar se estudia un enfoque basado en el aprendizaje para la reutilización y revisión de soluciones. Este enfoque de reutilización y revisión intenta identificar la diferencia entre la parte del problema de los casos comparados, para posteriormente actualizar la solución recuperada en base a las experiencias previas. Por último, se presenta un enfoque de retención del caso asociado. Además de las estrategias típicas de adición y eliminación, el enfoque de retención propuesto se refiere también a la actualización de las asociaciones existentes y a la generación nuevas asociaciones para los casos aprendidos. Al mejorar cada uno de los pasos del ciclo CBR, el algoritmo CBR propuesto puede generar soportes prometedores para la toma de decisiones, y además hacerlo con gran eficiencia y precisión. La tercera área considera un mecanismo híbrido de soporte a decisiones para la plataforma AFarCloud. Se observa que, aunque el algoritmo CBR mejorado puede generar resultados satisfactorios durante la mayor parte del tiempo, es posible que no pueda generar soportes a decisión cuando el algoritmo CBR no pueda recuperar una lista de casos pasados similares. Bajo esta circunstancia, el sistema de soporte a decisiones debe iniciar otros algoritmos registrados para llevar a cabo la tarea. Por lo tanto, para coordinar la interacción entre varios algoritmos de soporte a decisiones, se adopta un patrón de diseño tipo mediador en este mecanismo híbrido de soporte a decisiones. Gracias al diseño del componente mediador, los diferentes algoritmos de soporte a decisiones no necesitan interactuar entre sí directamente. En cambio, este componente mediador maneja el trabajo de comunicación entre el administrador de algoritmos y los algoritmos de soporte a decisiones. Este mecanismo híbrido de soporte a decisión se verifica a través de una prueba preliminar que utiliza el algoritmo CBR y una red neuronal artificial. El resultado sugiere que el mecanismo híbrido de soporte a decisiones puede mejorar la solidez del sistema general. Por último, el sistema de soporte a decisiones propuesto, junto con el algoritmo CBR mejorado, se verifican por simulación. Los resultados de la simulación demuestran que la propuesta en esta tesis es efectiva y logra un mejor rendimiento que los trabajos anteriores

A Triangular Similarity Measure for Case Retrieval in CBR and Its Application to an Agricultural Decision Support System

Case-based reasoning has been a widely-used approach to assist humans in making decisions through four steps: retrieve, reuse, revise, and retain. Among these steps, case retrieval plays a significant role because the rest of processes cannot proceed without successfully identifying the most similar past case beforehand. Some popular methods such as angle-based and distance-based similarity measures have been well explored for case retrieval. However, these methods may match inaccurate cases under certain extreme circumstances. Thus, a triangular similarity measure is proposed to identify commonalities between cases, overcoming the drawbacks of angle-based and distance-based measures. For verifying the effectiveness and performance of the proposed measure, case-based reasoning was applied to an agricultural decision support system for pest management and 300 new cases were used for testing purposes. Once a new pest problem is reported, its attributes are compared with historical data by the proposed triangular similarity measure. Farmers can obtain quick decision support on managing pest problems by learning from the retrieved solution of the most similar past case. The experimental result shows that the proposed measure can retrieve the most similar case with an average accuracy of 91.99% and it outperforms the other measures in the aspects of accuracy and robustness

Early Detection of Bacterial Blight in Hyperspectral Images Based on Random Forest and Adaptive Coherence Estimator

Rice disease detection is of great significance to rice disease management. It is difficult to identify the rice leaves with different colors in different disease periods by RGB image and without aided eyes. Traditional equipment and methods are relatively inefficient in meeting the needs of current disease detection. The accurate and efficient detection the infected areas from hyperspectral images has become a primary concern in current research. However, current spectral target detection research pays less attention to the time and computing resources consumed by detection. A disease detection method based on random forest (RF) and adaptive coherence estimator (ACE) is proposed here. Firstly, based on the spectral differences between diseased and healthy leaves, 18 characteristic spectral wavelengths with the highest importance were selected by an RF algorithm, and the spectral images of those characteristic wavelengths were synthesized. Then, the ACE model was established for the disease recognition of full wavelength spectral images, characteristic wavelength spectral images, and RGB images. At the same time, three other familiar target detection methods were selected as the control experiments. The detection results showed a similarity between the detection performance of the four detection methods for full wavelength spectral image and characteristic wavelength spectral image. This detection performance was higher than that of the RGB image, indicating that characteristic wavelength spectral image can replace full wavelength spectral image for disease detection. The detection performance of the ACE algorithm was better than other algorithms. The detection accuracy of 18 characteristic wavelengths was 97.41%. Compared with the hyperspectral full wavelength image detection results, the accuracy decreased by 1.12%, and the detection time decreased by 2/3, which greatly reduced the detection time. Based on these results, the target detection method combining the RF algorithm and the ACE algorithm can effectively and accurately detect rice bacterial blight disease, which provides a new method for automatic detection of plant disease in the field

Identification of Fish Hunger Degree with Deformable Attention Transformer

Feeding is a critical process in aquaculture, as it has a direct impact on the quantity and quality of fish. With advances in convolutional neural network (CNN) and vision transformer (ViT), intelligent feeding has been widely adopted in aquaculture, as the real-time monitoring of fish behavior can lead to better feeding decisions. However, existing models still have the problem of insufficient accuracy in the fish behavior-recognition task. In this study, the largemouth bass (Micropterus salmoides) was selected as the research subject, and three categories (weakly, moderately, and strongly hungry) were defined. We applied the deformable attention to the vision transformer (DeformAtt-ViT) to identify the fish hunger degree. The deformable attention module was extremely powerful in feature extraction because it improved the fixed geometric structure of the receptive fields with data-dependent sparse attention, thereby guiding the model to focus on more important regions. In the experiment, the proposed DeformAtt-ViT was compared with the state-of-the-art transformers. Among them, DeformAtt-ViT achieved optimal performance in terms of accuracy, F1-score, recall, and precision at 95.50%, 94.13%, 95.87%, and 92.45%, respectively. Moreover, a comparative evaluation between DeformAtt-ViT and CNNs was conducted, and DeformAtt-ViT still dominated the others. We further visualized the important pixels that contributed the most to the classification result, enabling the interpretability of the model. As a prerequisite for determining the feed time, the proposed DeformAtt-ViT could identify the aggregation level of the fish and then trigger the feeding machine to be turned on. Also, the feeding machine will stop working when the aggregation disappears. Conclusively, this study was of great significance, as it explored the field of intelligent feeding in aquaculture, enabling precise feeding at a proper time

ResViT-Rice: A Deep Learning Model Combining Residual Module and Transformer Encoder for Accurate Detection of Rice Diseases

Rice is a staple food for over half of the global population, but it faces significant yield losses: up to 52% due to leaf blast disease and brown spot diseases, respectively. This study aimed at proposing a hybrid architecture, namely ResViT-Rice, by taking advantage of both CNN and transformer for accurate detection of leaf blast and brown spot diseases. We employed ResNet as the backbone network to establish a detection model and introduced the encoder component from the transformer architecture. The convolutional block attention module was also integrated to ResViT-Rice to further enhance the feature-extraction ability. We processed 1648 training and 104 testing images for two diseases and the healthy class. To verify the effectiveness of the proposed ResViT-Rice, we conducted comparative evaluation with popular deep learning models. The experimental result suggested that ResViT-Rice achieved promising results in the rice disease-detection task, with the highest accuracy reaching 0.9904. The corresponding precision, recall, and F1-score were all over 0.96, with an AUC of up to 0.9987, and the corresponding loss rate was 0.0042. In conclusion, the proposed ResViT-Rice can better extract features of different rice diseases, thereby providing a more accurate and robust classification output

DC2Net: An Asian Soybean Rust Detection Model Based on Hyperspectral Imaging and Deep Learning

Asian soybean rust (ASR) is one of the major diseases that causes serious yield loss worldwide, even up to 80%. Early and accurate detection of ASR is critical to reduce economic losses. Hyperspectral imaging, combined with deep learning, has already been proved as a powerful tool to detect crop diseases. However, current deep learning models are limited to extract both spatial and spectral features in hyperspectral images due to the use of fixed geometric structure of the convolutional kernels, leading to the fact that the detection accuracy of current models remains further improvement. In this study, we proposed a deformable convolution and dilated convolution neural network (DC2Net) for the ASR detection. The deformable convolution module was used to extract the spatial features, while the dilated convolution module was applied to extract features from the spectral dimension. We also adopted the Shapley value and the channel attention methods to evaluate the importance of each wavelength during decision-making, thereby identifying the most contributing ones. The proposed DC2Net can realize early asymptomatic detection of ASR even when visual symptoms have not appeared. The results of the experiment showed that the detection performance of DC2Net dominated state-of-the-art methods, reaching an overall accuracy at 96.73%. Meanwhile, the experimental result suggested that the Shapley Additive exPlanations method was able to extract feature wavelengths correctly, thereby helping DC2Net achieve reasonable performance with less input data. The research result of this study could provide early warning of ASR outbreak in advance, even at the asymptomatic period

Decision support systems for agriculture 4.0: Survey and challenges

Undoubtedly, high demands for food from the world-wide growing population are impacting the environment and putting many pressures on agricultural productivity. Agriculture 4.0, as the fourth evolution in farming technology, puts forward four essential requirements: increasing productivity, allocating resources reasonably, adapting to climate change, and avoiding food waste. As advanced information systems and Internet technologies are adopted in Agriculture 4.0, enormous farming data, such as temperature, humidity, soil conditions, marketing demands, and land uses, can be collected, analyzed, and processed for assisting farmers in making appropriate decisions and obtaining higher profits. Therefore, agricultural decision support systems for Agriculture 4.0 has become a very attractive topic for the research community. The objective of this paper aims at improving future agricultural decision support systems to better serve Agriculture 4.0. In this paper, the systematic literature review technique is used to survey thirteen representative decision support systems, including their applications for agricultural mission planning, water resources management, climate change adaptation, and food waste control. Each decision support system is introduced and analyzed under a systematic manner. A comprehensive evaluation is conducted from the aspects of interoperability, scalability, accessibility, usability, etc. Based on the evaluation result, upcoming challenges of employing agricultural decision support systems in Agriculture 4.0 are detected and summarized, suggesting the development trends and highlighting direct insights for future research

An Efficient Case Retrieval Algorithm for Agricultural Case-Based Reasoning Systems, with Consideration of Case Base Maintenance

Case-based reasoning has considerable potential to model decision support systems for smart agriculture, assisting farmers in managing farming operations. However, with the explosive amount of sensing data, these systems may achieve poor performance in knowledge management like case retrieval and case base maintenance. Typical approaches of case retrieval have to traverse all past cases for matching similar ones, leading to low efficiency. Thus, a new case retrieval algorithm for agricultural case-based reasoning systems is proposed in this paper. At the initial stage, an association table is constructed, containing the relationships between all past cases. Afterwards, attributes of a new case are compared with an entry case. According to the similarity measurement, associated similar or dissimilar cases are then compared preferentially, instead of traversing the whole case base. The association of the new case is generated through case retrieval and added in the association table at the step of case retention. The association table is also updated when a closer relationship is detected. The experiment result demonstrates that our proposal enables rapid case retrieval with promising accuracy by comparing a fewer number of past cases. Thus, the retrieval efficiency of our proposal outperforms typical approaches