Prompting Visual-Language Models for Dynamic Facial Expression Recognition

This paper presents a novel visual-language model called DFER-CLIP, which is based on the CLIP model and designed for in-the-wild Dynamic Facial Expression Recognition (DFER). Specifically, the proposed DFER-CLIP consists of a visual part and a textual part. For the visual part, based on the CLIP image encoder, a temporal model consisting of several Transformer encoders is introduced for extracting temporal facial expression features, and the final feature embedding is obtained as a learnable "class" token. For the textual part, we use as inputs textual descriptions of the facial behaviour that is related to the classes (facial expressions) that we are interested in recognising – those descriptions are generated using large language models, like ChatGPT. This, in contrast to works that use only the class names and more accurately captures the relationship between them. Alongside the textual description, we introduce a learnable token which helps the model learn relevant context information for each expression during training. Extensive experiments demonstrate the effectiveness of the proposed method and show that our DFER-CLIP also achieves state-of-the-art results compared with the current supervised DFER methods on the DFEW, FERV39k, and MAFW benchmarks

Machine vision applications in agriculture

Keynote paper. [Abstract]: With the trend of computers towards convergence with multimedia entertainment, tools for vision processing are becoming commonplace. This has led to the pursuit of a host of unusual applications in the National Centre for Engineering in Agriculture, in addition to work on vision guidance. These range from the identification of animal species, through the location of macadamia nuts as they are harvested and visual tracking for behaviour analysis of small marsupials to the measurement of the volume of dingo teeth

Meat color recognition using machine vision

New technologies are being developed to give an ease to the human in a variety of different field each and every day. Food industry is the key of development that led to the rise of human civilization. The development of food industry dealt with the husbandry of domesticated animal and plants creating food surpluses that enabled the development of more densely populated and stratified societies. The study of food is very important that improves the quality of human's life. When it comes to classify and grade a meat, the color of fresh meat is a sensory indicator of which affects the consumers behavior, especially the consistency of meat color and musculature. Other factors that influence consumers purchasing include security, nutrition and taste. There has been no report that grades the meat freshness in the process of meat delivery. Most of the meat freshness is grading manually by using the human eyesight at the meat's color and quantity of fats. A parameter to show the freshness of meat has only been analyzed manually using a human's eyes. This is some kind of difficult method when making a right decision whether the meat is fresh or not. In order to overcome this problem, meat grading method has been studied to show the mathematical calculation on the change of color hue, saturation, and intensity (HSI) values. This study focuses on grading system design that helps to characterize the meat freshness according to its color. Using a MATLAB Graphical User Interface (GUI) program, it can analyzes the color of the meat that being inspected. The theory of this program includes the calculation of the mean values and histograms, and the final result. This system is capable of classifying meat freshness

Development of Moire machine vision

Three dimensional perception is essential to the development of versatile robotics systems in order to handle complex manufacturing tasks in future factories and in providing high accuracy measurements needed in flexible manufacturing and quality control. A program is described which will develop the potential of Moire techniques to provide this capability in vision systems and automated measurements, and demonstrate artificial intelligence (AI) techniques to take advantage of the strengths of Moire sensing. Moire techniques provide a means of optically manipulating the complex visual data in a three dimensional scene into a form which can be easily and quickly analyzed by computers. This type of optical data manipulation provides high productivity through integrated automation, producing a high quality product while reducing computer and mechanical manipulation requirements and thereby the cost and time of production. This nondestructive evaluation is developed to be able to make full field range measurement and three dimensional scene analysis