Guiding Image Classifier Using a Neuro-fuzzy Controller

This disclosure describes a neuro-fuzzy controller that can be utilized to guide image classifier networks for classification of subjective attributes. Per techniques of this disclosure, linguistic expert rules for memberships of an image to various output categories of the subjective attribute(s) are framed and the classification is analyzed as a fuzzy system. Fuzzy rules and fuzzy inference output from this system are used to guide a neural network to effectively incorporate the expert rules. Specific loss functions are utilized to guide the image classifier. A fuzzy-rule contradiction loss is utilized to capture a weighted deviation of image classifier prediction from expert rules. A fuzzy inference loss is utilized to capture overall deviation from fuzzy inference output. Utilization of the neuro-fuzzy controller can enable image classifier models to classify images according to subjective attributes, e.g., to provide accurate labels for family friendliness of a restaurant based on images of the restaurant

Learning Subjective Attributes of Images from Auxiliary Sources

10.1145/3343031.3350574ACM MM 20192263-227

Learning subjective attributes of images from auxiliary sources

Recent years have seen unprecedented research on using artificial intelligence to understand the subjective attributes of images and videos. These attributes are not objective properties of the content but are highly dependent on the perception of the viewers. Subjective attributes are extremely valuable in many applications where images are tailored to the needs of a large group, which consists of many individuals with inherently different ideas and preferences. For instance, marketing experts choose images to establish specific associations in the consumers' minds, while psychologists look for pictures with adequate emotions for therapy. Unfortunately, most of the existing frameworks either focus on objective attributes or rely on large scale datasets of annotated images, making them costly and unable to clearly measure multiple interpretations of a single input. Meanwhile, we can see that users or organizations often interact with images in a multitude of real-life applications, such as the sharing of photographs by brands on social media or the re-posting of image microblogs by users. We argue that these aggregated interactions can serve as auxiliary information to infer image interpretations. To this end, we propose a probabilistic learning framework capable of transferring such subjective information to the image-level labels based on a known aggregated distribution. We use our framework to rank images by subjective attributes from the domain knowledge of social media marketing and personality psychology. Extensive studies and visualizations show that using auxiliary information is a viable line of research for the multimedia community to perform subjective attributes prediction