AMNet: Memorability Estimation with Attention

In this paper we present the design and evaluation of an end-to-end trainable, deep neural network with a visual attention mechanism for memorability estimation in still images. We analyze the suitability of transfer learning of deep models from image classification to the memorability task. Further on we study the impact of the attention mechanism on the memorability estimation and evaluate our network on the SUN Memorability and the LaMem datasets. Our network outperforms the existing state of the art models on both datasets in terms of the Spearman's rank correlation as well as the mean squared error, closely matching human consistency

ResMem-Net: memory based deep CNN for image memorability estimation

Image memorability is a very hard problem in image processing due to its subjective nature. But due to the introduction of Deep Learning and the large availability of data and GPUs, great strides have been made in predicting the memorability of an image. In this paper, we propose a novel deep learning architecture called ResMem-Net that is a hybrid of LSTM and CNN that uses information from the hidden layers of the CNN to compute the memorability score of an image. The intermediate layers are important for predicting the output because they contain information about the intrinsic properties of the image. The proposed architecture automatically learns visual emotions and saliency, shown by the heatmaps generated using the GradRAM technique. We have also used the heatmaps and results to analyze and answer one of the most important questions in image memorability: ‘‘What makes an image memorable?“. The model is trained and evaluated using the publicly available Large-scale Image Memorability dataset (LaMem) from MIT. The results show that the model achieves a rank correlation of 0.679 and a mean squared error of 0.011, which is better than the current state-of-the-art models and is close to human consistency (p = 0.68). The proposed architecture also has a significantly low number of parameters compared to the state-of-the-art architecture, making it memory efficient and suitable for production

Predicting and Modifying Memorability of Images

Everyday, we are bombarded with many photographs of faces, whether on social media, television, or smartphones. From an evolutionary perspective, faces are intended to be remembered, mainly due to survival and personal relevance. However, all these faces do not have the equal opportunity to stick in our minds. It has been shown that memorability is an intrinsic feature of an image but yet, it is largely unknown what attributes make an image more memorable. In this work, we first proposed new models for predicting memorability of face and object images. Subsequently, we proposed a fast approach to modify and control the memorability of face images. In our proposed method, we first found a hyperplane in the latent space of StyleGAN to separate high and low memorable images. We then modified the image memorability (while maintaining the identity and other facial features such as age, emotion, etc.) by moving in the positive or negative direction of this hyperplane normal vector. We further analyzed how different layers of the StyleGAN augmented latent space contribute to face memorability. These analyses showed how each individual face attribute makes an image more or less memorable. Most importantly, we evaluated our proposed method for both real and unreal (generated) face images. The proposed method successfully modifies and controls the memorability of real human faces as well as unreal (generated) faces. Our proposed method can be employed in photograph editing applications for social media, learning aids, or advertisement purposes

Automatsko povećanje pamtljivosti slika

The dissertation considers the problem of automatic increase of image memorability. The problem-solving approach is based on editing-byapplying-filters paradigm. Given an arbitrary input image, the proposed deep learning model is able to automatically retrieve a set of “style seeds”, i.e., a set of style images which, applied to the input image through a neural style transfer algorithm, provide the highest increase in memorability. We show the effectiveness of the approach with experiments, performing both a quantitative evaluation and a user study.Дисертација разматра проблем аутоматског повећања памтљивости фотографије на основу модела дубоког учења. Овој проблематици се приступа са аспекта развоја иновативног приступа заснованог на парадигми уређивања слике применом филтера. Арбитрарна улазна слика аутоматски преузима сет стилских карактеристика који се преносе путем алгоритма неуронског стила, омогућавајући на овај начин пораст памтљивости целокупне слике. Ефикасност предложеног приступа евалуирана је експерименталнo уз изведбу корисничке студије.Disertacija razmatra problem automatskog povećanja pamtljivosti fotografije na osnovu modela dubokog učenja. Ovoj problematici se pristupa sa aspekta razvoja inovativnog pristupa zasnovanog na paradigmi uređivanja slike primenom filtera. Arbitrarna ulazna slika automatski preuzima set stilskih karakteristika koji se prenose putem algoritma neuronskog stila, omogućavajući na ovaj način porast pamtljivosti celokupne slike. Efikasnost predloženog pristupa evaluirana je eksperimentalno uz izvedbu korisničke studije

Biopsychosocial Assessment and Ergonomics Intervention for Sustainable Living: A Case Study on Flats

This study proposes an ergonomics-based approach for those who are living in small housings (known as flats) in Indonesia. With regard to human capability and limitation, this research shows how the basic needs of human beings are captured and analyzed, followed by proposed designs of facilities and standard living in small housings. Ninety samples were involved during the study through in- depth interview and face-to-face questionnaire. The results show that there were some proposed of modification of critical facilities (such as multifunction ironing work station, bed furniture, and clothesline) and validated through usability testing. Overall, it is hoped that the proposed designs will support biopsychosocial needs and sustainability

More than meets the eye: the conceptual essence of intrinsic memorability

In a world where sensory threads weave an endless tapestry of multi-modal data, the human brain stands as the masterful weaver of meaning. As we wade through this tempest of input, our brain spins these threads into an intelligible internal representation and holds on tight to what it deems important. But what, exactly, makes certain threads more important than others? And how can we predict their significance? Memorability is the tensile strength of the threads that tie us to the world. It is a proxy for human importance, indicating which threads the human brain will curate and retain with exceptional fidelity. This research investigates these multisensory threads by exploring the influence of audio, visual, and textual modalities on predicting video memorability, and how the interplay between them can influence the overall memorability of a given piece of content. The findings suggest that, while visual data may dominate our sensory experience, it is the underlying conceptual essence that truly holds the key to memorability. This thesis leverages state-of-the-art image synthesis techniques to distill and examine this essence, creating surrogate dreams of video scenes to facilitate the disentanglement of conceptual and perceptual elements of memorability. The work also leverages human EEG data to explore the possibility of a moment of memorability—a moment of encoding that corresponds to a remembering moment—which we expect to exist due to the temporal nature of the world and the natural encoding limits of our brains. The previously murky relationship between the two core means of remembrance---recognition and recall---are reconciled by conducting a novel video memorability drawing task. The research sheds new light on the nature of multi-modal memorability, providing a deeper understanding of how our brain processes and retains information in a complex sensory world. By uncovering the conceptual essence that lies at the heart of memorability, it opens up new avenues for predicting and curating more meaningful media content, and ultimately deepen our connection to the world around us

A psycho-ethological approach to social signal processing

The emerging field of social signal processing can benefit from a theoretical framework to guide future research activities. The present article aims at drawing attention to two areas of research that devoted considerable efforts to the understanding of social behaviour: ethology and social psychology. With a long tradition in the study of animal signals, ethology and evolutionary biology have developed theoretical concepts to account for the functional significance of signalling. For example, the consideration of divergent selective pressures responsible for the evolution of signalling and social cognition emphasized the importance of two classes of indicators: informative cues and communicative signals. Social psychology, on the other hand, investigates emotional expression and interpersonal relationships, with a focus on the mechanisms underlying the production and interpretation of social signals and cues. Based on the theoretical considerations developed in these two fields, we propose a model that integrates the processing of perceivable individual features (social signals and cues) with contextual information, and we suggest that output of computer-based processing systems should be derived in terms of functional significance rather than in terms of absolute conceptual meanin

Tourism experience co-creation: a service-dominant logic-driven model.

336 p.For some time now, new marketing paradigms are trying to set aside traditional and product-focused views of management and marketing strategies. Technological challenges, volatile customer needs, and alternative marketing application subjects and fields have pushed the discipline of marketing towards more adapted and generalizable marketing perspectives, leaving behind the long-established four Ps of marketing (Product, Place, Promotion, Price) and similar. These new paradigms emphasize especially the active role of customers in value creation, the inherent service-for-service nature of exchanges, the relevance of multisided resources, the formation of both planned and unintentional institutionally handled networks, and the interest on `experiences'. Among the propositions found, value co-creation is one of the most appealing and recurring. However, the lack of specification has transformed co-creation into an overused interpretation, a jumble of many different ideas (e.g., co-production, interactions, citizenship behaviors), and a concept at risk of turning into an "all and nothing"-type useless expression. Therefore, we need first, a conceptual underlying support that would lead the definition and implications of co-creation, and second, a practical backing that would demonstrate its empirical and contextual convenience.The objective of this study is to provide an empirical and context-driven approach of value co-creation, based on the service-dominant logic (SDL). In the conceptual concern, we believe that though in a meta-theoretical level, SDL provides a well-founded baseline framework for value co-creation, supplying a complete narrative and a number of premises centered on a comprehensive view of value co-creation. In the empirical concern, we chose place marketing in general and tourism in particular to situate our co-creation framework. Specifically, the research is focused on the tourism experience co-creation, which is thought to match the investigation requirements due to the experiential character of vacations, the multiple actors involved along the whole travel-related process, the main position of tourists in value creation, and the need of a marketing cut off from the traditional and commercial views in the tourism field.The study carries out an extensive and systematic literature review about value co-creation, SDL and similar concepts in place marketing, including urban, hospitality and destination environments. The literature review is then used, together with the SDL assumptions, to build an extended conceptual model of tourism experience co-creation, including antecedents and outcomes, represented by tourist and destination resources, and by functional, emotional, and social value dimensions, respectively. Value co-creation is defined as service exchange and resource integration represented by a set of tourist-driven processes, including interactional, behavioral, attitudinal, and mental processes before, during, and after the travel experience. Results show that tourist's specific travel-related knowledge and skills facilitate value co-creation processes, especially those connected with self-arrangement behaviors. At the same time, tourist co-creation processes are found to affect predominantly the emotional and social value dimensions, even more than destination resources. On the contrary, the latter positively affect the functional value of the experience. Among the co-creation processes, interactional and behavioral processes influence the tourism experience value perceived by the consumer. Concretely, interaction with local people is found to be one of the most influential. Mental co-creation processes deserve especial attention. Memorability is found to significantly and positively affect value, as well as representing a moderator role between other co-creative processes and value. This means that recalling the lived experience helps increasing the tourist's perceived value. On the contrary, attitudinal processes related to customer citizenship behaviors do not present any relevance. Therefore, the study contributes mostly in four aspects. First, an exhaustive literature review of place marketing co-creation allows acknowledging the efforts made previously on the field, in terms of identifying the different co-creation approaches, and registering the scarce variables used to precisely measure value co-creation (proxies are prevailing). Second, a verified measurement tool of value co-creation set up on a conceptually-based definition and composed of nine variables contributes to unfold the black box of "service exchange and resource integration". Third, the proposition and testing of various study hypotheses build on an expertise co-creation processes value chain helps identifying the most relevant tourist co-creation processes that increase perceived value, thus assisting destination and hospitality managers. Fourth, the two-phase data collection methodology used in the empirical part of the study provides a more rigorous research method that encourages alternative information gathering processes