Who is the director of this movie? Automatic style recognition based on shot features

We show how low-level formal features, such as shot duration, meant as length of camera takes, and shot scale, i.e. the distance between the camera and the subject, are distinctive of a director's style in art movies. So far such features were thought of not having enough varieties to become distinctive of an author. However our investigation on the full filmographies of six different authors (Scorsese, Godard, Tarr, Fellini, Antonioni, and Bergman) for a total number of 120 movies analysed second by second, confirms that these shot-related features do not appear as random patterns in movies from the same director. For feature extraction we adopt methods based on both conventional and deep learning techniques. Our findings suggest that feature sequential patterns, i.e. how features evolve in time, are at least as important as the related feature distributions. To the best of our knowledge this is the first study dealing with automatic attribution of movie authorship, which opens up interesting lines of cross-disciplinary research on the impact of style on the aesthetic and emotional effects on the viewers

Comparing Neural and Attractiveness-based Visual Features for Artwork Recommendation

Advances in image processing and computer vision in the latest years have brought about the use of visual features in artwork recommendation. Recent works have shown that visual features obtained from pre-trained deep neural networks (DNNs) perform very well for recommending digital art. Other recent works have shown that explicit visual features (EVF) based on attractiveness can perform well in preference prediction tasks, but no previous work has compared DNN features versus specific attractiveness-based visual features (e.g. brightness, texture) in terms of recommendation performance. In this work, we study and compare the performance of DNN and EVF features for the purpose of physical artwork recommendation using transactional data from UGallery, an online store of physical paintings. In addition, we perform an exploratory analysis to understand if DNN embedded features have some relation with certain EVF. Our results show that DNN features outperform EVF, that certain EVF features are more suited for physical artwork recommendation and, finally, we show evidence that certain neurons in the DNN might be partially encoding visual features such as brightness, providing an opportunity for explaining recommendations based on visual neural models.Comment: DLRS 2017 workshop, co-located at RecSys 201

Numerical analysis of a fin-tube plate heat exchanger with winglets

In this presented work, numerical analysis of heat transfer and flow characteristic using longitudinal vortex generators (LVGS) in fin and flat tube heat exchanger has been presented. Conjugate heat transfer 3D numerical model has been developed and successfully carried out. Rectangular winglets were set in pairs, with downstream orientation. The effects of impact angles of (20⁰ , 30⁰, and 40⁰ ) as well as tubes and winglets were placed in one row lined arrangement and air flow by forward arrangement and backward arrangement. Reynolds number is ranged from 500 to 5000. The numerical results showed that in the range of the present study, the variation of these parameters can result in the increase of heat transfer. The study focuses on the Influence of the different parameters of VGs on heat transfer and fluid flow characteristics of one row lined circular-tube banks. The characteristics of average Nu number and skin friction coefficient are studied numerically by the aid of the computational fluid dynamics (CFD) commercial code of FLUENT ANSYS 14. The results showed increasing in the heat transfer and skin friction coefficient with the increasing of Re number. It has been observed that the overall Nuav number of one circular tubes increases by 23-31% ,by 23-43% and by 23-47% with angles of (20⁰, 30°, and 40⁰) respectively, in forward arrangement and the overall Nuav number of one circular tubes increases by 23-42%, by 23-46% and 23-52%with angles of (20⁰, 30°, and 40⁰) respectively, in backward arrangement, with increasing in the overall average of skin friction coefficient. Also the results showed that the rectangular winglet pairs (RWPs) can significantly improve the heat transfer performance of the fin and-tube heat exchangers with a moderate pressure loss penalty

Novel Methods Using Human Emotion and Visual Features for Recommending Movies

Postponed access: the file will be accessible after 2022-06-01This master thesis investigates novel methods using human emotion as contextual information to estimate and elicit ratings when watching movie trailers. The aim is to acquire user preferences without the intrusive and time-consuming behavior of Explicit Feedback strategies, and generate quality recommendations. The proposed preference-elicitation technique is implemented as an Emotion-based Filtering technique (EF) to generate recommendations, and is evaluated against two other recommendation techniques. One Visual-based Filtering technique, using low-level visual features of movies, and one Collaborative Filtering (CF) using explicit ratings. In terms of \textit{Accuracy}, we found the Emotion-based Filtering technique (EF) to perform better than the two other filtering techniques. In terms of \textit{Diversity}, the Visual-based Filtering (VF) performed best. We further analyse the obtained data to see if movie genres tend to induce specific emotions, and the potential correlation between emotional responses of users and visual features of movie trailers. When investigating emotional responses, we found that \textit{joy} and \textit{disgust} tend to be more prominent in movie genres than other emotions. Our findings also suggest potential correlations on a per movie level. The proposed Visual-based Filtering technique can be adopted as an Implicit Feedback strategy to obtain user preferences. For future work, we will extend the experiment with more participants and build stronger affective profiles to be studied when recommending movies.Masteroppgave i informasjonsvitenskapINFO390MASV-INF

Leveraging the multimodal information from video content for video recommendation

Since the popularisation of media streaming, a number of video streaming services are continually buying new video content to mine the potential profit. As such, newly added content has to be handled appropriately to be recommended to suitable users. In this dissertation, the new item cold-start problem is addressed by exploring the potential of various deep learning features to provide video recommendations. The deep learning features investigated include features that capture the visual-appearance, as well as audio and motion information from video content. Different fusion methods are also explored to evaluate how well these feature modalities can be combined to fully exploit the complementary information captured by them. Experiments on a real-world video dataset for movie recommendations show that deep learning features outperform hand crafted features. In particular, it is found that recommendations generated with deep learning audio features and action-centric deep learning features are superior to Mel-frequency cepstral coefficients (MFCC) and state-of-the-art improved dense trajectory (iDT) features. It was also found that the combination of various deep learning features with textual metadata and hand-crafted features provide significant improvement in recommendations, as compared to combining only deep learning and hand-crafted features.Dissertation (MEng (Computer Engineering))--University of Pretoria, 2021.The MultiChoice Research Chair of Machine Learning at the University of PretoriaUP Postgraduate Masters Research bursaryElectrical, Electronic and Computer EngineeringMEng (Computer Engineering)Unrestricte

Adversarial Training Towards Robust Multimedia Recommender System

With the prevalence of multimedia content on the Web, developing recommender solutions that can effectively leverage the rich signal in multimedia data is in urgent need. Owing to the success of deep neural networks in representation learning, recent advance on multimedia recommendation has largely focused on exploring deep learning methods to improve the recommendation accuracy. To date, however, there has been little effort to investigate the robustness of multimedia representation and its impact on the performance of multimedia recommendation. In this paper, we shed light on the robustness of multimedia recommender system. Using the state-of-the-art recommendation framework and deep image features, we demonstrate that the overall system is not robust, such that a small (but purposeful) perturbation on the input image will severely decrease the recommendation accuracy. This implies the possible weakness of multimedia recommender system in predicting user preference, and more importantly, the potential of improvement by enhancing its robustness. To this end, we propose a novel solution named Adversarial Multimedia Recommendation (AMR), which can lead to a more robust multimedia recommender model by using adversarial learning. The idea is to train the model to defend an adversary, which adds perturbations to the target image with the purpose of decreasing the model's accuracy. We conduct experiments on two representative multimedia recommendation tasks, namely, image recommendation and visually-aware product recommendation. Extensive results verify the positive effect of adversarial learning and demonstrate the effectiveness of our AMR method. Source codes are available in https://github.com/duxy-me/AMR.Comment: TKD

Exploring the Semantic Gap for Movie Recommendations

In the last years, there has been much attention given to the semantic gap problem in multimedia retrieval systems. Much effort has been devoted to bridge this gap by building tools for the extraction of high-level, semantics-based features from multimedia content, as low-level features are not considered useful because they deal primarily with representing the perceived content rather than the semantics of it. In this paper, we explore a different point of view by leveraging the gap between low-level and high-level features. We experiment with a recent approach for movie recommendation that extract low-level Mise-en-Scéne features from multimedia content and combine it with high-level features provided by the wisdom of the crowd. To this end, we first performed an offline performance assessment by implementing a pure content-based recommender system with three different versions of the same algorithm, respectively based on (i) conventional movie attributes, (ii) mise-en-scene features, and (iii) a hybrid method that interleaves recommendations based on movie attributes and mise-en-scene features. In a second study, we designed an empirical study involving 100 subjects and collected data regarding the quality perceived by the users. Results from both studies show that the introduction of mise-en-scéne features in conjunction with traditional movie attributes improves both offline and online quality of recommendations