Deep Multi-View Learning for Visual Understanding

PhD ThesisMulti-view data is the result of an entity being perceived or represented from multiple perspectives. Plenty of applications in visual understanding contain multi-view data. For example, the face images for training a recognition system are usually captured by different devices from multiple angles. This thesis focuses on the cross-view visual recognition problems, e.g., identifying the face images of the same person across different cameras. Several representative multi-view settings, from the supervised multi-view learning to the more challenging unsupervised domain adaptive (UDA) multi-view learning, are investigated. Novel multi-view learning algorithms are proposed correspondingly. To be more specific, the proposed methods are based on the advanced deep neural network (DNN) architectures for better handling visual data. However, directly combining the multi-view learning objectives with DNN can result in different issues, e.g., on scalability, and limit the application scenarios and model performance. Corresponding novelties in DNN methods are thus required to solve them. This thesis is organised into three parts. Each chapter focuses on a multi-view learning setting with novel solutions and is detailed as follows: Chapter 3 A supervised multi-view learning setting with two different views are studied. To recognise the data samples across views, one strategy is aligning them in a common feature space via correlation maximisation. It is also known as canonical correlation analysis (CCA). Deep CCA has been proposed for better performance with the non-linear projection via deep neural networks. Existing deep CCA models typically decorrelate the deep feature dimensions of each view before their Euclidean distances are minimised in the common space. This feature decorrelation is achieved by enforcing an exact decorrelation constraint which is computationally expensive due to the matrix inversion or SVD operations. Therefore, existing deep CCA models are inefficient and have scalability issues. Furthermore, the exact decorrelation is incompatible with the gradient based deep model training and results in sub-optimal solution. To overcome these aforementioned issues, a novel deep CCA model Soft CCA is introduced in this thesis. Specifically, the exact decorrelation is replaced by soft decorrelation via a mini-batch based Stochastic Decorrelation Loss (SDL). It can be jointly optimised with the other training objectives. In addition, our SDL loss can be applied to other deep models beyond multi-view learning. Chapter 4 The supervised multi-view learning setting, whereby more than two views exist, are studied in this chapter. Recently developed deep multi-view learning algorithms either learn a latent visual representation based on a single semantic level and/or require laborious human annotation of these factors as attributes. A novel deep neural network architecture, called Multi- Level Factorisation Net (MLFN), is proposed to automatically factorise the visual appearance into latent discriminative factors at multiple semantic levels without manual annotation. The main purpose is forcing different views share the same latent factors so that they are can be aligned at all layers. Specifically, MLFN is composed of multiple stacked blocks. Each block contains multiple factor modules to model latent factors at a specific level, and factor selection modules that dynamically select the factor modules to interpret the content of each input image. The outputs of the factor selection modules also provide a compact latent factor descriptor that is complementary to the conventional deeply learned feature, and they can be fused efficiently. The effectiveness of the proposed MLFN is demonstrated by not only the large-scale cross-view recognition problems but also the general object categorisation tasks. Chapter 5 The last problem is a special unsupervised domain adaptation setting called unsupervised domain adaptive (UDA) multi-view learning. It contains a fully annotated dataset as the source domain and another unsupervised dataset with relevant tasks as the target domain. The main purpose is to improve the performance of the unlabelled dataset with the annotated data from the other dataset. More importantly, this setting further requires both the source and target domains are multi-view datasets with relevant tasks. Therefore, the assumption of the aligned label space across domains is inappropriate in the UDA multi-view learning. For example, the person re-identification (Re-ID) datasets built on different surveillance scenarios are with images of different people captured and should be given disjoint person identity labels. Existing methods for UDA multi-view learning problems are aligning different domains either in the raw image space or a feature embedding space for domain alignment. In this thesis, a different framework, multi-task learning, is adopted with the domain specific objectives for a common space learning. Specifically, such common space is proposed to enable the knowledge transfer. The conventional supervised losses can be used for the labelled source data while the unsupervised objectives for the target domain play the key roles in domain adaptation. Two novel unsupervised objectives are introduced for UDA multi-view learning and result in two models as below. The first model, termed common factorised space model (CFSM), is built on the assumptions that the semantic latent attributes are shared between the source and target domains since they are relevant multi-view learning tasks. Different from the existing methods that based on domain alignment, CFSM emphasizes on transferring the information across domains via discovering discriminative latent factors in the proposed common space. However, the multi-view data from target domain is without labels. Therefore, an unsupervised factorisation loss is derived and applied on the common space for latent factors discovery across domains. The second model still learns a shared embedding space with multi-view data from both domains but with a different assumption. It attempts to discover the latent correspondence of multi-view data in the unsupervised target data. The target data’s contribution comes from a clustering process. Each cluster thus reveals the underlying cross-view correspondences across multiple views in target domain. To this end, a novel Stochastic Inference for Deep Clustering (SIDC) method is proposed. It reduces self-reinforcing errors that lead to premature convergence to a sub-optimal solution by changing the conventional deterministic cluster assignment to a stochastic one

Investigating sensory-motor interactions to shape rehabilitation

Over the last decades, robotic devices for neurorehabilitation have been developed with the aim of providing better and faster improvement of motor performance. These devices are being used to help patients repeat movements and (re)learn different dynamic tasks. Over the years, these devices have become bigger and more complex, so as to provide the end user with a more realistic and sophisticated stimuli while still allowing the experimenter to have control over the interaction forces that can potentially shape the motor behaviour. However, experimental results have shown no clear advantage of these complex devices over simpler versions. In this context, this thesis investigates sensory-motor processes of human interaction, which can help us understand the main issues for rehabilitation devices and how to overcome the limitations of simple devices to train particular motor behaviours. Conventional neurorehabilitation of motor function relies on haptic interaction between the patient and physiotherapist. However, how humans deal with human-human interactions is largely unknown, and has been little studied. In this regard, experiments of the first section of the thesis investigate the mechanisms of interaction during human-human collaborative tasks. It goes from identifying the different strategies that dyads can take to proposing methods to measure and understand redundancy and synchrony in haptic interactions. It also shows that one can shape the interaction between partners by modifying only the visual information provided to each agent. Learning a novel skill requires integration of different sensory modalities, in particular vision and proprioception. Hence, one can expect that learning will depend on the mechanical characteristics of the device. For instance, a device with limited degrees of freedom will reduce the amount of information about the environment, modify the dynamics of the task and prevent certain error-based corrections. To investigate this, the second section of the thesis examines whether the lack of proprioceptive feedback that is created due to mechanical constraints or haptic guidance can be substituted with visual information. Psychophysical experiments with healthy subjects and some preliminary experiments with stroke patients presented in this thesis support the idea that by incorporating task-relevant visual feedback into simple devices, one could deliver effective neurorehabilitation protocols. The contributions of the thesis are not limited to the role of visual feedback to shape motor behaviour, but also advance our understanding on the mechanisms of learning and human-human interaction

Drones and the Creative Industry

This open access, interdisciplinary book presents innovative strategies in the use of civil drones in the cultural and creative industry. Specially aimed at small and medium-sized enterprises (SMEs), the book offers valuable insights from the fields of marketing, engineering, arts and management. With contributions from experts representing varied interests throughout the creative industry, including academic researchers, software developers and engineers, it analyzes the needs of the creative industry when using civil drones both outdoors and indoors. The book also provides timely recommendations to the industry, as well as guidance for academics and policymakers

Exploring Alternative Models of Localisation in Food Supply Chains: A Theory of Constraints Approach

Local food and the localisation of food are beset by many problems in the UK. We have still yet to agree on a consensus view of the term ‘local food’ despite the call for an enforceable definition. The continued absence of rules around products and their relative spatial determinacy has lead to the development of both fluid, and subjective interpretations around the term ‘local’, as well as a willingness by key actors to readily conflate ‘local’ with ‘regional’ as a pluralistic device in a market worth £4.6 billion in sales from farm shops and farmers’ markets alone. This research sets out to identify and diffuse the problems we have in defining what local food is, and presciently, what it may become. The research itself utilises a qualitative multiple case study approach, engaging with a final cohort of 23 producers of similar products, but at different scales of supply, and across a broad geographic spread of England. In encompassing areas which do not have a reputation for local food, the research mitigates against previous micro-analytical research and adds both construct and internal validity to its data, gathered by semi structured interviews, process mapping and questionnaires. Template analysis is used as a data extraction tool in this research, which seeks to provide disambiguation around the sector and suggest a way forward which has the potential to offer greater derived benefit to current and future stakeholders

Systems of State-Owned Enterprises: from Public Entrepreneurship to State Shareholding

This thesis outlines a new analytical perspective on state ownership through the original concept of systems of state-owned enterprises (SOSOEs). It is argued that the SOSOEs concept adequately captures the evolution of state-owned enterprises (SOEs) in modern capitalist economies, challenging and enriching existing economic theories as well as contributing to reinstate the policy instrumentality of state ownership. The concept is defined from a comparative case study analysis of two distinct SOSOEs, operating within the same national context in different time periods. The first case concerns the Istituto per la Ricostruzione Industriale (IRI), Italy’s former and most relevant state holding company, that played a central role in the Country’s post-WWII economic development. This thesis advances an interpretation of IRI’s economic function based on an original empirical investigation of its archival and documentary sources, focusing on its main public policy missions and on its display of industrial entrepreneurship features. The second case examines the current Italian system of SOEs, assessing the still relevant presence of SOEs in the Italian national context and evaluating the overall governance of the system through a set of interviews with leading executives. Despite the similarity in size and sectoral diversification, the two SOSOEs differ significantly in terms of their operating configurations. In fact, they could be assimilated to two dichotomous ideal types: the IRI SOSOEs represents a template for the policy-oriented and dynamic ‘public entrepreneurship’ model, while the current Italian SOSOEs resembles the policy-neutral and passive ‘state shareholding’ variant. Implicit in these results is the opportunity for current SOSOEs to embrace a public entrepreneurship configuration, in order to exploit the full policy potential of state ownership in driving economic change. The thesis concludes with a proposal for reforming Italy’s current SOSOEs via the creation of a state holding company

Music and Sound in Post-1989 Taiwan Cinema

Although film music research has been on the rise over the last decade, most research has focused on the Hollywood tradition. An increasing number of projects focusing on film-scoring traditions other than Hollywood are nevertheless beginning to reveal the richness of localised traditions and increase our understanding of the purpose of film music elsewhere. This thesis is a study of music in Taiwan cinema, focusing on the period 1989-2009. After the 1987 lifting of martial law and the 1989 release of HOU Hsiao-hsien's celebrated A City of Sadness, restrictions on cultural production in Taiwan were relaxed and cash from other Chinese-speaking communities as well as European and American companies began to be invested in Taiwan cinema. This influx of foreign capital has combined with the cultures of multiple colonisers to create the heterogeneous approach to filmmaking and the hybrid musical cultures found in Taiwan today. How do all these changes affect our understanding of the relationship between music and moving images in Taiwan cinema? How has music in Taiwan cinema reflected and responded to changes to its internal and external environments? I shall examine these questions from three perspectives. In Part One I briefly summarise the country's cultural history in order to flesh out an argument about the environment in which film musicians were working. I also define important conceptual terms such as Taiwan cinema, Chinese-language cinema, Cultural China, transnational cooperation, and so on. In Part Two I investigate the influences of Chinese aesthetics on music in Taiwan cinema, particularly on HOU Hsiao-hsien's use of silence. In doing so, I suggest that the changing philosophy of silence in HOU's films reflects his response to political and cultural currents over the past two decades. In Part Three I examine music in martial arts films, particularly in Ang 3 LEE's Crouching Tiger, Hidden Dragon (2000). As an important genre in China and the first to gain popularity in the West, martial arts films have long been subject to transnational exchange; to study the role of music and musicality in martial arts films is therefore to gain a useful perspective on the shifting forces that have influenced Taiwan cinema and Chinese-language cinema over several decades. The success of Crouching Tiger has given rise to more frequent transnational exchanges in Taiwan cinema than before. In Part Four, I will examine the music in two more recent films, Cape No. 7 (2008) and Secret (2007), to examine film music's ability to reflect the struggles in today's Taiwanese society to construct its own cultural identity. 4EThOS - Electronic Theses Online ServiceGBUnited Kingdo

The Provocative Joan Robinson

One of the most original and prolific economists of the twentieth century, Joan Robinson (1903–83) is widely regarded as the most important woman in the history of economic thought. Robinson studied economics at Cambridge University, where she made a career that lasted some fifty years. She was an unlikely candidate for success at Cambridge. A young woman in 1930 in a university dominated by men, she succeeded despite not having a remarkable academic record, a college fellowship, significant publications, or a powerful patron. In The Provocative Joan Robinson, Nahid Aslanbeigui and Guy Oakes trace the strategies and tactics Robinson used to create her professional identity as a Cambridge economist in the 1930s, examining how she recruited mentors and advocates, carefully defined her objectives, and deftly pursued and exploited opportunities

Defining the role of mast cells in guinea pig models of asthma

Asthma is a common respiratory disease characterized by several pathophysiological features, such as allergen induced bronchoconstriction (in allergic asthma), airway hyperresponsiveness (AHR), airway inflammation, airway remodeling and mast cell hyperplasia. An increase of mast cells has been found in asthma patients. However, how these cells are involved in the development of asthma are not well defined. To investigate the role of mast cells in the pathophysiological characteristics of asthma, we established asthma models in guinea pigs, which have many similarities with humans, by exposing the animals to human relevant allergens: house dust mite (HDM) and cat dander extract (CDE). The involvement of mast cells in asthma-like features was investigated either by the addition of mast cell mediator antagonists or inhibitors, or inducing mast cell death. In paper I, we repeatedly exposed guinea pigs to HDM via intranasal instillation for seven weeks and successfully recaptured the antigen induced bronchoconstriction, the production of HDM specific immunoglobulins, AHR, eosinophilic inflammation with an increase of IL-13, airway remodeling (e.g., subepithelial collagen deposition and goblet cell hyperplasia) and mast cell hyperplasia. This model can be further used to study the role of mast cells in asthma. In paper II, we exposed guinea pigs to HDM or CDE intranasally for different time. Both HDM and CDE induce airway inflammation and airway remodeling after 4 weeks’ antigen exposures. These increases maintained after 8- and 12-week exposures. Exposing to both antigens for 8 weeks and 12 weeks induced a clear expansion of mast cells which is predominated by mast cells expressing tryptase. An increase of mast cells expressing both tryptase and chymase were also observed. In paper III, we isolated guinea pig trachea for comparing the effect of different mast cell agonists (HDM and Compound 48/80 (C48/80)) on airway smooth muscle responses and mediator release. We found that histamine, prostaglandins and 5- lipoxygenase products mediated the bronchoconstriction induced by HDM and C48/80. Both agonists induced a release of histamine, prostaglandin D2 and leukotriene B4. However, distinct of lipid mediator profiles were observed. The leukotriene E4 was only elevated by HDM, whereas C48/80 induced a broader release of lipid mediators. In paper IV and V, we identified an antibiotic monensin that can induce mast cell death. To examine if monensin can be a tool for investigating the role of mast cells in asthma, we cultured guinea pig tracheal segments from HDM sensitized guinea pigs and human bronchi with different concentrations of monensin for different time. We found that monensin has robust effects on causing mast cell death and totally blocked the HDM (in guinea pig trachea) and anti-IgE (in human bronchi) induced bronchoconstriction after 2 to 72h exposure without affecting the general tissue viability at low concentration. In the in vivo investigations, we exposed the guinea pigs to HDM repeatedly with or without monensin interventions. Monensin reduced the AHR, airway inflammation and mast cell hyperplasia in the HDM induced guinea pig model. In conclusion, exposing to human relevant allergens (HDM and CDE) are suitable for modeling of allergic asthma in guinea pigs. The increase of mast cells by HDM and CDE helps to investigate the role of mast cells in asthma models. Mast cells in guinea pig airways can respond differently to antigen and non-antigen agonists. Monensin can be a robust tool to induce mast cell death. The antigen induced bronchoconstriction by HDM in guinea pig trachea and anti-IgE in human bronchi are purely mast cell mediated. Our findings emphasize that mast cells have important roles in the development of AHR and airway inflammation in the guinea pig model used in this PhD study. The findings in this thesis highlight the importance of mast cells in asthma and the models we developed can be used as important tools for defining the mechanisms behind asthma