Semantic multimedia modelling & interpretation for search & retrieval

With the axiomatic revolutionary in the multimedia equip devices, culminated in the proverbial proliferation of the image and video data. Owing to this omnipresence and progression, these data become the part of our daily life. This devastating data production rate accompanies with a predicament of surpassing our potentials for acquiring this data. Perhaps one of the utmost prevailing problems of this digital era is an information plethora. Until now, progressions in image and video retrieval research reached restrained success owed to its interpretation of an image and video in terms of primitive features. Humans generally access multimedia assets in terms of semantic concepts. The retrieval of digital images and videos is impeded by the semantic gap. The semantic gap is the discrepancy between a user’s high-level interpretation of an image and the information that can be extracted from an image’s physical properties. Content- based image and video retrieval systems are explicitly assailable to the semantic gap due to their dependence on low-level visual features for describing image and content. The semantic gap can be narrowed by including high-level features. High-level descriptions of images and videos are more proficient of apprehending the semantic meaning of image and video content. It is generally understood that the problem of image and video retrieval is still far from being solved. This thesis proposes an approach for intelligent multimedia semantic extraction for search and retrieval. This thesis intends to bridge the gap between the visual features and semantics. This thesis proposes a Semantic query Interpreter for the images and the videos. The proposed Semantic Query Interpreter will select the pertinent terms from the user query and analyse it lexically and semantically. The proposed SQI reduces the semantic as well as the vocabulary gap between the users and the machine. This thesis also explored a novel ranking strategy for image search and retrieval. SemRank is the novel system that will incorporate the Semantic Intensity (SI) in exploring the semantic relevancy between the user query and the available data. The novel Semantic Intensity captures the concept dominancy factor of an image. As we are aware of the fact that the image is the combination of various concepts and among the list of concepts some of them are more dominant then the other. The SemRank will rank the retrieved images on the basis of Semantic Intensity. The investigations are made on the LabelMe image and LabelMe video dataset. Experiments show that the proposed approach is successful in bridging the semantic gap. The experiments reveal that our proposed system outperforms the traditional image retrieval systems

Does the way museum staff define inspiration help them work with information from visitors' Social Media?

Since the early 2000s, Social Media has become part of the everyday activity of billions of people. Museums and galleries are part of this major cultural change - the largest museums attract millions of Social Media 'friends' and 'followers', and museums now use Social Media channels for marketing and audience engagement activities. Social Media has also become a more heavily-used source of data with which to investigate human behaviour. Therefore, this research investigated the potential uses of Social Media information to aid activities such as exhibition planning and development, or fundraising, in museums. Potential opportunities provided by the new Social Media platforms include the ability to capture data at high volume and then analyse them computationally. For instance, the links between entities on a Social Media platform can be analysed. Who follows who? Who created the content related to a specific event, and when? How did communication flow between people and organisations? The computerised analysis techniques used to answer such questions can generate statistics for measuring concepts such as the 'reach' of a message across a network (often equated simply with the potential size of the a message's audience) or the degree of 'engagement' with content (often a simple count of the number of responses, or the number of instances of communication between correspondents). Other computational analysis opportunities related to Social Media rely upon various Natural Language Processing (NLP) techniques; for example indexing content and counting term frequency, or using lexicons or online knowledge bases to relate content to concepts. Museums, galleries and other cultural organisations have known for some time, however, that simple quantifications of their audiences (the number of tickets sold for an exhibition, for example), while certainly providing indications of an event's success, do not tell the whole story. While it is important to know that thousands of people have visited an exhibition, it is also part of a museum's remit to inspire the audience, too. A budding world-class artist or ground-breaking engineer could have been one of the thousands in attendance, and the exhibition in question could have been key to the development of their artistic or technical ideas. It is potentially helpful to museums and galleries to know when they have inspired members of their audience, and to be able to tell convincing stories about instances of inspiration, if their full value to society is to be judged. This research, undertaken in participation with two museums, investigated the feasibility of using new data sources from Social Media to capture potential expressions of inspiration made by visitors. With a background in IT systems development, the researcher developed three prototype systems during three cycles of Action Research, and used them to collect and analyse data from the Twitter Social Media platform. This work had two outcomes: firstly, prototyping enabled investigation of the technical constraints of extracting data from a Social Media platform (Twitter), and the computing processes used to analyse that data. Secondly, and more importantly, the prototypes were used to assess potential changes to the work of museum staff information about events visited and experienced by visitors was synthesised, then investigated, discussed and evaluated with the collaborative partners, in order to assess the meaning and value of such information for them. Could the museums use the information in their event and exhibition planning? How might it fit in with event evaluation? Was it clear to the museum what the information meant? What were the risks of misinterpretation? The research made several contributions. Firstly, the research developed a definition of inspiration that resonated with museum staff. While this definition was similar to the definition of 'engagement' from the marketing literature, one difference was an emphasis upon creativity. The second set of contributions related to a deeper understanding of Social Media from museums' perspective, and included findings about how Social Media information could be used to segment current and potential audiences by 'special interest', and find potential expressions of creativity and innovation in the audience's responses to museum activities. These findings also considered some of the pitfalls of working with data from Social Media, in particular the tendency of museum staff to use the information to confirm positive biases, and the often hidden biases caused by the mediating effects of the platforms from which the data came. The final major contribution was a holistic analysis of the ways in which Social Media information could be integrated into the work of a museum, by helping to plan and evaluate audience development and engagement. This aspect of the research also highlighted some of the dangers of an over-dependency upon individual Social Media platforms which was previously absent from the museums literature

Natural Language Processing: Emerging Neural Approaches and Applications

This Special Issue highlights the most recent research being carried out in the NLP field to discuss relative open issues, with a particular focus on both emerging approaches for language learning, understanding, production, and grounding interactively or autonomously from data in cognitive and neural systems, as well as on their potential or real applications in different domains

Combining SOA and BPM Technologies for Cross-System Process Automation

This paper summarizes the results of an industry case study that introduced a cross-system business process automation solution based on a combination of SOA and BPM standard technologies (i.e., BPMN, BPEL, WSDL). Besides discussing major weaknesses of the existing, custom-built, solution and comparing them against experiences with the developed prototype, the paper presents a course of action for transforming the current solution into the proposed solution. This includes a general approach, consisting of four distinct steps, as well as specific action items that are to be performed for every step. The discussion also covers language and tool support and challenges arising from the transformation