Toward Large Scale Semantic Image Understanding and Retrieval

Semantic image retrieval is a multifaceted, highly complex problem. Not only does the solution to this problem require advanced image processing and computer vision techniques, but it also requires knowledge beyond what can be inferred from the image content alone. In contrast, traditional image retrieval systems are based upon keyword searches on filenames or metadata tags, e.g. Google image search, Flickr search, etc. These conventional systems do not analyze the image content and their keywords are not guaranteed to represent the image. Thus, there is significant need for a semantic image retrieval system that can analyze and retrieve images based upon the content and relationships that exist in the real world.In this thesis, I present a framework that moves towards advancing semantic image retrieval in large scale datasets. At a conceptual level, semantic image retrieval requires the following steps: viewing an image, understanding the content of the image, indexing the important aspects of the image, connecting the image concepts to the real world, and finally retrieving the images based upon the index concepts or related concepts. My proposed framework addresses each of these components in my ultimate goal of improving image retrieval. The first task is the essential task of understanding the content of an image. Unfortunately, typically the only data used by a computer algorithm when analyzing images is the low-level pixel data. But, to achieve human level comprehension, a machine must overcome the semantic gap, or disparity that exists between the image data and human understanding. This translation of the low-level information into a high-level representation is an extremely difficult problem that requires more than the image pixel information. I describe my solution to this problem through the use of an online knowledge acquisition and storage system. This system utilizes the extensible, visual, and interactable properties of Scalable Vector Graphics (SVG) combined with online crowd sourcing tools to collect high level knowledge about visual content.I further describe the utilization of knowledge and semantic data for image understanding. Specifically, I seek to incorporate knowledge in various algorithms that cannot be inferred from the image pixels alone. This information comes from related images or structured data (in the form of hierarchies and ontologies) to improve the performance of object detection and image segmentation tasks. These understanding tasks are crucial intermediate steps towards retrieval and semantic understanding. However, the typical object detection and segmentation tasks requires an abundance of training data for machine learning algorithms. The prior training information provides information on what patterns and visual features the algorithm should be looking for when processing an image. In contrast, my algorithm utilizes related semantic images to extract the visual properties of an object and also to decrease the search space of my detection algorithm. Furthermore, I demonstrate the use of related images in the image segmentation process. Again, without the use of prior training data, I present a method for foreground object segmentation by finding the shared area that exists in a set of images. I demonstrate the effectiveness of my method on structured image datasets that have defined relationships between classes i.e. parent-child, or sibling classes.Finally, I introduce my framework for semantic image retrieval. I enhance the proposed knowledge acquisition and image understanding techniques with semantic knowledge through linked data and web semantic languages. This is an essential step in semantic image retrieval. For example, a car class classified by an image processing algorithm not enhanced by external knowledge would have no idea that a car is a type of vehicle which would also be highly related to a truck and less related to other transportation methods like a train . However, a query for modes of human transportation should return all of the mentioned classes. Thus, I demonstrate how to integrate information from both image processing algorithms and semantic knowledge bases to perform interesting queries that would otherwise be impossible. The key component of this system is a novel property reasoner that is able to translate low level image features into semantically relevant object properties. I use a combination of XML based languages such as SVG, RDF, and OWL in order to link to existing ontologies available on the web. My experiments demonstrate an efficient data collection framework and novel utilization of semantic data for image analysis and retrieval on datasets of people and landmarks collected from sources such as IMDB and Flickr. Ultimately, my thesis presents improvements to the state of the art in visual knowledge representation/acquisition and computer vision algorithms such as detection and segmentation toward the goal of enhanced semantic image retrieval

A Necessary Sin : An Ethnographic Study Of Sex Selection In Western India

This dissertation analyzes sex-selective abortion in western India as a lived process with profound cultural, ethical, and demographic implications. Over the past three decades, selective elimination of female fetuses has emerged as a disturbing form of family planning across parts of Europe and Asia. In India, the practice remains widespread despite extensive efforts to combat it, with drastically skewed girl-to-boy ratios resulting in many locales. Drawing on eighteen months of ethnographic fieldwork with families and clinicians practicing sex selection, as well as with government officials and activists attempting to regulate it, this dissertation examines how prenatal sex determination marks fetuses with gender and incorporates them into local systems of kinship, biomedicine, and governance. Elucidating a kinship logic that renders daughters threatening and sons indispensable, I follow prospective parents and clinicians as they imagine divergent futures for children-to-be, navigate a clandestine black market, and employ specific biomedical techniques to produce and act on gendered fetuses. In the process of sex selection, fetuses become subject to complex ethical deliberations, familial struggles over reproductive decision-making, herbal and religious modalities of son production, and a host of public interventions aimed at saving daughters-to-be. I argue that the diverse actions around prenatal sex determination presuppose, shape, and intervene on a “gendered fetal subject”—an imagined or potential person whose liminal status (between human and non-human, between alive and un-alive) makes it a point of connection among households, clinics, and governance institutions. Tracing the production, transformation, and elimination of gendered fetal subjects reveals how kinship extends prenatally, as well as how fetuses become incorporated into social life. Furthermore, as suggested by prospective parents’ fraught reflections on the notion of a “necessary sin” (profoundly unethical but nonetheless unavoidable), understanding gendered fetal subjects provides an entry point for untangling the moral complexities of sex selection as a liminal form of violence—a gendered violation at the very threshold of human social existence