Toward Large Scale Semantic Image Understanding and Retrieval

Semantic image retrieval is a multifaceted, highly complex problem. Not only does the solution to this problem require advanced image processing and computer vision techniques, but it also requires knowledge beyond what can be inferred from the image content alone. In contrast, traditional image retrieval systems are based upon keyword searches on filenames or metadata tags, e.g. Google image search, Flickr search, etc. These conventional systems do not analyze the image content and their keywords are not guaranteed to represent the image. Thus, there is significant need for a semantic image retrieval system that can analyze and retrieve images based upon the content and relationships that exist in the real world.In this thesis, I present a framework that moves towards advancing semantic image retrieval in large scale datasets. At a conceptual level, semantic image retrieval requires the following steps: viewing an image, understanding the content of the image, indexing the important aspects of the image, connecting the image concepts to the real world, and finally retrieving the images based upon the index concepts or related concepts. My proposed framework addresses each of these components in my ultimate goal of improving image retrieval. The first task is the essential task of understanding the content of an image. Unfortunately, typically the only data used by a computer algorithm when analyzing images is the low-level pixel data. But, to achieve human level comprehension, a machine must overcome the semantic gap, or disparity that exists between the image data and human understanding. This translation of the low-level information into a high-level representation is an extremely difficult problem that requires more than the image pixel information. I describe my solution to this problem through the use of an online knowledge acquisition and storage system. This system utilizes the extensible, visual, and interactable properties of Scalable Vector Graphics (SVG) combined with online crowd sourcing tools to collect high level knowledge about visual content.I further describe the utilization of knowledge and semantic data for image understanding. Specifically, I seek to incorporate knowledge in various algorithms that cannot be inferred from the image pixels alone. This information comes from related images or structured data (in the form of hierarchies and ontologies) to improve the performance of object detection and image segmentation tasks. These understanding tasks are crucial intermediate steps towards retrieval and semantic understanding. However, the typical object detection and segmentation tasks requires an abundance of training data for machine learning algorithms. The prior training information provides information on what patterns and visual features the algorithm should be looking for when processing an image. In contrast, my algorithm utilizes related semantic images to extract the visual properties of an object and also to decrease the search space of my detection algorithm. Furthermore, I demonstrate the use of related images in the image segmentation process. Again, without the use of prior training data, I present a method for foreground object segmentation by finding the shared area that exists in a set of images. I demonstrate the effectiveness of my method on structured image datasets that have defined relationships between classes i.e. parent-child, or sibling classes.Finally, I introduce my framework for semantic image retrieval. I enhance the proposed knowledge acquisition and image understanding techniques with semantic knowledge through linked data and web semantic languages. This is an essential step in semantic image retrieval. For example, a car class classified by an image processing algorithm not enhanced by external knowledge would have no idea that a car is a type of vehicle which would also be highly related to a truck and less related to other transportation methods like a train . However, a query for modes of human transportation should return all of the mentioned classes. Thus, I demonstrate how to integrate information from both image processing algorithms and semantic knowledge bases to perform interesting queries that would otherwise be impossible. The key component of this system is a novel property reasoner that is able to translate low level image features into semantically relevant object properties. I use a combination of XML based languages such as SVG, RDF, and OWL in order to link to existing ontologies available on the web. My experiments demonstrate an efficient data collection framework and novel utilization of semantic data for image analysis and retrieval on datasets of people and landmarks collected from sources such as IMDB and Flickr. Ultimately, my thesis presents improvements to the state of the art in visual knowledge representation/acquisition and computer vision algorithms such as detection and segmentation toward the goal of enhanced semantic image retrieval

Driving to a fast IMS feature vector computing

La creciente cantidad de imágenes transmitidas a través de Internet ha llevado al desarrollo de Sistemas de Minería de Imágenes de propósito general. La performance de un SMI depende en gran medida de una rápida y buena especificación del vector característica que describe unívocamente a una imagen completa. El tamaño del vector y las relaciones existentes entre cada una de las características evaluadas y su tiempo de procesamiento son críticos, más aún cuando la cantidad de imágenes es lo suficientemente grande. Una posible solución consiste en el uso de paralelismo en las diferentes tareas involucradas en un SMI. Hoy en día, los clusters de computadoras son una opción ampliamente utilizada, con un bajo costo y alto rendimiento, principalmente para máquinas de propósitos específicos y se adaptan a la resolución de problemas de procesamiento de imágenes con un alto grado de paralelismo y localidad de datos. En este paper nos focalizaremos en el paralelismo de la etapa de procesamiento de un sistema SMI con la intención de acelerar el cálculo del vector característica por medio de una arquitectura cluster intentando brindar una mejor performance al sistema SMI en su totalidad.Increasing amount of image data transmitted via Internet has triggered the development of general purposes Image Mining Systems (IMS). An IMS performance relies on a good and fast feature vector specification that describes univocally an entire image. Vector size and the relationship between each evaluated feature and its computation time are critical, moreover when the image amount is big enough. Decreasing this IMS computational complexity by means of parallelism at the different involved tasks is one solution. Nowadays clusters of computers are already widely used as a low cost and high utility option to special-purpose machines, and suited to solve image processing problems with a high degree of data locality and parallelism. At this paper, we will focus on parallelism into the IMS processing stage trying to accelerate the feature vector calculus thru a cluster architecture attempting to give a better performance to the whole image mining system.Workshop de Computación Gráfica, Imágenes y Visualización (WCGIV)Red de Universidades con Carreras en Informática (RedUNCI

Mobile Wound Assessment and 3D Modeling from a Single Image

The prevalence of camera-enabled mobile phones have made mobile wound assessment a viable treatment option for millions of previously difficult to reach patients. We have designed a complete mobile wound assessment platform to ameliorate the many challenges related to chronic wound care. Chronic wounds and infections are the most severe, costly and fatal types of wounds, placing them at the center of mobile wound assessment. Wound physicians assess thousands of single-view wound images from all over the world, and it may be difficult to determine the location of the wound on the body, for example, if the wound is taken at close range. In our solution, end-users capture an image of the wound by taking a picture with their mobile camera. The wound image is segmented and classified using modern convolution neural networks, and is stored securely in the cloud for remote tracking. We use an interactive semi-automated approach to allow users to specify the location of the wound on the body. To accomplish this we have created, to the best our knowledge, the first 3D human surface anatomy labeling system, based off the current NYU and Anatomy Mapper labeling systems. To interactively view wounds in 3D, we have presented an efficient projective texture mapping algorithm for texturing wounds onto a 3D human anatomy model. In so doing, we have demonstrated an approach to 3D wound reconstruction that works even for a single wound image

Imparting 3D representations to artificial intelligence for a full assessment of pressure injuries.

During recent decades, researches have shown great interest to machine learning techniques in order to extract meaningful information from the large amount of data being collected each day. Especially in the medical field, images play a significant role in the detection of several health issues. Hence, medical image analysis remarkably participates in the diagnosis process and it is considered a suitable environment to interact with the technology of intelligent systems. Deep Learning (DL) has recently captured the interest of researchers as it has proven to be efficient in detecting underlying features in the data and outperformed the classical machine learning methods. The main objective of this dissertation is to prove the efficiency of Deep Learning techniques in tackling one of the important health issues we are facing in our society, through medical imaging. Pressure injuries are a dermatology related health issue associated with increased morbidity and health care costs. Managing pressure injuries appropriately is increasingly important for all the professionals in wound care. Using 2D photographs and 3D meshes of these wounds, collected from collaborating hospitals, our mission is to create intelligent systems for a full non-intrusive assessment of these wounds. Five main tasks have been achieved in this study: a literature review of wound imaging methods using machine learning techniques, the classification and segmentation of the tissue types inside the pressure injury, the segmentation of these wounds and the design of an end-to-end system which measures all the necessary quantitative information from 3D meshes for an efficient assessment of PIs, and the integration of the assessment imaging techniques in a web-based application

Web page cleaning for web mining

Ph.DDOCTOR OF PHILOSOPH

Land Use Identification of the Metropolitan Area of Guadalajara Using Bicycle Data: An Unsupervised Classification Approach

El siguiente trabajo propone diferentes maneras de resolver una problemática que se encuentra en la actualidad, que es el hacer la investigación en el área de land-use, mapeo y comportamiento humano evaluando su movimiento por medio de fuentes de información que contienen información geo referenciada, también se comparte la meta de clasificar diferentes secciones y su relación entre ellas. Se utilizó como fuente de información MiBici que es una plataforma de compartimiento de bicicleta que existe en la ciudad de Guadalajara, Jalisco, la cual comparte mes tras mes un archivo consolidado de los viajes que se realizan en cada mes, cabe mencionar que el acceso de esta información es totalmente libre. Las metodologías utilizadas fueron agile para planeación del proyecto, KNN, Decision Trees y KMeans para la cauterización de las zonas, el lenguaje de programación utilizado fue Python, además se anexo una propuesta de implementación utilizando la plataforma de Amazon Web Service con el objetivo de proponer una solución más “sencilla” de implementar, pero con el mismo valor que hacerlo con puros recursos libres. El proceso se dividió primordialmente en 3 partes en donde la primera fue limpiar datos y entenderlos, se aplicaron algoritmos machine learning que fueron Decision tree y KNN, para la segunda etapa evaluando los resultados de la etapa anterior se hicieron modificaciones a los datos en donde se agregaron nuevos campos para mejor los resultados y se aplicó KMeans para la creación de grupos y como último paso se creó un flujo que inicio con la limpieza de los datos en crudo utilizando herramientas de AWS y se terminó con la interpretación de los resultados finales. Los resultados obtenidos fueron demasiados alentadores ya que los grupos que se obtuvieron fueron demasiados marcados y revisándolo con las zonas relacionadas a los nodos se encontró una gran relación. Sin duda alguna queda aún demasiado trabajo a desarrollar en esta rama de investigación