Model-based learning of local image features for unsupervised texture segmentation

Features that capture well the textural patterns of a certain class of images are crucial for the performance of texture segmentation methods. The manual selection of features or designing new ones can be a tedious task. Therefore, it is desirable to automatically adapt the features to a certain image or class of images. Typically, this requires a large set of training images with similar textures and ground truth segmentation. In this work, we propose a framework to learn features for texture segmentation when no such training data is available. The cost function for our learning process is constructed to match a commonly used segmentation model, the piecewise constant Mumford-Shah model. This means that the features are learned such that they provide an approximately piecewise constant feature image with a small jump set. Based on this idea, we develop a two-stage algorithm which first learns suitable convolutional features and then performs a segmentation. We note that the features can be learned from a small set of images, from a single image, or even from image patches. The proposed method achieves a competitive rank in the Prague texture segmentation benchmark, and it is effective for segmenting histological images

People Matching for Transportation Planning Using Optimized Features and Texel Camera Data for Sequential Estimation

This thesis explores pattern recognition in the dynamic setting of public transportation, such as a bus, as people enter and later exit from a doorway. Matching the entrance and exit of each individual provides accurate information about individual riders such as how long a person is on a bus and which stops the person uses. At a higher level, matching exits to entries provides information about the distribution of traffic flow across the whole transportation system. A texel camera is implemented and multiple measures of people are made where the depth and color data are generated. A large number of features are generated and the sequential floating forward selection (SFFS) algorithm is used for selecting the optimized features. Criterion functions using marginal accuracy and maximization of minimum normalized Mahalanobis distance are designed and compared. Because of the particular case of the bus environment, which is a sequential estimation problem, a trellis optimization algorithm is designed based on a sequence of measurements from the texel camera. Since the number of states in the trellis grows exponentially with the number of people currently on the bus, a beam search pruning technique is employed to manage the computational and memory load. Experimental results using real texel camera measurements show good results for 68 people exiting from an initially full bus in a randomized order. In a bus route simulation where a true traffic flow distribution is used to randomly draw entry and exit events for simulated riders, the proposed sequential estimation algorithm produces an estimated traffic flow distribution which provides an excellent match to the true distribution

Image Understanding and Robotics Research at Columbia University

The research investigations of the Vision/Robotics Laboratory at Columbia University reflect the diversity of interests of its four faculty members, two staff programmers and 15 Ph.D. students. Several of the projects involve either a visiting computer science post-doc, other faculty members in the department or the university, or researchers at AT&T Bell Laboratories or Philips laboratories. We list below a summary of our interest and results, together with the principal researchers associated with them. Since it is difficult to separate those aspects of robotic research that are purely visual from those that are vision-like (for example, tactile sensing) or vision-related (for example, integrated vision-robotic systems), we have listed all robotic research that is not purely manipulative

Raw Depth Image Enhancement Using a Neural Network

The term image is often used to denote a data format that records information about a scene’s color. This dissertation object focuses on a similar format for recording distance information about a scene, “depth images”. Depth images have been used extensively in consumer-level applications, such as Apple’s Face ID, based on depth images for face recognition. However, depth images suffer from low precision and high errors, and some post-processing techniques need to be utilized to improve their quality. Deep learning, or neural networks, are frameworks that use a series of hierarchically arranged nonlinear networks to process input data. Although each layer of the network is limited in its capabilities, the learning capacity accumulated by the multilayer network becomes very powerful. This dissertation assembles two different deep learning frameworks to solve two different types of raw image preprocessing problems. The first network is the super-resolution network, a nonlinear interpolation of low-resolution deep images through the deep network to obtain high-resolution images. The second network is the inpainting network, which is used to mitigate the problem of losing specific pixel data in the original depth image for various reasons. This dissertation presents deep images processed by these two frameworks, and the quality of the processed images is significantly improved compared to the original images. The great potential of deep learning techniques in the field of deep image processing is shown

Indices of comparative cognition:Assessing animal models of human brain function

Understanding the cognitive capacities of animals is important, because (a) several animal models of human neurodegenerative disease are considered poor representatives of the human equivalent and (b) cognitive capacities may provide insight into alternative animal models. We used a three-stage process of cognitive and neuroanatomical comparison (using sheep as an example) to assess the appropriateness of a species to model human brain function. First, a cognitive task was defined via a reinforcement-learning algorithm where values/constants in the algorithm were taken as indirect measures of neurophysiological attributes. Second, cognitive data (values/constants) were generated for the example species (sheep) and compared to other species. Third, cognitive data were compared with neuroanatomical metrics for each species (endocranial volume, gyrification index, encephalisation quotient, and number of cortical neurons). Four breeds of sheep (n = 15/sheep) were tested using the two-choice discrimination-reversal task. The 'reversal index' was used as a measure of constants within the learning algorithm. Reversal index data ranked sheep as third in a table of species that included primates, dogs, and pigs. Across all species, number of cortical neurons correlated strongest against the reversal index (r2 = 0.66, p = 0.0075) followed by encephalization quotient (r2 = 0.42, p = 0.03), endocranial volume (r2 = 0.30, p = 0.08), and gyrification index (r2 = 0.16, p = 0.23). Sheep have a high predicted level of cognitive capacity and are thus a valid alternative model for neurodegenerative research. Using learning algorithms within cognitive tasks increases the resolution of methods of comparative cognition and can help to identify the most relevant species to model human brain function and dysfunction.CHDI In

Improved Encoding for Compressed Textures

For the past few decades, graphics hardware has supported mapping a two dimensional image, or texture, onto a three dimensional surface to add detail during rendering. The complexity of modern applications using interactive graphics hardware have created an explosion of the amount of data needed to represent these images. In order to alleviate the amount of memory required to store and transmit textures, graphics hardware manufacturers have introduced hardware decompression units into the texturing pipeline. Textures may now be stored as compressed in memory and decoded at run-time in order to access the pixel data. In order to encode images to be used with these hardware features, many compression algorithms are run offline as a preprocessing step, often times the most time-consuming step in the asset preparation pipeline. This research presents several techniques to quickly serve compressed texture data. With the goal of interactive compression rates while maintaining compression quality, three algorithms are presented in the class of endpoint compression formats. The first uses intensity dilation to estimate compression parameters for low-frequency signal-modulated compressed textures and offers up to a 3X improvement in compression speed. The second, FasTC, shows that by estimating the final compression parameters, partition-based formats can choose an approximate partitioning and offer orders of magnitude faster encoding speed. The third, SegTC, shows additional improvement over selecting a partitioning by using a global segmentation to find the boundaries between image features. This segmentation offers an additional 2X improvement over FasTC while maintaining similar compressed quality. Also presented is a case study in using texture compression to benefit two dimensional concave path rendering. Compressing pixel coverage textures used for compositing yields both an increase in rendering speed and a decrease in storage overhead. Additionally an algorithm is presented that uses a single layer of indirection to adaptively select the block size compressed for each texture, giving a 2X increase in compression ratio for textures of mixed detail. Finally, a texture storage representation that is decoded at runtime on the GPU is presented. The decoded texture is still compressed for graphics hardware but uses 2X fewer bytes for storage and network bandwidth.Doctor of Philosoph