Photo collage-based photograph display system on mobile computing platform

In the last few decades, mobile computing platform technology has grown rapidly, as observed from smart phones that have quickly become ubiquitous. The mobile computing platform is the most widely used platform in our life today, and digital photographs captured through these devices have become routine for most people. In this study, we propose a novel artistic method for displaying photographs in mobile devices as a photo collage. Using our system, users can view a representative photograph as a collage of photographs associated with a certain event and access each of photographs individually. To implement this, we employ centroidal Voronoi diagram to obtain an even distribution of tiles, and use the sites as the location of tiles. We use the edge avoidance technique to prevent tiles from being located across the edges. To obtain the direction of tiles that follow near a strong edge, we employ the Edge tangent Flow field and use the field as the directions of tiles. Finally, we search for photographs that best match the tiles calculated above by using a thumbnail difference metric

Scattered Mosaic Rendering Using Unit Images

An image mosaic method that can be used when creating advertisements or posters is proposed in this study. Mosaic is a method that expresses an entire image using an arbitrary number of cells. Photomosaic generates new images using a combination of photos. In this paper, we propose a new mosaic algorithm that generates an abstract artistic mosaic image by filling a region that is divided by a boundary using a unit image, which is an image that only has a shape and no allocated color. A unit image can be changed diversely through rotation or shifting, and the corresponding region is filled by using the gradient direction and edge information of the input image. For this, we extract and use information from input image such as color, edge and gradient. In result we can generate various abstractive images which can be used in advertisement and multimedia contents market

Repeatable semantic reef-mapping through photogrammetry and label-augmentation

In an endeavor to study natural systems at multiple spatial and taxonomic resolutions, there is an urgent need for automated, high-throughput frameworks that can handle plethora of information. The coalescence of remote-sensing, computer-vision, and deep-learning elicits a new era in ecological research. However, in complex systems, such as marine-benthic habitats, key ecological processes still remain enigmatic due to the lack of cross-scale automated approaches (mms to kms) for community structure analysis. We address this gap by working towards scalable and comprehensive photogrammetric surveys, tackling the profound challenges of full semantic segmentation and 3D grid definition. Full semantic segmentation (where every pixel is classified) is extremely labour-intensive and difficult to achieve using manual labeling. We propose using label-augmentation, i.e., propagation of sparse manual labels, to accelerate the task of full segmentation of photomosaics. Photomosaics are synthetic images generated from a projected point-of-view of a 3D model. In the lack of navigation sensors (e.g., a diver-held camera), it is difficult to repeatably determine the slope-angle of a 3D map. We show this is especially important in complex topographical settings, prevalent in coral-reefs. Specifically, we evaluate our approach on benthic habitats, in three different environments in the challenging underwater domain. Our approach for label-augmentation shows human-level accuracy in full segmentation of photomosaics using labeling as sparse as 0.1%, evaluated on several ecological measures. Moreover, we found that grid definition using a leveler improves the consistency in community-metrics obtained due to occlusions and topology (angle and distance between objects), and that we were able to standardise the 3D transformation with two percent error in size measurements. By significantly easing the annotation process for full segmentation and standardizing the 3D grid definition we present a semantic mapping methodology enabling change-detection, which is practical, swift, and cost-effective. Our workflow enables repeatable surveys without permanent markers and specialized mapping gear, useful for research and monitoring, and our code is available online. Additionally, we release the Benthos data-set, fully manually labeled photomosaics from three oceanic environments with over 4500 segmented objects useful for research in computer-vision and marine ecology

Perceptually Inspired Real-time Artistic Style Transfer for Video Stream

This study presents a real-time texture transfer method for artistic style transfer for video stream. We propose a parallel framework using a T-shaped kernel to enhance the computational performance. With regard to accelerated motion estimation, which is necessarily required for maintaining temporal coherence, we present a method using a downscaled motion field to successfully achieve high real-time performance for texture transfer of video stream. In addition, to enhance the artistic quality, we calculate the level of abstraction using visual saliency and integrate it with the texture transfer algorithm. Thus, our algorithm can stylize video with perceptual enhancements

Assessing placement efficiency of photovoltaic installations using Mask R-CNN

Photovoltaic (PV) energy production has experienced strong growth over the past years and is forecasted to greatly contribute to the successful transition to renewable energy production as demanded by Switzerland’s Energy Strategy 2050. Several studies attempted to estimate the national PV potential on building rooftops but arrived at strongly varying results ranging from 15 to 53 TWh annually. To a vast extent, the differences can be explained by the application of varying rooftop utilization ratios which were extrapolated by all previous studies. Moreover, no comparison of the placement of existing PV installations to the suitability categorization from the sonnendach.ch project was yet carried out. Therefore, the aim of this master thesis was to develop and evaluate a prototype methodology to close the research gaps regarding rooftop utilization ratio and the efficiency of PV panel placement. The prototype methodology to answer these questions was developed in Python and leverages publicly available data from the Swiss government in conjunction with a Mask R-CNN for the accurate segmentation of PV panels on high resolution aerial imagery. A total of 1130 individual images of building rooftop were thereby collected in the canton of Aargau of which 974 were used to train the Mask R-CNN model. After four training iterations with varying dataset sizes, the segmentation performance of the Mask R-CNN achieved an iou_score of 0.74. Overall, the rooftop utilization ratio found in this thesis equated to 29%, suggesting that all PV potential studies systematically overestimate the extent of rooftop utilization. Moreover, the findings of this thesis suggest that the more suitable a rooftop area is, the greater its extent of utilization whereas previous studies assumed a uniform distribution of utilization ratio across all suitability categorizations. From the assessed building rooftops, 2.8% have their PV panels suboptimally placed and therefore fail to efficiently exploit solar radiation. 71% of which were successfully detected by the model. Overall, the findings of this thesis proved that an automated, large-scale assessment of PV placement efficiency is technically feasible. This information could support national energy planning as well as PV incentive decision making. However, the segmentation performance of the Mask R-CNN achieved with the resources available to this thesis is currently insufficient for detailed quantitative analyses. Consequently, further studies to improve the Mask R-CNN performance should be conducted before applying the prototype methodology on a large scale

A protocol for the large‐scale analysis of reefs using Structure from Motion photogrammetry

1. Substrate complexity is an essential metric of reef health and a strong predictor of several ecological processes connected to the reef, including disturbance, resilience, and associated community abundance and diversity. / 2. Underwater Structure from Motion (SfM) photogrammetry has been growing rapidly in use over the last 5 years due to advances in computing power, reduced costs of underwater digital cameras and a push for reproducible data. This has led to the adaptation of an originally terrestrial survey technique into the marine realm, which can now be applied at the habitat scale. / 3. This technique allows researchers to make detailed 3D reconstructions of reef surfaces for morphometric analysis of reef physical structure and perform large‐scale image‐mosaic mapping. SfM is useful for both reef‐scale and colony‐scale assessments, where visual or acoustic methods are impractical or not sufficiently detailed. / 4. Here we provide a protocol for the collection, analysis and display of 3D reef data, focussing on large‐scale habitat assessments of coral reefs using primarily open‐source software. We further suggest applications for other underwater environments and scales of assessment, and hope this standardized protocol will help researchers apply this technology and inspire new avenues of ecological research

Browse-to-search

This demonstration presents a novel interactive online shopping application based on visual search technologies. When users want to buy something on a shopping site, they usually have the requirement of looking for related information from other web sites. Therefore users need to switch between the web page being browsed and other websites that provide search results. The proposed application enables users to naturally search products of interest when they browse a web page, and make their even causal purchase intent easily satisfied. The interactive shopping experience is characterized by: 1) in session - it allows users to specify the purchase intent in the browsing session, instead of leaving the current page and navigating to other websites; 2) in context - -the browsed web page provides implicit context information which helps infer user purchase preferences; 3) in focus - users easily specify their search interest using gesture on touch devices and do not need to formulate queries in search box; 4) natural-gesture inputs and visual-based search provides users a natural shopping experience. The system is evaluated against a data set consisting of several millions commercial product images. © 2012 Authors

Advanced perception, navigation and planning for autonomous in-water ship hull inspection

Inspection of ship hulls and marine structures using autonomous underwater vehicles has emerged as a unique and challenging application of robotics. The problem poses rich questions in physical design and operation, perception and navigation, and planning, driven by difficulties arising from the acoustic environment, poor water quality and the highly complex structures to be inspected. In this paper, we develop and apply algorithms for the central navigation and planning problems on ship hulls. These divide into two classes, suitable for the open, forward parts of a typical monohull, and for the complex areas around the shafting, propellers and rudders. On the open hull, we have integrated acoustic and visual mapping processes to achieve closed-loop control relative to features such as weld-lines and biofouling. In the complex area, we implemented new large-scale planning routines so as to achieve full imaging coverage of all the structures, at a high resolution. We demonstrate our approaches in recent operations on naval ships.United States. Office of Naval Research (Grant N00014-06-10043)United States. Office of Naval Research (Grant N00014-07-1-0791