Optimization for automated assembly of puzzles

The puzzle assembly problem has many application areas such as restoration and reconstruction of archeological findings, repairing of broken objects, solving jigsaw type puzzles, molecular docking problem, etc. The puzzle pieces usually include not only geometrical shape information but also visual information such as texture, color, and continuity of lines. This paper presents a new approach to the puzzle assembly problem that is based on using textural features and geometrical constraints. The texture of a band outside the border of pieces is predicted by inpainting and texture synthesis methods. Feature values are derived from these original and predicted images of pieces. An affinity measure of corresponding pieces is defined and alignment of the puzzle pieces is formulated as an optimization problem where the optimum assembly of the pieces is achieved by maximizing the total affinity measure. An fft based image registration technique is used to speed up the alignment of the pieces. Experimental results are presented on real and artificial data sets

A texture based approach to reconstruction of archaeological finds

Reconstruction of archaeological finds from fragments, is a tedious task requiring many hours of work from the archaeologists and restoration personnel. In this paper we present a framework for the full reconstruction of the original objects using texture and surface design information on the sherd. The texture of a band outside the border of pieces is predicted by inpainting and texture synthesis methods. The confidence of this process is also defined. Feature values are derived from these original and predicted images of pieces. A combination of the feature and confidence values is used to generate an affinity measure of corresponding pieces. The optimization of total affinity gives the best assembly of the piece. Experimental results are presented on real and artificial data

Local Matching of Surfaces Using Critical Points

The local matching problem on surfaces is: Given a pair of oriented surfaces in 3-space, find subsurfaces that are identical or complementary in shape. A heuristic method is presented for local matching that is intended for use on complex curved surfaces (rather than such surfaces as as cubes and cylinders). The method proceeds as follows: (1) Find a small set of points-called critical points -on the two surfaces with the property that if p is a critical point and p matches q, then q is also a critical point. The critical points are taken to be local extrema of either Gaussian or mean curvature. (2) Construct a rotation invariant representation around each critical point by intersecting the surface with spheres of standard radius centered around the critical point. For each of the resulting curves of intersection, compute a distance map function equal to the distance from a point on the curve to the center of gravity of the curve as a. function of arc length (normalized so that the domain of the function is the interval [0,1]). Cll the set of contours for a given critical point a distance profile. (3) Match distance profiles by computing a correlation between corresponding distance contours. (4) Use maximal compatible subsets of the set of matching profiles to induce a transformation that maps corresponding critical points together, then use a cellular spatial partitioning technique to find all points on each surface that are within a tolerance of the other surface

Semi-automatic Solving of "Jigsaw puzzles" for Material Reconstruction of Dead Sea Scrolls

Digital solving of jigsaw puzzles have been well researched throughout the years and multiple approaches to solve them have been proposed. But these approaches have not been applied to reconstructing ancient manuscripts out of transient material such as leather or parchment. The literature describes ways to reconstruct ancient artefacts but they describe the process for more durable objects like pottery. In this thesis we explore the usability of the existing state-of-the-art methods for the purpose of aiding reconstructing of the Dead Sea Scrolls, also known as Qumran scrolls. Our experiments show that the existing methods as such do not provide good results in this domain, but with modifications provide help through a semi-automated reconstruction process. We expect these modifications and the software that was created as a by-product of this thesis to ease the researchers' work by automating the previously laborious manual work

A Novel Hybrid Scheme Using Genetic Algorithms and Deep Learning for the Reconstruction of Portuguese Tile Panels

This paper presents a novel scheme, based on a unique combination of genetic algorithms (GAs) and deep learning (DL), for the automatic reconstruction of Portuguese tile panels, a challenging real-world variant of the jigsaw puzzle problem (JPP) with important national heritage implications. Specifically, we introduce an enhanced GA-based puzzle solver, whose integration with a novel DL-based compatibility measure (DLCM) yields state-of-the-art performance, regarding the above application. Current compatibility measures consider typically (the chromatic information of) edge pixels (between adjacent tiles), and help achieve high accuracy for the synthetic JPP variant. However, such measures exhibit rather poor performance when applied to the Portuguese tile panels, which are susceptible to various real-world effects, e.g., monochromatic panels, non-squared tiles, edge degradation, etc. To overcome such difficulties, we have developed a novel DLCM to extract high-level texture/color statistics from the entire tile information. Integrating this measure with our enhanced GA-based puzzle solver, we have demonstrated, for the first time, how to deal most effectively with large-scale real-world problems, such as the Portuguese tile problem. Specifically, we have achieved 82% accuracy for the reconstruction of Portuguese tile panels with unknown piece rotation and puzzle dimension (compared to merely 3.5% average accuracy achieved by the best method known for solving this problem variant). The proposed method outperforms even human experts in several cases, correcting their mistakes in the manual tile assembly

Computer aided puzzle assembly based on shape and texture information /

Puzzle assembly’s importance lies into application in many areas such as restoration and reconstruction of archeological findings, the repairing of broken objects, solving of the jigsaw type puzzles, molecular docking problem, etc. Puzzle pieces usually include not only geometrical shape information but also visual information of texture, color, continuity of lines, and so on. Moreover, textural information is mainly used to assembly pieces in some cases, such as classic jigsaw puzzles. This research presents a new approach in that pictorial assembly, in contrast to previous curve matching methods, uses texture information as well as geometric shape. The assembly in this study is performed using textural features and geometrical constraints. First, the texture of a band outside the border of pieces is predicted by inpainting and texture synthesis methods. The feature values are derived by these original and predicted images of pieces. A combination of the feature and confidence values is used to generate an affinity measure of corresponding pieces. Two new algorithms using Fourier based image registration techniques are developed to optimize the affinity. The algorithms for inpainting, affinity and Fourier based assembly are explained with experimental results on real and artificial data. The main contributions of this research are: The development of a performance measure that indicates the level of success of assembly of pieces based on textural features and geometrical shape. Solution of the assembly problem by using of the Fourier based methods

Leveraging facebook’s open graph to develop an environmental persuasive application

Dissertação para obtenção do Grau de Mestre em Engenharia InformáticaSocial networking sites persuade millions of users each day to adopt specific behaviors. Using the persuasive principles inherent to these sites to increase environmental awareness and reduce our ecological footprint can be challenging but certainly worthy. The DEAP project has already invested time and resources to address persuasion through different devices for a broad audience. However, there are still many obstacles when it comes to such a delicate subject as people’s routines. For many years, social factors have prevented people from adopting a way of living friendlier to our Environment. Whether it is due to lack of proper knowledge about this topic or simply because they are not willing to change, the truth is that we are eventually reaching a point where it will be too late to keep our planet as we know it. Consequently, the time has arrived when there is great need for a platform to bring existing efforts together no matter where they come from but the goal they share: change incorrect behaviors towards environmental sustainability. Towards this ambitious goal a board game was developed and integrated in Facebook capable of merging third-party applications and an important and valuable basis for future research in the field of persuasion

Characterization of Protein Folding Pathways and Structural Stability

Proteins are large, flexible molecules with an extremely large number of potential conformations. Proteins expressed in cells traverse available conformations to reach a consistent, thermodynamically stable, biologically active structure through a process known as protein folding. The atomic composition of the protein, defined by a sequence of amino acid residues encoded in DNA as a gene, determines the protein folding pathway and ultimate native structure of the protein molecule. Understanding the relationship between the sequence of amino acids and the resulting protein structure has been a central challenge in protein research for decades. To fill this knowledge gap, we test the hypothesis that the distribution of conformers observed for a short protein sequence across all known protein structures reflects that sequence\u27s intrinsic structural properties. Qualitative and quantitative predictions based on our model are tested against experimental data for protein stability and folding pathways. Replica-exchange Monte Carlo simulations, data mining of the Worldwide Protein Data Bank (wwPDB), analysis of published protein stability data, thermodynamic and kinetic folding experiments, and Xray crystallography were used to characterize the structural properties of amino acid sequences. The role of turn sequences in guiding the protein folding process was extensively characterized by the combined methods. Turn composition, structural preferences, and cooperation with neighboring residues determined whether a turn had an active, passive, or counter-active role in a protein\u27s folding process. Proline-rich turns, NPSNP and KPSDP, from the two-helix bundles found in bacterial type III secretion system needle proteins form native-like structure early in the folding process. Each of these turns are flanked by sequences with very high helix propensity that, when oriented by the turn, can actively nucleate the hydrophobic core of the protein. The hydrophobic turn, MGYE, from the three-helix bundle UBA(1) also forms native-like structure early in the folding process. This turn structure places the Met (M) and Tyr (Y) residues together, nucleating the hydrophobic core of UBA(1). These two residues can then stabilize the adjacent helices to form a Helix- Turn-Helix structure. The second, proline-containing turn in UBA(1), ASYNNP, forms non-native structure early in the folding process. This turn restructures late in the folding process when the third helix docks to the previous Helix-Turn-Helix structure. Each of the active turns characterized (NPSNP, KPSDP, and MGYE) direct the folding process by nucleating the protein\u27s hydrophobic core. A general purpose computational method to model the local structural properties of protein sequences was developed from data mined from the wwPDB. Turn mechanisms can be rapidly characterized using the tool, EmCAST, in conjunction with a PDB structure of the protein of interest. The impact of surface mutations on protein stability can also be scored by EmCAST. Models and calculations were extensively validated against experimental data for multiple protein and peptide systems. Calculations for stabilizing mutations at well-structured positions in UBA(1) produced a near perfect correlation with experimental measurements (R2 = 0.97). A user-friendly web interface to the software was developed to share the method with other protein researchers. Our model provides key insights into the protein sequence/structure relationship that can be used to characterize protein surface stability, identify regions with dynamic structure, and predict protein folding intermediates