Learning Gradient Fields for Scalable and Generalizable Irregular Packing

The packing problem, also known as cutting or nesting, has diverse applications in logistics, manufacturing, layout design, and atlas generation. It involves arranging irregularly shaped pieces to minimize waste while avoiding overlap. Recent advances in machine learning, particularly reinforcement learning, have shown promise in addressing the packing problem. In this work, we delve deeper into a novel machine learning-based approach that formulates the packing problem as conditional generative modeling. To tackle the challenges of irregular packing, including object validity constraints and collision avoidance, our method employs the score-based diffusion model to learn a series of gradient fields. These gradient fields encode the correlations between constraint satisfaction and the spatial relationships of polygons, learned from teacher examples. During the testing phase, packing solutions are generated using a coarse-to-fine refinement mechanism guided by the learned gradient fields. To enhance packing feasibility and optimality, we introduce two key architectural designs: multi-scale feature extraction and coarse-to-fine relation extraction. We conduct experiments on two typical industrial packing domains, considering translations only. Empirically, our approach demonstrates spatial utilization rates comparable to, or even surpassing, those achieved by the teacher algorithm responsible for training data generation. Additionally, it exhibits some level of generalization to shape variations. We are hopeful that this method could pave the way for new possibilities in solving the packing problem

Learning Physically Realizable Skills for Online Packing of General 3D Shapes

We study the problem of learning online packing skills for irregular 3D shapes, which is arguably the most challenging setting of bin packing problems. The goal is to consecutively move a sequence of 3D objects with arbitrary shapes into a designated container with only partial observations of the object sequence. Meanwhile, we take physical realizability into account, involving physics dynamics and constraints of a placement. The packing policy should understand the 3D geometry of the object to be packed and make effective decisions to accommodate it in the container in a physically realizable way. We propose a Reinforcement Learning (RL) pipeline to learn the policy. The complex irregular geometry and imperfect object placement together lead to huge solution space. Direct training in such space is prohibitively data intensive. We instead propose a theoretically-provable method for candidate action generation to reduce the action space of RL and the learning burden. A parameterized policy is then learned to select the best placement from the candidates. Equipped with an efficient method of asynchronous RL acceleration and a data preparation process of simulation-ready training sequences, a mature packing policy can be trained in a physics-based environment within 48 hours. Through extensive evaluation on a variety of real-life shape datasets and comparisons with state-of-the-art baselines, we demonstrate that our method outperforms the best-performing baseline on all datasets by at least 12.8% in terms of packing utility.Comment: ACM Transactions on Graphics (TOG

Doctor of Philosophy

dissertationVisualizing surfaces is a fundamental technique in computer science and is frequently used across a wide range of fields such as computer graphics, biology, engineering, and scientific visualization. In many cases, visualizing an interface between boundaries can provide meaningful analysis or simplification of complex data. Some examples include physical simulation for animation, multimaterial mesh extraction in biophysiology, flow on airfoils in aeronautics, and integral surfaces. However, the quest for high-quality visualization, coupled with increasingly complex data, comes with a high computational cost. Therefore, new techniques are needed to solve surface visualization problems within a reasonable amount of time while also providing sophisticated visuals that are meaningful to scientists and engineers. In this dissertation, novel techniques are presented to facilitate surface visualization. First, a particle system for mesh extraction is parallelized on the graphics processing unit (GPU) with a red-black update scheme to achieve an order of magnitude speed-up over a central processing unit (CPU) implementation. Next, extending the red-black technique to multiple materials showed inefficiencies on the GPU. Therefore, we borrow the underlying data structure from the closest point method, the closest point embedding, and the particle system solver is switched to hierarchical octree-based approach on the GPU. Third, to demonstrate that the closest point embedding is a fast, flexible data structure for surface particles, it is adapted to unsteady surface flow visualization at near-interactive speeds. Finally, the closest point embedding is a three-dimensional dense structure that does not scale well. Therefore, we introduce a closest point sparse octree that allows the closest point embedding to scale to higher resolution. Further, we demonstrate unsteady line integral convolution using the closest point method

Almost Isometric Mesh Parameterization through Abstract Domains

In this paper, we propose a robust, automatic technique to build a global hi-quality parameterization of a two-manifold triangular mesh. An adaptively chosen 2D domain of the parameterization is built as part of the process. The produced parameterization exhibits very low isometric distortion, because it is globally optimized to preserve both areas and angles. The domain is a collection of equilateral triangular 2D regions enriched with explicit adjacency relationships (it is abstract in the sense that no 3D embedding is necessary). It is tailored to minimize isometric distortion, resulting in excellent parameterization qualities, even when meshes with complex shape and topology are mapped into domains composed of a small number of large continuous regions. Moreover, this domain is, in turn, remapped into a collection of 2D square regions, unlocking many advantages found in quad-based domains (e. g., ease of packing). The technique is tested on a variety of cases, including challenging ones, and compares very favorably with known approaches. An open-source implementation is made available

A linear framework for character skinning

Character animation is the process of modelling and rendering a mobile character in a virtual world. It has numerous applications both off-line, such as virtual actors in films, and real-time, such as in games and other virtual environments. There are a number of algorithms for determining the appearance of an animated character, with different trade-offs between quality, ease of control, and computational cost. We introduce a new method, animation space, which provides a good balance between the ease-of-use of very simple schemes and the quality of more complex schemes, together with excellent performance. It can also be integrated into a range of existing computer graphics algorithms. Animation space is described by a simple and elegant linear equation. Apart from making it fast and easy to implement, linearity facilitates mathematical analysis. We derive two metrics on the space of vertices (the “animation space”), which indicate the mean and maximum distances between two points on an animated character. We demonstrate the value of these metrics by applying them to the problems of parametrisation, level-of-detail (LOD) and frustum culling. These metrics provide information about the entire range of poses of an animated character, so they are able to produce better results than considering only a single pose of the character, as is commonly done. In order to compute parametrisations, it is necessary to segment the mesh into charts. We apply an existing algorithm based on greedy merging, but use a metric better suited to the problem than the one suggested by the original authors. To combine the parametrisations with level-of-detail, we require the charts to have straight edges. We explored a heuristic approach to straightening the edges produced by the automatic algorithm, but found that manual segmentation produced better results. Animation space is nevertheless beneficial in flattening the segmented charts; we use least squares conformal maps (LSCM), with the Euclidean distance metric replaced by one of our animation-space metrics. The resulting parametrisations have significantly less overall stretch than those computed based on a single pose. Similarly, we adapt appearance preserving simplification (APS), a progressive mesh-based LOD algorithm, to apply to animated characters by replacing the Euclidean metric with an animation-space metric. When using the memoryless form of APS (in which local rather than global error is considered), the use of animation space for computations reduces the geometric errors introduced by LOD decomposition, compared to simplification based on a single pose. User tests, in which users compared video clips of the two, demonstrated a statistically significant preference for the animation-space simplifications, indicating that the visual quality is better as well. While other methods exist to take multiple poses into account, they are based on a sampling of the pose space, and the computational cost scales with the number of samples used. In contrast, our method is analytic and uses samples only to gather statistics. The quality of LOD approximations by improved further by introducing a novel approach to LOD, influence simplification, in which we remove the influences of bones on vertices, and adjust the remaining influences to approximate the original vertex as closely as possible. Once again, we use an animation-space metric to determine the approximation error. By combining influence simplification with the progressive mesh structure, we can obtain further improvements in quality: for some models and at some detail levels, the error is reduced by an order of magnitude relative to a pure progressive mesh. User tests showed that for some models this significantly improves quality, while for others it makes no significant difference. Animation space is a generalisation of skeletal subspace deformation (SSD), a popular method for real-time character animation. This means that there is a large existing base of models that can immediately benefit from the modified algorithms mentioned above. Furthermore, animation space almost entirely eliminates the well-known shortcomings of SSD (the so-called “candy-wrapper” and “collapsing elbow” effects). We show that given a set of sample poses, we can fit an animation-space model to these poses by solving a linear least-squares problem. Finally, we demonstrate that animation space is suitable for real-time rendering, by implementing it, along with level-of-detail rendering, on a PC with a commodity video card. We show that although the extra degrees of freedom make the straightforward approach infeasible for complex models, it is still possible to obtain high performance; in fact, animation space requires fewer basic operations to transform a vertex position than SSD. We also consider two methods of lighting LOD-simplified models using the original normals: tangent-space normal maps, an existing method that is fast to render but does not capture dynamic structures such as wrinkles; and tangent maps, a novel approach that encodes animation-space tangent vectors into textures, and which captures dynamic structures. We compare the methods both for performance and quality, and find that tangent-space normal maps are at least an order of magnitude faster, while user tests failed to show any perceived difference in quality between them

Surface Deformation Potentials on Meshes for Computer Graphics and Visualization

Shape deformation models have been used in computer graphics primarily to describe the dynamics of physical deformations like cloth draping, collisions of elastic bodies, fracture, or animation of hair. Less frequent is their application to problems not directly related to a physical process. In this thesis we apply deformations to three problems in computer graphics that do not correspond to physical deformations. To this end, we generalize the physical model by modifying the energy potential. Originally, the energy potential amounts to the physical work needed to deform a body from its rest state into a given configuration and relates material strain to internal restoring forces that act to restore the original shape. For each of the three problems considered, this potential is adapted to reflect an application specific notion of shape. Under the influence of further constraints, our generalized deformation results in shapes that balance preservation of certain shape properties and application specific objectives similar to physical equilibrium states. The applications discussed in this thesis are surface parameterization, interactive shape editing and automatic design of panorama maps. For surface parameterization, we interpret parameterizations over a planar domain as deformations from a flat initial configuration onto a given surface. In this setting, we review existing parameterization methods by analyzing properties of their potential functions and derive potentials accounting for distortion of geometric properties. Interactive shape editing allows an untrained user to modify complex surfaces, be simply grabbing and moving parts of interest. A deformation model interactively extrapolates the transformation from those parts to the rest of the surface. This thesis proposes a differential shape representation for triangle meshes leading to a potential that can be optimized interactively with a simple, tailored algorithm. Although the potential is not physically accurate, it results in intuitive deformation behavior and can be parameterized to account for different material properties. Panorama maps are blends between landscape illustrations and geographic maps that are traditionally painted by an artist to convey geographic surveyknowledge on public places like ski resorts or national parks. While panorama maps are not drawn to scale, the shown landscape remains recognizable and the observer can easily recover details necessary for self location and orientation. At the same time, important features as trails or ski slopes appear not occluded and well visible. This thesis proposes the first automatic panorama generation method. Its basis is again a surface deformation, that establishes the necessary compromise between shape preservation and feature visibility.Potentiale zur Flächendeformation auf Dreiecksnetzen für Anwendungen in der Computergrafik und Visualisierung Deformationsmodelle werden in der Computergrafik bislang hauptsächlich eingesetzt, um die Dynamik physikalischer Deformationsprozesse zu modellieren. Gängige Beispiele sind Bekleidungssimulationen, Kollisionen elastischer Körper oder Animation von Haaren und Frisuren. Deutlich seltener ist ihre Anwendung auf Probleme, die nicht direkt physikalischen Prozessen entsprechen. In der vorliegenden Arbeit werden Deformationsmodelle auf drei Probleme der Computergrafik angewandt, die nicht unmittelbar einem physikalischen Deformationsprozess entsprechen. Zu diesem Zweck wird das physikalische Modell durch eine passende Änderung der potentiellen Energie verallgemeinert. Die potentielle Energie entspricht normalerweise der physikalischen Arbeit, die aufgewendet werden muss, um einen Körper aus dem Ruhezustand in eine bestimmte Konfiguration zu verformen. Darüber hinaus setzt sie die aktuelle Verformung in Beziehung zu internen Spannungskräften, die wirken um die ursprüngliche Form wiederherzustellen. In dieser Arbeit passen wir für jedes der drei betrachteten Problemfelder die potentielle Energie jeweils so an, dass sie eine anwendungsspezifische Definition von Form widerspiegelt. Unter dem Einfluss weiterer Randbedingungen führt die so verallgemeinerte Deformation zu einer Fläche, die eine Balance zwischen der Erhaltung gewisser Formeigenschaften und Zielvorgaben der Anwendung findet. Diese Balance entspricht dem Equilibrium einer physikalischen Deformation. Die drei in dieser Arbeit diskutierten Anwendungen sind Oberflächenparameterisierung, interaktives Bearbeiten von Flächen und das vollautomatische Erzeugen von Panoramakarten im Stile von Heinrich Berann. Zur Oberflächenparameterisierung interpretieren wir Parameterisierungen über einem flachen Parametergebiet als Deformationen, die ein ursprünglich ebenes Flächenstück in eine gegebene Oberfläche verformen. Innerhalb dieses Szenarios vergleichen wir dann existierende Methoden zur planaren Parameterisierung, indem wir die resultierenden potentiellen Energien analysieren, und leiten weitere Potentiale her, die die Störung geometrischer Eigenschaften wie Fläche und Winkel erfassen. Verfahren zur interaktiven Flächenbearbeitung ermöglichen schnelle und intuitive Änderungen an einer komplexen Oberfläche. Dazu wählt der Benutzer Teile der Fläche und bewegt diese durch den Raum. Ein Deformationsmodell extrapoliert interaktiv die Transformation der gewählten Teile auf die restliche Fläche. Diese Arbeit stellt eine neue differentielle Flächenrepräsentation für diskrete Flächen vor, die zu einem einfach und interaktiv zu optimierendem Potential führt. Obwohl das vorgeschlagene Potential nicht physikalisch korrekt ist, sind die resultierenden Deformationen intuitiv. Mittels eines Parameters lassen sich außerdem bestimmte Materialeigenschaften einstellen. Panoramakarten im Stile von Heinrich Berann sind eine Verschmelzung von Landschaftsillustration und geographischer Karte. Traditionell werden sie so von Hand gezeichnet, dass bestimmt Merkmale wie beispielsweise Skipisten oder Wanderwege in einem Gebiet unverdeckt und gut sichtbar bleiben, was große Kunstfertigkeit verlangt. Obwohl diese Art der Darstellung nicht maßstabsgetreu ist, sind Abweichungen auf den ersten Blick meistens nicht zu erkennen. Dadurch kann der Betrachter markante Details schnell wiederfinden und sich so innerhalb des Gebietes orientieren. Diese Arbeit stellt das erste, vollautomatische Verfahren zur Erzeugung von Panoramakarten vor. Grundlage ist wiederum eine verallgemeinerte Oberflächendeformation, die sowohl auf Formerhaltung als auch auf die Sichtbarkeit vorgegebener geographischer Merkmale abzielt