Joint Demosaicking / Rectification of Fisheye Camera Images using Multi-color Graph Laplacian Regulation

To compose one 360 degrees image from multiple viewpoint images taken from different fisheye cameras on a rig for viewing on a head-mounted display (HMD), a conventional processing pipeline first performs demosaicking on each fisheye camera's Bayer-patterned grid, then translates demosaicked pixels from the camera grid to a rectified image grid. By performing two image interpolation steps in sequence, interpolation errors can accumulate, and acquisition noise in each captured pixel can pollute its neighbors, resulting in correlated noise. In this paper, a joint processing framework is proposed that performs demosaicking and grid-to-grid mapping simultaneously, thus limiting noise pollution to one interpolation. Specifically, a reverse mapping function is first obtained from a regular on-grid location in the rectified image to an irregular off-grid location in the camera's Bayer-patterned image. For each pair of adjacent pixels in the rectified grid, its gradient is estimated using the pair's neighboring pixel gradients in three colors in the Bayer-patterned grid. A similarity graph is constructed based on the estimated gradients, and pixels are interpolated in the rectified grid directly via graph Laplacian regularization (GLR). To establish ground truth for objective testing, a large dataset containing pairs of simulated images both in the fisheye camera grid and the rectified image grid is built. Experiments show that the proposed joint demosaicking / rectification method outperforms competing schemes that execute demosaicking and rectification in sequence in both objective and subjective measures

Joint Demosaicking / Rectification of Fisheye Camera Images using Multi-color Graph Laplacian Regulation

To compose one 360 degrees image from multiple viewpoint images taken from different fisheye cameras on a rig for viewing on a head-mounted display (HMD), a conventional processing pipeline first performs demosaicking on each fisheye camera's Bayer-patterned grid, then translates demosaicked pixels from the camera grid to a rectified image grid. By performing two image interpolation steps in sequence, interpolation errors can accumulate, and acquisition noise in each captured pixel can pollute its neighbors, resulting in correlated noise. In this paper, a joint processing framework is proposed that performs demosaicking and grid-to-grid mapping simultaneously, thus limiting noise pollution to one interpolation. Specifically, a reverse mapping function is first obtained from a regular on-grid location in the rectified image to an irregular off-grid location in the camera's Bayer-patterned image. For each pair of adjacent pixels in the rectified grid, its gradient is estimated using the pair's neighboring pixel gradients in three colors in the Bayer-patterned grid. A similarity graph is constructed based on the estimated gradients, and pixels are interpolated in the rectified grid directly via graph Laplacian regularization (GLR). To establish ground truth for objective testing, a large dataset containing pairs of simulated images both in the fisheye camera grid and the rectified image grid is built. Experiments show that the proposed joint demosaicking / rectification method outperforms competing schemes that execute demosaicking and rectification in sequence in both objective and subjective measures

DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands

Achieving human-like dexterous manipulation remains a crucial area of research in robotics. Current research focuses on improving the success rate of pick-and-place tasks. Compared with pick-and-place, throw-catching behavior has the potential to increase picking speed without transporting objects to their destination. However, dynamic dexterous manipulation poses a major challenge for stable control due to a large number of dynamic contacts. In this paper, we propose a Stability-Constrained Reinforcement Learning (SCRL) algorithm to learn to catch diverse objects with dexterous hands. The SCRL algorithm outperforms baselines by a large margin, and the learned policies show strong zero-shot transfer performance on unseen objects. Remarkably, even though the object in a hand facing sideward is extremely unstable due to the lack of support from the palm, our method can still achieve a high level of success in the most challenging task. Video demonstrations of learned behaviors and the code can be found on the supplementary website

A Policy Optimization Method Towards Optimal-time Stability

In current model-free reinforcement learning (RL) algorithms, stability criteria based on sampling methods are commonly utilized to guide policy optimization. However, these criteria only guarantee the infinite-time convergence of the system's state to an equilibrium point, which leads to sub-optimality of the policy. In this paper, we propose a policy optimization technique incorporating sampling-based Lyapunov stability. Our approach enables the system's state to reach an equilibrium point within an optimal time and maintain stability thereafter, referred to as "optimal-time stability". To achieve this, we integrate the optimization method into the Actor-Critic framework, resulting in the development of the Adaptive Lyapunov-based Actor-Critic (ALAC) algorithm. Through evaluations conducted on ten robotic tasks, our approach outperforms previous studies significantly, effectively guiding the system to generate stable patterns.Comment: 27 pages, 11 figues. 7th Annual Conference on Robot Learning. 202

Development of Mobile Manipulator Robot System with Embodied Intelligence

Embodied intelligence stands as a strategic technology in the ongoing scientific and technological revolution, forming a frontier in global competition. The mobile manipulator robot system, with its exceptional mobility, planning, and execution capabilities, has become the preferred hardware carrier for embodied intelligence. Moreover, the mobile manipulator robot system, rooted in embodied intelligence, emerges as a pivotal platform capable of cross-domain functionality. Positioned at the forefront of a new era in information technology and artificial intelligence, this system is integral for future development. Addressing the strategic demand for embodied-intelligence-based mobile manipulator robot systems, this study presents an overview of the current developmental landscape. It delves into the challenges faced by this field, proposing key common technologies such as multimodal perception, world cognition, intelligent autonomous decision-making, and joint planning for movement and manipulation. Furthermore, the study offers recommendations for advancing the field, encompassing national policy support, breakthroughs in common technologies, interdisciplinary collaboration, talent cultivation, and construction of comprehensive verification platforms. These suggestions aim to facilitate the rapid progress of mobile manipulator robots in China amid the wave of embodied intelligence development

Rapid fabrication of a microdevice with concave microwells and its application in embryoid body formation

Here, we report a novel method for the fabrication of polydimethylsiloxane microdevices with complicated 3-D structures, such as concave and crater shapes, using an easily machined polymethyl methacrylate mold combined with a one-step molding process. The procedure presented here enables rapid preparation of complex 3-D microstructures varying in shape and dimensions. To regulate embryoid body (EB) formation, we fabricated a microfluidic device with an array of concave microwells and found that EBs growing in microwells maintained their shape, viability, and a high degree of homogeneity. We believe that this novel method provides an alternative for rapid prototyping, especially in fabricating devices with curved 3-D microstructures

Preparation and Finite Element Analysis of Fly Ash/HDPE Composites for Large Diameter Bellows

In recent years, buried bellows have often had safety accidents such as pipeline bursts and ground subsidence due to the lack of adequate mechanical properties and other quality problems. In order to improve the mechanical properties of bellows, fly ash (FA) was used as a reinforced filler in high density polyethylene (HDPE) to develop composites. The FA was surface treated with a silane coupling agent and HDPE-g-maleic anhydride was used as compatibilizer. Dumbbell-shaped samples were prepared via extrusion blending and injection molding. The cross-section morphology, thermal stability and mechanical properties of the composites were studied. It was observed that when 10% modified FA and 5% compatibilizer were added to HDPE, the tensile yield strength and tensile breaking strength of the composites were nearly 30.2% and 40.4% higher than those of pure HDPE, respectively, and the Young’s modulus could reach 1451.07 MPa. In addition, the ring stiffness of the bellows was analyzed using finite element analysis. Compared with a same-diameter bellows fabricated from common commercially available materials, the ring stiffness increased by nearly 23%. The preparation method of FA/HDPE is simple, efficient, and low-cost. It is of great significance for the popularization of high-performance bellows and the high value-added utilization of FA