RGB-D datasets using microsoft kinect or similar sensors: a survey

RGB-D data has turned out to be a very useful representation of an indoor scene for solving fundamental computer vision problems. It takes the advantages of the color image that provides appearance information of an object and also the depth image that is immune to the variations in color, illumination, rotation angle and scale. With the invention of the low-cost Microsoft Kinect sensor, which was initially used for gaming and later became a popular device for computer vision, high quality RGB-D data can be acquired easily. In recent years, more and more RGB-D image/video datasets dedicated to various applications have become available, which are of great importance to benchmark the state-of-the-art. In this paper, we systematically survey popular RGB-D datasets for different applications including object recognition, scene classification, hand gesture recognition, 3D-simultaneous localization and mapping, and pose estimation. We provide the insights into the characteristics of each important dataset, and compare the popularity and the difficulty of those datasets. Overall, the main goal of this survey is to give a comprehensive description about the available RGB-D datasets and thus to guide researchers in the selection of suitable datasets for evaluating their algorithms

Real-time marker-less multi-person 3D pose estimation in RGB-Depth camera networks

This paper proposes a novel system to estimate and track the 3D poses of multiple persons in calibrated RGB-Depth camera networks. The multi-view 3D pose of each person is computed by a central node which receives the single-view outcomes from each camera of the network. Each single-view outcome is computed by using a CNN for 2D pose estimation and extending the resulting skeletons to 3D by means of the sensor depth. The proposed system is marker-less, multi-person, independent of background and does not make any assumption on people appearance and initial pose. The system provides real-time outcomes, thus being perfectly suited for applications requiring user interaction. Experimental results show the effectiveness of this work with respect to a baseline multi-view approach in different scenarios. To foster research and applications based on this work, we released the source code in OpenPTrack, an open source project for RGB-D people tracking.Comment: Submitted to the 2018 IEEE International Conference on Robotics and Automatio

GEMINI: A Generic Multi-Modal Natural Interface Framework for Videogames

In recent years videogame companies have recognized the role of player engagement as a major factor in user experience and enjoyment. This encouraged a greater investment in new types of game controllers such as the WiiMote, Rock Band instruments and the Kinect. However, the native software of these controllers was not originally designed to be used in other game applications. This work addresses this issue by building a middleware framework, which maps body poses or voice commands to actions in any game. This not only warrants a more natural and customized user-experience but it also defines an interoperable virtual controller. In this version of the framework, body poses and voice commands are respectively recognized through the Kinect's built-in cameras and microphones. The acquired data is then translated into the native interaction scheme in real time using a lightweight method based on spatial restrictions. The system is also prepared to use Nintendo's Wiimote as an auxiliary and unobtrusive gamepad for physically or verbally impractical commands. System validation was performed by analyzing the performance of certain tasks and examining user reports. Both confirmed this approach as a practical and alluring alternative to the game's native interaction scheme. In sum, this framework provides a game-controlling tool that is totally customizable and very flexible, thus expanding the market of game consumers.Comment: WorldCIST'13 Internacional Conferenc

Evidence for the prepattern/cooption model of vertebrate jaw evolution

The appearance of jaws was a turning point in vertebrate evolution because it allowed primitive vertebrates to capture and process large, motile prey. The vertebrate jaw consists of separate dorsal and ventral skeletal elements connected by a joint. How this structure evolved from the unjointed gill bar of a jawless ancestor is an unresolved question in vertebrate evolution. To understand the developmental bases of this evolutionary transition, we examined the expression of 12 genes involved in vertebrate pharyngeal patterning in the modern jawless fish lamprey. We find nested expression of Dlx genes, as well as combinatorial expression of Msx, Hand and Gsc genes along the dorso-ventral (DV) axis of the lamprey pharynx, indicating gnathostome-type pharyngeal patterning evolved before the appearance of the jaw. In addition, we find that Bapx and Gdf5/6/7, key regulators of joint formation in gnathostomes, are not expressed in the lamprey first arch, whereas Barx, which is absent from the intermediate first arch in gnathostomes, marks this domain in lamprey. Taken together, these data support a new scenario for jaw evolution in which incorporation of Bapx and Gdf5/6/7 into a preexisting DV patterning program drove the evolution of the jaw by altering the identity of intermediate first-arch chondrocytes. We present this “Pre-pattern/Cooption” model as an alternative to current models linking the evolution of the jaw to the de novo appearance of sophisticated pharyngeal DV patterning

Co-culture of Adult Mesenchymal Stem Cells and Nucleus Pulposus Cells in Bilaminar Pellets for Intervertebral Disc Regeneration

Background: Our goal is to optimize stem cell-based tissue engineering strategies in the context of the intervertebral disc environment. We explored the benefits of co-culturing nucleus pulposus cells (NPC) and adult mesenchymal stem cells (MSC) using a novel spherical bilaminar pellet culture system where one cell type is enclosed in a sphere of the other cell type. Our 3D system provides a structure that exploits embryonic processes such as tissue induction and condensation. We observed a unique phenomenon: the budding of co-culture pellets and the formation of satellite pellets that separate from the main pellet. Methods: MSC and NPC co-culture pellets were formed with three different structural organizations. The first had random organization. The other two had bilaminar organization with either MSC inside and NPC outside or NPC inside and MSC outside. Results: By 14 days, all co-culture pellets exhibited budding and spontaneously generated satellite pellets. The satellite pellets were composed of both cell types and, surprisingly, all had the same bilaminar organization with MSC on the inside and NPC on the outside. This organization was independent of the structure of the main pellet that the satellites stemmed from. Conclusion: The main pellets generated satellite pellets that spontaneously organized into a bilaminar structure. This implies that structural organization occurs naturally in this cell culture system and may be inherently favorable for cell-based tissue engineering strategies. The occurrence of budding and the organization of satellite pellets may have important implications for the use of co-culture pellets in cell-based therapies for disc regeneration. Clinical Relevance: From a therapeutic point of view, the generation of satellite pellets may be a beneficial feature that would serve to spread donor cells throughout the host matrix and restore normal matrix composition in a sustainable way, ultimately renewing tissue function. © 2009 The Spine Arthroplasy Society