Improving Surgical Training Phantoms by Hyperrealism: Deep Unpaired Image-to-Image Translation from Real Surgeries

Current `dry lab' surgical phantom simulators are a valuable tool for surgeons which allows them to improve their dexterity and skill with surgical instruments. These phantoms mimic the haptic and shape of organs of interest, but lack a realistic visual appearance. In this work, we present an innovative application in which representations learned from real intraoperative endoscopic sequences are transferred to a surgical phantom scenario. The term hyperrealism is introduced in this field, which we regard as a novel subform of surgical augmented reality for approaches that involve real-time object transfigurations. For related tasks in the computer vision community, unpaired cycle-consistent Generative Adversarial Networks (GANs) have shown excellent results on still RGB images. Though, application of this approach to continuous video frames can result in flickering, which turned out to be especially prominent for this application. Therefore, we propose an extension of cycle-consistent GANs, named tempCycleGAN, to improve temporal consistency.The novel method is evaluated on captures of a silicone phantom for training endoscopic reconstructive mitral valve procedures. Synthesized videos show highly realistic results with regard to 1) replacement of the silicone appearance of the phantom valve by intraoperative tissue texture, while 2) explicitly keeping crucial features in the scene, such as instruments, sutures and prostheses. Compared to the original CycleGAN approach, tempCycleGAN efficiently removes flickering between frames. The overall approach is expected to change the future design of surgical training simulators since the generated sequences clearly demonstrate the feasibility to enable a considerably more realistic training experience for minimally-invasive procedures.Comment: 8 pages, accepted at MICCAI 2018, supplemental material at https://youtu.be/qugAYpK-Z4

Augmented Reality

Augmented Reality (AR) is a natural development from virtual reality (VR), which was developed several decades earlier. AR complements VR in many ways. Due to the advantages of the user being able to see both the real and virtual objects simultaneously, AR is far more intuitive, but it's not completely detached from human factors and other restrictions. AR doesn't consume as much time and effort in the applications because it's not required to construct the entire virtual scene and the environment. In this book, several new and emerging application areas of AR are presented and divided into three sections. The first section contains applications in outdoor and mobile AR, such as construction, restoration, security and surveillance. The second section deals with AR in medical, biological, and human bodies. The third and final section contains a number of new and useful applications in daily living and learning

History of Computer Art

A large text presents the history of Computer Art. The history of the artistic uses of computers and computing processes is reconstructed from its beginnings in the fifties to its present state. It points out hypertextual, modular and generative modes to use computing processes in Computer Art and features examples of early developments in media like cybernetic sculptures, video tools, computer graphics and animation (including music videos and demos), video and computer games, pervasive games, reactive installations, virtual reality, evolutionary art and net art. The functions of relevant art works are explained more detailed than is usual in such histories. From October 2011 to December 2012 the chapters have been published successively in German (The English translation started in August 2013 and was completed in June 2014)

History of Computer Art

Die Entwicklung von Computer und Software von den fünfziger Jahren bis heute wird vorgestellt. Als Leitkriterien der Geschichte der Computerkunst werden ein Interface-Modell und drei Arten, Rechenprozesse einzusetzen (generativ, modular, hyptertextuell), vorgeschlagen. Die "Geschichte der Computerkunst"/"History of Computer Art" erörtert Beispiele aus frühen Entwicklungsphasen von Kunstformen wie Kybernetische Skulpturen, Computergraphik und -animation (einschließlich Musikvideos und Demos), Videokunst und Computerspielen, reaktive Installationen, Virtuelle Realität, Evolutionäre Kunst und Netzkunst. Die Funktionen der ausgewählten Werke werden detaillierter vorgestellt als dies in vergleichbaren Geschichten üblich ist. Die deutsche Version wurde bis Dezember 2012 kapitelweise in IASLonline Lektionen/Lessons in Net Art publiziert. Das letzte Kapitel der englischen Version wurde Juni 2014 veröffentlicht. Im September 2015 wurde ein erstes Update eingestellt

History of Computer Art

The development of the use of computers and software in art from the Fifties to the present is explained. As general aspects of the history of computer art an interface model and three dominant modes to use computational processes (generative, modular, hypertextual) are presented. The "History of Computer Art" features examples of early developments in media like cybernetic sculptures, computer graphics and animation (including music videos and demos), video and computer games, reactive installations, virtual reality, evolutionary art and net art. The functions of relevant art works are explained more detailed than usual in such histories. The German version was completed in December 2012. The last chapter of the English translation was published in June 2014. First update: September 2015