Unsupervised decoding of long-term, naturalistic human neural recordings with automated video and audio annotations

Fully automated decoding of human activities and intentions from direct neural recordings is a tantalizing challenge in brain-computer interfacing. Most ongoing efforts have focused on training decoders on specific, stereotyped tasks in laboratory settings. Implementing brain-computer interfaces (BCIs) in natural settings requires adaptive strategies and scalable algorithms that require minimal supervision. Here we propose an unsupervised approach to decoding neural states from human brain recordings acquired in a naturalistic context. We demonstrate our approach on continuous long-term electrocorticographic (ECoG) data recorded over many days from the brain surface of subjects in a hospital room, with simultaneous audio and video recordings. We first discovered clusters in high-dimensional ECoG recordings and then annotated coherent clusters using speech and movement labels extracted automatically from audio and video recordings. To our knowledge, this represents the first time techniques from computer vision and speech processing have been used for natural ECoG decoding. Our results show that our unsupervised approach can discover distinct behaviors from ECoG data, including moving, speaking and resting. We verify the accuracy of our approach by comparing to manual annotations. Projecting the discovered cluster centers back onto the brain, this technique opens the door to automated functional brain mapping in natural settings

Ipb Biodiversity Informatics (Ipbiotics) Untuk Pembangunan Berkelanjutan

Indonesia is the country with the second highest biodiversity in the world. It is not only the diversity of biodiversity, but also diversity of indigenous knowledge such as functional foods and other traditional ingredients. IPB as one of the leading university in Indonesia has important role in the management of natural resources of biodiversity. Currently, management of biodiversity resource require an integrated and holistic system using computer science and technology which develop rapidly at this time. This study developed a system of biodiversity informatics IPB (IPBiotics) for biodiversity information management of Indonesia's natural resources in order to improve the knowledge management (knowledge management), exploration, analysis, synthesis and interpretation of data ranging from the level of genomic biodiversity, species level to the ecosystem level. Activities undertaken in this research include exploration of organism, biodiversity database development and biodiversity informatics infrastructure using model Resources Descriptions framework RDF with biodiversity data standards. Taxonomic Databases Working Group (TDWG). IPBiotics participatory and integrated. Some of the features of the application that was developed in organism such as IPBiotics system, location mapping and exploration missions. IPBiotics also uses computer vision technology in application development. By IPBiotics we hope that the data information and knowledge of Indonesian natural wealth can be utilized appropriately and optimally, so that the preservation of natural resources can be properly maintained

Neural Encoding and Decoding with Deep Learning for Natural Vision

The overarching objective of this work is to bridge neuroscience and artificial intelligence to ultimately build machines that learn, act, and think like humans. In the context of vision, the brain enables humans to readily make sense of the visual world, e.g. recognizing visual objects. Developing human-like machines requires understanding the working principles underlying the human vision. In this dissertation, I ask how the brain encodes and represents dynamic visual information from the outside world, whether brain activity can be directly decoded to reconstruct and categorize what a person is seeing, and whether neuroscience theory can be applied to artificial models to advance computer vision. To address these questions, I used deep neural networks (DNN) to establish encoding and decoding models for describing the relationships between the brain and the visual stimuli. Using the DNN, the encoding models were able to predict the functional magnetic resonance imaging (fMRI) responses throughout the visual cortex given video stimuli; the decoding models were able to reconstruct and categorize the visual stimuli based on fMRI activity. To further advance the DNN model, I have implemented a new bidirectional and recurrent neural network based on the predictive coding theory. As a theory in neuroscience, predictive coding explains the interaction among feedforward, feedback, and recurrent connections. The results showed that this brain-inspired model significantly outperforms feedforward-only DNNs in object recognition. These studies have positive impact on understanding the neural computations under human vision and improving computer vision with the knowledge from neuroscience

IPB BIODIVERSITY INFORMATICS (IPBIOTICS) UNTUK PEMBANGUNAN BERKELANJUTAN

Indonesia is the country with the second highest biodiversity in the world. It is not only the diversity of biodiversity, but also diversity of indigenous knowledge such as functional foods and other traditional ingredients. IPB as one of the leading university in Indonesia has important role in the management of natural resources of biodiversity. Currently, management of biodiversity resource require an integrated and holistic system using computer science and technology which develop rapidly at this time. This study developed a system of biodiversity informatics IPB (IPBiotics) for biodiversity information management of indonesia’s natural resources in order to improve the knowledge management (knowledge management), exploration, analysis, synthesis and interpretation of data ranging from the level of genomic biodiversity, species level to the ecosystem level. Activities undertaken in this research include exploration of organism, biodiversity database development and biodiversity informatics infrastructure using model Resources Descriptions framework RDF with biodiversity data standards. Taxonomic Databases Working Group (TDWG). IPBiotics participatory and integrated. Some of the features of the application that was developed in organism such as IPBiotics system, location mapping and exploration missions. IPBiotics also uses computer vision technology in application development. By IPBiotics we hope that the data information and knowledge of indonesian natural wealth can be utilized appropriately and optimally, so that the preservation of natural resources can be properly maintained. Keywords: Biodiversity informatics, Computer vision, Databases, IPBiotics, Sustainability