Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Fluctuations between high- and low-modularity topology in time-resolved functional connectivity

Modularity is an important topological attribute for functional brain networks. Recent studies have reported that modularity of functional networks varies not only across individuals being related to demographics and cognitive performance, but also within individuals co-occurring with fluctuations in network properties of functional connectivity, estimated over short time intervals. However, characteristics of these time-resolved functional networks during periods of high and low modularity have remained largely unexplored. In this study we investigate spatiotemporal properties of time-resolved networks in the high and low modularity periods during rest, with a particular focus on their spatial connectivity patterns, temporal homogeneity and test-retest reliability. We show that spatial connectivity patterns of time-resolved networks in the high and low modularity periods are represented by increased and decreased dissociation of the default mode network module from task-positive network modules, respectively. We also find that the instances of time-resolved functional connectivity sampled from within the high (low) modularity period are relatively homogeneous (heterogeneous) over time, indicating that during the low modularity period the default mode network interacts with other networks in a variable manner. We confirmed that the occurrence of the high and low modularity periods varies across individuals with moderate inter-session test-retest reliability and that it is correlated with previously-reported individual differences in the modularity of functional connectivity estimated over longer timescales. Our findings illustrate how time-resolved functional networks are spatiotemporally organized during periods of high and low modularity, allowing one to trace individual differences in long-timescale modularity to the variable occurrence of network configurations at shorter timescales.Comment: Reorganized the paper; to appear in NeuroImage; arXiv abstract shortened to fit within character limit

Cognitive satellite communications and representation learning for streaming and complex graphs.

This dissertation includes two topics. The first topic studies a promising dynamic spectrum access algorithm (DSA) that improves the throughput of satellite communication (SATCOM) under the uncertainty. The other topic investigates distributed representation learning for streaming and complex networks. DSA allows a secondary user to access the spectrum that are not occupied by primary users. However, uncertainty in SATCOM causes more spectrum sensing errors. In this dissertation, the uncertainty has been addressed by formulating a DSA decision-making process as a Partially Observable Markov Decision Process (POMDP) model to optimally determine which channels to sense and access. Large-scale networks have attracted many attentions to discover the hidden information from big data. Particularly, representation learning embeds the network into a lower vector space while maximally preserving the similarity among nodes. I propose a real-time distributed graph embedding algorithm (RTDGE) which is capable of distributively embedding the streaming graph by combining a novel edge partition approach and an incremental negative sample approach. Furthermore, a platform is prototyped based on Kafka and Storm. Real-time Twitter network data can be retrieved, partitioned and processed for state-of-art tasks. For knowledge graphs, existing works cannot capture the complex connection patterns and never consider the impacts from complicated relations, due to the unquantifiable relationships. A novel embedding algorithm is proposed to hierarchically measure the structural similarity and the impacts from relations by constructing a multi-layer graph. Then, an advanced representation learning model is designed based on an entity\u27s context generated by random walks on the multi-layer content graph

Cognition-inspired 5G cellular networks: a review and the road ahead

Despite the evolution of cellular networks, spectrum scarcity and the lack of intelligent and autonomous capabilities remain a cause for concern. These problems have resulted in low network capacity, high signaling overhead, inefficient data forwarding, and low scalability, which are expected to persist as the stumbling blocks to deploy, support and scale next-generation applications, including smart city and virtual reality. Fifth-generation (5G) cellular networking, along with its salient operational characteristics - including the cognitive and cooperative capabilities, network virtualization, and traffic offload - can address these limitations to cater to future scenarios characterized by highly heterogeneous, ultra-dense, and highly variable environments. Cognitive radio (CR) and cognition cycle (CC) are key enabling technologies for 5G. CR enables nodes to explore and use underutilized licensed channels; while CC has been embedded in CR nodes to learn new knowledge and adapt to network dynamics. CR and CC have brought advantages to a cognition-inspired 5G cellular network, including addressing the spectrum scarcity problem, promoting interoperation among heterogeneous entities, and providing intelligence and autonomous capabilities to support 5G core operations, such as smart beamforming. In this paper, we present the attributes of 5G and existing state of the art focusing on how CR and CC have been adopted in 5G to provide spectral efficiency, energy efficiency, improved quality of service and experience, and cost efficiency. This main contribution of this paper is to complement recent work by focusing on the networking aspect of CR and CC applied to 5G due to the urgent need to investigate, as well as to further enhance, CR and CC as core mechanisms to support 5G. This paper is aspired to establish a foundation and to spark new research interest in this topic. Open research opportunities and platform implementation are also presented to stimulate new research initiatives in this exciting area

Mechanisms of cognitive-behavioral therapy for obsessive-compulsive disorder involve robust and extensive increases in brain network connectivity.

Cognitive-behavioral therapy (CBT) is effective for obsessive compulsive disorder (OCD); however, little is understood about its mechanisms related to brain network connectivity. We examined connectivity changes from resting-state functional magnetic resonance imaging data pre-to-post-CBT in 43 OCD participants, randomized to receive either 4 weeks of intensive CBT or 4 weeks waitlist followed by 4 weeks of CBT, and 24 healthy controls before and after 4 weeks of no treatment. Network-based-statistic analysis revealed large-magnitude increases in OCD connectivity in eight networks. Strongest increases involved connectivity between the cerebellum and caudate/putamen, and between the cerebellum and dorsolateral/ventrolateral prefrontal cortices. Connectivity increases were associated with increased resistance to compulsions. Mechanisms of CBT may involve enhanced cross-network integration, both within and outside of classical cortico-striatal-thalamo-cortical regions; those involving cerebellar to striatal and prefrontal regions may reflect acquisition of new non-compulsive goal-directed behaviors and thought patterns. Our findings have implications for identifying targets for enhancing treatment efficacy and monitoring treatment progress

MAC/PHY Co-Design of CSMA Wireless Networks Using Software Radios.

In the past decade, CSMA-based protocols have spawned numerous network standards (e.g., the WiFi family), and played a key role in improving the ubiquity of wireless networks. However, the rapid evolution of CSMA brings unprecedented challenges, especially the coexistence of different network architectures and communications devices. Meanwhile, many intrinsic limitations of CSMA have been the main obstacle to the performance of its derivatives, such as ZigBee, WiFi, and mesh networks. Most of these problems are observed to root in the abstract interface of the CSMA MAC and PHY layers --- the MAC simply abstracts the advancement of PHY technologies as a change of data rate. Hence, the benefits of new PHY technologies are either not fully exploited, or they even may harm the performance of existing network protocols due to poor interoperability. In this dissertation, we show that a joint design of the MAC/PHY layers can achieve a substantially higher level of capacity, interoperability and energy efficiency than the weakly coupled MAC/PHY design in the current CSMA wireless networks. In the proposed MAC/PHY co-design, the PHY layer exposes more states and capabilities to the MAC, and the MAC performs intelligent adaptation to and control over the PHY layer. We leverage the reconfigurability of software radios to design smart signal processing algorithms that meet the challenge of making PHY capabilities usable by the MAC layer. With the approach of MAC/PHY co-design, we have revisited the primitive operations of CSMA (collision avoidance, carrier signaling, carrier sensing, spectrum access and transmitter cooperation), and overcome its limitations in relay and broadcast applications, coexistence of heterogeneous networks, energy efficiency, coexistence of different spectrum widths, and scalability for MIMO networks. We have validated the feasibility and performance of our design using extensive analysis, simulation and testbed implementation.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/95944/1/xyzhang_1.pd

Real time unsupervised learning of visual stimuli in neuromorphic VLSI systems

Neuromorphic chips embody computational principles operating in the nervous system, into microelectronic devices. In this domain it is important to identify computational primitives that theory and experiments suggest as generic and reusable cognitive elements. One such element is provided by attractor dynamics in recurrent networks. Point attractors are equilibrium states of the dynamics (up to fluctuations), determined by the synaptic structure of the network; a `basin' of attraction comprises all initial states leading to a given attractor upon relaxation, hence making attractor dynamics suitable to implement robust associative memory. The initial network state is dictated by the stimulus, and relaxation to the attractor state implements the retrieval of the corresponding memorized prototypical pattern. In a previous work we demonstrated that a neuromorphic recurrent network of spiking neurons and suitably chosen, fixed synapses supports attractor dynamics. Here we focus on learning: activating on-chip synaptic plasticity and using a theory-driven strategy for choosing network parameters, we show that autonomous learning, following repeated presentation of simple visual stimuli, shapes a synaptic connectivity supporting stimulus-selective attractors. Associative memory develops on chip as the result of the coupled stimulus-driven neural activity and ensuing synaptic dynamics, with no artificial separation between learning and retrieval phases.Comment: submitted to Scientific Repor