Dimensions of Timescales in Neuromorphic Computing Systems

This article is a public deliverable of the EU project "Memory technologies with multi-scale time constants for neuromorphic architectures" (MeMScales, https://memscales.eu, Call ICT-06-2019 Unconventional Nanoelectronics, project number 871371). This arXiv version is a verbatim copy of the deliverable report, with administrative information stripped. It collects a wide and varied assortment of phenomena, models, research themes and algorithmic techniques that are connected with timescale phenomena in the fields of computational neuroscience, mathematics, machine learning and computer science, with a bias toward aspects that are relevant for neuromorphic engineering. It turns out that this theme is very rich indeed and spreads out in many directions which defy a unified treatment. We collected several dozens of sub-themes, each of which has been investigated in specialized settings (in the neurosciences, mathematics, computer science and machine learning) and has been documented in its own body of literature. The more we dived into this diversity, the more it became clear that our first effort to compose a survey must remain sketchy and partial. We conclude with a list of insights distilled from this survey which give general guidelines for the design of future neuromorphic systems

Reservoir Computing in Materio

Reservoir Computing first emerged as an efficient mechanism for training recurrent neural networks and later evolved into a general theoretical model for dynamical systems. By applying only a simple training mechanism many physical systems have become exploitable unconventional computers. However, at present, many of these systems require careful selection and tuning by hand to produce usable or optimal reservoir computers. In this thesis we show the first steps to applying the reservoir model as a simple computational layer to extract exploitable information from complex material substrates. We argue that many physical substrates, even systems that in their natural state might not form usable or "good" reservoirs, can be configured into working reservoirs given some stimulation. To achieve this we apply techniques from evolution in materio whereby configuration is through evolved input-output signal mappings and targeted stimuli. In preliminary experiments the combined model and configuration method is applied to carbon nanotube/polymer composites. The results show substrates can be configured and trained as reservoir computers of varying quality. It is shown that applying the reservoir model adds greater functionality and programmability to physical substrates, without sacrificing performance. Next, the weaknesses of the technique are addressed, with the creation of new high input-output hardware system and an alternative multi-substrate framework. Lastly, a substantial effort is put into characterising the quality of a substrate for reservoir computing, i.e its ability to realise many reservoirs. From this, a methodological framework is devised. Using the framework, radically different computing substrates are compared and assessed, something previously not possible. As a result, a new understanding of the relationships between substrate, tasks and properties is possible, outlining the way for future exploration and optimisation of new computing substrates

Co-designing the computational model and the computing substrate

Given a proposed unconventional computing substrate, we can ask: Does it actually compute? If so, how well does it compute? Can it be made to compute better? Given a proposed unconventional computational model we can ask: How powerful is the model? Can it be implemented in a substrate? How faithfully or efficiently can it be implemented? Given complete freedom in the choice of model and substrate, we can ask: Can we co-design a model and substrate to work well together? Here I propose an approach to posing and answering these questions, building on an existing definition of physical computing and framework for characterising the computing properties of given substrates

Black shale lithofacies prediction and distribution Pattern analysis of middle Devonian Marcellus Shale in the Appalachian Basin, northeastern U.S.A.

The Marcellus Shale, marine organic-rich mudrock deposited during Middle Devonian in the Appalachian basin, is considered the largest unconventional shale-gas resource in United State. Although homogeneous in the appearance, the mudstone shows heterogeneity in mineral composition, organic matter richness, gas content, and fracture density. Two critical factors for unconventional mudstone reservoirs are units amenable to hydraulic fracture stimulation and rich of organic matter. The effectiveness of hydraulic fracture stimulation is influenced by rock geomechanical properties, which are related to rock mineralogy. The natural gas content in mudrock reservoirs has a strong relationship with organic matter, which is measured by total organic carbon (TOC). In place of using petrographic information and sedimentary structures, Marcellus Shale lithofacies were based on mineral composition and organic matter richness and were predicted by conventional logs to make the lithofacies \u27meaningful’, ‘predictable’ and ‘mappable’ at multiple scales from the well bore to basin. Core X-ray diffraction (XRD) and TOC data was used to classify Marcellus Shale into seven lithofacies according to three criteria: clay volume, the ratio of quartz to carbonate, and TOC. Pulsed neutron spectroscopy (PNS) logs provide similar mineral concentration and TOC content, and were used to classify shale lithofacies by the same three criteria. Artificial neural network (ANN) with improvements (i.e., learning algorithms, performance function and topology design) was utilized to predict Marcellus Shale lithofacies in 707 wells with conventional logs. To improve the effectiveness of wireline logs to predict lithofacies, the effects of barite and pyrite were partly removed and eight petrophysical parameters commonly used for a conventional reservoir analysis were derived from conventional logs by petrophysical analysis. These parameters were used as input to the ANN analysis. Geostatistical analysis was used to develop the experimental variogram models and vertical proportion of each lithofacies. Indictor kriging, truncated Gaussian simulation (TGS), and sequential indicator simulation (SIS) were compared, and SIS algorithm performed well for modeling Marcellus Shale lithofacies in three-dimensions. Controlled primarily by sediment dilution, organic matter productivity, and organic matter preservation/decomposition, Marcellus Shale lithofacies distribution was dominantly affected by the water depth and the distance to shoreline. The Marcellus Shale lithofacies with the greatest organic content and highest measure of brittleness is concentrated along a crescent shape region paralleling the inferred shelf and shoreline, showing shape of crescent paralleling with shoreline. The normalized average gas production rate from horizontal wells supported the proposed approach to modeling Marcellus Shale lithofacies. The proposed 3-D modeling approach may be helpful for (1) investigating the distribution of each lithofacies at a basin-scale; (2) developing a better understanding of the factors controlling the deposition and preservation of organic matter and the depositional model of marine organic-rich mudrock; (3) identifying organic-rich units and areas and brittle units and areas in shale-gas reservoirs; (4) assisting in the design of horizontal drilling trajectories and location of stimulation activity; and (5) providing input parameters for the simulation of gas flow and production in mudrock (e.g., porosity, permeability and fractures)