Recurrent neural networks for hydrodynamic imaging using a 2D-sensitive artificial lateral line

The lateral line is a mechanosensory organ found in fish and amphibians that allows them to sense and act on their near-field hydrodynamic environment. We present a 2D-sensitive Artificial lateral line (ALL) comprising eight all-optical flow sensors, which we use to measure hydrodynamic velocity profiles along the sensor array in response to a moving object in its vicinity. We then use the measured velocity profiles to reconstruct the objects location, via two types of neural networks: feed-forward and recurrent. Several implementations of feed-forward neural networks for ALL source localisation exist, while recurrent neural networks may be more appropriate for this task. The performance of a recurrent neural network (the Long Short-Term Memory, LSTM) is compared to that of a feed-forward neural network (the Online-Sequential Extreme Learning Machine, OS-ELM) via localizing a 6 cm sphere moving at 13 cm/s. Results show that, in a 62 cm × 9.5 cm area of interest, the LSTM outperforms the OS-ELM with an average localisation error of 0.72 cm compared to 4.27 cm respectively. Furthermore, the recurrent network is relatively less affected by noise, indicating that recurrent connections can be beneficial for hydrodynamic object localisation

Three-dimensional multi-source localization of underwater objects using convolutional neural networks for artificial lateral lines

This research focuses on the signal processing required for a sensory system that can simultaneously localize multiple moving underwater objects in a three-dimensional (3D) volume by simulating the hydrodynamic flow caused by these objects. We propose a method for localization in a simulated setting based on an established hydrodynamic theory founded in fish lateral line organ research. Fish neurally concatenate the information of multiple sensors to localize sources. Similarly, we use the sampled fluid velocity via two parallel lateral lines to perform source localization in three dimensions in two steps. Using a convolutional neural network, we first estimate a two-dimensional image of the probability of a present source. Then we determine the position of each source, via an automated iterative 3D-aware algorithm. We study various neural network architectural designs and different ways of presenting the input to the neural network; multi-level amplified inputs and merged convolutional streams are shown to improve the imaging performance. Results show that the combined system can exhibit adequate 3D localization of multiple sources

Bio-inspired all-optical artificial neuromast for 2D flow sensing

We present the design, fabrication and testing of a novel all-optical 2D flow velocity sensor, inspired by a fish lateral line neuromast. This artificial neuromast consists of optical fibres inscribed with Bragg gratings supporting a fluid force recipient sphere. Its dynamic response is modelled based on the Stokes solution for unsteady flow around a sphere and found to agree with experimental results. Tuneable mechanical resonance is predicted, allowing a deconvolution scheme to accurately retrieve fluid flow speed and direction from sensor readings. The optical artificial neuromast achieves a low frequency threshold flow sensing of 5 mm s(-1) and 5 mu m s(-1) at resonance, with a typical linear dynamic range of 38 dB at 100 Hz sampling. Furthermore, the optical artificial neuromast is shown to determine flow direction within a few degrees

The sensitivity of lateral line receptors and their role in the behavior of Mexican blind cavefish (Astyanax mexicanus)

The characid fish species Astyanax mexicanus offers a classic comparative model for the evolution of sensory systems. Populations of this species evolved in caves and became blind while others remained in streams (i.e. surface fish) and retained a functional visual system. The flow-sensitive lateral line receptors, called superficial neuromasts, are more numerous in cavefish than in surface fish, but it is unclear whether individual neuromasts differ in sensitivity between these populations. The aims of this study were to determine whether the neuromasts in cavefish impart enhanced sensitivity relative to surface fish and to test whether this aids their ability to sense flow in the absence of visual input. Sensitivity was assessed by modeling the mechanics and hydrodynamics of a flow stimulus. This model required that we measure the dimensions of the transparent cupula of a neuromast, which was visualized with fluorescent microspheres. We found that neuromasts within the eye orbit and in the suborbital region were larger and consequently about twice as sensitive in small adult cavefish as in surface fish. Behavioral experiments found that these cavefish, but not surface fish, were attracted to a 35 Hz flow stimulus. These results support the hypothesis that the large superficial neuromasts of small cavefish aid in flow sensing. We conclude that the morphology of the lateral line could have evolved in cavefish to permit foraging in a cave environment

Performance of neural networks for localizing moving objects with an artificial lateral line

Fish are able to sense water flow velocities relative to their body with their mechanoreceptive lateral line organ. This organ consists of an array of flow detectors distributed along the fish body. Using the excitation of these individual detectors, fish can determine the location of nearby moving objects. Inspired by this sensory modality, it is shown here how neural networks can be used to extract an object's location from simulated excitation patterns, as can be measured along arrays of stationary artificial flow velocity sensors. The applicability, performance and robustness with respect to input noise of different neural network architectures are compared. When trained and tested under high signal to noise conditions (46 dB), the Extreme Learning Machine architecture performs best with a mean Euclidean error of 0.4% of the maximum depth of the field D, which is taken half the length of the sensor array. Under lower signal to noise conditions Echo State Networks, having recurrent connections, enhance the performance while the Multilayer Perceptron is shown to be the most noise robust architecture. Neural network performance decreased when the source moves close to the sensor array or to the sides of the array. For all considered architectures, increasing the number of detectors per array increased localization performance and robustness

Frequency-Dependent Properties of a Fluid Jet Stimulus: Calibration, Modeling, and Application to Cochlear Hair Cell Bundles

The investigation of small physiological mechano-sensory systems, such as hair cells or their accessory structures in the inner ear or lateral line organ, requires mechanical stimulus equipment that allows spatial manipulation with micrometer precision and stimulation with amplitudes down to the nanometer scale. Here, we describe the calibration of a microfluid jet produced by a device that was designed to excite individual cochlear hair cell bundles or cupulae of the fish superficial lateral line system. The calibration involves a precise definition of the linearity and time- and frequency-dependent characteristics of the fluid jet as produced by a pressurized fluid-filled container combined with a glass pipette having a microscopically sized tip acting as an orifice. A procedure is described that can be applied during experiments to obtain a fluid jet’s frequency response, which may vary with each individual glass pipette. At small orifice diameters (<15 μm), the fluid velocity of the jet is proportional to the displacement of the piezoelectric actuator pressurizing the container’s volume and is suitable to stimulate the hair bundles of sensory hair cells. With increasing diameter, the fluid jet velocity becomes proportional to the actuator’s velocity. The experimentally observed characteristics can be described adequately by a dynamical model of damped fluid masses coupled by elastic components

d-Tubocurarine and Berbamine: Alkaloids That Are Permeant Blockers of the Hair Cell's Mechano-Electrical Transducer Channel and Protect from Aminoglycoside Toxicity

Aminoglycoside antibiotics are widely used for the treatment of life-threatening bacterial infections, but cause permanent hearing loss in a substantial proportion of treated patients. The sensory hair cells of the inner ear are damaged following entry of these antibiotics via the mechano-electrical transducer (MET) channels located at the tips of the hair cell’s stereocilia. d-Tubocurarine (dTC) is a MET channel blocker that reduces the loading of gentamicin-Texas Red (GTTR) into rat cochlear hair cells and protects them from gentamicin treatment. Berbamine is a structurally related alkaloid that reduces GTTR labeling of zebrafish lateral-line hair cells and protects them from aminoglycoside-induced cell death. Both compounds are thought to reduce aminoglycoside entry into hair cells through the MET channels. Here we show that dTC (≥6.25 µM) or berbamine (≥1.55 µM) protect zebrafish hair cells in vivo from neomycin (6.25 µM, 1 h). Protection of zebrafish hair cells against gentamicin (10 µM, 6 h) was provided by ≥25 µM dTC or ≥12.5 µM berbamine. Hair cells in mouse cochlear cultures are protected from longer-term exposure to gentamicin (5 µM, 48 h) by 20 µM berbamine or 25 µM dTC. Berbamine is, however, highly toxic to mouse cochlear hair cells at higher concentrations (≥30 µM) whilst dTC is not. The absence of toxicity in the zebrafish assays prompts caution in extrapolating results from zebrafish neuromasts to mammalian cochlear hair cells. MET current recordings from mouse outer hair cells (OHCs) show that both compounds are permeant open-channel blockers, rapidly and reversibly blocking the MET channel with half-blocking concentrations of 2.2 µM (dTC) and 2.8 µM (berbamine) in the presence of 1.3 mM Ca2+ at −104 mV. Berbamine, but not dTC, also blocks the hair cell’s basolateral K + current, IK,neo, and modeling studies indicate that berbamine permeates the MET channel more readily than dTC. These studies reveal key properties of MET-channel blockers required for the future design of successful otoprotectants

Hydrodynamic detection by cupulae in a lateral line canal: functional relations between physics and physiology.

In the present review, signal-processing capabilities of the canal lateral line organ imposed by its peripheral architecture are quantified in terms of a limited set of measurable physical parameters. It is demonstrated that cupulae in the lateral line canal organ can only partly be described as canal fluid velocity detectors. Deviation from velocity detection may result from resonance, and can be characterized by the extent to which a single dimensionless resonance number, N ( r ), exceeds 1. This number depends on four physical parameters: it is proportional to cupular size, cupular sliding stiffness and canal fluid density, and inversely proportional to the square of fluid viscosity. Situated in a canal, a cupula may benefit from its resonance by compensating for the limited frequency range of water motion that is efficiently transferred into the lateral line canal. The peripheral transfer of hydrodynamic signals, via canal and cupula, leads to a nearly constant sensitivity to outside water acceleration in a bandwidth that ranges from d.c. to a cut-off frequency of up to several hundreds of Hertz, significantly exceeding the cut-off frequency of the lateral line canal. Threshold values of hydrodynamic detection by the canal lateral line organ are derived in terms of water displacement, water velocity, water acceleration and water pressure gradients and are shown to be close to the detection limits imposed by hair cell mechano-transduction in combination with the physical constraints of peripheral lateral line signal transfer. The notion that the combination of canal- and cupular hydrodynamics effectively provides the lateral line canal organ with a constant sensitivity to water acceleration at low frequencies so that it consequently functions as a low-pass detector of pressure gradients, supports the appropriateness of describing it as a sense organ that "feels at a distance" (Dijkgraaf in Biol Rev 38:51-105, 1963)

The flexural stiffness of superficial neuromasts in the zebrafish (Danio rerio) lateral line

Superficial neuromasts are structures that detect water flow on the surface of the body of fish and amphibians. As a component of the lateral line system, these receptors are distributed along the body, where they sense flow patterns that mediate a wide variety of behaviors. Their ability to detect flow is governed by their structural properties, yet the micromechanics of superficial neuromasts are not well understood. The aim of this study was to examine these mechanics in zebrafish (Danio rerio) larvae by measuring the flexural stiffness of individual neuromasts. Each neuromast possesses a gelatinous cupula that is anchored to hair cells by kinocilia. Using quasi-static bending tests of the proximal region of the cupula, we found that flexural stiffness is proportional to the number of hair cells, and consequently the number of kinocilia, within a neuromast. From this relationship, the flexural stiffness of an individual kinocilium was found to be 2.4x10(-20) N m(2). Using this value, we estimate that the 11 kinocilia in an average cupula generate more than four-fifths of the total flexural stiffness in the proximal region. The relatively minor contribution of the cupular matrix may be attributed to its highly compliant material composition (Young's modulus of similar to 21 Pa). The distal tip of the cupula is entirely composed of this material and is consequently predicted to be at least an order of magnitude more flexible than the proximal region. These findings suggest that the transduction of flow by a superficial neuromast depends on structural dynamics that are dominated by the number and height of kinocilia