A Simple Method to Simultaneously Detect and Identify Spikes from Raw Extracellular Recordings

The ability to track when and which neurons fire in the vicinity of an electrode, in an efficient and reliable manner can revolutionize the neuroscience field. The current bottleneck lies in spike sorting algorithms; existing methods for detecting and discriminating the activity of multiple neurons rely on inefficient, multi-step processing of extracellular recordings. In this work, we show that a single-step processing of raw (unfiltered) extracellular signals is sufficient for both the detection and identification of active neurons, thus greatly simplifying and optimizing the spike sorting approach. The efficiency and reliability of our method is demonstrated in both real and simulated data

VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias

Multimedia content has become ubiquitous on social media platforms, leading to the rise of multimodal misinformation (MM) and the urgent need for effective strategies to detect and prevent its spread. In recent years, the challenge of multimodal misinformation detection (MMD) has garnered significant attention by researchers and has mainly involved the creation of annotated, weakly annotated, or synthetically generated training datasets, along with the development of various deep learning MMD models. However, the problem of unimodal bias in MMD benchmarks -- where biased or unimodal methods outperform their multimodal counterparts on an inherently multimodal task -- has been overlooked. In this study, we systematically investigate and identify the presence of unimodal bias in widely-used MMD benchmarks (VMU-Twitter, COSMOS), raising concerns about their suitability for reliable evaluation. To address this issue, we introduce the "VERification of Image-TExtpairs" (VERITE) benchmark for MMD which incorporates real-world data, excludes "asymmetric multimodal misinformation" and utilizes "modality balancing". We conduct an extensive comparative study with a Transformer-based architecture that shows the ability of VERITE to effectively address unimodal bias, rendering it a robust evaluation framework for MMD. Furthermore, we introduce a new method -- termed Crossmodal HArd Synthetic MisAlignment (CHASMA) -- for generating realistic synthetic training data that preserve crossmodal relations between legitimate images and false human-written captions. By leveraging CHASMA in the training process, we observe consistent and notable improvements in predictive performance on VERITE; with a 9.2% increase in accuracy. We release our code at: https://github.com/stevejpapad/image-text-verificatio

MindSpaces:Art-driven Adaptive Outdoors and Indoors Design

MindSpaces provides solutions for creating functionally and emotionally appealing architectural designs in urban spaces. Social media services, physiological sensing devices and video cameras provide data from sensing environments. State-of-the-Art technology including VR, 3D design tools, emotion extraction, visual behaviour analysis, and textual analysis will be incorporated in MindSpaces platform for analysing data and adapting the design of spaces.</p

EEG-based emotion recognition using advanced signal processing techniques

The aim of the present thesis is to develop advanced signal processing techniques for Electroencephalogram (EEG)-based emotion recognition. The proposed methodologies lie in two major lines: (i) the extraction of valuable information, related with emotions, that would lead to the enhancement of the emotion recognition task and (i) the evaluation of emotion elicitation procedures, as far as their ability to induct emotions is concerned. Moreover, the aforementioned methodologies are based on the basic principles that govern the emotional expression in EEG signals, as are proved from the current findings in neuroscience, and intend to incorporate those principles in the relative emotion recognition systems. In the first part of the thesis, the recognition of the six basic emotions is examined, i.e., the emotions of happiness, surprise, anger, fear, disgust, and sadness. For the elicitation of the aforementioned discrete emotions, the theory behind the Mirror Neuron System (MNS) is adopted. According to MNS, the EEG activity that is expressed when a subject feels an emotion, is the same or akin when a subject is actually watches another person expressing this emotion through face or body gesture, via a mechanism that is responsible for decoding other people's intentions. A new technique for the classification of the six basic emotions is proposed, where a feature set drawn from Higher Order Crossings (HOC) analysis is employed. Subsequently, an attempt to efficiently isolate the emotion-related EEG characteristics, a novel algorithmic tool is proposed, namely Hybrid Adaptive Filtering (HAF). High classification rates are accomplished, reaching 85%. In the second part of this thesis, the recognition of generalized affective states is studied. These affective states are not described as discrete emotions but rather as states characterized by two fundamental dimensions, arousal and valence. The subjective criteria that govern the emotional evaluation of the pictures emotional content, revealed the need of a measure that would evaluate the emotion elicitation trial in terms of its ability to induct emotional experience. Towards that, an index is proposed, namely Asymmetry Index (AsI). Furthermore, in order to localize the emotional experience in time and better isolate the frequency components that incorporate emotional information, four different implementations of the AsI measure are proposed. The resulted methodologies accomplish, as the HAF algorithm does, a kind of “emotional” filtering that boosts the emotional content of the filtered signals and leads to better classification results. Finally, for the first time in the EEG-based emotion recognition area, the detection of emotional transitions is studied. The conduct of a specifically designed experiment, that aims to elicit emotional transitions, is described. Moreover the methodology to detect these transitions is presented. The method proposed, is based on a variation of the AsI index, i.e., the transitional AsI (TAsI). The effectiveness of TAsI is tested for various subjects' mood scenarios and its dependence from mood is revealed. The whole approach shows promising results and lays the foundations for a more realistic implementation of EEG-based emotion recognition systems.Σκοπός της διατριβής είναι ανάπτυξη μεθοδολογιών επεξεργασίας σήματος για την αναγνώριση συναισθημάτων από ηλεκτροεγκεφαλογράφημα (ΗΕΓ). Οι προτεινόμενες μεθοδολογίες βασίζονται σε δύο κύριους άξονες: (i) την εξόρυξη της χρήσιμης πληροφορίας από τα σήματα ΗΕΓ για την όσο το δυνατόν πιο αποτελεσματική αναγνώριση των συναισθημάτων και (ii) την αξιολόγηση των διαδικασιών εκμαίευσης συναισθημάτων ως προς την αποτελεσματικότητά τους να προκαλούν συναισθήματα. Κάθε μέθοδος που προτείνεται βασίζεται στις αρχές που διέπουν την έκφραση του συναισθήματος στα σήματα ΗΕΓ, όπως περιγράφονται από τις σύγχρονες μελέτες των νευροεπιστημών, και σκοπό έχουν την ενσωμάτωση αυτών των αρχών στα συστήματα αναγνώρισης συναισθημάτων. Αρχικά, εξετάζεται η αναγνώριση διακριτών συναισθημάτων και συγκεκριμένα των έξι βασικών συναισθημάτων, δηλαδή, της χαράς, της έκπληξης, του θυμού, του φόβου, της απέχθειας και της λύπης. Η εκμαίευση των συναισθημάτων αυτών βασίζεται στη θεωρία του Συστήματος Νευρώνων Καθρεπτών σύμφωνα με την οποία, η πρόκληση του συναισθήματος προέρχεται μέσα από μια διαδικασία υποσυνείδητης αποκωδικοποίησης των συναισθηματικών εκφράσεων των άλλων ατόμων. Για την αναγνώριση των διακριτών συναισθημάτων προτείνεται η δημιουργία ενός διανύσματος χαρακτηριστικών που βασίζεται στην ανάλυση των διαβάσεων από το Μηδέν Ανώτερης Τάξης (ΜΑΤ). Στη συνέχεια, σε μια προσπάθεια να απομονωθεί από τα σήματα ΗΕΓ η πληροφορία που σχετίζεται με τα συναισθήματα και να απορριφθούν τυχόν φαινόμενα θορύβου (μη συναισθηματική πληροφορία), αναπτύχθηκε ένα εργαλείο «συναισθηματικού» φιλτραρίσματος, το Υβριδικό Προσαρμοζόμενο Φιλτράρισμα (ΥΠΦ). Τα ποσοστά ταξινόμησης είναι αρκετά υψηλά, αγγίζοντας το 85%. Στο δεύτερο μέρος της διατριβής εξετάζεται η αναγνώριση πιο γενικευμένων συναισθηματικών καταστάσεων, οι οποίες περιγράφονται από δύο παραμέτρους, αυτή της συναισθηματικής έντασης και αυτή της συναισθηματικής πολικότητας (θετικό ή αρνητικό συναίσθημα). Ο υποκειμενικός χαρακτήρας της συναισθηματικής αξιολόγησης των εικόνων αυτών οδήγησε στην ανάγκη ανάπτυξης ενός δείκτη, του δείκτη AsI, που χαρακτηρίζει την εκάστοτε δοκιμή εκμαίευσης συναισθήματος ως προς την αποτελεσματικότητά της. Επιπλέον, σε μια προσπάθεια για τον εντοπισμό των τμημάτων στο χρόνο και των συχνοτικών συνιστωσών που ενέχουν σε μεγάλο βαθμό συναισθηματική πληροφορία, εξετάστηκαν διαφορετικές παραλλαγές εφαρμογής του δείκτη AsI. Οι διαφορετικοί τρόποι εφαρμογής του AsI στο χρόνο και στη συχνότητα οδηγούν στην απομόνωση της χρήσιμης πληροφορίας για την αναγνώριση των συναισθηματικών καταστάσεων, επιτελώντας πάλι ένα είδος «συναισθηματικού» φιλτραρίσματος. Τέλος, μελετάται για πρώτη φορά στην περιοχή της αναγνώρισης συναισθημάτων από ΗΕΓ, ο εντοπισμός των συναισθηματικών μεταβάσεων. Περιγράφεται η διεξαγωγή κατάλληλου πειράματος, που σχεδιάστηκε για την εκμαίευση συναισθηματικών μεταβάσεων, και κατόπιν αναλύεται η μεθοδολογία για τον εντοπισμό των μεταβάσεων αυτών. Η μέθοδος που προτείνεται στηρίζεται σε μια παραλλαγή του δείκτη AsI, το μεταβατικό (transitional) AsI (TAsI). Η αποτελεσματικότητα του TAsI εξετάζεται για διάφορα σενάρια συναισθηματικής διάθεσης (mood) των υποκειμένων και αποκαλύπτεται η εξάρτησή του από αυτήν. Η όλη προσέγγιση παρουσιάζει ελπιδοφόρα αποτελέσματα και θέτει τις βάσεις για μια πιο ρεαλιστική υλοποίηση των συστημάτων αναγνώρισης συναισθημάτων από σήματα ΗΕΓ

Intra- and inter- cluster inhibition in Dentate Gyrus.

<p>In intra-cluster inhibition(first column of GCs), the most excited GC excites an interneuron which projects back to inhibit other GC within the same cluster. The same mechanism holds for the inter-cluster inhibition mediated by MCs. (MC: Mossy Cells, GC: Granule Cells, INT: Interneurons).</p

Dentate Gyrus Circuitry Features Improve Performance of Sparse Approximation Algorithms

<div><p>Memory-related activity in the Dentate Gyrus (DG) is characterized by sparsity. Memory representations are seen as activated neuronal populations of granule cells, the main encoding cells in DG, which are estimated to engage 2–4% of the total population. This sparsity is assumed to enhance the ability of DG to perform pattern separation, one of the most valuable contributions of DG during memory formation. In this work, we investigate how features of the DG such as its excitatory and inhibitory connectivity diagram can be used to develop theoretical algorithms performing Sparse Approximation, a widely used strategy in the Signal Processing field. Sparse approximation stands for the algorithmic identification of few components from a dictionary that approximate a certain signal. The ability of DG to achieve pattern separation by sparsifing its representations is exploited here to improve the performance of the state of the art sparse approximation algorithm “Iterative Soft Thresholding” (IST) by adding new algorithmic features inspired by the DG circuitry. Lateral inhibition of granule cells, either direct or indirect, via mossy cells, is shown to enhance the performance of the IST. Apart from revealing the potential of DG-inspired theoretical algorithms, this work presents new insights regarding the function of particular cell types in the pattern separation task of the DG.</p></div

Evolution of approximation for purple-highlighted elements in Fig. 2B, using DG-IST with d = opt = 171 (left panel) and DG-IST with d = inf (right panel).

<p>First row of each panel shows the approximation evolution of the two elements, black horizontal lines declare the original elements to be approximated and vertical pink lines (only left panel) show the iterations at which elimination of inhibition takes place for the second largest element in the corresponding column of <i>x^m</i>. The second row of each panel illustrates the Input to each GC through the iterative process, <math><mrow><mi>i</mi><mi>n</mi><mi>p</mi><mi>u</mi><mi>t</mi><mo>=</mo><mi>κ</mi><mo>⋅</mo><mrow><mo>[</mo><mrow><mrow><mrow><mo>(</mo><mrow><mi>A</mi><mi>T</mi><mrow><mo>(</mo><mrow><mi>y</mi><mo>−</mo><mi>A</mi><msubsup><mi>x</mi><mi>i</mi><mi>m</mi></msubsup></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow><mi>m</mi><mo>−</mo><msub><mi>I</mi><mi>s</mi></msub><mo>−</mo><msub><mi>M</mi><mi>s</mi></msub></mrow><mo>]</mo></mrow><mo>=</mo><mi>κ</mi><mo>⋅</mo><mo stretchy="false">(</mo><mi>E</mi><mi>r</mi><mi>r</mi><mi>o</mi><mi>r</mi><mo>+</mo><mi>I</mi><mi>N</mi><mi>T</mi><mo>+</mo><mi>M</mi><mi>C</mi><mo stretchy="false">)</mo></mrow></math>. The Error, MC, and INT values that add up to form the Input value are shown in the remaining rows of each panel.</p

DG-IST without INT- or MC- mediated inhibition.

<p>(A) Mean MSE of 100 instances of <i>x</i> vectors with: <i>N</i> = 1000, <i>a</i> = 2%, <i>M</i> ≥ <i>a</i> ⋅ log(<i>N</i>/<i>a</i>) estimated using IST (blue), DG-IST with <i>d</i> = 96 (red), DG-IST with <i>d</i> = 96 without INT-mediated inhibition (green), and DG-IST with <i>d</i> = 96 without MC-mediated inhibition (black) (Inset: magnification of the last 100 iterations). (B) Boxplots of the MSE of 100 instances of <i>x</i> vectors with: <i>N</i> = 1000, <i>a</i> = 2%, <i>M</i> ≥ <i>a</i> ⋅ log(<i>N</i>/<i>a</i>) estimated using IST, DG-IST with <i>d</i> = 96, DG-IST with <i>d</i> = 96 without INT-mediated inhibition, and DG-IST with <i>d</i> = 96 without MC-mediated inhibition. (C) MSE of a specific instance of vector <i>x</i> with: <i>N</i> = 1000, <i>a</i> = 2%, <i>M</i> ≥ <i>a</i> ⋅ log(<i>N</i>/<i>a</i>) estimated using IST (blue), DG-IST with <i>d</i> = 96 (red), DG-IST with <i>d</i> = 96 without INT-mediated inhibition (green), and DG-IST with <i>d</i> = 96 without MC-mediated inhibition (black).</p