A Big Data Platform for Real Time Analysis of Signs of Depression in Social Media

In this paper we propose a scalable platform for real-time processing of Social Media data. The platform ingests huge amounts of contents, such as Social Media posts or comments, and can support Public Health surveillance tasks. The processing and analytical needs of multiple screening tasks can easily be handled by incorporating user-defined execution graphs. The design is modular and supports different processing elements, such as crawlers to extract relevant contents or classifiers to categorise Social Media. We describe here an implementation of a use case built on the platform that monitors Social Media users and detects early signs of depressionThis work was funded by FEDER/Ministerio de Ciencia, Innovación y Universidades—Agencia Estatal de Investigación/ Project (RTI2018-093336-B-C21). Our research also receives financial support from the Consellería de Educación, Universidade e Formación Profesional (accreditation 2019–2022 ED431G-2019/04, ED431C 2018/29, ED431C 2018/19) and the European Regional Development Fund (ERDF), which acknowledges the CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University SystemS

Framework para la construcción y despliegue de sistemas de procesamiento en tiempo real

Traballo Fin de Máster en Tecnoloxías de Análise de Datos Masivos: Big Data. Curso 2017-2018En los últimos años se han desarrollado numerosas tecnologías destinadas al procesamiento de datos masivos, muchas de ellas de código abierto y de uso libre. Estas plataformas se centran en la escalabilidad horizontal, lo que implica que para el procesamiento de una mayor cantidad de datos sin grandes distorsiones en el ritmo, no es necesario aumentar o actualizar los recursos de una máquina (escalabilidad vertical), sino que es su ficiente con añadir más nodos con similares características a un clúster. La proliferación de este tipo de tecnologías de código abierto han democratizado y condicionado el gran número de aplicaciones que hacen uso de estas plataformas en multitud de ámbitos, tanto profesionales como académicos. Centrándonos en los frameworks de procesamiento, nos encontramos con una importante limitación: los datos han de poder dividirse en grupos independientes, de tal modo que sea posible paralelizar el trabajo en diferentes máquinas aunque existan puntos de procesamiento secuencial. Existen dos grandes tipos de tecnologías de procesamiento de este tipo: procesamiento de lotes (batch processing) y procesamiento de flujos (stream processing). En el primer caso, los resultados finales se obtienen juntos al fi nalizar el procesamiento del lote de datos compuesto por una o más etapas. Para de finir el trabajo a realizar, se define una topología de procesamiento que indica el flujo de los datos a través de las distintas etapas. Cada nodo (físico o virtual) puede ejecutar una instancia de la topología (aislada del resto de instancias), repartiéndose los datos de forma equitativa entre las instancias existentes. En las tecnologías de procesamiento de flujos, las distintas etapas de una topología son independientes y no pertenecen a una instancia concreta. Por tanto, las distintas etapas pueden ser paralelizadas de forma individual sin aumentar el grado de paralelismo de toda la topología. Estas tecnologías son adecuadas para aplicaciones que obtienen información en tiempo real y deben dar una respuesta inmediata, ya que cuando un dato completa su camino a través de las distintas etapas, el resultado puede obtenerse de forma instantánea. Sin embargo, con el procesamiento de lotes los resultados se obtienen cuando un lote de datos es procesado por completo. Un caso de aplicación de procesamiento en tiempo real es el análisis de contenidos en redes sociales para la detección temprana de riesgos. Este será el objetivo principal de este proyecto

git2net - Mining Time-Stamped Co-Editing Networks from Large git Repositories

Data from software repositories have become an important foundation for the empirical study of software engineering processes. A recurring theme in the repository mining literature is the inference of developer networks capturing e.g. collaboration, coordination, or communication from the commit history of projects. Most of the studied networks are based on the co-authorship of software artefacts defined at the level of files, modules, or packages. While this approach has led to insights into the social aspects of software development, it neglects detailed information on code changes and code ownership, e.g. which exact lines of code have been authored by which developers, that is contained in the commit log of software projects. Addressing this issue, we introduce git2net, a scalable python software that facilitates the extraction of fine-grained co-editing networks in large git repositories. It uses text mining techniques to analyse the detailed history of textual modifications within files. This information allows us to construct directed, weighted, and time-stamped networks, where a link signifies that one developer has edited a block of source code originally written by another developer. Our tool is applied in case studies of an Open Source and a commercial software project. We argue that it opens up a massive new source of high-resolution data on human collaboration patterns.Comment: MSR 2019, 12 pages, 10 figure

The state of peer-to-peer network simulators

Networking research often relies on simulation in order to test and evaluate new ideas. An important requirement of this process is that results must be reproducible so that other researchers can replicate, validate and extend existing work. We look at the landscape of simulators for research in peer-to-peer (P2P) networks by conducting a survey of a combined total of over 280 papers from before and after 2007 (the year of the last survey in this area), and comment on the large quantity of research using bespoke, closed-source simulators. We propose a set of criteria that P2P simulators should meet, and poll the P2P research community for their agreement. We aim to drive the community towards performing their experiments on simulators that allow for others to validate their results

A Comprehensive Survey of Deep Learning: Advancements, Applications, and Challenges

Artificial intelligence's "deep learning" discipline has taken off, revolutionizing a variety of industries, from computer vision and natural language processing to healthcare and finance. Deep learning has shown extraordinary effectiveness in resolving complicated issues, and it has a wide range of potential applications, from autonomous vehicles to healthcare. The purpose of the survey to study deep learning's present condition, including recent advancements, difficulties, and constraints since the subject is currently fast growing. The basic ideas of deep learning, such as neural networks, activation functions, and optimization algorithms, are first introduced. We next explore numerous topologies, emphasizing their distinct properties and uses, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs). Further concepts, applications, and difficulties of deep learning are all covered in this survey paper's thorough review. This survey aid the academics, professionals, and individuals who want to learn more about deep learning and explore its applications to challenging situations in the real world

A Priority-based Fair Queuing (PFQ) Model for Wireless Healthcare System

Healthcare is a very active research area, primarily due to the increase in the elderly population that leads to increasing number of emergency situations that require urgent actions. In recent years some of wireless networked medical devices were equipped with different sensors to measure and report on vital signs of patient remotely. The most important sensors are Heart Beat Rate (ECG), Pressure and Glucose sensors. However, the strict requirements and real-time nature of medical applications dictate the extreme importance and need for appropriate Quality of Service (QoS), fast and accurate delivery of a patient’s measurements in reliable e-Health ecosystem. As the elderly age and older adult population is increasing (65 years and above) due to the advancement in medicine and medical care in the last two decades; high QoS and reliable e-health ecosystem has become a major challenge in Healthcare especially for patients who require continuous monitoring and attention. Nevertheless, predictions have indicated that elderly population will be approximately 2 billion in developing countries by 2050 where availability of medical staff shall be unable to cope with this growth and emergency cases that need immediate intervention. On the other side, limitations in communication networks capacity, congestions and the humongous increase of devices, applications and IOT using the available communication networks add extra layer of challenges on E-health ecosystem such as time constraints, quality of measurements and signals reaching healthcare centres. Hence this research has tackled the delay and jitter parameters in E-health M2M wireless communication and succeeded in reducing them in comparison to current available models. The novelty of this research has succeeded in developing a new Priority Queuing model ‘’Priority Based-Fair Queuing’’ (PFQ) where a new priority level and concept of ‘’Patient’s Health Record’’ (PHR) has been developed and integrated with the Priority Parameters (PP) values of each sensor to add a second level of priority. The results and data analysis performed on the PFQ model under different scenarios simulating real M2M E-health environment have revealed that the PFQ has outperformed the results obtained from simulating the widely used current models such as First in First Out (FIFO) and Weight Fair Queuing (WFQ). PFQ model has improved transmission of ECG sensor data by decreasing delay and jitter in emergency cases by 83.32% and 75.88% respectively in comparison to FIFO and 46.65% and 60.13% with respect to WFQ model. Similarly, in pressure sensor the improvements were 82.41% and 71.5% and 68.43% and 73.36% in comparison to FIFO and WFQ respectively. Data transmission were also improved in the Glucose sensor by 80.85% and 64.7% and 92.1% and 83.17% in comparison to FIFO and WFQ respectively. However, non-emergency cases data transmission using PFQ model was negatively impacted and scored higher rates than FIFO and WFQ since PFQ tends to give higher priority to emergency cases. Thus, a derivative from the PFQ model has been developed to create a new version namely “Priority Based-Fair Queuing-Tolerated Delay” (PFQ-TD) to balance the data transmission between emergency and non-emergency cases where tolerated delay in emergency cases has been considered. PFQ-TD has succeeded in balancing fairly this issue and reducing the total average delay and jitter of emergency and non-emergency cases in all sensors and keep them within the acceptable allowable standards. PFQ-TD has improved the overall average delay and jitter in emergency and non-emergency cases among all sensors by 41% and 84% respectively in comparison to PFQ model

High Energy Physics Forum for Computational Excellence: Working Group Reports (I. Applications Software II. Software Libraries and Tools III. Systems)

Computing plays an essential role in all aspects of high energy physics. As computational technology evolves rapidly in new directions, and data throughput and volume continue to follow a steep trend-line, it is important for the HEP community to develop an effective response to a series of expected challenges. In order to help shape the desired response, the HEP Forum for Computational Excellence (HEP-FCE) initiated a roadmap planning activity with two key overlapping drivers -- 1) software effectiveness, and 2) infrastructure and expertise advancement. The HEP-FCE formed three working groups, 1) Applications Software, 2) Software Libraries and Tools, and 3) Systems (including systems software), to provide an overview of the current status of HEP computing and to present findings and opportunities for the desired HEP computational roadmap. The final versions of the reports are combined in this document, and are presented along with introductory material.Comment: 72 page