Self-Adaptive Architecture for Multi-sensor Embedded Vision System

International audienceArchitectural optimization for heterogeneous multi-sensor processing is a real technological challenge. Most of the vision systems involve only one single color sensor and they do not address the heterogeneous sensors challenge. However, more and more applications require other types of sensor in addition, such as infrared or low-light sensor, so that the vision system could face various luminosity conditions. These heterogeneous sensors could differ in the spectral band, the resolution or even the frame rate. Such sensor variety needs huge computing performance , but embedded systems have stringent area and power constraints. Reconfigurable architecture makes possible flexible computing while respecting the latter constraints. Many reconfigurable architectures for vision application have been proposed in the past. Yet, few of them propose a real dynamic adaptation capability to manage sensor heterogeneity. In this paper, a self-adaptive architecture is proposed to deal with heterogeneous sensors dynamically. This architecture supports on-the-fly sensor switch. Architecture of the system is self-adapted thanks to a system monitor and an adaptation controller. A stream header concept is used to convey sensor information to the self-adaptive architecture. The proposed architecture was implemented in Altera Cyclone V FPGA. In this implementation, adaptation of the architecture consists in Dynamic and Partial Reconfiguration of FPGA. The self-adaptive ability of the architecture has been proved with low resource overhead and an average global adaptation time of 75 ms

Auto-Adaptive Multi-Sensor Architecture

International audienceTo overcome luminosity problems, modern embedded vision systems often integrate technologically heterogeneous sensors. Also, it has to provide different functionalities such as photo or video mode, image improvement or data fusion, according to the user environment. Therefore, nowadays vision systems should be context-aware and adapt their performance parameters automatically. In this context, we propose a novel auto-adaptive architecture enabling on-the-fly and automatic frame rate and resolution adaptation by a frequency tuning method. This method also intends to reduce power consumption as an alternative to existing power gating method. Performance evaluation in a FPGA implementation demonstrates an inter-frame adaptation capability with a relative low area overhead. I. INTRODUCTION From decades, the ability of computer vision systems increases thanks to the multiplication of integrated sensors. Multi-sensor systems enable many high-level vision applications such as stereo vision, data fusion [1] or 3D stereo view [2]. Also smart camera networks take advantage of the multi-sensor concept for large-scale surveillance applications [3]. More and more vision systems involve several heterogeneous sensors such as color, infrared or intensified low-light sensor [4] to overcome the variable luminosity conditions or improve the application robustness. Frequently, the considered vision system accomplishes various tasks such as video streaming, photo capture or high level processing (i.e. face detection, object tracking, ...). Each one of these tasks imposes different performance computing ability to the hardware resources, according to the applicative context and used sensor. That is why, nowadays vision systems have to be context-aware and to possess the ability to adapt their performance according to the user environment [5]. Fig. 1 illustrates the differences between video and photo user mode parameters: latency, frame rate, resolution, image quality and power consumption. While a video mode needs a high frame rate and low latency, a photo mode rather expects a higher resolution and higher image quality. In this context, we expect the system architecture adapt itself on-the-fly to the required frame rate or resolution while minimizing the use-case transition time when the user mode changes. In addition, the frame rate and the resolution of the involved sensors are not supposed to be known in advance. Numerous adaptable architectures exist for high-performance image processing [6]–[8] and also even for energy aware heterogeneous vision systems [2], they do not enable such dynamic adaptation of the frame rate or the resolution. In this paper, we propose a novel pixel frequency tuning approach for heterogeneous multi-sensor vision systems. Th

Fast and efficient FPGA implementation of connected operators

International audienceThe Connected Component Tree (CCT)-based operators play a central role in the development of new algorithms related to image processing applications such as pattern recognition, video-surveillance or motion extraction. The CCT construction, being a time consuming task (about 80% of the application time), these applications remain far-off mobile embedded systems. This paper presents its efficient FPGA implementation suited for embedded systems. Three main contributions are discussed: an efficient data structure proposal adapted to representing the CCT in embedded systems, a memory organization suitable for FPGA implementation by using on-chip memory and a customizable hardware accelerator architecture for CCT-based applications

Memory System for a Dynamically Adaptable Pixel Stream Architecture

International audienceNowadays, embedded vision systems have to face new hard requirements involved by modern applications: realtime processing of high resolution images issued by multiple image sensors. Recently, a new adaptable ring-based interconnection network on chip has been proposed. Based on adaptive datapath, it allows handling of multiple parallel pixel streams. In this paper, we present a new hierarchical memory system proposed for this adaptable ring-based architecture. The design of its different levels is discussed and we show how the memory system adapts dynamically with respect to the datapath and data access management in the interconnection network. We also present the timing performance and area occupation measured on an FPGA prototype

Concurrent capecitabine and upper abdominal radiation therapy is well tolerated

We retrospectively evaluated acute toxicity in 88 patients that were treated with capecitabine and concurrent radiotherapy to the upper abdomen. These patients included 28 (32%) with pancreatic adenocarcinoma, 18 (20%) with cholangiocarcinoma, 11 (13%) with ampullary carcinoma, 11 (13%) with other primary tumors, 14 (16%) with liver metastases, and 6 (7%) with metastases at other sites. The median dose of radiotherapy was 45 Gy (range 30–72 Gy). The median dose of capecitabine was 850 mg/m(2 )twice daily, with 77% receiving 800–900 mg/m(2 )twice daily. The highest grade of acute toxicity was Common Terminology Criteria (CTC) grade 0 in 5 (6%), grade 1 in 60 (68%), grade 2 in 18 (20%), and grade 3 in 5 (6%) patients. No patient had CTC grade 4 toxicity. The most common grade 2 toxicities were nausea, hand-foot syndrome, fatigue, anorexia and diarrhea. The grade 3 toxicities included nausea, vomiting and fatigue. Three patients (3%) required hospitalization due to grade 3 acute toxicity. Capecitabine was interrupted, discontinued or given at an adjusted dose in 13 (15%) patients because of acute toxicity. Therefore, capecitabine and concurrent radiotherapy to the upper abdomen appears to be well tolerated. Capecitabine may serve as an alternative to bolus or infusional 5-FU during chemoradiation for upper gastrointestinal malignancies

Altimetry for the future: Building on 25 years of progress

In 2018 we celebrated 25 years of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology. The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the ‘‘Green” Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instruments’ development and satellite missions’ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion

Peri-operative red blood cell transfusion in neonates and infants: NEonate and Children audiT of Anaesthesia pRactice IN Europe: A prospective European multicentre observational study

BACKGROUND: Little is known about current clinical practice concerning peri-operative red blood cell transfusion in neonates and small infants. Guidelines suggest transfusions based on haemoglobin thresholds ranging from 8.5 to 12 g dl-1, distinguishing between children from birth to day 7 (week 1), from day 8 to day 14 (week 2) or from day 15 (≥week 3) onwards. OBJECTIVE: To observe peri-operative red blood cell transfusion practice according to guidelines in relation to patient outcome. DESIGN: A multicentre observational study. SETTING: The NEonate-Children sTudy of Anaesthesia pRactice IN Europe (NECTARINE) trial recruited patients up to 60 weeks' postmenstrual age undergoing anaesthesia for surgical or diagnostic procedures from 165 centres in 31 European countries between March 2016 and January 2017. PATIENTS: The data included 5609 patients undergoing 6542 procedures. Inclusion criteria was a peri-operative red blood cell transfusion. MAIN OUTCOME MEASURES: The primary endpoint was the haemoglobin level triggering a transfusion for neonates in week 1, week 2 and week 3. Secondary endpoints were transfusion volumes, 'delta haemoglobin' (preprocedure - transfusion-triggering) and 30-day and 90-day morbidity and mortality. RESULTS: Peri-operative red blood cell transfusions were recorded during 447 procedures (6.9%). The median haemoglobin levels triggering a transfusion were 9.6 [IQR 8.7 to 10.9] g dl-1 for neonates in week 1, 9.6 [7.7 to 10.4] g dl-1 in week 2 and 8.0 [7.3 to 9.0] g dl-1 in week 3. The median transfusion volume was 17.1 [11.1 to 26.4] ml kg-1 with a median delta haemoglobin of 1.8 [0.0 to 3.6] g dl-1. Thirty-day morbidity was 47.8% with an overall mortality of 11.3%. CONCLUSIONS: Results indicate lower transfusion-triggering haemoglobin thresholds in clinical practice than suggested by current guidelines. The high morbidity and mortality of this NECTARINE sub-cohort calls for investigative action and evidence-based guidelines addressing peri-operative red blood cell transfusions strategies. TRIAL REGISTRATION: ClinicalTrials.gov, identifier: NCT02350348

Altimetry for the future: building on 25 years of progress

In 2018 we celebrated 25 years of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology. The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the “Green” Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instruments’ development and satellite missions’ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion

31st Annual Meeting and Associated Programs of the Society for Immunotherapy of Cancer (SITC 2016) : part two

Background The immunological escape of tumors represents one of the main ob- stacles to the treatment of malignancies. The blockade of PD-1 or CTLA-4 receptors represented a milestone in the history of immunotherapy. However, immune checkpoint inhibitors seem to be effective in specific cohorts of patients. It has been proposed that their efficacy relies on the presence of an immunological response. Thus, we hypothesized that disruption of the PD-L1/PD-1 axis would synergize with our oncolytic vaccine platform PeptiCRAd. Methods We used murine B16OVA in vivo tumor models and flow cytometry analysis to investigate the immunological background. Results First, we found that high-burden B16OVA tumors were refractory to combination immunotherapy. However, with a more aggressive schedule, tumors with a lower burden were more susceptible to the combination of PeptiCRAd and PD-L1 blockade. The therapy signifi- cantly increased the median survival of mice (Fig. 7). Interestingly, the reduced growth of contralaterally injected B16F10 cells sug- gested the presence of a long lasting immunological memory also against non-targeted antigens. Concerning the functional state of tumor infiltrating lymphocytes (TILs), we found that all the immune therapies would enhance the percentage of activated (PD-1pos TIM- 3neg) T lymphocytes and reduce the amount of exhausted (PD-1pos TIM-3pos) cells compared to placebo. As expected, we found that PeptiCRAd monotherapy could increase the number of antigen spe- cific CD8+ T cells compared to other treatments. However, only the combination with PD-L1 blockade could significantly increase the ra- tio between activated and exhausted pentamer positive cells (p= 0.0058), suggesting that by disrupting the PD-1/PD-L1 axis we could decrease the amount of dysfunctional antigen specific T cells. We ob- served that the anatomical location deeply influenced the state of CD4+ and CD8+ T lymphocytes. In fact, TIM-3 expression was in- creased by 2 fold on TILs compared to splenic and lymphoid T cells. In the CD8+ compartment, the expression of PD-1 on the surface seemed to be restricted to the tumor micro-environment, while CD4 + T cells had a high expression of PD-1 also in lymphoid organs. Interestingly, we found that the levels of PD-1 were significantly higher on CD8+ T cells than on CD4+ T cells into the tumor micro- environment (p < 0.0001). Conclusions In conclusion, we demonstrated that the efficacy of immune check- point inhibitors might be strongly enhanced by their combination with cancer vaccines. PeptiCRAd was able to increase the number of antigen-specific T cells and PD-L1 blockade prevented their exhaus- tion, resulting in long-lasting immunological memory and increased median survival

Dynamically adaptable Network-on-Chip for embedded vision systems

Un équipement portable moderne intègre plusieurs capteurs d'image qui peuvent être de différents types. On peut citer en guise d'exemple un capteur couleur, un capteur infrarouge ou un capteur basse lumière. Cet équipement doit alors supporter différentes sources qui peuvent être hétérogènes en terme de résolution, de granularité de pixels et de fréquence d'émission des images. Cette tendance à multiplier les capteurs, est motivée par des besoins applicatifs dans un but de complémentarité en sensibilité (fusion des images), en position (panoramique) ou en champ de vision. Le système doit par conséquent être capable de supporter des applications de plus en plus complexes et variées, nécessitant d'utiliser une seule ou plusieurs sources d'image. Du fait de cette variété de fonctionnalités embarquées, le système électronique doit pouvoir s'adapter constamment pour garantir des performances en terme de latence et de temps de traitement en fonction des applications, tout en respectant des contraintes d'encombrement.% Même si depuis de nombreuses années, un grand nombre de solutions architecturales ont été proposées pour améliorer l'adaptabilité des unités de calcul, un problème majeur persiste au niveau du réseau d'interconnexion qui n'est pas suffisamment adaptable, en particulier pour le transfert des flux de pixels et l'accès aux données. Nous proposons dans cette thèse un nouveau réseau de communication sur puce (NoC) pour un SoC dédié à la vision. Ce réseau permet de gérer dynamiquement différents types de flux en parallèle en auto-adaptant le chemin de donnée entre les unités de calcul, afin d'exécuter de manière efficace différentes applications. La proposition d'une nouvelle structure de paquets de données, facilite les mécanismes d'adaptation du système grâce à la combinaison d'instructions et de données à traiter dans un même paquet. Nous proposons également un système de mémorisation de trames à adressage indirecte, capable de gérer dynamiquement plusieurs trames image de différentes sources d'image. Cet adressage indirect est réalisé par l'intermédiaire d'une couche d'abstraction matérielle qui se charge de traduire des requêtes de lecture et d'écriture, réalisées suivant des indicateurs de la trame requise (source de l'image, indice temporel et dernière opération effectuée). Afin de valider notre proposition, nous définissons une nouvelle architecture, appelée Multi Data Flow Ring (MDFR) basée sur notre réseau avec une topologie en anneau. Les performances de cette architecture, en temps et en surface, ont été évaluées dans le cadre d'une implémentation sur une cible FPGAModern portable vision systems include several types of image sensors such as colour, low-light or infrared sensor. Such system has to support heterogeneous image sources with different spatial resolutions, pixel granularities and working frequencies. This trend to multiply sensors is motivated by needs to complete sensor sensibilities with image fusion processing techniques, or sensor positions in the system. Moreover, portable vision systems implement image applications which require several images sources with a growing computing complexity. To face those challenges in integrating such a variety of functionalities, the embedded electronic computing system has to adapt permanently to preserve application timing performance in latency and processing, and to respect area and low-power constraints. In this thesis, we propose a new Network-On-Chip (NoC) adapted for a System-On-Chip (SoC) dedicated to image applications. This NoC can manage several pixel streams in parallel by adapting dynamically the datapatah between processing elements and memories. The new header packet structure enables adaptation mechanisms in routers by combining instructions and data in a same packet. To manage efficiently the frames storage required for an application, we propose a frame buffer system with an indirect frame addressing, which is able to manage several frames from different sensors. It features a hardware abstraction layer which is in charge to collect reading and writing requests, according to specific frame indicators such as the image source ID. The NoC has been validated in a complete processing architecture called Multi Data Flow Ring (MDFR) with a ring topology. The MDFR performances in time and area has been demonstrated for an FPGA targe