Towards Guidelines for Assessing Qualities of Machine Learning Systems

Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary quality aspects of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to adjust quality aspects or add additional ones (such as trustworthiness) and be very precise about which aspect is really relevant for which object of interest (such as completeness of training data), and how to objectively assess adherence to quality requirements. In this article, we present the construction of a quality model (i.e., evaluation objects, quality aspects, and metrics) for an ML system based on an industrial use case. This quality model enables practitioners to specify and assess quality requirements for such kinds of ML systems objectively. In the future, we want to learn how the term quality differs between different types of ML systems and come up with general guidelines for specifying and assessing qualities of ML systems.Comment: Has been accepted at the 13th International Conference on the Quality of Information and Communications Technology QUATIC2020 (https://2020.quatic.org/). QUATIC 2020 proceedings will be included in a volume of Springer CCIS Series (Communications in Computer and Information Science

Exploring the relation between remotely sensed vertical canopy structure and tree species diversity in Gabon

Mapping tree species diversity is increasingly important in the face of environmental change and biodiversity conservation. We explore a potential way of mapping this diversity by relating forest structure to tree species diversity in Gabon. First, we test the relation between canopy height, as a proxy for niche volume, and tree species diversity. Then, we test the relation between vertical canopy structure, as a proxy for vertical niche occupation, and tree species diversity. We use large footprint full-waveform airborne lidar data collected across four study sites in Gabon (Lopé, Mabounié, Mondah, and Rabi) in combination with in situ estimates of species richness (S) and Shannon diversity (H'). Linear models using canopy height explained 44% and 43% of the variation in S and H' at the 0.25 ha resolution. Linear models using canopy height and the plant area volume density profile explained 71% of this variation. We demonstrate applications of these models by mapping S and H' in Mondah using a simulated GEDI-TanDEM-X fusion height product, across the four sites using wall-to-wall airborne lidar data products, and across and between the study sites using ICESat lidar waveforms. The modeling results are encouraging in the context of developing pan-tropical structure-diversity models applicable to data from current and upcoming spaceborne remote sensing missions

Evaluating the potential of full-waveform lidar for mapping pan-tropical tree species richness

AIM: Mapping tree species richness across the tropics is of great interest for effective conservation and biodiversity management. In this study, we evaluated the potential of full‐waveform lidar data for mapping tree species richness across the tropics by relating measurements of vertical canopy structure, as a proxy for the occupation of vertical niche space, to tree species richness. LOCATION: Tropics. TIME PERIOD: Present. MAJOR TAXA STUDIED: Trees. METHODS: First, we evaluated the characteristics of vertical canopy structure across 15 study sites using (simulated) large‐footprint full‐waveform lidar data (22 m diameter) and related these findings to in‐situ tree species information. Then, we developed structure–richness models at the local (within 25–50 ha plots), regional (biogeographical regions) and pan‐tropical scale at three spatial resolutions (1.0, 0.25 and 0.0625 ha) using Poisson regression. RESULTS: The results showed a weak structure–richness relationship at the local scale. At the regional scale (within a biogeographical region) a stronger relationship between canopy structure and tree species richness across different tropical forest types was found, for example across Central Africa and in South America [R^{2} ranging from .44–.56, root mean squared difference as a percentage of the mean (RMSD%) ranging between 23–61%]. Modelling the relationship pan‐tropically, across four continents, 39% of the variation in tree species richness could be explained with canopy structure alone (R^{2} = .39 and RMSD% = 43%, 0.25‐ha resolution). MAIN CONCLUSIONS: Our results may serve as a basis for the future development of a set of structure–richness models to map high resolution tree species richness using vertical canopy structure information from the Global Ecosystem Dynamics Investigation (GEDI). The value of this effort would be enhanced by access to a larger set of field reference data for all tropical regions. Future research could also support the use of GEDI data in frameworks using environmental and spectral information for modelling tree species richness across the tropics

Evaluating the potential of full-waveform lidar for mapping pan-tropical tree species richness

© 2020 John Wiley & Sons Ltd Aim: Mapping tree species richness across the tropics is of great interest for effective conservation and biodiversity management. In this study, we evaluated the potential of full-waveform lidar data for mapping tree species richness across the tropics by relating measurements of vertical canopy structure, as a proxy for the occupation of vertical niche space, to tree species richness. Location: Tropics. Time period: Present. Major taxa studied: Trees. Methods: First, we evaluated the characteristics of vertical canopy structure across 15 study sites using (simulated) large-footprint full-waveform lidar data (22 m diameter) and related these findings to in-situ tree species information. Then, we developed structure–richness models at the local (within 25–50 ha plots), regional (biogeographical regions) and pan-tropical scale at three spatial resolutions (1.0, 0.25 and 0.0625 ha) using Poisson regression. Results: The results showed a weak structure–richness relationship at the local scale. At the regional scale (within a biogeographical region) a stronger relationship between canopy structure and tree species richness across different tropical forest types was found, for example across Central Africa and in South America [R2 ranging from.44–.56, root mean squared difference as a percentage of the mean (RMSD%) ranging between 23–61%]. Modelling the relationship pan-tropically, across four continents, 39% of the variation in tree species richness could be explained with canopy structure alone (R2 =.39 and RMSD% = 43%, 0.25-ha resolution). Main conclusions: Our results may serve as a basis for the future development of a set of structure–richness models to map high resolution tree species richness using vertical canopy structure information from the Global Ecosystem Dynamics Investigation (GEDI). The value of this effort would be enhanced by access to a larger set of field reference data for all tropical regions. Future research could also support the use of GEDI data in frameworks using environmental and spectral information for modelling tree species richness across the tropics

Aboveground biomass density models for NASA's Global Ecosystem Dynamics Investigation (GEDI) lidar mission

NASA's Global Ecosystem Dynamics Investigation (GEDI) is collecting spaceborne full waveform lidar data with a primary science goal of producing accurate estimates of forest aboveground biomass density (AGBD). This paper presents the development of the models used to create GEDI's footprint-level (similar to 25 m) AGBD (GEDI04_A) product, including a description of the datasets used and the procedure for final model selection. The data used to fit our models are from a compilation of globally distributed spatially and temporally coincident field and airborne lidar datasets, whereby we simulated GEDI-like waveforms from airborne lidar to build a calibration database. We used this database to expand the geographic extent of past waveform lidar studies, and divided the globe into four broad strata by Plant Functional Type (PFT) and six geographic regions. GEDI's waveform-to-biomass models take the form of parametric Ordinary Least Squares (OLS) models with simulated Relative Height (RH) metrics as predictor variables. From an exhaustive set of candidate models, we selected the best input predictor variables, and data transformations for each geographic stratum in the GEDI domain to produce a set of comprehensive predictive footprint-level models. We found that model selection frequently favored combinations of RH metrics at the 98th, 90th, 50th, and 10th height above ground-level percentiles (RH98, RH90, RH50, and RH10, respectively), but that inclusion of lower RH metrics (e.g. RH10) did not markedly improve model performance. Second, forced inclusion of RH98 in all models was important and did not degrade model performance, and the best performing models were parsimonious, typically having only 1-3 predictors. Third, stratification by geographic domain (PFT, geographic region) improved model performance in comparison to global models without stratification. Fourth, for the vast majority of strata, the best performing models were fit using square root transformation of field AGBD and/or height metrics. There was considerable variability in model performance across geographic strata, and areas with sparse training data and/or high AGBD values had the poorest performance. These models are used to produce global predictions of AGBD, but will be improved in the future as more and better training data become available

Aboveground biomass density models for NASA's Global Ecosystem Dynamics Investigation (GEDI) lidar mission

NASA's Global Ecosystem Dynamics Investigation (GEDI) is collecting spaceborne full waveform lidar data with a primary science goal of producing accurate estimates of forest aboveground biomass density (AGBD). This paper presents the development of the models used to create GEDI's footprint-level (~25 m) AGBD (GEDI04_A) product, including a description of the datasets used and the procedure for final model selection. The data used to fit our models are from a compilation of globally distributed spatially and temporally coincident field and airborne lidar datasets, whereby we simulated GEDI-like waveforms from airborne lidar to build a calibration database. We used this database to expand the geographic extent of past waveform lidar studies, and divided the globe into four broad strata by Plant Functional Type (PFT) and six geographic regions. GEDI's waveform-to-biomass models take the form of parametric Ordinary Least Squares (OLS) models with simulated Relative Height (RH) metrics as predictor variables. From an exhaustive set of candidate models, we selected the best input predictor variables, and data transformations for each geographic stratum in the GEDI domain to produce a set of comprehensive predictive footprint-level models. We found that model selection frequently favored combinations of RH metrics at the 98th, 90th, 50th, and 10th height above ground-level percentiles (RH98, RH90, RH50, and RH10, respectively), but that inclusion of lower RH metrics (e.g. RH10) did not markedly improve model performance. Second, forced inclusion of RH98 in all models was important and did not degrade model performance, and the best performing models were parsimonious, typically having only 1-3 predictors. Third, stratification by geographic domain (PFT, geographic region) improved model performance in comparison to global models without stratification. Fourth, for the vast majority of strata, the best performing models were fit using square root transformation of field AGBD and/or height metrics. There was considerable variability in model performance across geographic strata, and areas with sparse training data and/or high AGBD values had the poorest performance. These models are used to produce global predictions of AGBD, but will be improved in the future as more and better training data become available

Communicative memory of irregular migration : The re-circulation of news images on YouTube

This article analyses user-generated YouTube cut and mix videos of irregular migration as producing communicative memory of those who have suffered at Europe’s external borders. Visual and textual analyses examine a neglected perspective on the study of media representations of migration by examining a particular practice through which people engage with news images and participate in (re)construction of collective memory in relation to irregular migration. The analysis shows that while hegemonic Eurocentric imagery prevails also in the vernacular amateur productions, re-mixing different cultural productions nevertheless complicates the representation of irregular migration and affords alternative positions. The article examines an emerging area in memory studies: practices of vernacular commemoration and its convergence with more institutionalized and professional media and memory practices. The videos are communicative reactions to emotionally disturbing news images of suffering – the public’s re-articulations of migration and migration control, which nevertheless are constrained into the existing mainstream media framings and entertainment economies. This type of performative citizenship is crucial for social equality, particularly since non-performance of memory in relation to contemporary migration tragedies has dominated European public spheres.peerReviewe

Aboveground biomass density models for NASA's Global Ecosystem Dynamics Investigation (GEDI) lidar mission

NASA's Global Ecosystem Dynamics Investigation (GEDI) is collecting spaceborne full waveform lidar data with a primary science goal of producing accurate estimates of forest aboveground biomass density (AGBD). This paper presents the development of the models used to create GEDI's footprint-level (~25 m) AGBD (GEDI04_A) product, including a description of the datasets used and the procedure for final model selection. The data used to fit our models are from a compilation of globally distributed spatially and temporally coincident field and airborne lidar datasets, whereby we simulated GEDI-like waveforms from airborne lidar to build a calibration database. We used this database to expand the geographic extent of past waveform lidar studies, and divided the globe into four broad strata by Plant Functional Type (PFT) and six geographic regions. GEDI's waveform-to-biomass models take the form of parametric Ordinary Least Squares (OLS) models with simulated Relative Height (RH) metrics as predictor variables. From an exhaustive set of candidate models, we selected the best input predictor variables, and data transformations for each geographic stratum in the GEDI domain to produce a set of comprehensive predictive footprint-level models. We found that model selection frequently favored combinations of RH metrics at the 98th, 90th, 50th, and 10th height above ground-level percentiles (RH98, RH90, RH50, and RH10, respectively), but that inclusion of lower RH metrics (e.g. RH10) did not markedly improve model performance. Second, forced inclusion of RH98 in all models was important and did not degrade model performance, and the best performing models were parsimonious, typically having only 1-3 predictors. Third, stratification by geographic domain (PFT, geographic region) improved model performance in comparison to global models without stratification. Fourth, for the vast majority of strata, the best performing models were fit using square root transformation of field AGBD and/or height metrics. There was considerable variability in model performance across geographic strata, and areas with sparse training data and/or high AGBD values had the poorest performance. These models are used to produce global predictions of AGBD, but will be improved in the future as more and better training data become available