Reading the news through its structure: new hybrid connectivity based approaches

In this thesis a solution for the problem of identifying the structure of news published by online newspapers is presented. This problem requires new approaches and algorithms that are capable of dealing with the massive number of online publications in existence (and that will grow in the future). The fact that news documents present a high degree of interconnection makes this an interesting and hard problem to solve. The identification of the structure of the news is accomplished both by descriptive methods that expose the dimensionality of the relations between different news, and by clustering the news into topic groups. To achieve this analysis this integrated whole was studied using different perspectives and approaches. In the identification of news clusters and structure, and after a preparatory data collection phase, where several online newspapers from different parts of the globe were collected, two newspapers were chosen in particular: the Portuguese daily newspaper Público and the British newspaper The Guardian. In the first case, it was shown how information theory (namely variation of information) combined with adaptive networks was able to identify topic clusters in the news published by the Portuguese online newspaper Público. In the second case, the structure of news published by the British newspaper The Guardian is revealed through the construction of time series of news clustered by a kmeans process. After this approach an unsupervised algorithm, that filters out irrelevant news published online by taking into consideration the connectivity of the news labels entered by the journalists, was developed. This novel hybrid technique is based on Qanalysis for the construction of the filtered network followed by a clustering technique to identify the topical clusters. Presently this work uses a modularity optimisation clustering technique but this step is general enough that other hybrid approaches can be used without losing generality. A novel second order swarm intelligence algorithm based on Ant Colony Systems was developed for the travelling salesman problem that is consistently better than the traditional benchmarks. This algorithm is used to construct Hamiltonian paths over the news published using the eccentricity of the different documents as a measure of distance. This approach allows for an easy navigation between published stories that is dependent on the connectivity of the underlying structure. The results presented in this work show the importance of taking topic detection in large corpora as a multitude of relations and connectivities that are not in a static state. They also influence the way of looking at multi-dimensional ensembles, by showing that the inclusion of the high dimension connectivities gives better results to solving a particular problem as was the case in the clustering problem of the news published online.Neste trabalho resolvemos o problema da identificação da estrutura das notícias publicadas em linha por jornais e agências noticiosas. Este problema requer novas abordagens e algoritmos que sejam capazes de lidar com o número crescente de publicações em linha (e que se espera continuam a crescer no futuro). Este facto, juntamente com o elevado grau de interconexão que as notícias apresentam tornam este problema num problema interessante e de difícil resolução. A identificação da estrutura do sistema de notícias foi conseguido quer através da utilização de métodos descritivos que expõem a dimensão das relações existentes entre as diferentes notícias, quer através de algoritmos de agrupamento das mesmas em tópicos. Para atingir este objetivo foi necessário proceder a ao estudo deste sistema complexo sob diferentes perspectivas e abordagens. Após uma fase preparatória do corpo de dados, onde foram recolhidos diversos jornais publicados online optou-se por dois jornais em particular: O Público e o The Guardian. A escolha de jornais em línguas diferentes deve-se à vontade de encontrar estratégias de análise que sejam independentes do conhecimento prévio que se tem sobre estes sistemas. Numa primeira análise é empregada uma abordagem baseada em redes adaptativas e teoria de informação (nomeadamente variação de informação) para identificar tópicos noticiosos que são publicados no jornal português Público. Numa segunda abordagem analisamos a estrutura das notícias publicadas pelo jornal Britânico The Guardian através da construção de séries temporais de notícias. Estas foram seguidamente agrupadas através de um processo de k-means. Para além disso desenvolveuse um algoritmo que permite filtrar de forma não supervisionada notícias irrelevantes que apresentam baixa conectividade às restantes notícias através da utilização de Q-analysis seguida de um processo de clustering. Presentemente este método utiliza otimização de modularidade, mas a técnica é suficientemente geral para que outras abordagens híbridas possam ser utilizadas sem perda de generalidade do método. Desenvolveu-se ainda um novo algoritmo baseado em sistemas de colónias de formigas para solução do problema do caixeiro viajante que consistentemente apresenta resultados melhores que os tradicionais bancos de testes. Este algoritmo foi aplicado na construção de caminhos Hamiltonianos das notícias publicadas utilizando a excentricidade obtida a partir da conectividade do sistema estudado como medida da distância entre notícias. Esta abordagem permitiu construir um sistema de navegação entre as notícias publicadas que é dependente da conectividade observada na estrutura de notícias encontrada. Os resultados apresentados neste trabalho mostram a importância de analisar sistemas complexos na sua multitude de relações e conectividades que não são estáticas e que influenciam a forma como tradicionalmente se olha para sistema multi-dimensionais. Mostra-se que a inclusão desta dimensões extra produzem melhores resultados na resolução do problema de identificar a estrutura subjacente a este problema da publicação de notícias em linha

Detecção de comunidades no sistema de correio electrónico universitário

O estudo de sistemas estruturados em redes sociais conheceu inúmeros desenvolvimentos na aplicação da teoria de grafos às ciências sociais. Um dos aspectos recentes tem sido o da detecção de módulos, ou comunidades, em redes sociais. Diversos algoritmos e estratégias tem sido desenvolvidos para identificar a estrutura existente por detrás das interacções sociais. Atrav´es de um estudo de caso, mostrámos a existência de comunidades de comunicação informal que utiliza a rede de correio electrónico do ISCTE, através da aplicação de algoritmos hierárquicos de detecção de comunidades. Analisámos a estrutura hierárquica da rede através de k-cores e verificámos que a as comunidades de comunicação informal formadas ultrapassam as fronteiras dos departamentos institucionais através do método de percolação de cliques. `As comunidades detectadas aplicámos uma medida de variação de informação para determinar a distancia entre os diversos departamentos. Construímos um modelo de simulação multi-agente, para mimar o sistema de comunicação informal através de correio electrónico, CIUCEU, que nos permitiu verificar a influencia da vizinhança “social” dos agentes na criação e manutenção da estrutura da rede de professores do ISCTE. Analisámos ainda a utilização de simulações alimentadas por dados reais, concluindo sobre as implicações da utilização de dados reais sobre o desenho da simulação.The study of structured systems in social networks has gone through several developments by the use of graph theory in social sciences. On aspect that has been given considerable attention in recent years is the module or community detection in social networks. Several algorithms and strategies have been developed to identify the structure behind social interaction. Through a case study we show the existence of communities based on informal communication that use the email system at ISCTE. We applied a set of hierarchical algorithms to detect communities. Also, we analyzed the hierarchical structure through the k-cores method and verified the transitivity of the communities detected through clique percolation to put in evidence that informal communities are transversal to the institution departments. We also used a information variation measure to compare distances between different clusterings. We built a multi-agent simulation to model the informal communication mechanism of the email system, CIUCEU. This is used to verify the dependence of the system on the notion of social neighborhood, in the teachers network of ISCTE. We also analyzed the usage of real data and concluded on its implications of the sampling and drawing os multi-agent simulations

Cuidados biomédicos de saúde em Angola e na Companhia de Diamantes de Angola, c. 1910-1970

Pretende-se caracterizar a prestação de cuidados biomédicos em Angola durante a atividade da Companhia de Diamantes de Angola. Uma análise comparativa de políticas e práticas de saúde pública de vários atores coloniais, como os serviços de saúde da Companhia, sua congénere do Estado e outras empresas coloniais, revelará diferenças de investimento na saúde, isto é, instalações e pessoal de saúde, e tratamentos. Este escrutínio bem como as condições de vida iluminarão o carácter idiossincrático e central dos serviços de saúde da Companhia em termos de morbimortalidade em Angola, e a centralidade destes para as representações de um império cuidador

Local hydrological conditions influence tree diversity and composition across the Amazon basin

Tree diversity and composition in Amazonia are known to be strongly determined by the water supplied by precipitation. Nevertheless, within the same climatic regime, water availability is modulated by local topography and soil characteristics (hereafter referred to as local hydrological conditions), varying from saturated and poorly drained to well-drained and potentially dry areas. While these conditions may be expected to influence species distribution, the impacts of local hydrological conditions on tree diversity and composition remain poorly understood at the whole Amazon basin scale. Using a dataset of 443 1-ha non-flooded forest plots distributed across the basin, we investigate how local hydrological conditions influence 1) tree alpha diversity, 2) the community-weighted wood density mean (CWM-wd) – a proxy for hydraulic resistance and 3) tree species composition. We find that the effect of local hydrological conditions on tree diversity depends on climate, being more evident in wetter forests, where diversity increases towards locations with well-drained soils. CWM-wd increased towards better drained soils in Southern and Western Amazonia. Tree species composition changed along local soil hydrological gradients in Central-Eastern, Western and Southern Amazonia, and those changes were correlated with changes in the mean wood density of plots. Our results suggest that local hydrological gradients filter species, influencing the diversity and composition of Amazonian forests. Overall, this study shows that the effect of local hydrological conditions is pervasive, extending over wide Amazonian regions, and reinforces the importance of accounting for local topography and hydrology to better understand the likely response and resilience of forests to increased frequency of extreme climate events and rising temperatures

Geography and ecology shape the phylogenetic composition of Amazonian tree communities

Aim: Amazonia hosts more tree species from numerous evolutionary lineages, both young and ancient, than any other biogeographic region. Previous studies have shown that tree lineages colonized multiple edaphic environments and dispersed widely across Amazonia, leading to a hypothesis, which we test, that lineages should not be strongly associated with either geographic regions or edaphic forest types. Location: Amazonia. Taxon: Angiosperms (Magnoliids; Monocots; Eudicots). Methods: Data for the abundance of 5082 tree species in 1989 plots were combined with a mega-phylogeny. We applied evolutionary ordination to assess how phylogenetic composition varies across Amazonia. We used variation partitioning and Moran\u27s eigenvector maps (MEM) to test and quantify the separate and joint contributions of spatial and environmental variables to explain the phylogenetic composition of plots. We tested the indicator value of lineages for geographic regions and edaphic forest types and mapped associations onto the phylogeny. Results: In the terra firme and várzea forest types, the phylogenetic composition varies by geographic region, but the igapó and white-sand forest types retain a unique evolutionary signature regardless of region. Overall, we find that soil chemistry, climate and topography explain 24% of the variation in phylogenetic composition, with 79% of that variation being spatially structured (R$^{2}$ = 19% overall for combined spatial/environmental effects). The phylogenetic composition also shows substantial spatial patterns not related to the environmental variables we quantified (R$^{2}$ = 28%). A greater number of lineages were significant indicators of geographic regions than forest types. Main Conclusion: Numerous tree lineages, including some ancient ones (>66 Ma), show strong associations with geographic regions and edaphic forest types of Amazonia. This shows that specialization in specific edaphic environments has played a long-standing role in the evolutionary assembly of Amazonian forests. Furthermore, many lineages, even those that have dispersed across Amazonia, dominate within a specific region, likely because of phylogenetically conserved niches for environmental conditions that are prevalent within regions

Geography and ecology shape the phylogenetic composition of Amazonian tree communities

AimAmazonia hosts more tree species from numerous evolutionary lineages, both young and ancient, than any other biogeographic region. Previous studies have shown that tree lineages colonized multiple edaphic environments and dispersed widely across Amazonia, leading to a hypothesis, which we test, that lineages should not be strongly associated with either geographic regions or edaphic forest types.LocationAmazonia.TaxonAngiosperms (Magnoliids; Monocots; Eudicots).MethodsData for the abundance of 5082 tree species in 1989 plots were combined with a mega-phylogeny. We applied evolutionary ordination to assess how phylogenetic composition varies across Amazonia. We used variation partitioning and Moran's eigenvector maps (MEM) to test and quantify the separate and joint contributions of spatial and environmental variables to explain the phylogenetic composition of plots. We tested the indicator value of lineages for geographic regions and edaphic forest types and mapped associations onto the phylogeny.ResultsIn the terra firme and várzea forest types, the phylogenetic composition varies by geographic region, but the igapó and white-sand forest types retain a unique evolutionary signature regardless of region. Overall, we find that soil chemistry, climate and topography explain 24% of the variation in phylogenetic composition, with 79% of that variation being spatially structured (R2 = 19% overall for combined spatial/environmental effects). The phylogenetic composition also shows substantial spatial patterns not related to the environmental variables we quantified (R2 = 28%). A greater number of lineages were significant indicators of geographic regions than forest types.Main ConclusionNumerous tree lineages, including some ancient ones (>66 Ma), show strong associations with geographic regions and edaphic forest types of Amazonia. This shows that specialization in specific edaphic environments has played a long-standing role in the evolutionary assembly of Amazonian forests. Furthermore, many lineages, even those that have dispersed across Amazonia, dominate within a specific region, likely because of phylogenetically conserved niches for environmental conditions that are prevalent within regions

Geographic patterns of tree dispersal modes in Amazonia and their ecological correlates

Aim: To investigate the geographic patterns and ecological correlates in the geographic distribution of the most common tree dispersal modes in Amazonia (endozoochory, synzoochory, anemochory and hydrochory). We examined if the proportional abundance of these dispersal modes could be explained by the availability of dispersal agents (disperser-availability hypothesis) and/or the availability of resources for constructing zoochorous fruits (resource-availability hypothesis). Time period: Tree-inventory plots established between 1934 and 2019. Major taxa studied: Trees with a diameter at breast height (DBH) ≥ 9.55 cm. Location: Amazonia, here defined as the lowland rain forests of the Amazon River basin and the Guiana Shield. Methods: We assigned dispersal modes to a total of 5433 species and morphospecies within 1877 tree-inventory plots across terra-firme, seasonally flooded, and permanently flooded forests. We investigated geographic patterns in the proportional abundance of dispersal modes. We performed an abundance-weighted mean pairwise distance (MPD) test and fit generalized linear models (GLMs) to explain the geographic distribution of dispersal modes. Results: Anemochory was significantly, positively associated with mean annual wind speed, and hydrochory was significantly higher in flooded forests. Dispersal modes did not consistently show significant associations with the availability of resources for constructing zoochorous fruits. A lower dissimilarity in dispersal modes, resulting from a higher dominance of endozoochory, occurred in terra-firme forests (excluding podzols) compared to flooded forests. Main conclusions: The disperser-availability hypothesis was well supported for abiotic dispersal modes (anemochory and hydrochory). The availability of resources for constructing zoochorous fruits seems an unlikely explanation for the distribution of dispersal modes in Amazonia. The association between frugivores and the proportional abundance of zoochory requires further research, as tree recruitment not only depends on dispersal vectors but also on conditions that favour or limit seedling recruitment across forest types

Mapping density, diversity and species-richness of the Amazon tree flora

Using 2.046 botanically-inventoried tree plots across the largest tropical forest on Earth, we mapped tree species-diversity and tree species-richness at 0.1-degree resolution, and investigated drivers for diversity and richness. Using only location, stratified by forest type, as predictor, our spatial model, to the best of our knowledge, provides the most accurate map of tree diversity in Amazonia to date, explaining approximately 70% of the tree diversity and species-richness. Large soil-forest combinations determine a significant percentage of the variation in tree species-richness and tree alpha-diversity in Amazonian forest-plots. We suggest that the size and fragmentation of these systems drive their large-scale diversity patterns and hence local diversity. A model not using location but cumulative water deficit, tree density, and temperature seasonality explains 47% of the tree species-richness in the terra-firme forest in Amazonia. Over large areas across Amazonia, residuals of this relationship are small and poorly spatially structured, suggesting that much of the residual variation may be local. The Guyana Shield area has consistently negative residuals, showing that this area has lower tree species-richness than expected by our models. We provide extensive plot meta-data, including tree density, tree alpha-diversity and tree species-richness results and gridded maps at 0.1-degree resolution

Consistent patterns of common species across tropical tree communities

Trees structure the Earth’s most biodiverse ecosystem, tropical forests. The vast number of tree species presents a formidable challenge to understanding these forests, including their response to environmental change, as very little is known about most tropical tree species. A focus on the common species may circumvent this challenge. Here we investigate abundance patterns of common tree species using inventory data on 1,003,805 trees with trunk diameters of at least 10 cm across 1,568 locations1,2,3,4,5,6 in closed-canopy, structurally intact old-growth tropical forests in Africa, Amazonia and Southeast Asia. We estimate that 2.2%, 2.2% and 2.3% of species comprise 50% of the tropical trees in these regions, respectively. Extrapolating across all closed-canopy tropical forests, we estimate that just 1,053 species comprise half of Earth’s 800 billion tropical trees with trunk diameters of at least 10 cm. Despite differing biogeographic, climatic and anthropogenic histories7, we find notably consistent patterns of common species and species abundance distributions across the continents. This suggests that fundamental mechanisms of tree community assembly may apply to all tropical forests. Resampling analyses show that the most common species are likely to belong to a manageable list of known species, enabling targeted efforts to understand their ecology. Although they do not detract from the importance of rare species, our results open new opportunities to understand the world’s most diverse forests, including modelling their response to environmental change, by focusing on the common species that constitute the majority of their trees.Publisher PDFPeer reviewe

Long-term thermal sensitivity of Earth’s tropical forests

The sensitivity of tropical forest carbon to climate is a key uncertainty in predicting global climate change. Although short-term drying and warming are known to affect forests, it is unknown if such effects translate into long-term responses. Here, we analyze 590 permanent plots measured across the tropics to derive the equilibrium climate controls on forest carbon. Maximum temperature is the most important predictor of aboveground biomass (−9.1 megagrams of carbon per hectare per degree Celsius), primarily by reducing woody productivity, and has a greater impact per °C in the hottest forests (>32.2°C). Our results nevertheless reveal greater thermal resilience than observations of short-term variation imply. To realize the long-term climate adaptation potential of tropical forests requires both protecting them and stabilizing Earth’s climate