A geometric view on Pearson’s correlation coefficient and a generalization of it to non-linear dependencies

Measuring strength or degree of statistical dependence between two random variables is a common problem in many domains. Pearson’s correlation coefficient ρ is an accurate measure of linear dependence. We show that ρ is a normalized, Euclidean type distance between joint probability distribution of the two random variables and that when their independence is assumed while keeping their marginal distributions. And the normalizing constant is the geometric mean of two maximal distances; each between the joint probability distribution when the full linear dependence is assumed while preserving respective marginal distribution and that when the independence is assumed. Usage of it is restricted to linear dependence because it is based on Euclidean type distances that are generally not metrics and considered full dependence is linear. Therefore, we argue that if a suitable distance metric is used while considering all possible maximal dependences then it can measure any non-linear dependence. But then, one must define all the full dependences. Hellinger distance that is a metric can be used as the distance measure between probability distributions and obtain a generalization of ρ for the discrete case

Cheaper, cleaner, more reliable: why invest in cross-border power-trading

Despite improvements to energy supply over the years, many Indian states still face frequent power shortages. Meanwhile, neighboring Nepal and Bhutan have large reserves of untapped hydro power with the potential to meet unserved demand for energy in major load centres. Investing in interconnections could also contribute to significant reductions in CO2 emissions. Today’s blog quantifies potential gains from an integrated South Asian power

Recent advances in multimodal artificial intelligence for disease diagnosis, prognosis and prevention.

Artificial Intelligence (AI) has gained huge attention in computer-aided decision-making in the healthcare domain. Many novel AI methods have been developed for disease diagnosis and prognosis which may support in the prevention of disease. Most diseases can be cured early and managed better if timely diagnosis is made. The AI models can aid clinical diagnosis; thus, they make the processes more efficient by reducing the workload of physicians, nurses, radiologists, and others. However, the majority of AI methods rely on the use of single-modality data. For example, brain tumor detection uses brain MRI, skin lesion detection uses skin pathology images, and lung cancer detection uses lung CT or x-ray imaging (1). Single-modality AI models lack the much-needed integration of complex features available from different modality data, such as electronic health records (EHR), unstructured clinical notes, and different medical imaging modalities– otherwise form the backbone of clinical decision-making

On Associative Confounder Bias

Conditioning on some set of confounders that causally affect both treatmentand outcome variables can be sufficient for eliminating bias introduced by allsuch confounders when estimating causal effect of the treatment on the outcomefrom observational data. It is done by including them in propensity score modelin so-called potential outcome framework for causal inference whereas in causalgraphical modeling framework usual conditioning on them is done. However inthe former framework, it is confusing when modeler finds a variable that is noncausallyassociated with both the treatment and the outcome. Some argue that suchvariables should also be included in the analysis for removing bias. But others arguethat they introduce no bias so they should be excluded and conditioning onthem introduces spurious dependence between the treatment and the outcome, thusresulting extra bias in the estimation. We show that there may be errors in boththe arguments in different contexts. When such a variable is found neither of theactions may give the correct causal effect estimate. Selecting one action over theother is needed in order to be less wrong.We discuss how to select the better action