Inability of spatial transformations of CNN feature maps to support invariant recognition

A large number of deep learning architectures use spatial transformations of CNN feature maps or filters to better deal with variability in object appearance caused by natural image transformations. In this paper, we prove that spatial transformations of CNN feature maps cannot align the feature maps of a transformed image to match those of its original, for general affine transformations, unless the extracted features are themselves invariant. Our proof is based on elementary analysis for both the single- and multi-layer network case. The results imply that methods based on spatial transformations of CNN feature maps or filters cannot replace image alignment of the input and cannot enable invariant recognition for general affine transformations, specifically not for scaling transformations or shear transformations. For rotations and reflections, spatially transforming feature maps or filters can enable invariance but only for networks with learnt or hardcoded rotation- or reflection-invariant featuresComment: 22 pages, 3 figure

Understanding the effectiveness of government interventions against the resurgence of COVID-19 in Europe.

Funder: European and Developing Countries Clinical Trials Partnership (EDCTP); doi: https://doi.org/10.13039/501100001713Funder: MRC Centre for Global Infectious Disease Analysis (MR/R015600/1), jointly funded by the U.K. Medical Research Council (MRC) and the U.K. Foreign, Commonwealth and Development Office (FCDO), under the MRC/FCDO Concordat agreement. Community Jameel. The UK Research and Innovation (MR/V038109/1), the Academy of Medical Sciences Springboard Award (SBF004/1080), The MRC (MR/R015600/1), The BMGF (OPP1197730), Imperial College Healthcare NHS Trust- BRC Funding (RDA02), The Novo Nordisk Young Investigator Award (NNF20OC0059309) and The NIHR Health Protection Research Unit in Modelling Methodology. S. Bhatt thanks Microsoft AI for Health and Amazon AWS for computational credits.Funder: EA FundsFunder: University of Oxford (Oxford University); doi: https://doi.org/10.13039/501100000769Funder: DeepMindFunder: OpenPhilanthropyFunder: UKRI Centre for Doctoral Training in Interactive Artificial Intelligence (EP/S022937/1)Funder: Augustinus Fonden (Augustinus Foundation); doi: https://doi.org/10.13039/501100004954Funder: Knud Højgaards Fond (Knud Højgaard Fund); doi: https://doi.org/10.13039/501100009938Funder: Kai Lange og Gunhild Kai Langes Fond (Kai Lange and Gunhild Kai Lange Foundation); doi: https://doi.org/10.13039/501100008206Funder: Aage og Johanne Louis-Hansens Fond (Aage and Johanne Louis-Hansen Foundation); doi: https://doi.org/10.13039/501100010344Funder: William Demant FoundationFunder: Boehringer Ingelheim Fonds (Stiftung für medizinische Grundlagenforschung); doi: https://doi.org/10.13039/501100001645Funder: Imperial College COVID-19 Research FundFunder: Cancer Research UK (CRUK); doi: https://doi.org/10.13039/501100000289European governments use non-pharmaceutical interventions (NPIs) to control resurging waves of COVID-19. However, they only have outdated estimates for how effective individual NPIs were in the first wave. We estimate the effectiveness of 17 NPIs in Europe's second wave from subnational case and death data by introducing a flexible hierarchical Bayesian transmission model and collecting the largest dataset of NPI implementation dates across Europe. Business closures, educational institution closures, and gathering bans reduced transmission, but reduced it less than they did in the first wave. This difference is likely due to organisational safety measures and individual protective behaviours-such as distancing-which made various areas of public life safer and thereby reduced the effect of closing them. Specifically, we find smaller effects for closing educational institutions, suggesting that stringent safety measures made schools safer compared to the first wave. Second-wave estimates outperform previous estimates at predicting transmission in Europe's third wave

En undersökning av strategisk, hierarkisk information

Multi-agent strategic planning is a field that seeks to model groups of agents who must cooperate to fulfil some goal. It is closely connected to the field of game theory, as games are used to model the interactions between players. An important problem is to determine, for a given game and win condition, whether there exists any strategy that guarantees victory. Under conditions of imperfect information, where some players do not know the current state of the game, this problem is known to be undecidable. This means that there cannot exist any general algorithm that correctly determines whether there exists a winning strategy, for any game and any win condition. To circumvent this problem, researchers have identified some conditions such that, if a game fulfils one of the conditions, the problem of finding a winning strategy is decidable. One such condition is that of hierarchical information. This requires that, if you put the players in some total order, p1  p2  …  pn, then players earlier in the order should always know at least as much about the game as players later in the order. For example, if pi  pj, then pi should know everything that pj knows, regardless of what happens in the game. There exist multiple variations of this hierarchical principle, but none of them consider that players can be aware of the strategies that other players have been using, or even that they can remember the actions they themselves have taken. Thus, when evaluating what the players know, information about what actions have been taken is disregarded. In this thesis, we define strategic hierarchical information as a total preorder between players where the earlier players know at least as much as later players, if they know their own and later players’ strategies (regardless of what these strategies are). We prove that it is decidable whether a game with strategic hierarchical information has a winning strategy. We also consider games where strategic hierarchical information is not always present, but can be guaranteed for some strategies. We say that a strategy maintains strategic hierarchical information if, given that all players are following it and that each player knows that all later players have been following it, any player will know everything that players later in the hierarchy knows. We show that is is decidable whether any game contains a winning strategy that maintains strategic hierarchical information, and describe a way of synthesizing such strategies.Strategisk planering för spel med flera spelare handlar om att modellera en grupp agenter som måste samarbeta för att nå något mål. Liksom spelteori så använder sig fältet av formellt specificerade spel för att modellera interaktioner mellan spelarna. Ett viktigt problem inom fältet är att avgöra huruvida det finns någon vinnande strategi, givet ett spel och ett mål. Under ofullständig information, där vissa spelare inte vet spelets nuvarande tillstånd, så är detta problem oavgörbart. Detta innebär att det inte finns någon generell algoritm som korrekt avgör om det finns en vinnande strategy, för vilket spel och mål som helst. För att komma runt detta problem så har några villkor identifierats, sådana att om ett spel uppfyller något av villkoren, så är det ett avgörbart problem huruvida det finns en vinnande strategi, eller ej. Ett sådant villkor är hierarkisk information. Ett spel har hierarkisk information om det är möjligt att placera spelarna i någon ordning, p1  p2  …  pn, så att spelare som kommer tidigare i ordningen vet åtminstone lika mycket som spelare som kommer senare i ordningen. Om vi till exempel har spelare pi och pj så att  pi  pj så ska pi alltid veta minst lika mycket som pj om vad som har hänt i spelet, oavsett vad som har hänt. Det finns flera variationer på denna hierarkiska princip, men ingen av dem tar hänsyn till att spelare kan vara medvetna om strategierna som andra spelare använder sig av, eller ens att spelare kan komma ihåg vad de har gjort för val i tidigare rundor. Därmed så ignoreras all information om hur spelarna agerar, när det avgörs vad spelare kan sägas veta, med hänsyn till den hierarkiska principen. I den här rapporten så definerar vi strategisk hierarisk information som en ordning mellan spelarna där tidigare spelare vet minst lika mycket som senare spelare, under förutsättningen att de vet sin egen och alla senare spelares strategier (oavsett vilka dessa strategier är). Vi visar att det är avgörbart om ett spel med strategisk hierarkisk information har en vinnande strategi. Vi studerar också spel där strategisk hierarkisk information inte gäller för alla strategier, men kan garanteras för några. Vi säger att en strategi bibehåller strategisk hierarkisk information om, givet att alla spelare följer strategin och varje spelare vet att alla senare spelare följer strategin, så kommer tidigare spelare att veta minst lika mycket som senare spelare. Vi visar att det är avgörbart om ett spel innehåller en vinnande strategi som bibehåller strategisk hierarkisk information, och beskriver en metod för att hitta sådana strategier

En undersökning av strategisk, hierarkisk information

Multi-agent strategic planning is a field that seeks to model groups of agents who must cooperate to fulfil some goal. It is closely connected to the field of game theory, as games are used to model the interactions between players. An important problem is to determine, for a given game and win condition, whether there exists any strategy that guarantees victory. Under conditions of imperfect information, where some players do not know the current state of the game, this problem is known to be undecidable. This means that there cannot exist any general algorithm that correctly determines whether there exists a winning strategy, for any game and any win condition. To circumvent this problem, researchers have identified some conditions such that, if a game fulfils one of the conditions, the problem of finding a winning strategy is decidable. One such condition is that of hierarchical information. This requires that, if you put the players in some total order, p1  p2  …  pn, then players earlier in the order should always know at least as much about the game as players later in the order. For example, if pi  pj, then pi should know everything that pj knows, regardless of what happens in the game. There exist multiple variations of this hierarchical principle, but none of them consider that players can be aware of the strategies that other players have been using, or even that they can remember the actions they themselves have taken. Thus, when evaluating what the players know, information about what actions have been taken is disregarded. In this thesis, we define strategic hierarchical information as a total preorder between players where the earlier players know at least as much as later players, if they know their own and later players’ strategies (regardless of what these strategies are). We prove that it is decidable whether a game with strategic hierarchical information has a winning strategy. We also consider games where strategic hierarchical information is not always present, but can be guaranteed for some strategies. We say that a strategy maintains strategic hierarchical information if, given that all players are following it and that each player knows that all later players have been following it, any player will know everything that players later in the hierarchy knows. We show that is is decidable whether any game contains a winning strategy that maintains strategic hierarchical information, and describe a way of synthesizing such strategies.Strategisk planering för spel med flera spelare handlar om att modellera en grupp agenter som måste samarbeta för att nå något mål. Liksom spelteori så använder sig fältet av formellt specificerade spel för att modellera interaktioner mellan spelarna. Ett viktigt problem inom fältet är att avgöra huruvida det finns någon vinnande strategi, givet ett spel och ett mål. Under ofullständig information, där vissa spelare inte vet spelets nuvarande tillstånd, så är detta problem oavgörbart. Detta innebär att det inte finns någon generell algoritm som korrekt avgör om det finns en vinnande strategy, för vilket spel och mål som helst. För att komma runt detta problem så har några villkor identifierats, sådana att om ett spel uppfyller något av villkoren, så är det ett avgörbart problem huruvida det finns en vinnande strategi, eller ej. Ett sådant villkor är hierarkisk information. Ett spel har hierarkisk information om det är möjligt att placera spelarna i någon ordning, p1  p2  …  pn, så att spelare som kommer tidigare i ordningen vet åtminstone lika mycket som spelare som kommer senare i ordningen. Om vi till exempel har spelare pi och pj så att  pi  pj så ska pi alltid veta minst lika mycket som pj om vad som har hänt i spelet, oavsett vad som har hänt. Det finns flera variationer på denna hierarkiska princip, men ingen av dem tar hänsyn till att spelare kan vara medvetna om strategierna som andra spelare använder sig av, eller ens att spelare kan komma ihåg vad de har gjort för val i tidigare rundor. Därmed så ignoreras all information om hur spelarna agerar, när det avgörs vad spelare kan sägas veta, med hänsyn till den hierarkiska principen. I den här rapporten så definerar vi strategisk hierarisk information som en ordning mellan spelarna där tidigare spelare vet minst lika mycket som senare spelare, under förutsättningen att de vet sin egen och alla senare spelares strategier (oavsett vilka dessa strategier är). Vi visar att det är avgörbart om ett spel med strategisk hierarkisk information har en vinnande strategi. Vi studerar också spel där strategisk hierarkisk information inte gäller för alla strategier, men kan garanteras för några. Vi säger att en strategi bibehåller strategisk hierarkisk information om, givet att alla spelare följer strategin och varje spelare vet att alla senare spelare följer strategin, så kommer tidigare spelare att veta minst lika mycket som senare spelare. Vi visar att det är avgörbart om ett spel innehåller en vinnande strategi som bibehåller strategisk hierarkisk information, och beskriver en metod för att hitta sådana strategier

En undersökning av strategisk, hierarkisk information

Multi-agent strategic planning is a field that seeks to model groups of agents who must cooperate to fulfil some goal. It is closely connected to the field of game theory, as games are used to model the interactions between players. An important problem is to determine, for a given game and win condition, whether there exists any strategy that guarantees victory. Under conditions of imperfect information, where some players do not know the current state of the game, this problem is known to be undecidable. This means that there cannot exist any general algorithm that correctly determines whether there exists a winning strategy, for any game and any win condition. To circumvent this problem, researchers have identified some conditions such that, if a game fulfils one of the conditions, the problem of finding a winning strategy is decidable. One such condition is that of hierarchical information. This requires that, if you put the players in some total order, p1  p2  …  pn, then players earlier in the order should always know at least as much about the game as players later in the order. For example, if pi  pj, then pi should know everything that pj knows, regardless of what happens in the game. There exist multiple variations of this hierarchical principle, but none of them consider that players can be aware of the strategies that other players have been using, or even that they can remember the actions they themselves have taken. Thus, when evaluating what the players know, information about what actions have been taken is disregarded. In this thesis, we define strategic hierarchical information as a total preorder between players where the earlier players know at least as much as later players, if they know their own and later players’ strategies (regardless of what these strategies are). We prove that it is decidable whether a game with strategic hierarchical information has a winning strategy. We also consider games where strategic hierarchical information is not always present, but can be guaranteed for some strategies. We say that a strategy maintains strategic hierarchical information if, given that all players are following it and that each player knows that all later players have been following it, any player will know everything that players later in the hierarchy knows. We show that is is decidable whether any game contains a winning strategy that maintains strategic hierarchical information, and describe a way of synthesizing such strategies.Strategisk planering för spel med flera spelare handlar om att modellera en grupp agenter som måste samarbeta för att nå något mål. Liksom spelteori så använder sig fältet av formellt specificerade spel för att modellera interaktioner mellan spelarna. Ett viktigt problem inom fältet är att avgöra huruvida det finns någon vinnande strategi, givet ett spel och ett mål. Under ofullständig information, där vissa spelare inte vet spelets nuvarande tillstånd, så är detta problem oavgörbart. Detta innebär att det inte finns någon generell algoritm som korrekt avgör om det finns en vinnande strategy, för vilket spel och mål som helst. För att komma runt detta problem så har några villkor identifierats, sådana att om ett spel uppfyller något av villkoren, så är det ett avgörbart problem huruvida det finns en vinnande strategi, eller ej. Ett sådant villkor är hierarkisk information. Ett spel har hierarkisk information om det är möjligt att placera spelarna i någon ordning, p1  p2  …  pn, så att spelare som kommer tidigare i ordningen vet åtminstone lika mycket som spelare som kommer senare i ordningen. Om vi till exempel har spelare pi och pj så att  pi  pj så ska pi alltid veta minst lika mycket som pj om vad som har hänt i spelet, oavsett vad som har hänt. Det finns flera variationer på denna hierarkiska princip, men ingen av dem tar hänsyn till att spelare kan vara medvetna om strategierna som andra spelare använder sig av, eller ens att spelare kan komma ihåg vad de har gjort för val i tidigare rundor. Därmed så ignoreras all information om hur spelarna agerar, när det avgörs vad spelare kan sägas veta, med hänsyn till den hierarkiska principen. I den här rapporten så definerar vi strategisk hierarisk information som en ordning mellan spelarna där tidigare spelare vet minst lika mycket som senare spelare, under förutsättningen att de vet sin egen och alla senare spelares strategier (oavsett vilka dessa strategier är). Vi visar att det är avgörbart om ett spel med strategisk hierarkisk information har en vinnande strategi. Vi studerar också spel där strategisk hierarkisk information inte gäller för alla strategier, men kan garanteras för några. Vi säger att en strategi bibehåller strategisk hierarkisk information om, givet att alla spelare följer strategin och varje spelare vet att alla senare spelare följer strategin, så kommer tidigare spelare att veta minst lika mycket som senare spelare. Vi visar att det är avgörbart om ett spel innehåller en vinnande strategi som bibehåller strategisk hierarkisk information, och beskriver en metod för att hitta sådana strategier

Understanding when spatial transformer networks do not support invariance, and what to do about it

Spatial transformer networks (STNs) were designed to enable convolutional neural networks (CNNs) to learn invariance to image transformations. STNs were originally proposed to transform CNN feature maps as well as input images. This enables the use of more complex features when predicting transformation parameters. However, since STNs perform a purely spatial transformation, they do not, in the general case, have the ability to align the feature maps of a transformed image with those of its original. STNs are therefore unable to support invariance when transforming CNN feature maps. We present a simple proof for this and study the practical implications, showing that this inability is coupled with decreased classification accuracy. We therefore investigate alternative STN architectures that make use of complex features. We find that while deeper localization networks are difficult to train, localization networks that share parameters with the classification network remain stable as they grow deeper, which allows for higher classification accuracy on difficult datasets. Finally, we explore the interaction between localization network complexity and iterative image alignment.Not duplicate with DiVA 1516191QC 20200511</p

The problems with using STNs to align CNN feature maps

Spatial transformer networks (STNs) were designed to enable CNNs to learn invariance to image transformations. STNs were originally proposed to transform CNN feature maps as well as input images. This enables the use of more complex features when predicting transformation parameters. However, since STNs perform a purely spatial transformation, they do not, in the general case, have the ability to align the feature maps of a transformed image and its original. We present a theoretical argument for this and investigate the practical implications, showing that this inability is coupled with decreased classification accuracy. We advocate taking advantage of more complex features in deeper layers by instead sharing parameters between the classification and the localisation network.QC 20200123</p

The problems with using STNs to align CNN feature maps

Spatial transformer networks (STNs) were designed to enable CNNs to learn invariance to image transformations. STNs were originally proposed to transform CNN feature maps as well as input images. This enables the use of more complex features when predicting transformation parameters. However, since STNs perform a purely spatial transformation, they do not, in the general case, have the ability to align the feature maps of a transformed image and its original. We present a theoretical argument for this and investigate the practical implications, showing that this inability is coupled with decreased classification accuracy. We advocate taking advantage of more complex features in deeper layers by instead sharing parameters between the classification and the localisation network.QC 20200123</p

Understanding when spatial transformer networks do not support invariance, and what to do about it

Spatial transformer networks (STNs) were designed to enable convolutional neural networks (CNNs) to learn invariance to image transformations. STNs were originally proposed to transform CNN feature maps as well as input images. This enables the use of more complex features when predicting transformation parameters. However, since STNs perform a purely spatial transformation, they do not, in the general case, have the ability to align the feature maps of a transformed image with those of its original. STNs are therefore unable to support invariance when transforming CNN feature maps. We present a simple proof for this and study the practical implications, showing that this inability is coupled with decreased classification accuracy. We therefore investigate alternative STN architectures that make use of complex features. We find that while deeper localization networks are difficult to train, localization networks that share parameters with the classification network remain stable as they grow deeper, which allows for higher classification accuracy on difficult datasets. Finally, we explore the interaction between localization network complexity and iterative image alignment.Not duplicate with DiVA 1428271QC 20210831</p

Understanding when spatial transformer networks do not support invariance, and what to do about it

Spatial transformer networks (STNs) were designed to enable convolutional neural networks (CNNs) to learn invariance to image transformations. STNs were originally proposed to transform CNN feature maps as well as input images. This enables the use of more complex features when predicting transformation parameters. However, since STNs perform a purely spatial transformation, they do not, in the general case, have the ability to align the feature maps of a transformed image with those of its original. STNs are therefore unable to support invariance when transforming CNN feature maps. We present a simple proof for this and study the practical implications, showing that this inability is coupled with decreased classification accuracy. We therefore investigate alternative STN architectures that make use of complex features. We find that while deeper localization networks are difficult to train, localization networks that share parameters with the classification network remain stable as they grow deeper, which allows for higher classification accuracy on difficult datasets. Finally, we explore the interaction between localization network complexity and iterative image alignment.QC 20200511</p