Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation

Despite the tremendous achievements of deep convolutional neural networks (CNNs) in many computer vision tasks, understanding how they actually work remains a significant challenge. In this paper, we propose a novel two-step understanding method, namely Salient Relevance (SR) map, which aims to shed light on how deep CNNs recognize images and learn features from areas, referred to as attention areas, therein. Our proposed method starts out with a layer-wise relevance propagation (LRP) step which estimates a pixel-wise relevance map over the input image. Following, we construct a context-aware saliency map, SR map, from the LRP-generated map which predicts areas close to the foci of attention instead of isolated pixels that LRP reveals. In human visual system, information of regions is more important than of pixels in recognition. Consequently, our proposed approach closely simulates human recognition. Experimental results using the ILSVRC2012 validation dataset in conjunction with two well-established deep CNN models, AlexNet and VGG-16, clearly demonstrate that our proposed approach concisely identifies not only key pixels but also attention areas that contribute to the underlying neural network's comprehension of the given images. As such, our proposed SR map constitutes a convenient visual interface which unveils the visual attention of the network and reveals which type of objects the model has learned to recognize after training. The source code is available at https://github.com/Hey1Li/Salient-Relevance-Propagation.Comment: 35 pages, 15 figure

Self-organization and phase transition in financial markets with multiple choices

Market confidence is essential for successful investing. By incorporating multi-market into the evolutionary minority game, we investigate the effects of investor beliefs on the evolution of collective behaviors and asset prices. When there exists another investment opportunity, market confidence, including overconfidence and under-confidence, is not always good or bad for investment. The roles of market confidence is closely related to market impact. For low market impact, overconfidence in a particular asset makes an investor become insensitive to losses and a delayed strategy adjustment leads to a decline in wealth, and thereafter, one's runaway from the market. For high market impact, under-confidence in a particular asset makes an investor over-sensitive to losses and one's too frequent strategy adjustment leads to a large fluctuation in asset prices, and thereafter, a decrease in the number of agents. At an intermediate market impact, the phase transition occurs. No matter what the market impact is, an equilibrium between different markets exists, which is reflected in the occurrence of similar price fluctuations in different markets. A theoretical analysis indicates that such an equilibrium results from the coupled effects of strategy updating and shift in investment. The runaway of the agents trading a specific asset will lead to a decline in the asset price volatility and such a decline will be inhibited by the clustering of the strategies. A uniform strategy distribution will lead to a large fluctuation in asset prices and such a fluctuation will be suppressed by the decrease in the number of agents in the market. A functional relationship between the price fluctuations and the numbers of agents is found

A generalized public goods game with coupling of individual ability and project benefit

Facing a heavy task, any single person can only make a limited contribution and team cooperation is needed. As one enjoys the benefit of the public goods, the potential benefits of the project are not always maximized and may be partly wasted. By incorporating individual ability and project benefit into the original public goods game, we study the coupling effect of the four parameters, the upper limit of individual contribution, the upper limit of individual benefit, the needed project cost and the upper limit of project benefit on the evolution of cooperation. Coevolving with the individual-level group size preferences, an increase in the upper limit of individual benefit promotes cooperation while an increase in the upper limit of individual contribution inhibits cooperation. The coupling of the upper limit of individual contribution and the needed project cost determines the critical point of the upper limit of project benefit, where the equilibrium frequency of cooperators reaches its highest level. Above the critical point, an increase in the upper limit of project benefit inhibits cooperation. The evolution of cooperation is closely related to the preferred group-size distribution. A functional relation between the frequency of cooperators and the dominant group size is found

Coupled effects of local movement and global interaction on contagion

By incorporating segregated spatial domain and individual-based linkage into the SIS (susceptible-infected-susceptible) model, we investigate the coupled effects of random walk and intragroup interaction on contagion. Compared with the situation where only local movement or individual-based linkage exists, the coexistence of them leads to a wider spread of infectious disease. The roles of narrowing segregated spatial domain and reducing mobility in epidemic control are checked, these two measures are found to be conducive to curbing the spread of infectious disease. Considering heterogeneous time scales between local movement and global interaction, a log-log relation between the change in the number of infected individuals and the timescale $\tau$ is found. A theoretical analysis indicates that the evolutionary dynamics in the present model is related to the encounter probability and the encounter time. A functional relation between the epidemic threshold and the ratio of shortcuts, and a functional relation between the encounter time and the timescale $\tau$ are found

Before Becoming a World Heritage: Spatiotemporal Dynamics and Spatial Dependency of the Soundscapes in Kulangsu Scenic Area, China

Kulangsu is a famous scenic area in China and a World Heritage Site. It is important to obtain knowledge with regard to the status of soundscape and landscape resources and their interrelationships in Kulangsu before it became a World Heritage. The objective of this study was to explore the spatial dependency of the soundscapes in Kulangsu, based on the spatiotemporal dynamics of soundscape and landscape perceptions, including perceived sound sources, soundscape quality, and landscape satisfaction degree, and the spatial landscape characteristics, including the distance to green spaces, normalized difference vegetation index, and landscape spatial patterns. The results showed that perception of soundscape and landscape were observed in significant spatiotemporal dynamics, and the dominance of biological sounds in all sampling periods and human sounds in the evening indicated that Kulangsu scenic area had a good natural environment and a developed night-time economy, respectively. The green spaces and commercial lands may contribute to both the soundscape pleasantness and eventfulness. Moreover, the soundscape quality was dependent on the sound dominant degree and landscape satisfaction degree but not on the landscape characteristics. The GWR model had better goodness of fit than the OLS model, and possible non-linear relationships were found between the soundscape pleasantness and the variables of perceived sound sources and landscape satisfaction degree. The GWR models with spatial stationarity were found to be more effective in understanding the spatial dependence of soundscapes. In particular, the data applied should ideally include a complete temporal dimension to obtain a relatively high fitting accuracy of the model. These findings can provide useful data support and references for future planning and design practices, and management strategies for the soundscape resources in scenic areas and World Heritage Sites