Recurrent Multimodal Interaction for Referring Image Segmentation

In this paper we are interested in the problem of image segmentation given natural language descriptions, i.e. referring expressions. Existing works tackle this problem by first modeling images and sentences independently and then segment images by combining these two types of representations. We argue that learning word-to-image interaction is more native in the sense of jointly modeling two modalities for the image segmentation task, and we propose convolutional multimodal LSTM to encode the sequential interactions between individual words, visual information, and spatial information. We show that our proposed model outperforms the baseline model on benchmark datasets. In addition, we analyze the intermediate output of the proposed multimodal LSTM approach and empirically explain how this approach enforces a more effective word-to-image interaction.Comment: To appear in ICCV 2017. See http://www.cs.jhu.edu/~cxliu/ for code and supplementary materia

Efficient market versus regulatory capture: a political economy assessment of power market reform in China

China began implementing market-based economic dispatch through power sector reform in 2015, but the reform has encountered some political and economic challenges. This paper identifies the reform’s efficiency changes and explores and quantifies the influences of market-driven and politically driven mechanisms behind these changes, employing a partial market equilibrium model integrating high-frequency data in southern China. We found that the dispatch transition improves the overall efficiency, but regulatory capture in provincial markets limits its full potential. The preference for local enterprises over central state-owned enterprises (SOEs) by local governments, in the form of allocated generation quotas, demonstrates the political challenge for market reform. The allocated generation quota protects small coal-fired and natural gas generators owned by local SOEs, lessening their motivation to improve generation efficiency, even after the reform. As a result, nearly half of the potential carbon dioxide emission reduction and social welfare gains through market reform is not realized

Problems in the design and operation of uncertain complex engineering systems

In this dissertation, we consider two problems. The first one is a general approach to the optimal design of uncertain dynamical systems, where the uncertainty is represented by a random parameter. The problem is formulated using two types of performance criteria, that result in two different optimal design methods. However, both of them are difficult to solve analytically for most uncertain complex dynamical systems. A numerical scheme is developed for the optimal design that involves two steps. First, in order to obtain a numerical algorithm for the optimal solution, we apply randomized algorithms for average performance synthesis to approximate the optimal solution. Second, using the properties of the Perron-Frobenius operator we develop an efficient computation approach for calculating the stationary distribution for the uncertain dynamical systems and the average performance criteria. The proposed approach is demonstrated through numerical examples. The second problem is a novel approach for evaluating the short-term Loss of Load Probability (LOLP) in power systems that include wind generation resources that vary stochastically in time. We firstly introduce a mathematical model for calculating the short-term LOLP, and then a novel quantitative measure of its behavior when converging to its steady-state level is derived. In addition, the corresponding empirical formulas are offered which can be used in practice to estimate the convergence time of LOLP under different conditions. Finally, an application of the outcomes of the analytical work in estimation of the dynamic behavior of short-term LOLP with an actual wind generation profile is presented to show the significance of the developed measures

High spatial-resolution imaging of label-free in vivo protein aggregates by VISTA

Amyloid aggregation, formed by aberrant proteins, is a pathological hallmark for neurodegenerative diseases, including Alzheimer's disease and Huntington's disease. High-resolution holistic mapping of the fine structures from these aggregates should facilitate our understanding of their pathological roles. Here, we achieved label-free high-resolution imaging of the polyQ and the amyloid-beta (Aβ) aggregates in cells and tissues utilizing a sample-expansion stimulated Raman strategy. We further focused on characterizing the Aβ plaques in 5XFAD mouse brain tissues. 3D volumetric imaging enabled visualization of the whole plaques, resolving both the fine protein filaments and the surrounding components. Coupling our expanded label-free Raman imaging with machine learning, we obtained specific segmentation of aggregate cores, peripheral filaments together with cell nuclei and blood vessels by pre-trained convolutional neural network models. Combining with 2-channel fluorescence imaging, we achieved a 6-color holistic view of the same sample. This ability for precise and multiplex high-resolution imaging of the protein aggregates and their micro-environment without the requirement of labeling would open new biomedical applications

Neural Collapse Inspired Federated Learning with Non-iid Data

One of the challenges in federated learning is the non-independent and identically distributed (non-iid) characteristics between heterogeneous devices, which cause significant differences in local updates and affect the performance of the central server. Although many studies have been proposed to address this challenge, they only focus on local training and aggregation processes to smooth the changes and fail to achieve high performance with deep learning models. Inspired by the phenomenon of neural collapse, we force each client to be optimized toward an optimal global structure for classification. Specifically, we initialize it as a random simplex Equiangular Tight Frame (ETF) and fix it as the unit optimization target of all clients during the local updating. After guaranteeing all clients are learning to converge to the global optimum, we propose to add a global memory vector for each category to remedy the parameter fluctuation caused by the bias of the intra-class condition distribution among clients. Our experimental results show that our method can improve the performance with faster convergence speed on different-size datasets.Comment: 11 pages, 5 figure

OBMO: One Bounding Box Multiple Objects for Monocular 3D Object Detection

Compared to typical multi-sensor systems, monocular 3D object detection has attracted much attention due to its simple configuration. However, there is still a significant gap between LiDAR-based and monocular-based methods. In this paper, we find that the ill-posed nature of monocular imagery can lead to depth ambiguity. Specifically, objects with different depths can appear with the same bounding boxes and similar visual features in the 2D image. Unfortunately, the network cannot accurately distinguish different depths from such non-discriminative visual features, resulting in unstable depth training. To facilitate depth learning, we propose a simple yet effective plug-and-play module, \underline{O}ne \underline{B}ounding Box \underline{M}ultiple \underline{O}bjects (OBMO). Concretely, we add a set of suitable pseudo labels by shifting the 3D bounding box along the viewing frustum. To constrain the pseudo-3D labels to be reasonable, we carefully design two label scoring strategies to represent their quality. In contrast to the original hard depth labels, such soft pseudo labels with quality scores allow the network to learn a reasonable depth range, boosting training stability and thus improving final performance. Extensive experiments on KITTI and Waymo benchmarks show that our method significantly improves state-of-the-art monocular 3D detectors by a significant margin (The improvements under the moderate setting on KITTI validation set are $\mathbf{1.82\sim 10.91\%}$ \textbf{mAP in BEV} and $\mathbf{1.18\sim 9.36\%}$ \textbf{mAP in 3D}). Codes have been released at \url{https://github.com/mrsempress/OBMO}.Comment: 10 pages, 7 figure