43 research outputs found
S2SNet: A Pretrained Neural Network for Superconductivity Discovery
Superconductivity allows electrical current to flow without any energy loss,
and thus making solids superconducting is a grand goal of physics, material
science, and electrical engineering. More than 16 Nobel Laureates have been
awarded for their contribution to superconductivity research. Superconductors
are valuable for sustainable development goals (SDGs), such as climate change
mitigation, affordable and clean energy, industry, innovation and
infrastructure, and so on. However, a unified physics theory explaining all
superconductivity mechanism is still unknown. It is believed that
superconductivity is microscopically due to not only molecular compositions but
also the geometric crystal structure. Hence a new dataset, S2S, containing both
crystal structures and superconducting critical temperature, is built upon
SuperCon and Material Project. Based on this new dataset, we propose a novel
model, S2SNet, which utilizes the attention mechanism for superconductivity
prediction. To overcome the shortage of data, S2SNet is pre-trained on the
whole Material Project dataset with Masked-Language Modeling (MLM). S2SNet
makes a new state-of-the-art, with out-of-sample accuracy of 92% and Area Under
Curve (AUC) of 0.92. To the best of our knowledge, S2SNet is the first work to
predict superconductivity with only information of crystal structures. This
work is beneficial to superconductivity discovery and further SDGs. Code and
datasets are available in https://github.com/zjuKeLiu/S2SNetComment: Accepted to IJCAI 202
-Equivariant Vision Transformer
Vision Transformer (ViT) has achieved remarkable performance in computer
vision. However, positional encoding in ViT makes it substantially difficult to
learn the intrinsic equivariance in data. Initial attempts have been made on
designing equivariant ViT but are proved defective in some cases in this paper.
To address this issue, we design a Group Equivariant Vision Transformer
(GE-ViT) via a novel, effective positional encoding operator. We prove that
GE-ViT meets all the theoretical requirements of an equivariant neural network.
Comprehensive experiments are conducted on standard benchmark datasets,
demonstrating that GE-ViT significantly outperforms non-equivariant
self-attention networks. The code is available at
https://github.com/ZJUCDSYangKaifan/GEVit.Comment: Accept to UAI202