Decoupled Contrastive Learning

Contrastive learning (CL) is one of the most successful paradigms for self-supervised learning (SSL). In a principled way, it considers two augmented "views" of the same image as positive to be pulled closer, and all other images as negative to be pushed further apart. However, behind the impressive success of CL-based techniques, their formulation often relies on heavy-computation settings, including large sample batches, extensive training epochs, etc. We are thus motivated to tackle these issues and establish a simple, efficient, yet competitive baseline of contrastive learning. Specifically, we identify, from theoretical and empirical studies, a noticeable negative-positive-coupling (NPC) effect in the widely used InfoNCE loss, leading to unsuitable learning efficiency concerning the batch size. By removing the NPC effect, we propose decoupled contrastive learning (DCL) loss, which removes the positive term from the denominator and significantly improves the learning efficiency. DCL achieves competitive performance with less sensitivity to sub-optimal hyperparameters, requiring neither large batches in SimCLR, momentum encoding in MoCo, or large epochs. We demonstrate with various benchmarks while manifesting robustness as much less sensitive to suboptimal hyperparameters. Notably, SimCLR with DCL achieves 68.2% ImageNet-1K top-1 accuracy using batch size 256 within 200 epochs pre-training, outperforming its SimCLR baseline by 6.4%. Further, DCL can be combined with the SOTA contrastive learning method, NNCLR, to achieve 72.3% ImageNet-1K top-1 accuracy with 512 batch size in 400 epochs, which represents a new SOTA in contrastive learning. We believe DCL provides a valuable baseline for future contrastive SSL studies.Comment: Accepted by ECCV202

Design, Synthesis, and Properties of Polyimides and Their Hybrid Materials for Optical Device Applications

本文藉由理論與實驗探討聚醯亞胺之分子結構與其近紅外光區光學性質之關係。並合成相容性更佳之新型聚醯亞胺/氧化矽混成材料。最後製備可進行直接微影程序之感光性聚醯亞胺/氧化矽混成材料以製作通道性光波導。 本文的第一部份由量子化學的角度探討分子結構對分子於近紅外光的振動吸收之影響。由實驗與量子化學計算的結果顯示，當聚醯亞胺之雙胺部份的橋接官能基之拉電子能力不同時，會造成苯環上碳氫鍵長之變化，而使得碳氫鍵之振動頻率（包含基頻及倍頻）產生偏移。當橋接官能基的拉電子能力越強時，會使得碳氫鍵縮短而提高振動頻率。僅藉由改變橋基官能基由-CH2-至-SO2-即可使碳氫鍵於近紅外光區的第一倍頻吸收位置偏移約20nm，第二倍頻吸收位置偏移約10 nm。且由於分子結構使倍頻吸收位置產生偏移，會影響到聚醯亞胺所製備平面光波導之光傳損失。以PMDA/DPM與PMDA/ODA為例，雖然PMDA/DPM具有較高的C-H鍵密度，但由於其C-H鍵較長使得其振動吸收位置往長波長處偏移，造成以PMDA/DPM所製備的光波導在波長1310nm處的光傳損失反而比PMDA/ODA來得低。 本文第二部份探討聚醯亞胺/氧化矽混成材料的合成與性質。聚醯亞胺為傳統之高性能高分子，被廣泛地使用在微電子等產業中，聚醯亞胺/氧化矽混成材料可進一步提高其熱性質與機械性質而獲得廣泛地研究。本研究利用GOTMS作為分子間作用力（氫鍵）之耦合基與APrTMS作為分子內耦合基（共價鍵），可以同時保留聚醯亞胺的優秀熱穩定性同時使混成的氧化矽顆粒均勻的分散其內。藉由TEM照片可知本研究逤製備之混成材料，其氧化矽顆粒大小僅有5nm，可將非必要之光散射損失降至最低。此外聚醯亞胺的熱穩定性亦可藉由加入氧化矽而提高。由於奈米等級分散的氧化矽顆粒與聚醯亞胺藉由分子內與分子間耦合基形成類似交聯構造，使得聚醯亞胺的熱膨脹係數在加入氧化矽後從原本的123.4-726.9 ppm/℃ 大幅降低為16.6-44 pm/℃。所合成的聚醯亞胺/氧化矽混成材料之介電常數為2.85-3.73，波長1310nm處的光傳損失為0.5-1.9 dB/cm，具有應用在低介電常數材料或光波導材料上之潛力。 本文第三部份探討感光性聚醯亞胺/氧化矽混成材料的合成與性質。聚醯亞胺/氧化矽混成材料雖已受到廣泛的研究，但傳統的微影製程需要較多的製程步驟與較長的製程時間。本文在聚醯亞胺鏈段上導入離子型之未飽和基團MDAE，在氧化矽部份則導入未飽和基團MPTMS，加入光起始劑反應後可使其為負型之感光性材料，以直接進行曝光顯影與圖型轉移之步驟。感光性聚醯亞胺/氧化矽混成材料具有較原本之聚醯亞胺更為優秀之耐熱性質，同時在其熱固化的過程中，體積的收縮極小，可避免固化後圖型產生變型進而提高材料的微影解析度。以感光性聚醯亞胺/氧化矽混成材料製作通道型光學波導具有折射率易調控，熱穩定性佳，熱膨脹係數較低，與低光傳損失等優點。且此一方法可直接應用於目前大部份之聚醯亞胺/金屬氧化物混成材料中，使其能夠直接進行光學微影圖型化以節省製程所需之時間。且由於此感光性聚醯亞胺/氧化矽混成材料對有機與無機材料之相容性俱佳，可以之掺雜有機發光基團（dye），稀土元素（Lanthanide），或是半導體奈米晶體等（e.g. CdS），使其成為可直接進行光學微影之主動性材料以應用於非線性光學（NLO-EO）、EL、PL、Amplifier等領域上。In this dissertation, both theoretical and experimental studies were used to correlate the near-infrared optical properties with the molecular structures of polyimides. Besides, new polyimide/silica hybrid materials were developed to improve the physical properties of their parent polyimides. Low loss, low volume shrinkage photosensitive polyimide/silica hybrid materials were also synthesized for fabricating channel optical waveguides. In the first part of this dissertation, “the chemical structure-near infrared optical property” relationship was investigated by both ab initio calculations and experiments. In Chapter 2, the substituents effects of several benzene derivatives on their C-H vibrational frequencies including fundamentals and overtones were investigated. It was found that the electron-withdrawing ability of substituents would influence the bond lengths of other C-H bonds on the aromatic ring: the stronger the electron-withdrawing ability, the shorter the adjacent C-H bond length; and the shorter C-H bonds will vibrate at higher frequencies. This characteristic can be utilized to tune the absorption position of C-H vibrational overtones in the near-infrared region. The maximum shifting of absorption positions was 30nm for the first overtones, and 15nm for the second overtones. In Chapter 3, the similar concept was extended to the polyimides. Polyimides stand for a potential candidate of materials for high performance and low cost optical waveguides. For polymers, their C-H absorption loss in the near-infrared region is critical for optical waveguide applications. In this chapter, several diamines bearing different bridge groups with different electron-withdrawing ability were used to investigate the effects of these bridge groups on the near-infrared optical properties of diamines and polyimides (synthesized with the dianhydride PMDA). It was found that the electron-withdrawing ability also could influence the C-H vibrational overtone absorption positions in the near-infrared region. Their absorption positions can be shifted up to 20nm for the first overtone, and 10nm for the second overtone. The planar waveguides of these polyimides were fabricated to investigate if the shifting of absorption band would influence their apparent optical loss. And it was found that PMDA/DPM stands for a lower optical loss at the wavelength of 1310nm than PMDA/ODA though its higher C-H number density than PMDA/ODA. In the second part of this dissertation (Chapter 4), a new method to produce nano-sized silica particles in polyimide/silica hybrid materials was proposed. By using both inter- and intra-molecular interactions between polyimide and silica can improve their compatibility most. APrTEOS was used as the intermolecular coupling agents by covalently bonded to the ends of polyimides, and GOTMS was used as intramolecular coupling agents by the formation of hydrogen bonds between the end diols of GOTMS and carbonyl groups of polyimides. By this method, it was found that the size of silica particles can be reduced to 5nm. And their coefficients of thermal expansion can be reduced to about 2 orders in comparison with their parent polyimides. In the third part of this dissertation (Chapter 5), a new synthesis scheme was proposed to produce photo-patternable polyimide/silica hybrid materials. By introducing the photosensitive compound of MDAE via the formation of acid/base ionic salts between tertiary amines of MDAE and carboxylic acid of poly(amic acid), the polyimide/silica hybrid materials can be patterned by direct photo-lithography acting as a negative tone photoresist. Besides MDAE, a trifunctional silane bearing a unsaturated methacrylate substituents, MPTMS, was incorporated to the silica moieties of polyimide/silica hybrid materials. In conventional ionic salt photosensitive polyimide, the volume shrinkage is considerable because all MDAE will be removed from films after the complete imidization (no more ionic salts). The incorporation of MPTMS can reduce the volume shrinkage of polyimide/silica hybrid materials to low during the curing processes, because part of MDAE was photo-crosslinked with MPTMS after the exposure step. The volume shrinkage was reduced to <10% for Polyimide/SiO2/MPTMS in comparison with >20% of that for polyimide. The prepared low shrinkage Polyimide/SiO2/MPTMS hybrid materials exhibit good lithographic resolution, excellent thermal stability, and tunable refractive indices.中文摘要.p1 Abstract.p3 Contents.p6 Table Captions.p10 Figure Captions.p12 Chapter 1 Design, Synthesis, and Properties of Polymers and Their Hybrids for Optical Waveguide Applications.p1 Chapter 2 Model Compound Studies: Effect of Molecular Structures on the C-H Overtone Spectra and Bond Lengths of Benzene Derivatives.p103 Chapter 3 Monomer and Polymer Studies: Effects of Molecular Structures on the Near-Infrared Optical Properties of Polyimides and Their Corresponding Optical Planar Waveguides.p137 Chapter 4 Synthesis and Characterization of Polyimide/Silica Hybrid Materials: An Approach through Both Interchain and Intrachain Interactions toward Monodispersed Nanosize Silica Particles. p160 Chapter 5 Low Volume Shrinkage Photo-patternable Polyimide/Silica Hybrid Materials for Low-loss Optical Waveguides.p184 Chapter 6 Conclusions and Future Works.p211 Autobiography and publication list.p21

Query-Driven Multi-Instance Learning

We introduce a query-driven approach (qMIL) to multi-instance learning where the queries aim to uncover the class labels embodied in a given bag of instances. Specifically, it solves a multi-instance multi-label learning (MIML) problem with a more challenging setting than the conventional one. Each MIML bag in our formulation is annotated only with a binary label indicating whether the bag contains the instance of a certain class and the query is specified by the word2vec of a class label/name. To learn a deep-net model for qMIL, we construct a network component that achieves a generalized compatibility measure for query-visual co-embedding and yields proper instance attentions to the given query. The bag representation is then formed as the attention-weighted sum of the instances' weights, and passed to the classification layer at the end of the network. In addition, the qMIL formulation is flexible for extending the network to classify unseen class labels, leading to a new technique to solve the zero-shot MIML task through an iterative querying process. Experimental results on action classification over video clips and three MIML datasets from MNIST, CIFAR10 and Scene are provided to demonstrate the effectiveness of our method

Altered stiffness of microchamber and macrochamber layers in the aged heel pad: Shear wave ultrasound elastography evaluation

Background/Purpose: To compare shear modulus of heel pad microchamber and macrochamber layers between young and elderly people using shear wave ultrasound elastography (SWUE), with the intent to clarify age-related changes. Methods: This single-center prospective cross-sectional study was conducted between March, 2014 and March, 2016. Shear modulus of entire heel pad (Gentire), macrochamber layer (Gmac), and microchamber layer (Gmic) were measured with SWUE. Results: Elderly participants (15 men, 15 women; age = 66.9 ± 6.2 years) had significantly higher Gmic (103.8 ± 20.7 vs. 60.1 ± 9.8 kPa; p < 0.001) and Gentire (39.4 ± 10.5 vs. 34.1 ± 5.4 kPa; p = 0.005), but a significantly lower Gmac (21.7 ± 7.5 vs. 27.9 ± 4.9 kPa; p < 0.001) compared with those of young participants (15 men, 15 women; age = 26.4 ± 2.9 years). Positive correlations were observed between age and Gmic (r = 0.79, p < 0.001) and between age and Gentire (r = 0.28, p = 0.03), and negative correlation between age and Gmac (r = −0.46, p = 0.001). Conclusion: SWUE revealed that the heel pad macrochamber layer was slightly softer but the microchamber layer was exaggeratedly stiffer, making the entire heel pad stiffer in the elderly group than in the younger group, implying age-related compensation in heel pad layers to retain foot function. Keywords: Age, Elastography, Heel pad, Ultrasoun

E-Cigarette Use in a Country With Prevalent Tobacco Smoking: A Population-Based Study in Taiwan

Background: The different profiles of e-cigarette users in different age groups have seldom been investigated, particularly in populations facing a high prevalence of cigarette smoking. This study aims to examine the prevalence and correlates of e-cigarette use separately for adolescents and adults in nationally representative samples in Taiwan. Methods: Among 17,837 participants in the 2014 National Survey of Substance Use in Taiwan, 4445 were aged 12 to 17 years and 13,392 were aged 18 to 64 years. Individuals’ lifetime tobacco use was divided into four groups: non-use, exclusive e-cigarette use, exclusive cigarette use, and dual use. Questions on sociodemographic features, use and problematic use of tobacco, alcohol, and other drugs, and psychosocial distress, among others, were administered using a computer-assisted self-interview on tablet computers. Results: Among lifetime users of e-cigarette (2.2% for adults and 0.8% for adolescents), 4.5% for adults and 36.6% for adolescents were exclusive e-cigarette users. From use of exclusive e-cigarettes to use of exclusive cigarettes to dual use, those usage groups were related to an increasing trend of adjusted odds ratios for use of other psychoactive substances, particularly problematic use of alcohol or drugs, and with more depressive symptoms. Two correlates were specific to e-cigarette use: alcohol use had stronger relationships with e-cigarette use among adolescents, and younger adults (18–34) were more likely to try e-cigarettes compared to older adults. Conclusions: These results provide essential information regarding e-cigarette use in the general population, and future prevention strategies should account for its specific correlates in young people

Concordance of SVR12, SVR24 and SVR durability in Taiwanese chronic hepatitis C patients with direct-acting antivirals.

Background/aimsUndetectable HCV RNA 12 weeks after the end of treatment (SVR12) has been the valid efficacy endpoint in the era of direct-acting antivirals (DAAs). Its concordance with SVR4 and SVR24 and long-term durability is unknown in Taiwanese chronic hepatitis C (CHC) patients.MethodsA total of 1080 CHC patients who received all-oral DAAs and an achieved end-of-treatment virological response (EOTVR), defined as undetectable HCV RNA at the end of therapy, were consecutively enrolled. HCV RNA was monitored 4, 12, and 24 weeks after EOT. Patients who achieved SVR24, defined as undetectable HCV RNA 24 weeks after EOT, were followed annually for assessing SVR durability.ResultsEleven (1.02%) patients experienced HCV RNA reappearance after EOT. The most frequent timing of RNA reappearance was observed at SVR4 (n = 7), followed by SVR12 (n = 3) and SVR 24 (n = 1). The positive predictive value (PPV) and negative predictive value (NPV) of SVR4 in predicting SVR12 were 99.7% and 100%, respectively, whereas the PPV and NPV of SVR12 in predicting SVR24 were 99.9% and 100%, respectively. Pyrosequencing confirmed delayed relapse rather than reinfection for the patient who had detectable HCV RNA at SVR24. Among 978 patients who achieved SVR24, after a median follow-up period of 17.3±8.2 months, the SVR durability is 100% up to a 4-year follow-up.ConclusionAchievement of SVR12 provides excellent durability of HCV seroclearance after DAA therapy. On-demand HCV RNA beyond SVR12 should be recommended for patients with unexplainable abnormal liver function or high-risk behaviors