LidarCLIP or: How I Learned to Talk to Point Clouds

Research connecting text and images has recently seen several breakthroughs, with models like CLIP, DALL-E 2, and Stable Diffusion. However, the connection between text and other visual modalities, such as lidar data, has received less attention, prohibited by the lack of text-lidar datasets. In this work, we propose LidarCLIP, a mapping from automotive point clouds to a pre-existing CLIP embedding space. Using image-lidar pairs, we supervise a point cloud encoder with the image CLIP embeddings, effectively relating text and lidar data with the image domain as an intermediary. We show the effectiveness of LidarCLIP by demonstrating that lidar-based retrieval is generally on par with image-based retrieval, but with complementary strengths and weaknesses. By combining image and lidar features, we improve upon both single-modality methods and enable a targeted search for challenging detection scenarios under adverse sensor conditions. We also explore zero-shot classification and show that LidarCLIP outperforms existing attempts to use CLIP for point clouds by a large margin. Finally, we leverage our compatibility with CLIP to explore a range of applications, such as point cloud captioning and lidar-to-image generation, without any additional training. Code and pre-trained models are available at https://github.com/atonderski/lidarclip

Farming systems education: Case study of Swedish test pilots

We describe and analyze a pedagogical experiment that introduced a broad and holistic perspective on complete farming systems, systemic learning tools, and a participatory learning strategy at an early stage in agronomy education. The paper describes the adventure of three students, who came from a conventional agronomy program at the Swedish University of Agricultural Sciences (SLU), who were frustrated with the lack of integrated approaches to the study of agricultural systems and a strong focus on molecular-level processes in their first year of education. They encountered a narrow focus in most courses and the overall curricula of agricultural education that is a function of specialization and university organization in unique departments that concentrate on small pieces of the large puzzle, that is the production milieu. In the current educational environment, it is difficult for students to make connections, integrate information and theories, and to create relevance to the challenges they observe in the practical world of farming and food systems. The three students agreed to put on pilots’ costumes and climb into an experimental vehicle called experiential learning, one that provides just-in-time education and a very high degree of self-responsibility for the learning process. The paper describes, analyzes, and evaluates the comprehensive and exhausting pedagogical process they followed in one semester in Sweden and Viet Nam, with positive and negative aspects of the program. We provide reflective recommendations from students and advisors for future agronomic education programs with the focus on developing renewable agriculture, selecting students and evaluating performance, and designing practical programs that will motivate highly committed and action-oriented students

You can have your ensemble and run it too -- Deep Ensembles Spread Over Time

Ensembles of independently trained deep neural networks yield uncertainty estimates that rival Bayesian networks in performance. They also offer sizable improvements in terms of predictive performance over single models. However, deep ensembles are not commonly used in environments with limited computational budget -- such as autonomous driving -- since the complexity grows linearly with the number of ensemble members. An important observation that can be made for robotics applications, such as autonomous driving, is that data is typically sequential. For instance, when an object is to be recognized, an autonomous vehicle typically observes a sequence of images, rather than a single image. This raises the question, could the deep ensemble be spread over time? In this work, we propose and analyze Deep Ensembles Spread Over Time (DESOT). The idea is to apply only a single ensemble member to each data point in the sequence, and fuse the predictions over a sequence of data points. We implement and experiment with DESOT for traffic sign classification, where sequences of tracked image patches are to be classified. We find that DESOT obtains the benefits of deep ensembles, in terms of predictive and uncertainty estimation performance, while avoiding the added computational cost. Moreover, DESOT is simple to implement and does not require sequences during training. Finally, we find that DESOT, like deep ensembles, outperform single models for out-of-distribution detection

A fast and verified software stack for secure function evaluation

We present a high-assurance software stack for secure function evaluation (SFE). Our stack consists of three components: i. a verified compiler (CircGen) that translates C programs into Boolean circuits; ii. a verified implementation of Yao’s SFE protocol based on garbled circuits and oblivious transfer; and iii. transparent application integration and communications via FRESCO, an open-source framework for secure multiparty computation (MPC). CircGen is a general purpose tool that builds on CompCert, a verified optimizing compiler for C. It can be used in arbitrary Boolean circuit-based cryptography deployments. The security of our SFE protocol implementation is formally verified using EasyCrypt, a tool-assisted framework for building high-confidence cryptographic proofs, and it leverages a new formalization of garbled circuits based on the framework of Bellare, Hoang, and Rogaway (CCS 2012). We conduct a practical evaluation of our approach, and conclude that it is competitive with state-of-the-art (unverified) approaches. Our work provides concrete evidence of the feasibility of building efficient, verified, implementations of higher-level cryptographic systems. All our development is publicly available.POCI-01-0145-FEDER-006961, FCT-PD/BD/113967/2015info:eu-repo/semantics/publishedVersio

Fundamental characterization, photophysics and photocatalysis of a base metal iron(II)-cobalt(III) dyad

A new base metal iron-cobalt dyad has been obtained by connection between a heteroleptic tetra-NHC iron(II) photosensitizer combining a 2,6-bis[3-(2,6-diisopropylphenyl)imidazol-2-ylidene]pyridine with 2,6-bis(3-methyl-imidazol-2-ylidene)-4,4′-bipyridine ligand, and a cobaloxime catalyst. This novel iron(II)-cobalt(III) assembly has been extensively characterized by ground- and excited-state methods like X-ray crystallography, X-ray absorption spectroscopy, (spectro-)electrochemistry, and steady-state and time-resolved optical absorption spectroscopy, with a particular focus on the stability of the molecular assembly in solution and determination of the excited-state landscape. NMR and UV/Vis spectroscopy reveal dissociation of the dyad in acetonitrile at concentrations below 1 mM and high photostability. Transient absorption spectroscopy after excitation into the metal-to-ligand charge transfer absorption band suggests a relaxation cascade originating from hot singlet and triplet MLCT states, leading to the population of the $^{3}$MLCT state that exhibits the longest lifetime. Finally, decay into the ground state involves a $^{3}$MC state. Attachment of cobaloxime to the iron photosensitizer increases the $^{3}$MLCT lifetime at the iron centre. Together with the directing effect of the linker, this potentially makes the dyad more active in photocatalytic proton reduction experiments than the analogous two-component system, consisting of the iron photosensitizer and Co(dmgH)$_2$(py)Cl. This work thus sheds new light on the functionality of base metal dyads, which are important for more efficient and sustainable future proton reduction systems

Low Detection Rates of Genetic FH in Cohort of Patients With Severe Hypercholesterolemia in the United Arabic Emirates

Background: Programs to screen for Familial hypercholesterolemia (FH) are conducted worldwide. In Western societies, these programs have been shown to be cost-effective with hit/detection rates of 1 in 217–250. Thus far, there is no published data on genetic FH in the Gulf region. Using United Arab Emirates as a proxy for the Gulf region, we assessed the prevalence of genetically confirmed FH in the Emirati population sample. Materials and Methods: We recruited 229 patients with LDL-C >95(th) percentile and employed a customized next generation sequencing pipeline to screen canonical FH genes (LDLR, APOB, PCSK9, LDLRAP1). Results: Participants were characterized by mean total cholesterol and low-density lipoprotein cholesterol (LDL-c) of 6.3 ± 1.1 and 4.7 ± 1.1 mmol/L respectively. Ninety-six percent of the participants were using lipid-lowering medication with mean corrected LDL-c values of 10.0 ± 3.0 mmol/L 15 out of 229 participants were found to suffer from genetically confirmed FH. Carriers of causal genetic variants for FH had higher on-treatment LDL-c compared to those without causal variants (5.7 ± 1.5 vs 4.7 ± 1.0; p = 3.7E-04). The groups did not differ regarding high-density lipoprotein cholesterol, triglycerides, body mass index, blood pressure, glucose, and glycated haemoglobin. Conclusion: This study reveals a low 7% prevalence of genetic FH in Emiratis with marked hypercholesterolemia as determined by correcting LDL-c for the use of lipid-lowering treatment. The portfolio of mutations identified is, to a large extent, unique and includes gene duplications. Our findings warrant further studies into origins of hypercholesterolemia in these patients. This is further supported by the fact that these patients are also characterized by high prevalence of type 2 diabetes (42% in the current study cohort) which already puts them at an increased risk of atherosclerotic cardiovascular disease. These results may also be useful in public health initiatives for FH cascade screening programs in the UAE and maybe the Gulf region

Self-supervised machine learning to characterise step counts from wrist-worn accelerometers in the UK Biobank

Purpose: Step count is an intuitive measure of physical activity frequently quantified in health-related studies; however, accurate step counting is difficult in the free-living environment, with error routinely above 20% in wrist-worn devices against camera-annotated ground truth. This study aims to describe the development and validation of step count derived from a wrist-worn accelerometer and assess its association with cardiovascular and all-cause mortality in a large prospective cohort. Methods: We developed and externally validated a self-supervised machine learning step detection model, trained on an open-source and step-annotated free-living dataset. 39 individuals will free-living ground-truth annotated step counts were used for model development. An open-source dataset with 30 individuals was used for external validation. Epidemiological analysis was performed using 75,263 UK Biobank participants without prevalent cardiovascular disease (CVD) or cancer. Cox regression was used to test the association of daily step count with fatal CVD and all-cause mortality after adjustment for potential confounders. Results: The algorithm substantially outperformed reference models (free-living mean absolute percent error of 12.5%, versus 65-231%). Our data indicate an inverse dose-response association, where taking 6,430-8,277 daily steps was associated with 37% [25-48%] and 28% [20-35%] lower risk of fatal CVD and all-cause mortality up to seven years later, compared to those taking fewer steps each day. Conclusions: We have developed an open and transparent method that markedly improves the measurement of steps in large-scale wrist-worn accelerometer datasets. The application of this method demonstrated expected associations with CVD and all-cause mortality, indicating excellent face validity. This reinforces public health messaging for increasing physical activity and can help lay the groundwork for the inclusion of target step counts in future public health guidelines

Alkyne derivatives of SARS-CoV-2 main protease inhibitors including nirmatrelvir inhibit by reacting covalently with the nucleophilic cysteine

Nirmatrelvir (PF-07321332) is a nitrile-bearing small-molecule inhibitor that, in combination with ritonavir, is used to treat infections by severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2). Nirmatrelvir interrupts the viral life cycle by inhibiting the SARS-CoV-2 main protease (Mpro), which is essential for processing viral polyproteins into functional nonstructural proteins. We report studies which reveal that derivatives of nirmatrelvir and other Mpro inhibitors with a nonactivated terminal alkyne group positioned similarly to the electrophilic nitrile of nirmatrelvir can efficiently inhibit isolated Mpro and SARS-CoV-2 replication in cells. Mass spectrometric and crystallographic evidence shows that the alkyne derivatives inhibit Mpro by apparent irreversible covalent reactions with the active site cysteine (Cys145), while the analogous nitriles react reversibly. The results highlight the potential for irreversible covalent inhibition of Mpro and other nucleophilic cysteine proteases by alkynes, which, in contrast to nitriles, can be functionalized at their terminal position to optimize inhibition and selectivity, as well as pharmacodynamic and pharmacokinetic properties

Machine-checked proofs for cryptographic standards indifferentiability of SPONGE and secure high-assurance implementations of SHA-3

We present a high-assurance and high-speed implementation of the SHA-3 hash function. Our implementation is written in the Jasmin programming language, and is formally verified for functional correctness, provable security and timing attack resistance in the EasyCrypt proof assistant. Our implementation is the first to achieve simultaneously the four desirable properties (efficiency, correctness, provable security, and side-channel protection) for a non-trivial cryptographic primitive.Concretely, our mechanized proofs show that: 1) the SHA-3 hash function is indifferentiable from a random oracle, and thus is resistant against collision, first and second preimage attacks; 2) the SHA-3 hash function is correctly implemented by a vectorized x86 implementation. Furthermore, the implementation is provably protected against timing attacks in an idealized model of timing leaks. The proofs include new EasyCrypt libraries of independent interest for programmable random oracles and modular indifferentiability proofs.This work received support from the National Institute of Standards and Technologies under agreement number 60NANB15D248.This work was partially supported by Office of Naval Research under projects N00014-12-1-0914, N00014-15-1-2750 and N00014-19-1-2292.This work was partially funded by national funds via the Portuguese Foundation for Science and Technology (FCT) in the context of project PTDC/CCI-INF/31698/2017. Manuel Barbosa was supported by grant SFRH/BSAB/143018/2018 awarded by the FCT.This work was supported in part by the National Science Foundation under grant number 1801564.This work was supported in part by the FutureTPM project of the Horizon 2020 Framework Programme of the European Union, under GA number 779391.This work was supported by the ANR Scrypt project, grant number ANR-18-CE25-0014.This work was supported by the ANR TECAP project, grant number ANR-17-CE39-0004-01