65 research outputs found
Classification of integers based on residue classes via modern deep learning algorithms
Judging whether an integer can be divided by prime numbers such as 2 or 3 may
appear trivial to human beings, but can be less straightforward for computers.
Here, we tested multiple deep learning architectures and feature engineering
approaches on classifying integers based on their residues when divided by
small prime numbers. We found that the ability of classification critically
depends on the feature space. We also evaluated Automated Machine Learning
(AutoML) platforms from Amazon, Google and Microsoft, and found that they
failed on this task without appropriately engineered features. Furthermore, we
introduced a method that utilizes linear regression on Fourier series basis
vectors, and demonstrated its effectiveness. Finally, we evaluated Large
Language Models (LLMs) such as GPT-4, GPT-J, LLaMA and Falcon, and demonstrated
their failures. In conclusion, feature engineering remains an important task to
improve performance and increase interpretability of machine-learning models,
even in the era of AutoML and LLMs.Comment: Accepted at Pattern
Enhancing Phenotype Recognition in Clinical Notes Using Large Language Models: PhenoBCBERT and PhenoGPT
We hypothesize that large language models (LLMs) based on the transformer
architecture can enable automated detection of clinical phenotype terms,
including terms not documented in the HPO. In this study, we developed two
types of models: PhenoBCBERT, a BERT-based model, utilizing Bio+Clinical BERT
as its pre-trained model, and PhenoGPT, a GPT-based model that can be
initialized from diverse GPT models, including open-source versions such as
GPT-J, Falcon, and LLaMA, as well as closed-source versions such as GPT-3 and
GPT-3.5. We compared our methods with PhenoTagger, a recently developed HPO
recognition tool that combines rule-based and deep learning methods. We found
that our methods can extract more phenotype concepts, including novel ones not
characterized by HPO. We also performed case studies on biomedical literature
to illustrate how new phenotype information can be recognized and extracted. We
compared current BERT-based versus GPT-based models for phenotype tagging, in
multiple aspects including model architecture, memory usage, speed, accuracy,
and privacy protection. We also discussed the addition of a negation step and
an HPO normalization layer to the transformer models for improved HPO term
tagging. In conclusion, PhenoBCBERT and PhenoGPT enable the automated discovery
of phenotype terms from clinical notes and biomedical literature, facilitating
automated downstream tasks to derive new biological insights on human diseases
Recommended from our members
Contrasting size-resolved hygroscopicity of fine particles derived by HTDMA and HR-ToF-AMS measurements between summer and winter in Beijing: the impacts of aerosol aging and local emissions
LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model
Universally modeling all typical information extraction tasks (UIE) with one
generative language model (GLM) has revealed great potential by the latest
study, where various IE predictions are unified into a linearized hierarchical
expression under a GLM. Syntactic structure information, a type of effective
feature which has been extensively utilized in IE community, should also be
beneficial to UIE. In this work, we propose a novel structure-aware GLM, fully
unleashing the power of syntactic knowledge for UIE. A heterogeneous structure
inductor is explored to unsupervisedly induce rich heterogeneous structural
representations by post-training an existing GLM. In particular, a structural
broadcaster is devised to compact various latent trees into explicit high-order
forests, helping to guide a better generation during decoding. We finally
introduce a task-oriented structure fine-tuning mechanism, further adjusting
the learned structures to most coincide with the end-task's need. Over 12 IE
benchmarks across 7 tasks our system shows significant improvements over the
baseline UIE system. Further in-depth analyses show that our GLM learns rich
task-adaptive structural bias that greatly resolves the UIE crux, the
long-range dependence issue and boundary identifying. Source codes are open at
https://github.com/ChocoWu/LasUIE.Comment: NeurIPS2022 conference pape
Proteomics analysis of rough endoplasmic reticulum in pancreatic beta cells
Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/111127/1/pmic8033.pd
Ultra-Low-Frequency Radio Astronomy Observations from a Selenocentric Orbit: first results of the Longjiang-2 experiment
This paper introduces the first results of observations with the
Ultra-Long-Wavelength (ULW) -- Low Frequency Interferometer and Spectrometer
(LFIS) on board the selenocentric satellite Longjiang-2. We present a brief
description of the satellite and focus on the LFIS payload. The in-orbit
commissioning confirmed a reliable operational status of the instrumentation.
We also present results of a transition observation, which offers unique
measurements on several novel aspects. We estimate the RFI suppression required
for such a radio astronomy instrumentation at the Moon distances from Earth to
be of the order of 80 dB. We analyse a method of separating Earth- and
satellite-originated radio frequency interference (RFI). It is found that the
RFI level at frequencies lower than a few MHz is smaller than the receiver
noise floor.Comment: Accepted for publication in Experimental Astronomy; 22 pages, 11
figure
Kosmos-2.5: A Multimodal Literate Model
We present Kosmos-2.5, a multimodal literate model for machine reading of
text-intensive images. Pre-trained on large-scale text-intensive images,
Kosmos-2.5 excels in two distinct yet cooperative transcription tasks: (1)
generating spatially-aware text blocks, where each block of text is assigned
its spatial coordinates within the image, and (2) producing structured text
output that captures styles and structures into the markdown format. This
unified multimodal literate capability is achieved through a shared Transformer
architecture, task-specific prompts, and flexible text representations. We
evaluate Kosmos-2.5 on end-to-end document-level text recognition and
image-to-markdown text generation. Furthermore, the model can be readily
adapted for any text-intensive image understanding task with different prompts
through supervised fine-tuning, making it a general-purpose tool for real-world
applications involving text-rich images. This work also paves the way for the
future scaling of multimodal large language models
- âŠ