Improving Medical Dialogue Generation with Abstract Meaning Representations

Medical Dialogue Generation serves a critical role in telemedicine by facilitating the dissemination of medical expertise to patients. Existing studies focus on incorporating textual representations, which have limited their ability to represent the semantics of text, such as ignoring important medical entities. To enhance the model's understanding of the textual semantics and the medical knowledge including entities and relations, we introduce the use of Abstract Meaning Representations (AMR) to construct graphical representations that delineate the roles of language constituents and medical entities within the dialogues. In this paper, We propose a novel framework that models dialogues between patients and healthcare professionals using AMR graphs, where the neural networks incorporate textual and graphical knowledge with a dual attention mechanism. Experimental results show that our framework outperforms strong baseline models in medical dialogue generation, demonstrating the effectiveness of AMR graphs in enhancing the representations of medical knowledge and logical relationships. Furthermore, to support future research in this domain, we provide the corresponding source code at https://github.com/Bernard-Yang/MedDiaAMR

Improving Medical Dialogue Generation with Abstract Meaning Representations

Medical Dialogue Generation serves a critical role in telemedicine by facilitating the dissemination of medical expertise to patients. Existing studies focus on incorporating textual representations, which have limited their ability to represent the semantics of text, such as ignoring important medical entities. To enhance the model's understanding of the textual semantics and the medical knowledge including entities and relations, we introduce the use of Abstract Meaning Representations (AMR) to construct graphical representations that delineate the roles of language constituents and medical entities within the dialogues. In this paper, We propose a novel framework that models dialogues between patients and healthcare professionals using AMR graphs, where the neural networks incorporate textual and graphical knowledge with a dual attention mechanism. Experimental results show that our framework outperforms strong baseline models in medical dialogue generation, demonstrating the effectiveness of AMR graphs in enhancing the representations of medical knowledge and logical relationships. Furthermore, to support future research in this domain, we provide the corresponding source code at https://github.com/Bernard-Yang/MedDiaAMR.Comment: Submitted to ICASSP 202

Effective Distillation of Table-based Reasoning Ability from LLMs

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, their remarkable parameter size and their impressive high requirement of computing resources pose challenges for their practical deployment. Recent research has revealed that specific capabilities of LLMs, such as numerical reasoning, can be transferred to smaller models through distillation. Some studies explore the potential of leveraging LLMs to perform table-based reasoning. Nevertheless, prior to our work, there has been no investigation into the prospect of specialising table reasoning skills in smaller models specifically tailored for table-to-text generation tasks. In this paper, we propose a novel table-based reasoning distillation, with the aim of distilling distilling LLMs into tailored, smaller models specifically designed for table-based reasoning task. Experimental results have shown that a 0.22 billion parameter model (Flan-T5-base) fine-tuned using distilled data, not only achieves a significant improvement compared to traditionally fine-tuned baselines but also surpasses specific LLMs like gpt-3.5-turbo on the scientific table-to-text generation dataset (SciGen). The code and data are released in https://github.com/Bernard-Yang/TableDistill

Identification and profiling of microRNA between back and belly Skin in Rex rabbits (Oryctolagus cuniculus)

[EN] Skin is an important trait for Rex rabbits and skin development is influenced by many processes, including hair follicle cycling, keratinocyte differentiation and formation of coat colour and skin morphogenesis. We identified differentially expressed microRNAs (miRNAs) between the back and belly skin in Rex rabbits. In total, 211 miRNAs (90 upregulated miRNAs and 121 downregulated miRNAs) were identified with a |log2 (fold change)|>1 and P-value<0.05. Using target gene prediction for the miRNAs, differentially expressed predicted target genes were identified and the functional enrichment and signalling pathways of these target genes were processed to reveal their biological functions. A number of differentially expressed miRNAs were found to be involved in regulation of the cell cycle, skin epithelium differentiation, keratinocyte proliferation, hair follicle development and melanogenesis. In addition, target genes regulated by miRNAs play key roles in the activities of the Hedgehog signalling pathway, Wnt signalling pathway, Osteoclast differentiation and MAPK pathway, revealing mechanisms of skin development. Nine candidate miRNAs and 5 predicted target genes were selected for verification of their expression by quantitative reverse transcription polymerase chain reaction. A regulation network of miRNA and their target genes was constructed by analysing the GO enrichment and signalling pathways. Further studies should be carried out to validate the regulatory relationships between candidate miRNAs and their target genes.This study was supported by the Modern Agricultural Industrial System Special Funding (CARS-44-A-1), the Priority Academic Programme Development of Jiangsu Higher Education Institutions (2014-134) and the General Programme of Natural Science Foundation of the Higher Education Institutions of Jiangsu Province (16KJB230001).Zhao, B.; Chen, Y.; Mu, L.; Hu, S.; Wu, X. (2018). Identification and profiling of microRNA between back and belly Skin in Rex rabbits (Oryctolagus cuniculus). World Rabbit Science. 26(2):179-190. https://doi.org/10.4995/wrs.2018.7058SWORD179190262Adamidi C. 2008. Discovering microRNAs from deep sequencing data using miRDeep. Nature Biotechnol., 26: 407-415. https://doi.org/10.1038/nbt1394Adijanto J., Castorino J.J., Wang Z.X., Maminishkis A., Grunwald G.B., Philp N.J. 2012. Microphthalmia-associated transcription factor (MITF) promotes differentiation of human retinal pigment epithelium (RPE) by regulating microRNAs-204/211 expression. J. Biol. Chem., 287: 20491-https://doi.org/10.1074/jbc.M112.354761Ahmed M.I., Alam M., Emelianov V.U., Poterlowicz K., Patel A., Sharov A.A., Mardaryev A.N., Botchkareva N.V. 2014. MicroRNA-214 controls skin and hair follicle development by modulating the activity of the Wnt pathway. J. Cell Biol., 207: 549-567. https://doi.org/10.1083/jcb.201404001Alexander M., Kawahara G., Motohashi N., Casar J., Eisenberg I., Myers J., Gasperini M., Estrella E., Kho A., Mitsuhashi S. 2013. MicroRNA-199a is induced in dystrophic muscle and affects WNT signaling, cell proliferation, and myogenic differentiation. Cell Death Diff., 20: 1194-1208. https://doi.org/10.1038/cdd.2013.62Anders S. 2010. Analysing RNA-Seq data with the DESeq package. Mol. Biol., 43: 1-17.Andl T., Botchkareva N.V. 2015. MicroRNAs (miRNAs) in the control of HF development and cycling: the next frontiers in hair research. Exp. Dermatol., 24: 821-826. https://doi.org/10.1111/exd.12785Andl T., Reddy S.T., Gaddapara T., Millar S.E. 2002. WNT signals are required for the initiation of hair follicle development. Develop. Cell, 2: 643-653. https://doi.org/10.1016/S1534-5807(02)00167-3Antonini D., Russo MT., De Rosa L., Gorrese M., Del Vecchio L., Missero C. 2010. Transcriptional repression of miR-34 family contributes to p63-mediated cell cycle progression in epidermal cells. J. Invest. Dermatol., 130: 1249-1257. https://doi.org/10.1038/jid.2009.438Athar M., Tang X., Lee J.L., Kopelovich L., Kim AL. 2006. Hedgehog signalling in skin development and cancer. Exp. Dermatol., 15: 667-677. https://doi.org/10.1111/j.1600-0625.2006.00473.xBartel D.P. 2004. MicroRNAs: genomics, biogenesis, mechanism, and function. Cell, 116: 281-297.https://doi.org/10.1016/S0092-8674(04)00045-5Bashirullah A., Pasquinelli A.E., Kiger A.A., Perrimon N., Ruvkun G., Thummel C.S. 2003. Coordinate regulation of small temporal RNAs at the onset of Drosophila metamorphosis. Dev. Biol., 259: 1-8. https://doi.org/10.1016/S0012-1606(03)00063-0Bommer GT., Gerin I., Feng Y., Kaczorowski AJ., Kuick R., Love RE., Zhai Y., Giordano TJ., Qin ZS., Moore BB. 2007. p53-mediated activation of miRNA34 candidate tumor-suppressor genes. Curr. Biol., 17: 1298-1307. https://doi.org/10.1016/j.cub.2007.06.068Braun C.J., Zhang X., Savelyeva I., Wolff S., Moll U.M., Schepeler T., Ørntoft T.F., Andersen C.L., Dobbelstein M. 2008. p53-Responsive micrornas 192 and 215 are capable of inducing cell cycle arrest. Cancer Res., 68: 10094-10104.https://doi.org/10.1158/0008-5472.CAN-08-1569Callis T.E., Chen J.F., Wang D.Z. 2007. MicroRNAs in skeletal and cardiac muscle development. Dna Cell Biol., 26: 219-225. https://doi.org/10.1089/dna.2006.0556Caramuta S., Egyházi S., Rodolfo M., Witten D., Hansson J., Larsson C., Lui W.O. 2010. MicroRNA expression profiles associated with mutational status and survival in malignant melanoma. J. Invest. Dermatol., 130: 2062-2070. https://doi.org/10.1038/jid.2010.63Chen C.H., Sakai Y., Demay M.B. 2001. Targeting expression of the human vitamin D receptor to the keratinocytes of vitamin D receptor null mice prevents alopecia. Endocrinology, 142: 5386-5386. https://doi.org/10.1210/endo.142.12.8650D'Juan T.F., Shariat N., Park C.Y., Liu H.J., Mavropoulos A., McManus M.T. 2013. Partially penetrant postnatal lethality of an epithelial specific MicroRNA in a mouse knockout. Plos One 8: e76634. https://doi.org/10.1371/journal.pone.0076634DeYoung M.P., Johannessen C.M., Leong C.O., Faquin W., Rocco J.W., Ellisen L.W. 2006. Tumor-specific p73 up-regulation mediates p63 dependence in squamous cell carcinoma. Cancer Res., 66: 9362-9368. https://doi.org/10.1158/0008-5472.CAN-06-1619Eckert R.L., Welter J.F. 1996. Transcription factor regulation of epidermal keratinocyte gene expression. Mol. Biol. Rep., 23: 59-70. https://doi.org/10.1007/BF00357073Enright A.J., Bino J., Ulrike G., Thomas T., Chris S., Marks D.S. 2004. MicroRNA targets in Drosophila. Gen. Biol., 5: R1-R1. https://doi.org/10.1186/gb-2003-5-1-r1Fontanesi L., Scotti E., Allain D., Dall'Olio S. 2014. A frameshift mutation in the melanophilin gene causes the dilute coat colour in rabbit (Oryctolagus cuniculus) breeds. Anim. Genet., 45: 248-255. https://doi.org/10.1111/age.12104Fontanesi L., Vargiolu M., Scotti E., Latorre R., Pellegrini M.S.F., Mazzoni M., Asti M., Chiocchetti R., Romeo G., Clavenzani P. 2014. The KIT gene is associated with the English spotting coat color locus and congenital megacolon in Checkered Giant rabbits (Oryctolagus cuniculus). Plos One 9: e93750. https://doi.org/10.1371/journal.pone.0093750Fuchs E. 2007. Scratching the surface of skin development. Nature, 445: 834-842. https://doi.org/10.1038/nature05659Georges S.A., Chau B.N., Braun C.J., Zhang X., Dobbelstein M. 2009. Cell cycle arrest or apoptosis by p53: are microRNAs-192/215 and-34 making the decision? Cell Cycle 8: 677-682. https://doi.org/10.4161/cc.8.5.8076Jackson S.J., Zhang Z., Feng D., Flagg M., O'Loughlin E., Wang D., Stokes N., Fuchs E., Yi R. 2013. Rapid and widespread suppression of self-renewal by microRNA-203 during epidermal differentiation. Development, 140: 1882-1891. https://doi.org/10.1242/dev.089649Katoh Y., Katoh M. 2008. Hedgehog signaling, epithelial-tomesenchymal transition and miRNA (review). Int. J. Mol. Med., 22: 271-275. https://doi.org/10.3892/ijmm_00000019Kim K., Vinayagam A., Perrimon N. 2014. A rapid genomewide microRNA screen identifies miR-14 as a modulator of Hedgehog signaling. Cell Rep., 7: 2066-2077. https://doi.org/10.1016/j.celrep.2014.05.025Kochegarov A., Moses A., Lian W., Meyer J., Hanna M.C., Lemanski L.F. 2013. A new unique form of microRNA from human heart, microRNA-499c, promotes myofibril formation and rescues cardiac development in mutant axolotl embryos. J. Biomed. Sci., 20: 1. https://doi.org/10.1186/1423-0127-20-20Kozomara, A., Griffiths J. 2014. miRBase: annotating high confidence microRNAs using deep sequencing data. Nucleic Acids Res., 42: 68-73. https://doi.org/10.1093/nar/gkt1181Kureel J., Dixit M., Tyagi A., Mansoori M., Srivastava K., Raghuvanshi A., Maurya R., Trivedi R., Goel A., Singh D. 2014. miR-542-3p suppresses osteoblast cell proliferation and differentiation, targets BMP-7 signaling and inhibits bone formation. Cell Death Dis., 5: e1050. https://doi.org/10.1038/cddis.2014.4Langmead B., Salzberg S.L. 2012. Fast gapped-read alignment with Bowtie 2. Nat. Methods, 9: 357-359. https://doi.org/10.1038/nmeth.1923Lim X., Nusse R. 2013. Wnt signaling in skin development, homeostasis, and disease. CSH Perspect. Biol., 5: a008029. https://doi.org/10.1101/cshperspect.a008029Liu Z., Xiao H., Li H., Zhao Y., Lai S., Yu X., Cai T., Du C., Zhang W., Li J. 2012. Identification of conserved and novel microRNAs in cashmere goat skin by deep sequencing. Plos One 7: e50001. https://doi.org/10.1371/journal.pone.0050001Mardaryev A.N., Ahmed M.I., Vlahov N.V., Fessing M.Y., Gill J.H., Sharov A.A., Botchkareva N.V. 2010. Micro-RNA-31 controls hair cycle-associated changes in gene expression programs of the skin and hair follicle. FASEB J. 24: 3869-3881. https://doi.org/10.1096/fj.10-160663Mills A.A., Zheng B., Wang X.J., Vogel H., Roop D.R., Bradley A. 1999. p63 is a p53 homologue required for limb and epidermal morphogenesis. Nature, 398: 708-713. https://doi.org/10.1038/19531Mueller D.W., Rehli M., Bosserhoff A.K. 2009. miRNA expression profiling in melanocytes and melanoma cell lines reveals miRNAs associated with formation and progression of malignant melanoma. J. Invest. Dermatol., 129: 1740-1751. https://doi.org/10.1038/jid.2008.452Naeem H., Küffner R., Csaba G., Zimmer R. 2010. miRSel: Automated extraction of associations between microRNAs and genes from the biomedical literature. Bmc Bioinformatics, 11: 135. https://doi.org/10.1186/1471-2105-11-135Neilson J.R., Zheng G.X., Burge CB., Sharp P.A. 2007. Dynamic regulation of miRNA expression in ordered stages of cellular development. Gene. Dev., 21: 578-589. https://doi.org/10.1101/gad.1522907Oda Y., Ishikawa M.H., Hawker N.P., Yun Q.C., Bikle D.D. 2007. Differential role of two VDR coactivators, DRIP205 and SRC-3, in keratinocyte proliferation and differentiation. J. Steroid Biochem., 103: 776-780. https://doi.org/10.1016/j.jsbmb.2006.12.069Pan L., Liu Y., Wei Q., Xiao C., Ji Q., Bao G., Wu X. 2015. Solexa-Sequencing Based Transcriptome Study of Plaice Skin Phenotype in Rex Rabbits (Oryctolagus cuniculus). Plos One: 10. https://doi.org/10.1371/journal.pone.0124583Rosenfield R.L., Deplewski D., Greene M.E. 2001. Peroxisome proliferator-activated receptors and skin development. Horm. Res. Paediat., 54: 269-274. https://doi.org/10.1159/000053270Schneider M.R. 2012. MicroRNAs as novel players in skin development, homeostasis and disease. Brit. J. Dermatol., 166: 22-28. https://doi.org/10.1111/j.1365-2133.2011.10568.xSenoo M., Pinto F., Crum C.P., McKeon F. 2007. p63 Is essential for the proliferative potential of stem cells in stratified epithelia. Cell, 129: 523-536. https://doi.org/10.1016/j.cell.2007.02.045Song B., Wang Y., Kudo K., Gavin E.J., Xi Y., Ju J. 2008. miR-192 Regulates dihydrofolate reductase and cellular proliferation through the p53-microRNA circuit. Clin. Cancer Res., 14: 8080-8086. https://doi.org/10.1158/1078-0432.CCR-08-1422Suh K.S., Mutoh M., Mutoh T., Li L., Ryscavage A., Crutchley J.M., Dumont R.A., Cheng C., Yuspa S.H. 2007. CLIC4 mediates and is required for Ca2+-induced keratinocyte differentiation. J. Cell Sci., 120: 2631-2640. https://doi.org/10.1242/jcs.002741Tao Y. 2010. Studies on the quality of rex rabbit fur. World Rabbit Sci., 2: 21-24. https://doi.org/10.4995/wrs.1994.213Tian X., Jiang J., Fan R., Wang H., Meng X., He X., He J., Li H., Geng J., Yu X. 2012. Identification and characterization of microRNAs in white and brown alpaca skin. BMC genomics 13: 1.https://doi.org/10.1186/1471-2164-13-555Vadlakonda L., Pasupuleti M., Pallu R. 2014. Role of PI3K-AKTmTOR and Wnt signaling pathways in transition of G1-S phase of cell cycle in cancer cells. Front. Oncol., 3: 85. https://doi.org/10.3389/fonc.2013.00085van Amerongen R., Fuerer C., Mizutani M., Nusse R. 2012. Wnt5a can both activate and repress Wnt/β-catenin signaling during mouse embryonic development. Dev. Biol., 369: 101-114. https://doi.org/10.1016/j.ydbio.2012.06.020Vousden K.H., Lane D.P. 2007. p53 in health and disease. Nat. Rev. Mol. Cell Biol., 8: 275-283. https://doi.org/10.1038/nrm2147Wang P., Li Y., Hong W., Zhen J., Ren J., Li Z., Xu A. 2012. The changes of microRNA expression profiles and tyrosinase related proteins in MITF knocked down melanocytes. Mol. BioSyst., 8: 2924-2931. https://doi.org/10.1039/c2mb25228gWhelan J.T., Hollis S.E., Cha D.S., Asch A.S., Lee M.H. 2012. Post‐transcriptional regulation of the Ras‐ERK/MAPK signaling pathway. J. Cell Physiol., 227: 1235-1241. https://doi.org/10.1002/jcp.22899Xia H., Ooi L.L.P.J., Hui K.M. 2013. MicroRNA-216a/217-induced epithelial-mesenchymal transition targets PTEN and SMAD7 to promote drug resistance and recurrence of liver cancer. Hepatology, 58: 629-641. https://doi.org/10.1002/hep.26369Yang A., Schweitzer R., Sun D., Kaghad M., Walker N., Bronson R.T., Tabin C., Sharpe A., Caput D., Crum C. 1999. p63 is essential for regenerative proliferation in limb, craniofacial and epithelial development. Nature, 398: 714-718. https://doi.org/10.1038/19539Yu J., Peng H., Ruan Q., Fatima A., Getsios S., Lavker R.M. 2010. MicroRNA-205 promotes keratinocyte migration via the lipid phosphatase SHIP2. FASEB J. 24: 3950-3959. https://doi.org/10.1096/fj.10-157404Yu J., Ryan D.G., Getsios S., Oliveira-Fernandes M., Fatima A., Lavker R.M. 2008. MicroRNA-184 antagonizes microRNA-205 to maintain SHIP2 levels in epithelia. In Proc.: National Academy of Sciences 105: 19300-19305. https://doi.org/10.1073/pnas.0803992105Zhang L., Nie Q., Su Y., Xie X., Luo W., Jia X., Zhang X. 2013. MicroRNA profile analysis on duck feather follicle and skin with high-throughput sequencing technology. Gene, 519: 77-81. https://doi.org/10.1016/j.gene.2013.01.043Zhao Y., Wang P., Meng J., Ji Y., Xu D., Chen T., Fan R., Yu X., Yao J., Dong C. 2015. MicroRNA-27a-3p Inhibits Melanogenesis in Mouse Skin Melanocytes by Targeting Wnt3a. Int. J. Mol. Sci., 16: 10921-10933. https://doi.org/10.3390/ijms16051092

Effective Distillation of Table-based Reasoning Ability from LLMs

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, their enormous parameter size and extremely high requirements for compute power pose challenges for their practical deployment. Recent research has revealed that specific capabilities of LLMs, such as numerical reasoning, can be transferred to smaller models through distillation. Some studies explore the potential of leveraging LLMs to perform table-based reasoning. However, there has been no prior work focusing on table reasoning skills in smaller models specifically tailored for scientific table-to-text generation tasks. In this paper, we propose a novel table-based reasoning distillation approach, with the aim of distilling LLMs into tailored smaller models. Our experimental results have shown that a 220 million parameter model (Flan-T5-base) fine-tuned using distilled data, not only achieves a significant improvement compared to traditionally fine-tuned baselines, but also surpasses specific LLMs on a scientific table-to-text generation dataset. Our code is available at https://github.com/Bernard-Yang/DistillTableCoT

Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information

The long-standing one-to-many issue of the open-domain dialogues poses significant challenges for automatic evaluation methods, i.e., there may be multiple suitable responses which differ in semantics for a given conversational context. To tackle this challenge, we propose a novel learning-based automatic evaluation metric (CMN), which can robustly evaluate open-domain dialogues by augmenting Conditional Variational Autoencoders (CVAEs) with a Next Sentence Prediction (NSP) objective and employing Mutual Information (MI) to model the semantic similarity of text in the latent space. Experimental results on two open-domain dialogue datasets demonstrate the superiority of our method compared with a wide range of baselines, especially in handling responses which are distant to the golden reference responses in semantics.Comment: Accepted at ACL202

Characterisation and functional analysis of the WIF1 gene and its role in hair follicle growth and development of the Angora rabbit

[EN] Growth and development of hair follicles (HF) is a complex and dynamic process in most mammals. As HF growth and development regulate rabbit wool yield, exploring the role of genes involved in HF growth and development may be relevant. In this study, the coding sequence of the Angora rabbit (Oryctolagus cuniculus) WIF1 gene was cloned. The length of the coding region sequence was found to be 1140 bp, which encodes 379 amino acids. Bioinformatics analysis indicated that the WIF1 protein was unstable, hydrophilic and located in the extracellular region, contained a putative signal peptide and exhibited a high homology in different mammals. Moreover, WIF1 was significantly downregulated in the high wool production in the Angora rabbit group. Overexpression and knockdown studies revealed that WIF1 regulates HF growth and development-related genes and proteins, such as LEF1 and CCND1. WIF1 activated β-catenin/TCF transcriptional activity, promoted cell apoptosis and inhibited cellular proliferation. These results indicate that WIF1 might be important for HF development. This study, therefore, provides a theoretical foundation for investigating WIF1 in HF growth and development.This research was funded by This research was funded by National Natural Science Foundation of China (Grant No. 32102529), China Agriculture Research System of MOF and MARA (CARS-43-A-1).Zhao, B.; Li, J.; Zhang, X.; Bao, Z.; Chen, Y.; Wu, X. (2022). Characterisation and functional analysis of the WIF1 gene and its role in hair follicle growth and development of the Angora rabbit. World Rabbit Science. 30(3):209-218. https://doi.org/10.4995/wrs.2022.1735320921830