Never Lost in the Middle: Improving Large Language Models via Attention Strengthening Question Answering

While large language models (LLMs) are equipped with longer text input capabilities than before, they are struggling to seek correct information in long contexts. The "lost in the middle" problem challenges most LLMs, referring to the dramatic decline in accuracy when correct information is located in the middle. To overcome this crucial issue, this paper proposes to enhance the information searching and reflection ability of LLMs in long contexts via specially designed tasks called Attention Strengthening Multi-doc QA (ASM QA). Following these tasks, our model excels in focusing more precisely on the desired information. Experimental results show substantial improvement in Multi-doc QA and other benchmarks, superior to state-of-the-art models by 13.7% absolute gain in shuffled settings, by 21.5% in passage retrieval task. We release our model, Ziya-Reader to promote related research in the community

Dlf1, a WRKY Transcription Factor, Is Involved in the Control of Flowering Time and Plant Height in Rice

<div><p>Flowering time and plant height are important agronomic traits for crop production. In this study, we characterized a semi-dwarf and late flowering (<i>dlf1</i>) mutation of rice that has pleiotropic effects on these traits. The <i>dlf1</i> mutation was caused by a T-DNA insertion and the cloned <i>Dlf1</i> gene was found to encode a WRKY transcription factor (OsWRKY11). The <i>dlf1</i> mutant contains a T-DNA insertion at the promoter region, leading to enhanced accumulation of <i>Dlf1</i> transcripts, resulting in a semidominant mutation. The <i>dlf1</i> mutation suppressed the transcription of <i>Ehd2/RID1/OsId1</i> and its downstream flowering-time genes including <i>Hd1</i>, <i>Ehd1</i> and <i>Hd3a</i> under both long-day (LD) and short-day (SD) conditions. Knock-down of <i>Dlf1</i> expression exhibited early flowering at LD condition related to the wild-type plants. Accumulation of <i>Dlf1</i> mRNA was observed in most tissues, and two splicing forms of <i>Dlf1</i> cDNAs were obtained (<i>OsWRKY11.1</i> and <i>OsWRKY11.2</i>). These two proteins showed transactivation activity in yeast cells. Dlf1 protein was found to be localized in the nucleus. Enhanced expression of <i>OsWRKY11.2</i> or its 5′ truncated gene showed similar phenotypes to the <i>dlf1</i> mutant, suggesting that it might function as a negative regulator. We conclude that Dlf1 acts as a transactivator to downregulate <i>Ehd2/RID1/OsId1</i> in the signal transduction pathway of flowering and plays an important role in the regulation of plant height in rice.</p></div

Dlf1 suppresses the expression of <i>Hd3a</i>, <i>Hd1</i>, <i>Ehd1</i> and <i>Ehd2</i>.

<p>Diurnal expression patterns of <i>Dlf1</i>, <i>Hd1</i>, <i>Hd3a, Ehd2</i> and <i>Ehd1</i> in wild-type ZH11 (filled squares) and <i>dlf1</i> (open squares) plants under SD (10/14 h light/dark) and LD (14/10 h light/dark) conditions by qPCR analysis. The expression levels are relative to the <i>ubiquitin</i> (<i>Ubq</i>) mRNA. The plants were grown at conditions as described in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0102529#pone-0102529-g001" target="_blank">Fig. 1</a>. Values are shown as means ± SD of two independent experiments. The open and filled bars at the bottom represent the light and dark periods, respectively.</p

Expression pattern of <i>Dlf1</i>.

<p>(<b>A</b>) Expression of <i>Dlf1</i> in different tissues was analyzed by qPCR. RNA was isolated from young panicles, sheathes, older leaves (leaf O) and younger leaves (leaf N). Tissue samples were collected at 4 h after dawn. Values are shown as means ± SD of two biological experiments. Values marked with different letters are significantly different (P<0.05, Duncan test). (<b>B</b>) Gus-fused constructs with different lengths of <i>Dlf1</i> promoter. (<b>C</b>) <i>Dlf1</i> promoter-driven <i>Gus</i> expression (<i>Cp-WP</i>) in (a) young panicles (b) leaves and (c) roots of three-week-old plants. (<b>D</b>) Gus histochemical staining of 6-day-old transgenic lines. (<b>E</b>) Gus enzyme activity was measured in six-day-old seedlings harboring different constructs. Transcription analysis of <i>Gus</i> gene in the transgenic plants. The gene expression was normalized to rice <i>ubiquitin</i> gene (<i>Ubq</i>) for each sample. Means and their standard deviations are shown from three independent experiments. (<b>F</b>) Different transcripts of <i>Dlf1</i>. Total RNAs were isolated from leaves of three-week-old ZH11 plants. PCR products were obtained by amplification using templates of reverse-transcribed RNA (T) and RNA (uT) and separated by electrophoresis. (<b>G</b>) Northern blot analysis of <i>Dlf1</i> expression in leaves (L) and roots (R), using the total RNAs isolated from three-week-old plants. The probes used were the <i>Dlf1</i> coding region. rRNA of ethidium bromide staining was used as the loading control. Bar  = 1 mm (<b>C</b>) and 2 mm (<b>D</b>).</p

Expression of <i>Dlf1</i> and other flowering-time genes during development.

<p>Changes of <i>Dlf1</i>, <i>Hd3a</i>, <i>Ehd1</i> and <i>Ehd2</i> transcription levels in wild-type (filled squares) and <i>dlf1</i> (open squares) plants during development under LD conditions. The expression levels are relative to the <i>ubiquitin</i> (<i>Ubq</i>) mRNA. The values of <i>Dlf1</i> expression in ZH11 were scaled up 10 times. The plants were grown at conditions as described in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0102529#pone-0102529-g001" target="_blank">Fig. 1</a>. Developing leaves were harvested 4 h after dawn. Values are shown as means ± SD of two independent experiments.</p

Phenotypes of Zhonghua 11 (ZH11) and <i>dlf1</i> mutant.

<p>(<b>A</b>) Photos of the wild-type ZH11 (left) and mutant <i>dlf1</i> (right), taken when ZH11 reached maturity. (<b>B</b>) Photos of main culms of ZH11 (left) and <i>dlf1</i> (right). (<b>C</b>) Main panicles of ZH11 (left) and <i>dlf1</i> (right). (<b>D</b>) Grains from main panicles of ZH11 (top) and <i>dlf1</i> (bottom). (<b>E</b>) Days to heading of ZH11 and <i>dlf1</i> under SD, LD and NLD (natural LD) conditions. Data are represented as mean values ± standard derivation (SD) of 20 plants. (F) Plant heights of ZH11 and <i>dlf1</i> under natural LD conditions. Values are means ± SD, n = 20. Experiment were performed three times, showing similar results. (<b>G</b>) Difference of internode lengths between matured ZH11 and <i>dlf1</i> plants. The plants were grown in the experimental field under natural LD conditions. Values are means ± SD, n = 20. The same experiments were repeated three times, and the similar results were obtained. For SD and LD treatments, the plants were grown in greenhouse under natural light conditions and shaded at the time designated. a and b in figure indicate ranking by Duncan test at P<0.05, starting from a. b is significantly different from a.</p

Nuclear localization of Dlf1-GFP fusion protein in onion epidermal cells.

<p>Onion epidermal cells were transformed with plasmids expressing GFP (top panel), or the Dlf1-GFP fusion protein (bottom panel) by bombardment and examined after 24 h. GFP fluorescence (left panel) and differential interference contrast image (middle panel) were compared to show the subcellular localization of GFP (cytoplasmic and nuclear) and Dlf1-GFP (nuclear). The images of the right panel were merged for each.</p

Dlf1 is transactivator in yeast.

<p>The full encoding sequence and deletion mutants of <i>Dlf1</i> were fused in frame to the <i>GAL4</i>-binding domain (BD) in <i>pBDKT7</i> to generate various vectors for yeast transformation. The constructed vectors were transformed into yeast AH109 strain, and grew on the selective medium at 30°C for 3 d. The β-galactosidase activity of transformants was determined using <i>o</i>-nitrophenyl β-D-galatopyranoside as a substrate. An empty vector <i>pGBKT7</i> (<i>pBD</i>) was used as the negative control. The values from three independent experiments were shown as means ± SD. Slash boxes represent BD in <i>pGBKT7</i> and the black boxes for the WRKY domain of Dlf1, whereas the white boxes represent the rest part of Dlf1, and the line indicates the deleted region. The numbers in the brackets are the start and end positions of each translation product of Dlf1 in the construct.</p

High level expression of the <i>OsWRKY11.2</i> leads to dwarfism and late flowering.

<p>(<b>A</b>) Schematic diagram of <i>Ubi:W11.2</i> (Ka-) and <i>Ubi:d4W11.2</i> (4S-, with 37-aa deletion at the N-terminus of W11.2) constructs. (<b>B</b>) and (<b>D</b>) Plant heights of those transformed with <i>Ubi:W11.2</i> or <i>Ubi:d4W11.2</i> in ZH11 or ZH17 genetic background, respectively. (<b>C</b>) Expression of total <i>OsWRKY11</i> (including the transferred and endogenous genes) in ZH11 and the <i>Ubi:W11.2</i> transgenic lines of T1 progenies under natural LD conditions. The first and second youngest leaves were sampled from 90-d-old plants for RNA isolation. Transcription levels were quantified by qPCR and the gene expression was normalized to rice <i>ubiquitin</i> gene (<i>Ubq</i>) for each sample. Transcription levels are expressed as ratio to the level of transcript in ZH11. The suffix A for dwarf and G for segregated non-transgenic plants. (<b>E</b>) Days to heading of the <i>Ubi:d4W11.2</i> plants of T₂ progenies under SD and LD conditions (the same treatments as in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0102529#pone-0102529-g006" target="_blank">Fig. 6</a>). (<b>F</b>) Expression of total <i>OsWRKY11</i> in ZH17 and the <i>Ubi:d4W11.2</i> transgenic plants of T₂ progenies under both LD and SD conditions (the same treatments as in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0102529#pone-0102529-g006" target="_blank">Fig. 6</a>). Transcription levels are expressed as ratio to the level of transcript in ZH17. (<b>G</b>) Analysis of the possible degraded mRNA of <i>Dlf1</i> using the RNA ligase-mediated amplification of 5′ cDNA ends (RLM-RACE). Total RNAs were isolated from the <i>dlf1</i> mutant (<i>dlf1</i>) and the wild-type (ZH11) of three-week-old plants. Two rounds of PCR were performed: 1) by using an outer primer from the adaptor and a <i>Dlf1</i>-specific reverse primer W5; and 2) by using the inner primer from the adaptor and a <i>Dlf1</i> gene primer W4 for 30 cycles. <i>Actin</i> gene was used as an internal standard. Asterisks indicate significant difference between ZH11 and the overexpression lines (P<0.05, Duncan test). a, b, c, d, and e indicate ranking by Duncan test at P<0.05, starting from a. Different letters indicate significantly difference from each other.</p

Alternation of <i>Dlf1</i> expression changes the heading day of rice.

<p>(<b>A</b>) The heading time of <i>Ubi:W11.1</i> transgenic lines in T₂ progenies (named as C-) and the ZH17 control under LD conditions. (<b>B</b>) Expression of total <i>OsWRKY11</i> (including the transferred and endogenous genes) in the <i>Ubi:W11.1</i> transgenic and ZH17 plants. (<b>C</b>) The heading day of ZH11 and the RNAi lines in T₂ progenies (named as dch-) under LD condition. (<b>D</b>) Expression of total <i>OsWRKY11</i> in the RNAi transgenic and ZH11 control plants. Leaves of the first and second youngest were sampled from 90-d old plants. Values are shown as means ± SD of two independent experiments. a and b in figure indicate ranking by Duncan test at P<0.05, starting from a. b is significantly different from a.</p