CITB: A Benchmark for Continual Instruction Tuning

Continual learning (CL) is a paradigm that aims to replicate the human ability to learn and accumulate knowledge continually without forgetting previous knowledge and transferring it to new tasks. Recent instruction tuning (IT) involves fine-tuning models to make them more adaptable to solving NLP tasks in general. However, it is still uncertain how instruction tuning works in the context of CL tasks. This challenging yet practical problem is formulated as Continual Instruction Tuning (CIT). In this work, we establish a CIT benchmark consisting of learning and evaluation protocols. We curate two long dialogue task streams of different types, InstrDialog and InstrDialog++, to study various CL methods systematically. Our experiments show that existing CL methods do not effectively leverage the rich natural language instructions, and fine-tuning an instruction-tuned model sequentially can yield similar or better results. We further explore different aspects that might affect the learning of CIT. We hope this benchmark will facilitate more research in this direction.Comment: EMNLP 2023 Finding

Turn-Level Active Learning for Dialogue State Tracking

Dialogue state tracking (DST) plays an important role in task-oriented dialogue systems. However, collecting a large amount of turn-by-turn annotated dialogue data is costly and inefficient. In this paper, we propose a novel turn-level active learning framework for DST to actively select turns in dialogues to annotate. Given the limited labelling budget, experimental results demonstrate the effectiveness of selective annotation of dialogue turns. Additionally, our approach can effectively achieve comparable DST performance to traditional training approaches with significantly less annotated data, which provides a more efficient way to annotate new dialogue data.Comment: EMNLP 2023 Main Conferenc

Punctuality Improvement in Australian Rail Freight Network by Transit Time Management

Abstract: With rapid development in product globalization and just-in-time production over the last two decades, area-specific reliable, responsive and customer-oriented rail freight services are in demand and are of increasing interest. Having a proper understanding of underlying factors in the evaluation of the quality of rail freight services is a key challenge in the short-term and long-term regional and metropolitan freight mobility planning, particularly within the context of a competitive rail freight market as in Australia. Among the fundamental attributes of rail freight services, transit time and reliability/punctuality are of utmost importance, which also tend to be inevitably correlated. This paper discusses the potential opportunities for service improvement in the Australian non-bulk interstate network through managing the underlying factors. The paper also addresses the conditions under which these factors can be combined to enhance the utilisation and efficiency of rail freight services in the national rail infrastructure. Citation: Ghaderi, H. & Namazi-Rad, M-R. (2014). Punctuality Improvement in Australian Rail Freight Network by Transit Time Management. In: Campbell P. and Perez P. (Eds), Proceedings of the International Symposium of Next Generation Infrastructure, 1-4 October 2013, SMART Infrastructure Facility, University of Wollongong, Australia

Pengujian Efisiensi Bentuk Setengah Kuat di Indonesia

Efficient market is the market in which when there is an announcement the market get a reac- tion quickly from the investors. This finally influences the price movement of securities to- ward the new equilibrium price. Some of action dealing with announcement and that it is be- lieved to have trustable information, this information can be considered feasible to get re- sponse technically so as to influence the transaction in the capital market outside. Further- more, a market can be efficient also when it is a strong form and the security prices fully ex- presses all information widespread. This study attempts to find out to what extend the effi- ciency for capital market information in Indonesia by testing some actions done by the com- panies announced on the stock split, reverse split, profit announcement, and dividend shar- ing. The sample was taken by means of purposive sampling. Each consists of 26 samples of events for stock split and 19 sample of event for stock reverse. For announcement of the profit consists of 28 companies with 45 events and dividend announcement 26 companies for 52 events. The Expected is calculated using 3 models (Market Model, Mean Adjusted Model, and Market Adjusted Model). Using estimation period of 100 with five day observation pe- riod after event analysis, it shows that Indonesia capital markets has different reactions to- ward each event. In general, the results show that only profit announcement is responded by the capital market

COVID-19 related stigma among the general population in Iran

Funding Information: GT is supported by the National Institute for Health Research (NIHR) Applied Research Collaboration South London at King’s College London NHS Foundation Trust, and by the NIHR Asset Global Health Unit award. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. GT is also supported by the Guy’s and St Thomas’ Charity for the On Trac project (EFT151101), and by the UK Medical Research Council (UKRI) in relation to the Emilia (MR/S001255/1) and Indigo Partnership (MR/R023697/1) awards. Publisher Copyright: © 2022, The Author(s).Peer reviewedPublisher PD

What Level of Statistical Model Should We Use in Small Domain Estimation?

If unit-level data are available, Small Area Estimation (SAE) is usually based on models formulated at the unit level, but they are ultimately used to produce estimates at the area level and thus involve area-level inferences. This paper investigates the circumstances when using an area-level model may be more effective. Linear mixed models fitted using different levels of data are applied in SAE to calculate synthetic estimators and Empirical Best Linear Unbiased Predictors (EBLUPs). The performance of area-level models is compared with unit-level models when both individual and aggregate data are available. A key factor is whether there are substantial contextual effects. Ignoring these effects in unit-level working models can cause biased estimates of regression parameters. The contextual effects can be automatically accounted for in the area-level models. Using synthetic and EBLUP techniques, small area estimates based on different levels of linear mixed models are studied in a simulation study

What level should we use in small area estimation?

Many different Small Area Estimation (SAE) methods have been proposed to overcome the challenge of finding reliable estimates for small domains. Often, the required data for various research purposes are available at different levels of aggregation. Based on the available data, individual-level or aggregated-level models are used in SAE. If unit-level data are available, SAE is usually based on models formulated at the unit level but they are ultimately used to produce estimates at the area level. However, parameter estimates obtained from individual and aggregated level analysis may be different in practice. Individual-level analysis usually results in small area estimates with smaller variances. However, if the unit-level working model is misspecified by exclusion of important auxiliary variables, parameter estimates obtained from the individual and aggregated level analysis will have different expectations. This thesis investigates the circumstances when using an area-level model may be more effective. This may happen due to some substantial contextual or area-level effects in the covariates which may be misspecified in an individual-level model. Ignoring these contextual effects leads to biased estimates. In particular, if an existing contextual variable is ignored, the parameter estimates calculated from an individual-level analysis will be biased, whereas an aggregated-level analysis can lead to small area estimates with less bias. Even if contextual variables are included in unit-level modeling, there may be an increase in the variance of parameter estimates due to increased number of variables in the working model. In this thesis, synthetic estimators and Empirical Best Linear Unbiased Predictors (EBLUPs) are evaluated in SAE based on different levels of linear mixed models. Using a numerical simulation study, the key role of contextual effects is examined for models used in SAE