Calibration Models for Real-World Deployment of Reinforcement Learning Agents

The sensitivity of reinforcement learning algorithm performance to hyperparameter choices poses a significant hurdle to the deployment of these algorithms in the real-world, where sampling can be limited by speed, safety, or other system constraints. To mitigate this, one approach is to learn a calibration model from offline data logs, and use this model to simulate trajectories for the purpose of hyperparameter tuning. While there has been preliminary success applying calibration models to simple simulated problems, more work is needed to understand the desirable properties of such models and to test their feasibility in a real-world setting. In this work, we take the first steps toward characterizing desirable properties of calibration models and provide the first application of a calibration model towards a real-world industrial prediction task. We investigate several measures that can be used to understand model quality and evaluate calibration model implementations according to these measures. The calibration models are then tested on a prediction task for sensors in a water treatment plant (WTP) located in Alberta, Canada. We find that various types of calibration models can be used to simulate simple environments, while generalizing models tend to collapse due to compounding prediction error in the more complex real-world setting. We show how a non-parametric k-nearest neighbors calibration model with a Laplacian distance metric is able to produce realistic rollouts over long-horizons in the WTP setting, and can be used successfully for hyperparameter tuning. Finally, we aim to bridge the gap towards real-world deployment and demonstrate how this model can be scaled to a year's worth of data

A phonological history of Amdo Tibetan rhymes

In this study, a reconstruction is offered for the phonetic evolution of rhymes from Old Tibetan to modern-day Amdo Tibetan dialects. The relevant sound changes are proposed, along with their relative chronological precedence and the dating of some specific changes. Most interestingly, although Amdo Tibetan, identically to its ancestor Old Tibetan, does not have phonemic length, this study shows that Amdo Tibetan derives from an intermediate stage which, like many other Tibetan dialects, does make the distinction

The Duhumbi Perspective On Proto-Western Kho-Bwa Onsets.

The eight Western Kho-Bwa varieties are spoken in western Arunachal Pradesh in Northeast India and form a small, coherent sub-group of the Tibeto-Burman (Trans-Himalayan / Sino-Tibetan) language family.This paper presents 96 sound correspondences, mainly between the two Western Kho-Bwa varieties Duhumbi and Khoitam, with additional evidence from other Western Kho-Bwa varieties and other Tibeto-Burman languages whenever deemed illustrative. On basis of these sound correspondences, I propose 282 Western Kho-Bwa proto-forms including a total of 92 onsets. The less common reconstructed Western Kho-Bwa onsets are the uvular onsets and the voiceless nasal and approximant onsets.A unique innovation of the Western Kho-Bwa languages, and indeed the Kho-Bwa languages in general, is the correspondence of initial *s- in other Tibeto-Burman languages to a vocal onset in Proto-Western Kho-Bwa and its descendent varieties. Another relatively unique innovation is the correspondence between Western Kho-Bwa obstruent onsets *b- and *g- ~ *kʰ- ~ *k- and other Tibeto-Burman nasal onsets *m- and *ŋ-, respectively

Retroflex initials in the history of Southern Guanhua phonology

During most of its history the Guānhuà koine is thought to have been rooted in the southern Mandarin dialects of the Yangtze watershed. However, the earliest alphabetic recordings of this language, dating from the mid-fifteenth century, show that it had certain distinctly northern characteristics. The present paper studies one of the features, i.e., retroflexion of certain syllable initials, and concludes that this is attributable to northern migration into the Nanking area from mid-Yuán to early Ming times. Subsequently, the koine came under increasingly heavy southern influence, resulting in competing non-northern forms in the lexicon. These probably became dominant later in the history of the koine. </jats:p

Retroflex initials in the history of Southern Guânhuà phonology [Un problème dans l'histoire de la phonologie du guânhuà du sud]

During most of its history the Guànhuà koiné is thought to have been rooted in the southern Mandarin dialects of the Yangtze watershed. However, the earliest alphabetic recordings of this language, dating from the mid-fifteenth century, show that it had certain distinctly northern characteristics. The present paper studies one of the features, i.e., retroflexion of certain syllable initials, and concludes that this is attributable to northern migration into the Nanking area from mid- Yuan to early Ming times. Subsequently, the koiné came under increasingly heavy southern influence, resulting in competing non-northern forms in the lexicon. These probably became dominant later in the history of the koiné.La koiné chinoise ou guànhuà, fondée pour l'essentiel sur les dialectes mandarins du bassin du Yangzi, présente pourtant dans ses premières transcriptions alphabétiques (datant du milieu du IS*1"6 siècle) certaines caractéristiques propres au mandarin du nord. Cet article analyse Tune d'elles, la retroflexion de certaines initiales, et l'attribue à une migration vers la région de Nankin de populations venues du nord, qui se serait produite entre le milieu des Yuan et le début des Ming. Une influence méridionale de plus en plus marquée aboutit ensuite à l'introduction dans la koiné de diverses formes différant de l'usage septentrional, qui rentrent en compétition dans le lexique avec les formes plus anciennes, et finissent par l'emporter.Coblin South W. Retroflex initials in the history of Southern Guânhuà phonology [Un problème dans l'histoire de la phonologie du guânhuà du sud]. In: Cahiers de linguistique - Asie orientale, vol. 38 1, 2009. pp. 125-162

Comparative Phonology of the Huīzhōu Dialects

The present article compares data from seven Huīzhōu dialects in an effort to construct a common phonological system for the family as a whole. In the final section of the paper it is concluded that this dialect group is probably an areal or geographical grouping rather than a genetically related dialect family.</jats:p

We Will Meet Somewhere In Heaven

Photograph of Vincent Lopes and Bruce Hayes; Illustration of sky shining down on soldiershttps://scholarsjunction.msstate.edu/cht-sheet-music/10902/thumbnail.jp

The phonetic values of the Tibetan letter r in Tibeto-Chinese transcriptional texts from Dunhuang

In Tibeto-Chinese transcriptions from Dūnhuáng, Tibetan r frequently occurs in syllable final position, where it corresponds to final -t of the traditional Qièyùn System. This use of Tibetan r in the texts has been well known for over a century and is often discussed in the literature. Much rarer and less well known is the use of r in syllable initial and medial positions. In this paper it is noted that syllable initial r- occurs exclusively in certain phrase-final particles and corresponds to Qièyùn System initial l-, while medial -r- is limited to a small number of Division II-type syllables. In these latter cases, it may indicate some sort of rhotacism.Published versio