Assessing agreement on classification tasks: the kappa statistic

Currently, computational linguists and cognitive scientists working in the area of discourse and dialogue argue that their subjective judgments are reliable using several different statistics, none of which are easily interpretable or comparable to each other. Meanwhile, researchers in content analysis have already experienced the same difficulties and come up with a solution in the kappa statistic. We discuss what is wrong with reliability measures as they are currently used for discourse and dialogue work in computational linguistics and cognitive science, and argue that we would be better off as a field adopting techniques from content analysis.Comment: 9 page

Reliability measurement without limits

In computational linguistics, a reliability measurement of 0.8 on some statistic such as $\kappa$ is widely thought to guarantee that hand-coded data is fit for purpose, with lower values suspect. We demonstrate that the main use of such data, machine learning, can tolerate data with a low reliability as long as any disagreement among human coders looks like random noise. When it does not, however, data can have a reliability of more than 0.8 and still be unsuitable for use: the disagreement may indicate erroneous patterns that machine-learning can learn, and evaluation against test data that contain these same erroneous patterns may lead us to draw wrong conclusions about our machine-learning algorithms. Furthermore, lower reliability values still held as acceptable by many researchers, between 0.67 and 0.8, may even yield inflated performance figures in some circumstances. Although this is a common sense result, it has implications for how we work that are likely to reach beyond the machine-learning applications we discuss. At the very least, computational linguists should look for any patterns in the disagreement among coders and assess what impact they will have

Modelling Participant Affect in Meetings with Turn-Taking Features

This paper explores the relationship between turn-taking and meeting affect. To investigate this, we model post-meeting ratings of satisfaction, cohesion and leadership from participants of AMI corpus meetings using group and individual turn-taking features. The results indicate that participants gave higher satisfaction and cohesiveness ratings to meetings with greater group turn-taking freedom and individual very short utterance rates, while lower ratings were associated with more silence and speaker overlap. Besides broad applicability to satisfaction ratings, turn-taking freedom was found to be a better predictor than equality of speaking time when considering whether participants felt that everyone they had a chance to contribute. If we include dialogue act information, we see that substantive feedback type turns like assessments are more predictive of meeting affect than information giving acts or backchannels. This work highlights the importance of feedback turns and modelling group level activity in multiparty dialogue for understanding the social aspects of speech

Dynamics and Control of Satellite Formations Invariant under the Zonal Harmonic Perturbation

A satellite formation operating in low-altitude orbits is subject to perturbations associated to the higher-order harmonics of the gravitational field, which cause a degradation of the formation configurations designed based on the unperturbed model of the Hill–Clohessy–Wiltshire equations. To compensate for these effects, periodic reconfiguration maneuvers are necessary, requiring the prior allocation of a propellant mass budget and, eventually, the use of resources from the ground segment, having a non-negligible impact on the complexity and cost of the mission. Using the Hamiltonian formalism and canonical transformations, a model is developed that allows designing configurations for formation flying invariant with respect to the zonal harmonic perturbation. Jn invariant configurations can be characterized, selecting the drift rate (or boundedness condition) and the amplitude of the oscillations, based on four parameters which can be easily converted in position and velocity components for the satellites of the formation. From this model, a guidance strategy is developed to inject a satellite approaching another spacecraft into a bounded relative trajectory about it and the optimal time for the maneuver, minimizing the total ΔV , is identified. The effectiveness of the model and of the guidance strategy is verified on some scenarios of interest for formations operating in a sun-synchronous and a medium-inclination low Earth orbit and a medium-inclination lunar orbit

Assessing College Students’ Perceived Risk for Developing Type 2 Diabetes Mellitus

This study assesses the effectiveness of an evidence-based educational intervention affecting the perception of the risk for developing type 2 diabetes (T2DM) in college students and examined relationships between demographic factors and perception of risk for T2DM. Seventy-seven students from the Georgia College Association of Nursing Students (GCANS) and the Personal Health and Fitness Class participated. Of these, 43 participants completed the post-educational email survey. Instruments measured demographic characteristics, perception of risk for developing diabetes, and diabetes risk. The Risk Perception Survey for Developing Diabetes (RPS-DD) assessed students’ perception of risk at baseline and one week post intervention. There was a significant increase for diabetes risk knowledge scores between participants who completed the intervention (M = 6.56, SD = 1.28) and those who did not complete the intervention (M = 7.38, SD = 1.65) t (75) = -2.47, p =.016. There was a significant increase in the worry scores from baseline (M = 2.21, SD = 0.64) to one week (M = 2.44, SD = 0.62), (t [42] = -2.89, p =.006). There was a significant increase in diabetes risk knowledge scores from pre-intervention (M = 6.56, SD = 1.28) to post-intervention (M = 8.35, 1.49) (t [42] = -7.09, p \u3c.001). There was a statistically significant relationship between educational intervention and body mass index (BMI), [F (1, 27) = 3.85, p =.034, η2 =.22] and educational intervention and college level, [F (1, 27) = 3.36, p =.033, η2 =.027]. Increased knowledge and raised awareness of risk for developing T2DM results supports the use of the T2DM educational intervention with college students

A Single-Launch Deployment Strategy for Lunar Constellations

Satellite constellations can provide communication and navigation services to support future lunar missions, and are attracting growing interest from both the scientific community and industry. The deployment of satellites in orbital planes that can have significantly different inclinations and right ascension of the ascending node requires dedicated launches and represents a non-trivial issue for lunar constellations, due to the complexity and low accessibility of launches to the Moon. In this work, a strategy to deploy multiple satellites in different orbital planes around the Moon in a single launch is examined. The launch vehicle moves along a conventional lunar escape trajectory, with parameters selected to take advantage of gravity-braking upon encountering the Moon. A maneuver at the periselenium allows the transfer of the spacecraft along a trajectory converging to the equilibrium region about the Earth–Moon libration point L1 , where the satellites are deployed. Providing a small ΔV , each satellite is transferred into a low-energy trajectory with the desired inclination, right ascension of the ascending node, and periselenium radius. A final maneuver, if required, allows the adjustment of the semimajor axis and the eccentricity. The method is verified using numerical integration using high-fidelity orbit propagators. The results indicate that the deployment could be accomplished within one sidereal month with a modest ΔV budget

Developing Meeting Support Technologies: From Data to Demonstration (and Beyond)

Proceedings of the 17th Nordic Conference of Computational Linguistics NODALIDA 2009. Editors: Kristiina Jokinen and Eckhard Bick. NEALT Proceedings Series, Vol. 4 (2009), 2-3. © 2009 The editors and contributors. Published by Northern European Association for Language Technology (NEALT) http://omilia.uio.no/nealt . Electronically published at Tartu University Library (Estonia) http://hdl.handle.net/10062/9206

Determining Best Method For Estimating Observed Level Of Maximum Convective Detrainment Based On Radar Reflectivity

Convective mass transport is the transport of mass from near the surface up to the upper troposphere and lower stratosphere (UTLS) by a deep convective updraft. This transport can alter the chemical makeup and water vapor balance of the UTLS, which can affect cloud formation and the radiative properties of the atmosphere. It is therefore important to understand the exact altitudes at which mass is detrained from convection. The purpose of this study is to improve upon previously published methodologies for estimating the level of maximum detrainment (LMD) within convection using data from individual radars. Three methods were used to identify the LMD and validated against dual-Doppler derived vertical mass divergence fields. The best method for locating the LMD was determined to be the method that uses a horizontal reflectivity texture-based technique to determine convective cores and a multi-layer echo identification to determine anvil locations. The methodology was found to work in many but not all cases. The methodology works best when applied to convective systems with mature updrafts, and is most accurate with convective lines and isolated cells. A time lag is present in the reflectivity based LMD compared to the vertical mass divergence based LMD because the reflectivity method is dependent on anvil growth. This methodology was then applied to archived NEXRAD 3D mosaic radar data. The regions of analysis were chosen to coincide with the observation regions for the Deep Convective Clouds and Chemistry Experiment (DC3): the Colorado Foothills, Southern Plains (OK/TX), and Southeast US (AL). These three regions provide a wide variety of convection. The dates analyzed were from May and June of 2012 so the results can be compared to future DC3 studie