Modeling peer assessment as a personalized predictor of teacher's grades: The case of OpenAnswer

Questions with open answers are rarely used as e-learning assessment tools because of the resulting high workload for the teacher/tutor that should grade them. This can be mitigated by having students grade each other's answers, but the uncertainty on the quality of the resulting grades could be high. In our OpenAnswer system we have modeled peer-assessment as a Bayesian network connecting a set of sub-networks (each representing a participating student) to the corresponding answers of her graded peers. The model has shown good ability to predict (without further info from the teacher) the exact teacher mark and a very good ability to predict it within 1 mark from the right one (ground truth). From the available datasets we noticed that different teachers sometimes disagree in their assessment of the same answer. For this reason in this paper we explore how the model can be tailored to the specific teacher to improve its prediction ability. To this aim, we parametrically define the CPTs (Conditional Probability Tables) describing the probabilistic dependence of a Bayesian variable from others in the modeled network, and we optimize the parameters generating the CPTs to obtain the smallest average difference between the predicted grades and the teacher's marks (ground truth). The optimization is carried out separately with respect to each teacher available in our datasets, or respect to the whole datasets. The paper discusses the results and shows that the prediction performance of our model, when optimized separately for each teacher, improves against the case in which our model is globally optimized respect to the whole dataset, which in turn improves against the predictions of the raw peer-assessment. The improved prediction would allow us to use OpenAnswer, without teacher intervention, as a class monitoring and diagnostic tool

Towards a quantitative evaluation of the relationship between the domain knowledge and the ability to assess peer work

In this work we present the preliminary results provided by the statistical modeling of the cognitive relationship between the knowledge about a topic a the ability to assess peer achievements on the same topic. Our starting point is Bloom's taxonomy of educational objectives in the cognitive domain, and our outcomes confirm the hypothesized ranking. A further consideration that can be derived is that meta-cognitive abilities (e.g., assessment) require deeper domain knowledge

Improved computation of individual ZPD in a distance learning system

This paper builds upon theoretical studies in the field of social constructivism. Lev Vygotsky is considered one of the greatest representatives of this research line, with his theory of the Zone of Proximal Development (ZPD). Our work aims at integrating this concept in the practice of a computer-assisted learning system. For each learner, the system stores a model summarizing the current Student Knowledge (SK). Each educational activity is specified through the deployed content, the skills required to tackle it, and those acquired, and is further annotated by the effort estimated for the task. The latter may change from one student to another, given the already achieved competence. A suitable weighting of the robustness (certainty) of student’s skills, stored in SK, and their combination are used to verify the inclusion of a learning activity in the student’s ZPD. With respect to our previous work, the algorithm for the calculation of the ZPD of the individual student has been optimized, by enhancing the certainty weighting policy, and a graphical display of the ZPD has been added. Thanks to the latter, the student can get a clear vision of the learning paths that he/she can presently tackle. This both facilitates the educational process, and helps developing the metacognitive ability self-assessment

Supporting mediated peer-evaluation to grade answers to open-ended questions

We show an approach to semi-automatic grading of answers given by students to open ended questions (open answers). We use both peer-evaluation and teacher evaluation. A learner is modeled by her Knowledge and her assessments quality (Judgment). The data generated by the peer- and teacher- evaluations, and by the learner models is represented by a Bayesian Network, in which the grades of the answers, and the elements of the learner models, are variables, with values in a probability distribution. The initial state of the network is determined by the peer-assessment data. Then, each teacher’s grading of an answer triggers evidence propagation in the network. The framework is implemented in a web-based system. We present also an experimental activity, set to verify the effectiveness of the approach, in terms of correctness of system grading, amount of required teacher's work, and correlation of system outputs with teacher’s grades and student’s final exam grade

Energy Retrofit of a Historic Building Using Simplified Dynamic Energy Modeling

Energy retro-commissioning of historical buildings is an important challenge that implies both historic-artistic and technological aspects concerning the improvement in energy efficiency and comfort. A critical analysis of each possibility is essential in order to preserve the balance between efficiency and architecture. The research focuses on a historical building owned by ANCE (Associazione Nazionale Costruttori Edili), situated in Rome in the Nomentano district. Retrofitting hypothesis were made in order to improve HVAC systems, building's envelope and building's management, always respecting its architectural features. An energy audit has been done in order to evaluate the possibilities. The first step of the study consisted of a measure campaign conducted by Avvenia to know more about the actual use of the building. Next, a dynamic simplified energy modeling of the building has been built using the software Archi Energy. This allowed to preview the effect of modifications on the HVAC and envelope systems. Although starting from an original medium energy performance, simulations showed that it would be possible to reach a further reduction of energy needs by making simple changes in the management/controls domain and, with higher costs, by upgrading envelope components. This study shows that a correct approach can lead to both relevant energetic results and the conservation of architectural characteristics of historical buildings

Zipf and Heaps laws from dependency structures in component systems

Complex natural and technological systems can be considered, on a coarse-grained level, as assemblies of elementary components: for example, genomes as sets of genes, or texts as sets of words. On one hand, the joint occurrence of components emerges from architectural and specific constraints in such systems. On the other hand, general regularities may unify different systems, such as the broadly studied Zipf and Heaps laws, respectively concerning the distribution of component frequencies and their number as a function of system size. Dependency structures (i.e., directed networks encoding the dependency relations between the components in a system) were proposed recently as a possible organizing principles underlying some of the regularities observed. However, the consequences of this assumption were explored only in binary component systems, where solely the presence or absence of components is considered, and multiple copies of the same component are not allowed. Here, we consider a simple model that generates, from a given ensemble of dependency structures, a statistical ensemble of sets of components, allowing for components to appear with any multiplicity. Our model is a minimal extension that is memoryless, and therefore accessible to analytical calculations. A mean-field analytical approach (analogous to the "Zipfian ensemble" in the linguistics literature) captures the relevant laws describing the component statistics as we show by comparison with numerical computations. In particular, we recover a power-law Zipf rank plot, with a set of core components, and a Heaps law displaying three consecutive regimes (linear, sub-linear and saturating) that we characterize quantitatively

Pauli Tomography: complete characterization of a single qubit device

The marriage of Quantum Physics and Information Technology, originally motivated by the need for miniaturization, has recently opened the way to the realization of radically new information-processing devices, with the possibility of guaranteed secure cryptographic communications, and tremendous speedups of some complex computational tasks. Among the many problems posed by the new information technology there is the need of characterizing the new quantum devices, making a complete identification and characterization of their functioning. As we will see, quantum mechanics provides us with a powerful tool to achieve the task easily and efficiently: this tools is the so called quantum entanglement, the basis of the quantum parallelism of the future computers. We present here the first full experimental quantum characterization of a single-qubit device. The new method, we may refer to as ''quantum radiography'', uses a Pauli Quantum Tomography at the output of the device, and needs only a single entangled state at the input, which works on the test channel as all possible input states in quantum parallel. The method can be easily extended to any n-qubits device

Cognition-Based Networks: A New Perspective on Network Optimization Using Learning and Distributed Intelligence

IEEE Access Volume 3, 2015, Article number 7217798, Pages 1512-1530 Open Access Cognition-based networks: A new perspective on network optimization using learning and distributed intelligence (Article) Zorzi, M.a , Zanella, A.a, Testolin, A.b, De Filippo De Grazia, M.b, Zorzi, M.bc a Department of Information Engineering, University of Padua, Padua, Italy b Department of General Psychology, University of Padua, Padua, Italy c IRCCS San Camillo Foundation, Venice-Lido, Italy View additional affiliations View references (107) Abstract In response to the new challenges in the design and operation of communication networks, and taking inspiration from how living beings deal with complexity and scalability, in this paper we introduce an innovative system concept called COgnition-BAsed NETworkS (COBANETS). The proposed approach develops around the systematic application of advanced machine learning techniques and, in particular, unsupervised deep learning and probabilistic generative models for system-wide learning, modeling, optimization, and data representation. Moreover, in COBANETS, we propose to combine this learning architecture with the emerging network virtualization paradigms, which make it possible to actuate automatic optimization and reconfiguration strategies at the system level, thus fully unleashing the potential of the learning approach. Compared with the past and current research efforts in this area, the technical approach outlined in this paper is deeply interdisciplinary and more comprehensive, calling for the synergic combination of expertise of computer scientists, communications and networking engineers, and cognitive scientists, with the ultimate aim of breaking new ground through a profound rethinking of how the modern understanding of cognition can be used in the management and optimization of telecommunication network

Thermo-economic assessment of a olive pomace gasifier for cogeneration applications

A thermo-economic analysis of a combined heat and power (CHP) plant fed by syngas produced through the gasification of dry olive pomace is presented. The plant is composed by a 800 kWtdowndraft gasifier, a gas clean-up system, a 200 kWemicroturbine (MGT) and a heat recovery system to cogenerate hot water. Surplus heat is used to dry olive pomace from 50% to 17% wb moisture content. The plant is modeled in ASPEN Plus. Real data from experimental tests are used to calibrate the gasifier model, while the technical specification and performance of the CHP plant are collected from commercial plants in operation and data from manufacturers. Mass and energy balances are reported throughout the paper. The thermodynamic simulation of the biomass gasifier coupled to the MGT, the thermal and electrical conversion efficiency and temperature of cogenerated heat available are also presented. A thermo-economic assessment is then proposed, to investigate the economic profitability of this small scale CHP plant in the Italian energy policy scenario and considering the subsidies available for renewable electricity in the form of feed-in tariffs. For this purpose, the case study of base load CHP plant operation and heat supplied to different typologies of energy end user is assumed. The results allow quantifying the most influencing economic and technical factors that affect the performance and profitability of such investment and the bottlenecks that should be faced to facilitate a broader implementation of such CHP schemes for on site generation