Screening for technical flaws in multiple-choice items. A generalizability study.

Construction errors in multiple-choice items are quite prevalent and constitute threats to test validity of multiple-choice tests. Currently very little research on the usefulness of systematic item screening by local review committees before test administration seem to exist. The aim of this study was therefore to examine validity and feasibility aspects of review committee screening for item flaws. We examined the reliability of item reviewers’ independent judgments of the presence/absence of item flaws with a generalizability study design and found only moderate reliability using five reviewers. Statistical analyses of actual exam scores could be a more efficient way of identifying flaws and improving average item discrimination of tests in local contexts. The question of validity of human judgments of item flaws is important - not just for sufficiently sound quality assurance procedures of tests in local test contexts - but also for the global research on item flaws

Screening for technical flaws in multiple-choice items. A generalizability study.

Construction errors in multiple-choice items are quite prevalent and constitute threats to test validity of multiple-choice tests. Currently very little research on the usefulness of systematic item screening by local review committees before test administration seem to exist. The aim of this study was therefore to examine validity and feasibility aspects of review committee screening for item flaws. We examined the reliability of item reviewers’ independent judgments of the presence/absence of item flaws with a generalizability study design and found only moderate reliability using five reviewers. Statistical analyses of actual exam scores could be a more efficient way of identifying flaws and improving average item discrimination of tests in local contexts. The question of validity of human judgments of item flaws is important - not just for sufficiently sound quality assurance procedures of tests in local test contexts - but also for the global research on item flaws

Validity assumptions for a multiple-choice test of medical knowledge with open-books and web access. A known groups comparison study.

Relatively little evidence about the validity threats in open-book multiple-choice tests exist. The aim of this study was to examine validity aspects relating to gener-alization, extrapolation and decision of a multiple-choice test of medical knowledge with aids (open-book and internet access). The theoretical framework was modern validity theory, and the study was designed as a ‘known groups com-parison’ study. Test performances of three known groups of test takers hypothe-sized to have different knowledge levels of the test content were compared, and analysis of pass/fail decisions was used to examine implications of decisions based on test scores. Results indicated that it was possible to discriminate between expert and non-expert test taker groups even with the access to aids. In contrast, an inde-fensible passing score was found to be the largest potential threat to test validity. Relatively little evidence about the validity threats in open-book multiple-choice tests exist. The aim of this study was to examine validity aspects relating to gener-alization, extrapolation and decision of a multiple-choice test of medical knowledge with aids (open-book and internet access). The theoretical framework was modern validity theory, and the study was designed as a ‘known groups com-parison’ study. Test performances of three known groups of test takers hypothe-sized to have different knowledge levels of the test content were compared, and analysis of pass/fail decisions was used to examine implications of decisions based on test scores. Results indicated that it was possible to discriminate between expert and non-expert test taker groups even with the access to aids. In contrast, an inde-fensible passing score was found to be the largest potential threat to test validity.&nbsp

Non-Standard Errors

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants

Bitcoin, Gold and the Dollar – a GARCH Volatility Analysis

This paper explores the financial asset capabilities of bitcoin using GARCH models. The initial model showed several similarities to gold and the dollar indicating hedging capabilities and advantages as a medium of exchange. The asymmetric GARCH showed that bitcoin may be useful in risk management and ideal for risk averse investors in anticipation of negative shocks to the market. Overall bitcoin has a place on the financial markets and in portfolio management as it can be classified as something in between gold and the American dollar on a scale from pure medium of exchange advantages to pure store of value advantages

A New Perspective on Financial Markets - What Cryptocurrencies and Blockchain Technology can Teach us About Market Microstructure

The applications of blockchain technology have grown rapidly in recent years. Governments, central banks, and companies are developing blockchain based projects to improve efficiency and remain competitive. In addition, the markets for cryptocurrencies have grown exponentially. It is therefore increasingly important to understand the market structure of these markets and understand how the design of blockchains affects the costs and usability. Such insights make it possible to gain new perspectives on financial issues, as aspects of their structures differ from traditional equity markets. First, I show that the intraday variation in liquidity on cryptocurrency exchanges resembles that of foreign exchange markets and that bitcoin is highly invenstible. Second, I give a new perspective on the discussion on the optimal tick size. I use the market setting on a cryptocurrency exchange, where the tick sizes are close to zero to show that small tick sizes are detrimental to market quality. I show that a dynamic tick size relative to the price of the asset may be more optimal than the "one tick size fits all" approach commonly found in stock exchanges. Third, I isolate and quantify market frictions and show that a long clearing and settlement time lowers the activities of arbitrageurs and leads to larger price differences. The results are relevant for central banks as they develop digital currencies to increase transaction speed. Lastly, I show that the bitcoin blockchain design inflates transaction volumes and adoption levels and obfuscates the transaction costs. This dissertation shows that the markets for cryptocurrencies can function as useful laboratories to provide new perspectives on long lasting finance issues. The findings have wide implications for users, companies, governments, banks, and regulators

Hedging Capabilities of Bitcoin. Is it the virtual gold?

This paper sets out to explore the hedging capabilities of bitcoin by applying the asymmetric GARCH methodology used in investigation of gold. The results show that bitcoin can clearly be used as a hedge against stocks in the Financial Times Stock Exchange Index. Additionally bitcoin can be used as a hedge against the American dollar in the short-term. Bitcoin thereby possess some of the same hedging abilities as gold and can be included in the variety of tools available to market analysts to hedge market specific risk

A Service of zbw Leibniz-Informationszentrum Wirtschaft Leibniz Information Centre for Economics -Attribution 4.0 International (CC BY 4.0) On the Return-volatility Relationship in the Bitcoin Market Around the Price Crash of 2013

Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in Abstract The authors examine the relation between price returns and volatility changes in the Bitcoin market using a daily database denominated in various currencies. The results for the entire period provide no evidence of an asymmetric return-volatility relation in the Bitcoin market. They test if there is a difference in the return-volatility relation before and after the price crash of 2013 and show a significant inverse relation between past shocks and volatility before the crash and no significant relation after. This finding shows that, prior to the price crash of December 2013, positive shocks increased the conditional volatility more than negative shocks. This inverted asymmetric reaction of Bitcoin to positive and negative shocks is contrary to what the authors observe in equities. As leverage effect and volatility feedback don't adequately explain this reaction, they propose the safe-haven effect (Baur, Asymmetric volatility in the gold market, 2012). The authors highlight the benefits of adding Bitcoin to a US equity portfolio, especially in the pre-crash period. Robustness analyses show, among others, a negative relation between the US implied volatility index (VIX) and Bitcoin volatility. Those additional analyses further support their findings and provide useful information for economic actors who are interested in adding Bitcoin to their equity portfolios or are curious about the capabilities of Bitcoin as a financial asset. JEL G11 G1