Linde, Maximilian

Meijer, Rob R.

Neumann, Marvin

Niessen, Susan

Tendeiro, Jorge N.

English

University of Groningen

   University of GroningenThe effects of autonomy in mechanical prediction on perceived stakeholder perceptionsNeumann, Marvin; Niessen, Susan; Linde, Maximilian; Tendeiro, Jorge N.; Meijer, Rob R.IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2022Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Neumann, M., Niessen, S., Linde, M., Tendeiro, J. N., & Meijer, R. R. (2022). The effects of autonomy inmechanical prediction on perceived stakeholder perceptions. Poster session presented at 37th AnnualConference of the Society for Industrial and Organizational Psychology, Seattle, Washington, UnitedStates.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 20-11-2022The Effects of Autonomy in Mechanical Prediction on Perceived Stakeholder PerceptionsMarvin Neumann*, A. Susan M. Niessen, Maximilian Linde, Jorge N. Tendeiro, & Rob R. Meijer*Department of Psychometrics and Statistics, Faculty of the Behavioral and Social Sciences, University of Groningen, The Netherlands. m.neumann@rug.nl1. Introduction• In personnel selection, applicant information can be combined in two ways [1]:– Holistic prediction (HP); information is subjectively combined in the mind– Mechanical prediction (MP); information is combined with an algorithm• MP is more valid than HP [2], but rarely used due to the "threat of technologicalunemployment" (TOTU) [3]; decision makers fear negative stakeholder evaluations.• We expected autonomy-enhancing prediction procedures (holistically adjusting predic-tions from a prescribed algorithm and self-designing the algorithm) to result in:– Improved decision-makers’ perceived stakeholder perceptions, compared to usinga prescribed algorithm.– Higher predictive validity, compared to pure holistic prediction.2. Method• Participants: N = 269 MTurk participants with experience in making hiring decisions.• Procedure: Participants made 40 performance predictions based on applicants’ GMAand conscientiousness test scores, and an unstructured interview rating.• Between-subjects design: The approach to making performance predictions wasvaried in four conditions.– Holistic: Predict performance holistically.– Holistic adjustment: Holistically adjust the prediction from a prescribed algorithm.– Self-designed algorithm: Determine predictor importance by assigning percentageweights to predictor scores.– Prescribed algorithm: Strictly use predictions from a prescribed algorithm.3.1 Results - Perceived stakeholder perceptions• Retaining autonomy (vs. using a prescribed algorithm) generally improved decision-makers’ perceived stakeholder perceptions:Variable Cohen’s d 95% HDIPersonal control 0.96 [0.67, 1.25]Locus of causality 0.38 [0.10, 0.66]Perceived competence 0.77 [0.49, 1.06]TOTU 0.14 [-0.14, 0.42]Use intentions 0.28 [0.00, 0.56]Figure 1: Mean scores per outcome measure and condition. Error bars are 95% CIs.3.2 Results - Predictive validity• Autonomy-enhancing prediction procedures resulted in more valid predictions thanholistic predictions (d = 1.10, 95% HDI [0.82, 1.37]).Condition r̄ 95% CIHolistic .16 [.11, .21]Holistic adjustment .28 [.25, .30]Self-designed algorithm .31 [.30, .32]Prescribed algorithm .36 -Figure 2: Mean predictive validity per condition with jittered individual correlations. Error bars are 95% CIs.4. Discussion and Conclusion• Holistically adjusting predictions from a prescribed algorithm or self-designing an al-gorithm can improve algorithm use and validity in practice• Wondering how to design your own algorithm? Check out this app!5. References[1] Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence (pp.83-128). University of Minnesota Press. https://doi.org/doi:10.1037/11281-008[2] Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data combina-tion in selection and admissions decisions: A meta-analysis. Journal of Applied Psychology, 98 (6), 1060-1072.https://doi.org/10.1037/a0034156[3] Nolan, K. P., Carter, N. T., & Dalal, D. K. (2016). Threat of technological unemployment: Are hiring managersdiscounted for using standardized employee selection practices? Personnel Assessment and Decisions, 2 (1),30–47. https://doi.org/10.25035/pad.2016.004.

The effects of autonomy in mechanical prediction on perceived stakeholder perceptions

ARTS repository - University of Groningen

   University of GroningenThe effects of autonomy in mechanical prediction on perceived stakeholder perceptionsNeumann, Marvin; Niessen, Susan; Linde, Maximilian; Tendeiro, Jorge N.; Meijer, Rob R.IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2022Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Neumann, M., Niessen, S., Linde, M., Tendeiro, J. N., & Meijer, R. R. (2022). The effects of autonomy inmechanical prediction on perceived stakeholder perceptions. Poster session presented at 37th AnnualConference of the Society for Industrial and Organizational Psychology, Seattle, Washington, UnitedStates.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 08-06-2022The Effects of Autonomy in Mechanical Prediction on Perceived Stakeholder PerceptionsMarvin Neumann*, A. Susan M. Niessen, Maximilian Linde, Jorge N. Tendeiro, & Rob R. Meijer*Department of Psychometrics and Statistics, Faculty of the Behavioral and Social Sciences, University of Groningen, The Netherlands. m.neumann@rug.nl1. Introduction• In personnel selection, applicant information can be combined in two ways [1]:– Holistic prediction (HP); information is subjectively combined in the mind– Mechanical prediction (MP); information is combined with an algorithm• MP is more valid than HP [2], but rarely used due to the "threat of technologicalunemployment" (TOTU) [3]; decision makers fear negative stakeholder evaluations.• We expected autonomy-enhancing prediction procedures (holistically adjusting predic-tions from a prescribed algorithm and self-designing the algorithm) to result in:– Improved decision-makers’ perceived stakeholder perceptions, compared to usinga prescribed algorithm.– Higher predictive validity, compared to pure holistic prediction.2. Method• Participants: N = 269 MTurk participants with experience in making hiring decisions.• Procedure: Participants made 40 performance predictions based on applicants’ GMAand conscientiousness test scores, and an unstructured interview rating.• Between-subjects design: The approach to making performance predictions wasvaried in four conditions.– Holistic: Predict performance holistically.– Holistic adjustment: Holistically adjust the prediction from a prescribed algorithm.– Self-designed algorithm: Determine predictor importance by assigning percentageweights to predictor scores.– Prescribed algorithm: Strictly use predictions from a prescribed algorithm.3.1 Results - Perceived stakeholder perceptions• Retaining autonomy (vs. using a prescribed algorithm) generally improved decision-makers’ perceived stakeholder perceptions:Variable Cohen’s d 95% HDIPersonal control 0.96 [0.67, 1.25]Locus of causality 0.38 [0.10, 0.66]Perceived competence 0.77 [0.49, 1.06]TOTU 0.14 [-0.14, 0.42]Use intentions 0.28 [0.00, 0.56]Figure 1: Mean scores per outcome measure and condition. Error bars are 95% CIs.3.2 Results - Predictive validity• Autonomy-enhancing prediction procedures resulted in more valid predictions thanholistic predictions (d = 1.10, 95% HDI [0.82, 1.37]).Condition r̄ 95% CIHolistic .16 [.11, .21]Holistic adjustment .28 [.25, .30]Self-designed algorithm .31 [.30, .32]Prescribed algorithm .36 -Figure 2: Mean predictive validity per condition with jittered individual correlations. Error bars are 95% CIs.4. Discussion and Conclusion• Holistically adjusting predictions from a prescribed algorithm or self-designing an al-gorithm can improve algorithm use and validity in practice• Wondering how to design your own algorithm? Check out this app!5. References[1] Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence (pp.83-128). University of Minnesota Press. https://doi.org/doi:10.1037/11281-008[2] Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data combina-tion in selection and admissions decisions: A meta-analysis. Journal of Applied Psychology, 98 (6), 1060-1072.https://doi.org/10.1037/a0034156[3] Nolan, K. P., Carter, N. T., & Dalal, D. K. (2016). Threat of technological unemployment: Are hiring managersdiscounted for using standardized employee selection practices? Personnel Assessment and Decisions, 2 (1),30–47. https://doi.org/10.25035/pad.2016.004.

https://pure.rug.nl/ws/files/213939118/SIOP_22_poster_Stakeholder_perceptions_in_mechanical_prediction.pdf

The effects of autonomy in mechanical prediction on perceived stakeholder perceptions

Abstract

Similar works

Full text

Available Versions

University of Groningen

ARTS repository - University of Groningen