Towards Systemic Evaluation

Problems of conventional evaluation models can be understood as an impoverished ‘conversation’ between realities (of non-linearity, indeterminate attributes, and ever-changing context), and models of evaluating such realities. Meanwhile, ideas of systems thinking and complexity science—grouped here under the acronym STCS—struggle to gain currency in the big ‘E’ world of institutionalized evaluation. Four evaluation practitioners familiar with evaluation tools associated with STCS offer perspectives on issues regarding mainstream uptake of STCS in the big ‘E’ world. The perspectives collectively suggest three features of practicing systemic evaluation: (i) developing value in conversing between bounded values (evaluations) and unbounded reality (evaluand), with humility; (ii) developing response-ability with evaluand stakeholders based on reflexivity, with empathy; and (iii) developing adaptive rather than mere contingent use(fulness) of STCS ‘tools’ as part of evaluation praxis, with inevitable fallibility and an orientation towards bricolage (adaptive use). The features hint towards systemic evaluation as core to a reconfigured notion of developmental evaluation

What If? The Art of Scenario Thinking for Nonprofits

Gives an overview of scenario thinking customized for a nonprofit audience. Outlines the basic phases of scenario development, and provides examples and advice for putting the process into practice. Includes an annotated bibliography of select readings

Tensions of Data-Driven Reflection: A Case Study of Real-Time Emotional Biosensing

Biosensing displays, increasingly enrolled in emotional reflection, promise authoritative insight by presenting users’ emotions as discrete categories. Rather than machines interpreting emotions, we sought to explore an alternative with emotional biosensing displays in which users formed their own interpretations and felt comfortable critiquing the display. So, we designed, implemented, and deployed, as a technology probe, an emotional biosensory display: Ripple is a shirt whose pattern changes color responding to the wearer’s skin conductance, which is associated with excitement. 17 participants wore Ripple over 2 days of daily life. While some participants appreciated the ‘physical connection’ Ripple provided between body and emotion, for others Ripple fostered insecurities about ‘how much’ feeling they had. Despite our design intentions, we found participants rarely questioned the display’s relation to their feelings. Using biopolitics to speculate on Ripple’s surprising authority, we highlight ethical stakes of biosensory representations for sense of self and ways of feeling

mFish Alpha Pilot: Building a Roadmap for Effective Mobile Technology to Sustain Fisheries and Improve Fisher Livelihoods.

In June 2014 at the Our Ocean Conference in Washington, DC, United States Secretary of State John Kerry announced the ambitious goal of ending overfishing by 2020. To support that goal, the Secretary's Office of Global Partnerships launched mFish, a public-private partnership to harness the power of mobile technology to improve fisher livelihoods and increase the sustainability of fisheries around the world. The US Department of State provided a grant to 50in10 to create a pilot of mFish that would allow for the identification of behaviors and incentives that might drive more fishers to adopt novel technology. In May 2015 50in10 and Future of Fish designed a pilot to evaluate how to improve adoption of a new mobile technology platform aimed at improving fisheries data capture and fisher livelihoods. Full report

An introduction to STRIKE : STRuctured Interpretation of the Knowledge Environment

Knowledge forms a critical part of the income generation of the system and the complex environment in which actors participate in the creation of knowledge assets merits robust, eclectic consideration. STRIKE - STRuctured Interpretation of the Knowledge Environment affords an unobtrusive and systematic framework to observe, record, evaluate and articulate concrete and abstract elements of a setting, across internal and external dimensions. Inter-relationships between actor and environment are preserved. STRIKE is supported by underlying techniques to enrich data and enhance the authenticity of its representation. Adoption of photography and videography tools provides illustrative and interpretive benefits and facilitates researcher reflexivity. This structured approach to data analysis and evaluation mitigates criticisms of methodological rigour in observational research and affords standardisation potential, germane for application in a verification or longitudinal capacity. Advancing exploratory validation studies, the method is employed to evaluate the knowledge environments of two enterprises in the UK creative sector. These occupy a critical role in fostering entrepreneurial innovation alongside participant self-efficacy. Access Space in Sheffield and the Bristol Hackspace are committed to open software, open knowledge and open participation; sharing peer learning, creativity and socio-technical aims to address broadly similar community needs. Drawing on Wittgenstein’s Picture Theory of Meaning, the knowledge management perspective is abstracted from the STRIKE assessment. It is argued that the tiered analytical approach which considers a breadth of dimensions enhances representation and interpretation of the knowledge environment and presents a diagnostic and prescriptive capability to actualise change. The paper concludes by evaluating framework effectiveness, findings application and future direction

"Challenge Current Practice and Assumptions! Make waves!!” : What Works Scotland Collaborative Learning Event 23 & 24 February 2016 Queens Hotel, Perth

No abstract available

Kairotic Situations: A Spatial Rethinking of the Burkean Parlor in the Writing Center

The Burkean parlor has been integrated into the lore of writing centers, showcasing how writing centers have both conversational and collaborative elements. However, the ease for students to enter into the academic conversation is not as simple as this metaphor suggests. To rethink this concept, kairos, or the opportune moment, must be considered. This article will investigate kairos as spatial and how that conceptualization can deepen the Burkean parlor and the conversations within it. Breaking down the Burkean parlor into three stages—questions, metacognition, and choices—can benefit the practicality of the tutoring session. Kairos complicates each of these three points of the student writer’s integration into the conversation. The creation of kairos depends upon the student and tutor being mindful of these conscious and unconscious interactions and understanding how to most effectively disrupt the spatial boundaries of the tutoring session. Connecting kairos into the Burkean parlor metaphor differentiates the perspective of the tutoring session, encouraging both student and tutor to become more aware of the spatiality of tutoring and to redefine these boundaries.University Writing Cente

A return to Teacherbot:Rethinking the development of educational technology at the University of Edinburgh

In the market discourses of technological disruption, higher education institutions have routinely been positioned in deficit models and of anachronistic approaches to teaching at odds with the types of educational futures being presented by commercial organisations. Predominantly, automation technologies in the form of artificial intelligence are being promoted as the future of teaching. In this paper, on the other hand, we explore the prospects for using non-artificial intelligence automated agents in teaching and its impact on the teacher function at the University of Edinburgh. Through engagement with teachers, staff and students at the university, this research has identified use cases for bots, in what spaces they would be situated, and how they would supplement the teacher function. This paper argues that a community-driven approach combined with a sociomaterial conceptualisation can generate a shift from market discourses and to collaborative development of educational technologies