Evaluating HCI research beyond usability

Evaluating research artefacts is an important step to showcase the validity of a chosen approach. The CHI community has developed and agreed upon a large variety of evaluation methods for HCI research; however, sometimes those methods are not applicable or not sufficient. This is especially the case when the contribution lies within the context of the application area, such as for research in sustainable HCI, HCI for development, or design fiction and futures studies. In this SIG, we invite the CHI community to share their insights from projects that encountered problems in evaluating research and aim to discuss solutions for this difficult topic. We invite researchers from all areas of HCI research who are interested to engage in a debate of issues in the process of validating research artefacts

A circular commons for digital devices: tools and services in ereuse.org

Circular economies are particularly relevant in the context of digital devices or electric and electronic equipment (EEE). Many digital devices built using scarce and potentially toxic materials have a too-short life, instead of being repaired or reused. In addition, informal recycling of electronics in the developed and developing world has emerged as a new global environmental concern. We describe the dimensions of the problem, the challenge to move to a circular economy, and the ecology for digital devices as well as how this depends on the traceability of devices and cooperation among all stakeholders locally and globally. Moreover we examine the need for support mechanisms to facilitate, standardise, and reduce the transaction cost of the processes and increase their added value. We present eReuse.org, a set of open-source tools, procedures, open data, and services organised as a common-pool resource (CPR) to reach the circular economy of electronics through promoting reuse and ensuring traceability until recycling. Further, eReuse.org envisions empowering and engaging people around the world to create local communities that bootstrap electronic reuse and to support the development of a globally recognised reuse quality and traceability standard.Peer ReviewedPostprint (author's final draft

The Revolution of Mobile Phone-Enabled Services for Agricultural Development (m-Agri Services) in Africa: The Challenges for Sustainability

The provision of information through mobile phone-enabled agricultural information services (m-Agri services) has the potential to revolutionise agriculture and significantly improve smallholder farmers’ livelihoods in Africa. Globally, the benefits of m-Agri services include facilitating farmers’ access to financial services and sourcing agricultural information about input use, practices, and market prices. There are very few published literature sources that focus on the potential benefits of m-Agri services in Africa and none of which explore their sustainability. This study, therefore, explores the evolution, provision, and sustainability of these m-Agri services in Africa. An overview of the current landscape of m-Agri services in Africa is provided and this illustrates how varied these services are in design, content, and quality. Key findings from the exploratory literature review reveal that services are highly likely to fail to achieve their intended purpose or be abandoned when implementers ignore the literacy, skills, culture, and demands of the target users. This study recommends that, to enhance the sustainability of m-Agri services, the implementers need to design the services with the users involved, carefully analyse, and understand the target environment, and design for scale and a long-term purpose. While privacy and security of users need to be ensured, the reuse or improvement of existing initiatives should be explored, and projects need to be data-driven and maintained as open source. Thus, the study concludes that policymakers can support the long-term benefit of m-Agri services by ensuring favourable policies for both users and implementers

A Proposed Framework for the Comprehensive Scalability Assessment of ICTD Projects

open access bookThe scalability of ICTD projects is an imperative topic that has been neglected in the field. Little has been written or investigated about the assessment of the scalability of ICTD projects due to factors, such as the lack of proven business models for success, the high failure rate of projects, undefined aspects of assessment, and the small number of projects that have scaled. Therefore, there are various factors that should be taken into consideration to alleviate the challenges experienced in the process of scaling up. This research study is guided by an investigation into how can the scalability of an ICTD project be assessed using a comprehensive evaluation approach that considers the impact and potential sustainability of the project. This research study proposes a Comprehensive Scalability Assessment Framework (CSAF), using systems theory and amplification theory to guide the theoretical analysis and empirical investigation. A theorizing approach is used to develop the framework, which is structured around three components: assessment guidelines and proceeding domains of evaluation; four scalability themes (stakeholder composition, models feasibility, resources sustainability and resilience) and judge scalability

Shifting the maturity needle of ICT for Sustainability

The ubiquity of ICT means the potential of ICT4S covers a broad range of sustainability topics and application domains. However, ICT4S research can be ill positioned with regard to the complexity of transforming society in such a way that people and environmental ecologies can coexist in a sustainable system. The danger is that ICT4S becomes partitioned into a small subset of sustainability and using a limited set of the levers at our disposal. Grounded in the Mann-Bates maturity scale for sustainability this paper performs an analysis of the ICT4S conference corpus to measure how mature the research is in our field with regard to sustainability. Based on this analysis we identify areas in which the ICT4S community can begin to shift the maturity of research in order to promote sustainable futures. By applying the Transformation Mindset our article demonstrates through a series of illustrative how ICT4S can apply this mindset to shift ICT4S research towards more sustainable trajectories. This is an essential first step in taking stock, highlighting shortcomings and identifying opportunities in ICT for sustainability

Evaluation Beyond Usability

The evaluation of research artefacts is an important step to validate research contributions. Sub-disciplines of HCI often pursue primary goals other than usability, such as Sustainable HCI (SHCI), HCI for development, or health and wellbeing. For such disciplines, established evaluation methods are not always appropriate or sufficient, and new conventions for identifying, discussing, and justifying suitable evaluation methods need to be established. In this paper, we revisit the purpose and goals of evaluation in HCI and SHCI, and elicit five key elements that can provide guidance to identifying evaluation methods for SHCI research. Our essay is meant as a starting point for discussing current and improving future evaluation practice in SHCI; we also believe it holds value for other subdisciplines in HCI that encounter similar challenges while evaluating their research

Preliminary thoughts on a taxonomy of value for sustainable computing

In anticipation of the various and imminent limits to global consumption-based growth, some quarters of the technology industry have begun to consider the novel use of computing to prevent, postpone, alleviate, or recover from a crisis in what could be called “sustainable computing.” There is, however, a great danger that certain inclinations of the technology sector will undermine the very goals of sustainability and resilience that such efforts claim to seek.This paper outlines a preliminary taxonomy of value for sustainable computing projects. The taxonomy suggests a three-dimensional classification of projects in terms of their impact on, intention toward, and effort required for sustainability. By making explicit an evaluative framework by which computing might and might not contribute to sustainability goals, the hope is that future work will tend toward projects that are genuinely helpful

Developing a Taxonomy for Sustainable ICT : An exploratory study of the feasibility of a taxonomy for sustainable ICT

Information and Communication Technology (ICT) can have a role to play in transitioning into a sustainable society but it is important to consider that implementation of an ICT-based solution not always lead to sustainability. Kentaro Toyama (2015) has proposed a framework, what he refers to as “Preliminary Thoughts on a Taxonomy of Value for Sustainable Computing”, to be used for classification of sustainable ICT-projects. A working taxonomy for ICT-projects could provide an explicit evaluative framework to be used for evaluating whether a given project might or might not contribute to sustainability goals. The taxonomy includes three dimensions to consider when classifying ICT-projects: Impact - On sustainability Intention - Towards sustainability Effort  - For achieving Impact This thesis picks up where Toyama left off with the purpose of exploring if the taxonomy is a feasible approach for addressing ICT and sustainability. This is done by first adding a Secondary Level to the taxonomy with the goal of making the classification process more accurate, but more importantly for the purpose of this thesis, enable a more detailed analysis of the overall feasibility of the taxonomy. Four interviews are conducted with representatives from ICT-projects addressing sustainability. As a result of the interviews four problems are identified that have implications for the feasibility of the taxonomy and that need to be considered in any future and further attempts to operationalize it