Perspectives on Large Language Models for Relevance Judgment

When asked, current large language models (LLMs) like ChatGPT claim that they can assist us with relevance judgments. Many researchers think this would not lead to credible IR research. In this perspective paper, we discuss possible ways for LLMs to assist human experts along with concerns and issues that arise. We devise a human-machine collaboration spectrum that allows categorizing different relevance judgment strategies, based on how much the human relies on the machine. For the extreme point of "fully automated assessment", we further include a pilot experiment on whether LLM-based relevance judgments correlate with judgments from trained human assessors. We conclude the paper by providing two opposing perspectives - for and against the use of LLMs for automatic relevance judgments - and a compromise perspective, informed by our analyses of the literature, our preliminary experimental evidence, and our experience as IR researchers. We hope to start a constructive discussion within the community to avoid a stale-mate during review, where work is dammed if is uses LLMs for evaluation and dammed if it doesn't

Emergency Services Workforce 2030: Changing landscape literature review

The Changing Landscape Literature Review collates a high-level evidence base around seven major themes in the changing landscape (i.e., the external environment) that fire, emergency service, and rural land management agencies operate in, and which will shape workforce planning and capability requirements over the next decade. It is an output of the Workforce 2030 project and is one of two literature reviews that summarise the research base underpinning a high-level integrative report of emerging workforce challenges and opportunities, Emergency Services Workforce 2030. Workforce 2030 aimed to highlight major trends and developments likely to impact the future workforces of emergency service organisations, and their potential implications. The starting point for the project was a question: What can research from outside the sphere of emergency management add to our knowledge of wider trends and developments likely to shape the future emergency services workforce, and their implications? The seven themes included in the Changing Landscape Literature Review are: 1) demographic changes, 2) changing nature of work, 3) changes in volunteering, 4) physical technology, 5) digital technology, 6) shifting expectations, and changing risk. A second, accompanying literature review, the Changing Work Literature Review, focuses on another nine themes related to emergency service organisation’s internal workforce management approaches and working environments

Finding Optimal 2-Packing Sets on Arbitrary Graphs at Scale

A 2-packing set for an undirected graph $G=(V,E)$ is a subset $\mathcal{S} \subset V$ such that any two vertices $v_1,v_2 \in \mathcal{S}$ have no common neighbors. Finding a 2-packing set of maximum cardinality is a NP-hard problem. We develop a new approach to solve this problem on arbitrary graphs using its close relation to the independent set problem. Thereby, our algorithm red2pack uses new data reduction rules specific to the 2-packing set problem as well as a graph transformation. Our experiments show that we outperform the state-of-the-art for arbitrary graphs with respect to solution quality and also are able to compute solutions multiple orders of magnitude faster than previously possible. For example, we are able to solve 63% of our graphs to optimality in less than a second while the competitor for arbitrary graphs can only solve 5% of the graphs in the data set to optimality even with a 10 hour time limit. Moreover, our approach can solve a wide range of large instances that have previously been unsolved

Can diagnosis-based capital allocation facilitate more appropriate, sustainable and innovative acute care?

Australians value access to public hospitals with technologically-appropriate clinical care. However, the Australian system of capital funding for public hospitals is not appropriate, effective, equitable, clinically-responsive, patient-centred, evidence-based or sustainable. A new model to effectively fund patient access to efficient public hospitals was developed from international evidence, Australian standards, clinical guidelines and expert clinical interviews. Capital was costed by patient diagnosis group to enable comprehensive funding for public hospital clinical care, for the first time

Dependency-Aware Software Requirements Selection using Fuzzy Graphs and Integer Programming

Software requirements selection aims to find an optimal subset of the requirements with the highest value while respecting the project constraints. But the value of a requirement may depend on the presence or absence of other requirements in the optimal subset. Such Value Dependencies, however, are imprecise and hard to capture. In this paper, we propose a method based on integer programming and fuzzy graphs to account for value dependencies and their imprecision in software requirements selection. The proposed method, referred to as Dependency-Aware Software Requirements Selection (DARS), is comprised of three components: (i) an automated technique for the identification of value dependencies from user preferences, (ii) a modeling technique based on fuzzy graphs that allows for capturing the imprecision of value dependencies, and (iii) an Integer Linear Programming (ILP) model that takes into account user preferences and value dependencies identified from those preferences to reduce the risk of value loss in software projects. Our work is verified by studying a real-world software project. The results show that our proposed method reduces the value loss in software projects and is scalable to large requirement sets.Comment: arXiv admin note: text overlap with arXiv:2003.0480

Discovering Smeaton : people, trade and finance, a study of imperialism and its heritage

Using Anderson’s Mill in the Victorian goldfield township of Smeaton as a case study, this thesis examines how the process of colonisation can be understood through the study of local history in the context of its imperial heritage. It also examines the transition of Crown sovereignty to colonial sovereignty in Victoria during the second half of the nineteenth century. This thesis explores the proposition that by discovering the history of Smeaton through the era of John Anderson, it is possible to trace how the Victorian gold rushes and the imperial legacy shaped the emerging Australian nation and constructions of identity during the era when the doctrine of terra nullius prevailed. The thesis sets out the argument that the history of Anderson's Mill and the township of Smeaton provides an original perspective into the Australian colonisation process, particularly in the colony of Victorian. It also contends that the Victorian gold rushes altered the balance of an imperial power struggle that influenced the colonial foundations of notions of sovereignty. This was underpinned by finance and trade, which were the driving forces that transferred the notion of empire through to local colonial communities. What emerges in this thesis is a critical narrative of colonial Victoria, which highlights the particular dynamic tension that was present between the colony and the imperial centre through a sharp focus on Anderson’s Mill and Smeaton, its associated townships.Doctor of Philosoph

Construction and statistical analysis of an industry wide ground control database of mechanical roof extensometer data from underground coal mine gateroads

The abundance of quantitative roof convergence measurements systematically recorded in Australian mines is distinctly different to all other major coal-producing countries, including the USA, South Africa, and China. The literature review identified that no industry-wide research using this data source had yet been completed. Nevertheless, roof failure still posed severe safety risks to coal mine workers and commercial risks to corporations. Based on these facts, this thesis had two objectives: (i) collate an extensive, novel, industry-wide ground control database fixated on tell-tale measurements and (ii) complete statistical analyses on this novel database to investigate ground response in various geotechnical environments. Real-world displacement measurements from >80,000 locations at 22 Australian longwall mines were collected. Python scripting was applied to transform this data into a query-friendly SQL database. Eventually, a final database of 5528 case histories including Total Displacement, CMRR, Depth, PRSUP, Stress, Roadway Type and Roof Type was analysed. Various statistics were applied to identify relationships between the independent variables. Results indicated that relationships between many different parameters were complex and mostly highly non-linear. Testing of many classic ML algorithms identified the artificial neural network most suitable. A MLP NN was then refined through a standard process of hyperparameter optimisation to arrive at Model 1. Model 1 was deemed to perform at an acceptable level, achieving evaluation metrics of 80.8% accuracy, 64.4% precision and 61.6% recall. Another five ANN model variants were created, with six models iteratively constructed and evaluated, with varying results. Due to the black-box nature of NN, further insight into how the parameters interacted was sought and found with SHapley Additive exPlanations (SHAP values), which is a common form of XAI (explainable artificial intelligence). The results from all ANN and SHAP analyses are delivered in the thesis, together with insights, applications, strengths, and limitations of each methodology. Finally, several recommendations are made for applying the burgeoning field of ML and XAI in future ground control research related to coal mine roadway stability

The effective and ethical development of artificial intelligence: An opportunity to improve our wellbeing

This project has been supported by the Australian Government through the Australian Research Council (project number CS170100008); the Department of Industry, Innovation and Science; and the Department of Prime Minister and Cabinet. ACOLA collaborates with the Australian Academy of Health and Medical Sciences and the New Zealand Royal Society Te Apārangi to deliver the interdisciplinary Horizon Scanning reports to government. The aims of the project which produced this report are: 1. Examine the transformative role that artificial intelligence may play in different sectors of the economy, including the opportunities, risks and challenges that advancement presents. 2. Examine the ethical, legal and social considerations and frameworks required to enable and support broad development and uptake of artificial intelligence. 3. Assess the future education, skills and infrastructure requirements to manage workforce transition and support thriving and internationally competitive artificial intelligence industries