Finding the Balance Between Guidance and Independence in Cybersecurity Exercises

Abstract In order to accomplish cyber security tasks, one needs to know how to analyze complex data and when and how to use tools. Many hands-on exercises for cybersecurity courses have been developed to teach these skills. There is a spectrum of ways that these exercises can be taught. On one end of the spectrum are prescriptive exercises, in which students follow step-by-step instructions to run scripted exploits, perform penetration testing, do security audits, etc. On the other end of the spectrum are open-ended exercises and capture-the-flag activities, where little guidance is given on how to proceed. This paper reports on our experience with trying to find a balance between these extremes in the context of one of the suite of cybersecurity exercises that we have developed in the EDURange framework 1 . The particular exercise that we present teaches students about dynamic analysis of binaries using strace. We have found that students are most successful in these exercises when they are given the right amount of prerequisite knowledge and guidance as well as some opportunity to find creative solutions. Our scenarios are specifically designed to develop analysis skills and the security mindset in students and to complement the theoretical aspects of the discipline and develop practical skills

Enhancing Situational Awareness for Tutors of Cybersecurity Capture the Flag Games

Supervised Capture the Flag games represent a popular method of practical hands-on training in cybersecurity education. However, as cybersecurity training sessions are process-oriented, tutors have only a limited insight into what trainees are doing and how they deal with the tasks. From their perspective, it is necessary to have situational awareness, enabling them to identify and react to any issues during a training session as soon as they emerge. We propose a tool designed in collaboration with cybersecurity educators. Based on user requirements, we developed the Progress Visualization Tool, which provides educators with timely feedback through the session. More specifically, the tool informs educators of the training progression, helps identify the students who might struggle with their tasks, and reveals overall deviation from the schedule. We validated the tool through formative and summative qualitative in-lab evaluations. The participants appraised the impact on the training workflow and gave further insights regarding the tool. We discuss the insights and recommendations that arose from the evaluations as they could aid the design of future tools for supporting educators, not only of CTFs but also in other domains.Supervised Capture the Flag games represent a popular method of practical hands-on training in cybersecurity education. However, as cybersecurity training sessions are process-oriented, tutors have only a limited insight into what trainees are doing and how they deal with the tasks. From their perspective, it is necessary to have situational awareness, enabling them to identify and react to any issues during a training session as soon as they emerge. We propose a tool designed in collaboration with cybersecurity educators. Based on user requirements, we developed the Progress Visualization Tool, which provides educators with timely feedback through the session. More specifically, the tool informs educators of the training progression, helps identify the students who might struggle with their tasks, and reveals overall deviation from the schedule. We validated the tool through formative and summative qualitative in-lab evaluations. The participants appraised the impact on the training workflow and gave further insights regarding the tool. We discuss the insights and recommendations that arose from the evaluations as they could aid the design of future tools for supporting educators, not only of CTFs but also in other domains

Reconstructing the progress of digital forensic evidence examination and analysis

Abstract. Many commands and tools are used during the evidence examination and analysis stages of digital forensics. If the need to replicate the exact steps from these stages arises later, doing so without proper documentation can be an arduous task. Thus, this thesis focuses on determining how the story of digital forensics progression could be told. To tell the story, this thesis contributes a three-piece system consisting of an updated version of the data collection tool titled Hardtrace, an Application Programming Interface (API) for summarizing and storing collected data to the cloud, and lastly a visualizer application allowing forensic researchers to visually inspect the steps taken during examination and analysis. To obtain data on digital forensics progression and to test the system, a case study was conducted. The study’s participants had to complete a memory forensics Capture the Flag challenge while using Hardtrace. Collected data from each participant was sent to the cloud API. The system’s ability to reconstruct and detail the progression of participants work was tested by performing visual and statistical analysis on the summarized data. System performance testing was also conducted. The results demonstrated that the presented system was able to detail, through visualization, the steps case study participants took while solving the challenge. Statistical summary analysis provided a large quantity of information on how each participant worked, deepening the understanding gained from just visual analysis. Finally, performance analysis showed that the system is able to summarize and visualize data in seconds. Updates to Hardtrace reduced command execution times significantly, nonetheless, the more system calls a tool or command performs, the more execution time overhead is still added by Hardtrace.Digitaaliforensiikan todisteiden tutkimisen ja analyysin rekonstruointi. Tiivistelmä. Digitaaliforensiikan todisteiden tutkimis- ja analyysivaiheissa käytetään useita komentoja ja työkaluja. Jos nämä vaiheet on myöhemmin toistettava samoin tai tutkijan on kerrottava, miten todisteita käsiteltiin, voi tehtyjen toimenpiteiden muistaminen olla haastavaa ilman kunnollista dokumentaatiota. Täten, tämä tutkielma keskittyy ratkaisemaan miten tutkimis- ja analyysivaiheiden eteneminen voitaisiin kertoa ohjelmallisesti. Tätä tarkoitusta varten tässä tutkielmassa toteutettiin kolmiosainen järjestelmä, jonka osat ovat Hardtrace, ohjelmointirajapinta ja visualisointiohjelma. Hardtrace on jo olemassa oleva datankeräystyökalu, jota tässä työssä päivitetään. Pilveen sijoitetun ohjelmointirajapinnan tehtävä on vastaanottaa ja säilyttää Hardtracen tuottamaa dataa, sekä luoda siitä tiivistelmiä. Visualisointiohjelman avulla forensiikkatutkija pystyy tarkastelemaan visuaalisesti tekemänsä forensiikkatutkimuksen etenemistä. Toteutetun järjestelmän kykyä rekonstruoida digitaaliforensiikan vaiheiden eteneminen testattiin tapaustutkimuksella. Tutkimuksen osallistujat suorittivat muistiforensiikka Capture the Flag -haasteen ja heidän suorituksista kerättiin dataa Hardtracella. Ohjelmointirajapinnan kerätystä datasta tuottamia tiivistelmiä analysoitiin visuaalisesti ja tilastollisesti. Tutkielman tulokset näyttivät, että järjestelmä kykeni kertomaan visualisoinnin keinoin, miten tapaustutkimuksen osallistujat selättivät heille annetun haasteen. Osallistujien suoritusten tilastollinen analyysi tuotti paljon lisätietoa osallistujien toiminnasta. Järjestelmän suorituskyvyn havaittiin olevan hyvä dataa tiivistäessä ja visualisoidessa. Hardtraceen tehdyt päivitykset laskivat komentojen ja työkalujen suoritusaikoja huomattavasti, mutta tästä huolimatta mitä enemmän järjestelmäkutsuja komento tai työkalu käyttää, sitä enemmän Hardtrace suoritusaikaa kasvattaa