A cyber exercise post assessment framework: In Malaysia perspectives

Critical infrastructures are based on complex systems that provide vital services to the nation. The complexities of the interconnected networks, each managed by individual organisations, if not properly secured, could offer vulnerabilities that threaten other organisations’ systems that depend on their services. This thesis argues that the awareness of interdependencies among critical sectors needs to be increased. Managing and securing critical infrastructure is not isolated responsibility of a government or an individual organisation. There is a need for a strong collaboration among critical service providers of public and private organisations in protecting critical information infrastructure. Cyber exercises have been incorporated in national cyber security strategies as part of critical information infrastructure protection. However, organising a cyber exercise involved multi sectors is challenging due to the diversity of participants’ background, working environments and incidents response policies. How well the lessons learned from the cyber exercise and how it can be transferred to the participating organisations is still a looming question. In order to understand the implications of cyber exercises on what participants have learnt and how it benefits participants’ organisation, a Cyber Exercise Post Assessment (CEPA) framework was proposed in this research. The CEPA framework consists of two parts. The first part aims to investigate the lessons learnt by participants from a cyber exercise using the four levels of the Kirkpatrick Training Model to identify their perceptions on reaction, learning, behaviour and results of the exercise. The second part investigates the Organisation Cyber Resilience (OCR) of participating sectors. The framework was used to study the impact of the cyber exercise called X Maya in Malaysia. Data collected through interviews with X Maya 5 participants were coded and categorised based on four levels according to the Kirkpatrick Training Model, while online surveys distributed to ten Critical National Information Infrastructure (CNII) sectors participated in the exercise. The survey used the C-Suite Executive Checklist developed by World Economic Forum in 2012. To ensure the suitability of the tool used to investigate the OCR, a reliability test conducted on the survey items showed high internal consistency results. Finally, individual OCR scores were used to develop the OCR Maturity Model to provide the organisation cyber resilience perspectives of the ten CNII sectors

Designing Digital Forensics Challenges for Multinational Cyber Defense Exercises

Töös püütakse kujundada ja hinnata digitaalse kohtuekspertiisi väljakutset, mida kasutada rahvusvahelisel küberkaitse õppusel. Eesmärk on fokusseerida põhioskustele, mida üks riiklik organisatsioon oma digitaalse kohtuekspertiisi ekspertidelt vajab ja disainida ning integreerida tehnilisi ülesandeid, mis adekvaatselt testivad neid oskusi suuremal küberkaitse õppusel. See töö kasutab NATO Locked Shields küberkaitse õppust test-näitena, mille jaoks väitekirja autor liitus digitaalse kohtuekspertiisi disainimeeskonnaga, NATO Cyber Defense Centre of Excellence juures, kui nad kavandasid ja rakendasid kolme-päevast digitaalse kohtuekspertiisi väljakutset. See lõputöö kehtestab rea tehnilisi ja protseduurilisi oskuseid, mida riiklikud organisatsioonid vajavad oma ekspertidelt, määrab viisid, kuidas testida neid oskusi ja arendab stsenaariumipõhist digitaalse kohtuekspertiisi väljakutset. Kasutades vahetult saadud tähelepanekuid, osaleja tagasisidet ja väljakutse tulemusi, et hinnata väljakutse efektiivsust, lõputöös leitakse, et stsenaarium testis piisavalt enamus oskusi õigel raskustasemel ja vajab parendamist ajastuses ning aruandlusstandardites. Lõpetuseks uuritakse erinevaid viise, kuidas parendada valitud meetodeid ja ülesandeid tuleviku õppusteks.This thesis seeks to design and evaluate a digital forensics challenge for inclusion in a multinational cyber defense exercise. The intent is to narrow down the key skills a state-based organization requires of its digital forensics experts and design and integrate technical tasks that adequately test these skills into a larger cyber defense exercise. It uses the NATO Locked Shields cyber defense exercise as a test case, for which the thesis author joined the digital forensics design team at the NATO Cyber Defense Centre of Excellence in designing and implementing a three day digital forensics challenge. This thesis establishes a series of technical and procedural skills state-based organizations require of their experts, determines ways to test these skills, and develops a scenario-based digital forensics challenge. Using first hand observations, participant feedback, and challenge scores to evaluate the effectiveness of the challenge, it finds that the scenario adequately tested a majority of the skills at the appropriate difficulty level and needs improvement in timing and reporting standards. Finally, it explores ways to improve upon the selected methods and tasks for future exercises

Due diligence in cyberspace: guidelines for international and European cyber policy and cybersecurity policy

Global cyberspace is undergoing fundamental change. There are now frequent references to a "fragmentation of the Internet", but many European and international working groups are also increasingly aware that "a free, open and at the same time secure Internet" is a global public good. However, the political rules adopted for International and European cyber policies and cybersecurity policies will always lag behind technological developments. It is the more important, therefore, to subject these rules to the over-arching norm of due diligence in cyberspace, and to do so on the national, European and international levels. This generates three requirements for Germany's future strategic orientation in cyberspace: European cooperation: integrating national policies into the European framework; inclusiveness: giving different interest groups broad and publicly accessible representation in formulating policies; civilian response: prioritising the civilian component over the military component, particularly in times of peace. However, Germany's major partners are confused as to what goals precisely it is pursuing in cyberspace. It is therefore advisable for Berlin to improve its coordination and communication of responsibilities at the national and EU levels, be it on issues of Internet Governance, the fight against cybercrime, or cyberdefence. (author's abstract

Actas de las VI Jornadas Nacionales (JNIC2021 LIVE)

Estas jornadas se han convertido en un foro de encuentro de los actores más relevantes en el ámbito de la ciberseguridad en España. En ellas, no sólo se presentan algunos de los trabajos científicos punteros en las diversas áreas de ciberseguridad, sino que se presta especial atención a la formación e innovación educativa en materia de ciberseguridad, y también a la conexión con la industria, a través de propuestas de transferencia de tecnología. Tanto es así que, este año se presentan en el Programa de Transferencia algunas modificaciones sobre su funcionamiento y desarrollo que han sido diseñadas con la intención de mejorarlo y hacerlo más valioso para toda la comunidad investigadora en ciberseguridad

Improving and Measuring Learning at Cyber Defence Exercises

Küberõppusi peetakse üheks efektiivseimaks meetodiks erinevate sihtgruppide koolitamisel, see sobib nii (sõjaväelistele) professionaalsetele meeskondadele kui individuaalsetele õpilastele. Samas põhinevad teadmised õppustel saavutatud õpitulemustest peamiselt suulisel infol ja metoodika efektiivsust pole tõestatud. Käesolev töö käsitleb õppimist küberkaitseõppustel ning keskendub õpitulemuste hindamisele. Erinevate õppuste formaatide seast on antud töö aluseks valitud tehnilised küberkaitseõppused, milles on esindatud punaste ja siniste meeskonnad. Töös analüüsitakse kübekaitseõppusi lähtuvalt täiskasvanu õpiteooriatest ja õpitulemuste mõõtmise hetkeolukorda küberkaitseõppuste raamistikus. Õpitulemusi mõõdeti kahel küberkaitseõppusel, Locked Shields ja Crossed Swords. Neist esimene on suurim avalik küberkaitseõppus maailmas peaaegu 900 osalejaga ning peamiseks koolitusgrupiks on siniste meeskonnad. Teine õppus on väiksemahuline punaste meeskonna õppus. Locked Shields ja Crossed Swords on korraldatud NATO küberkaitsekeskuse poolt. Sellised õppused on tehniliselt väga komplekssed ning nii korraldajatele kui osalejatele keerukad. Seetõttu vajavad nii õppuse disain kui õpitulemuste mõõtmine suuremat tähelepanu. Käesolev töö pakub välja uudse ja skaleeritava õpitulemuste mõõtmise metoodika, nn. “5-ajatempli metoodika”. Metoodika hõlmab nii efektiivset tagasisidet (s.h. võrdlusvõimalus) kui õpitulemuste mõõtmist. See võimaldab hinnata meeskondade tegevustulemust, ja väidab, et tulemuste muutus ajas näitab ka õpitulemusi. Ajatempleid saab koguda nii traditsiooniliste meetoditega (nt. intervjuud, vaatlused ja küsimustikud), aga ka potentsiaalselt mitte-intrusiivselt võrgulogidest (nt. pcap’id). Metoodika aitab parandada tagasisidet, tuvastada õppuse disaininõrkusi ja näidata kübekaitseõppuste õpiväärtust. Crossed Swords õppuse hindamisel keskenduti eelkõige osalejatele (punaste meeskond) kohese tagasiside andmisele nende tegevuste kohta. Käesolev töö annab olulise panuse küberkaitseõppuste õpitulemuste hindamise teoreetiliste ja praktiliste aluste kohta ning pakub välja praktilised soovitused õpikogemuse parendamiseks.Cyber security exercises are believed to be the most effective training for all training audiences from top (military) professional teams to individual students. However, evidence of learning outcomes for those exercises are often anecdotal and not validated. This thesis takes a fresh look at learning in Cyber Defence Exercises (CDXs) and focuses on measuring learning outcomes. As such exercises come in a variety of formats, this thesis focuses on technical CDXs with Red and Blue teaming elements. The review of adult learning theories and current state of learning measurement in CDXs context are presented. The learning measurements are performed at two CDXs: Locked Shields and Crossed Swords. First one is the largest unclassified live-fire CDX in the world with nearly 900 participants (with Blue teams as main training audience). Second one is a small scale exercise designed to train Red teams. Both exercises are organised by the NATO Cooperative Cyber Defence Centre of Excellence (CCD COE). Such top-end CDXs are highly complex, which makes it hard for organisers and participants to handle. Therefore, both learning design and measurement need careful consideration. This work proposes a novel and scalable learning measurement methodology, called the “5-timestamp methodology”. This method aims at accommodating for both—effective feedback (including benchmarking opportunity) and learning measurement. The method is capable of assessing team performance, and argues that changes in performance over time equal learning. The timestamps can either be collected using traditional methods, such as interviews, observations and surveys, but also potentially be obtained non-obtrusively from raw network traces (such as pcaps). The method enhances the feedback loop, allows identifying learning design flaws, and provides solid evidence of learning value for CDXs. Crossed Swords measurement focused on providing the training audience (Red team) with instant feedback about their actions to ensure effective learning. This work contributes to theoretical foundations and in practical terms by providing practical recommendations readily applicable for improvement of learning experience in CDXs

Cyber Security of Critical Infrastructures

Critical infrastructures are vital assets for public safety, economic welfare, and the national security of countries. The vulnerabilities of critical infrastructures have increased with the widespread use of information technologies. As Critical National Infrastructures are becoming more vulnerable to cyber-attacks, their protection becomes a significant issue for organizations as well as nations. The risks to continued operations, from failing to upgrade aging infrastructure or not meeting mandated regulatory regimes, are considered highly significant, given the demonstrable impact of such circumstances. Due to the rapid increase of sophisticated cyber threats targeting critical infrastructures with significant destructive effects, the cybersecurity of critical infrastructures has become an agenda item for academics, practitioners, and policy makers. A holistic view which covers technical, policy, human, and behavioural aspects is essential to handle cyber security of critical infrastructures effectively. Moreover, the ability to attribute crimes to criminals is a vital element of avoiding impunity in cyberspace. In this book, both research and practical aspects of cyber security considerations in critical infrastructures are presented. Aligned with the interdisciplinary nature of cyber security, authors from academia, government, and industry have contributed 13 chapters. The issues that are discussed and analysed include cybersecurity training, maturity assessment frameworks, malware analysis techniques, ransomware attacks, security solutions for industrial control systems, and privacy preservation methods