2 research outputs found
Measuring Accuracy of Automated Parsing and Categorization Tools and Processes in Digital Investigations
This work presents a method for the measurement of the accuracy of evidential
artifact extraction and categorization tasks in digital forensic
investigations. Instead of focusing on the measurement of accuracy and errors
in the functions of digital forensic tools, this work proposes the application
of information retrieval measurement techniques that allow the incorporation of
errors introduced by tools and analysis processes. This method uses a `gold
standard' that is the collection of evidential objects determined by a digital
investigator from suspect data with an unknown ground truth. This work proposes
that the accuracy of tools and investigation processes can be evaluated
compared to the derived gold standard using common precision and recall values.
Two example case studies are presented showing the measurement of the accuracy
of automated analysis tools as compared to an in-depth analysis by an expert.
It is shown that such measurement can allow investigators to determine changes
in accuracy of their processes over time, and determine if such a change is
caused by their tools or knowledge.Comment: 17 pages, 2 appendices, 1 figure, 5th International Conference on
Digital Forensics and Cyber Crime; Digital Forensics and Cyber Crime, pp.
147-169, 201