SoK:Prudent Evaluation Practices for Fuzzing

Fuzzing has proven to be a highly effective approach to uncover software bugs over the past decade. After AFL popularized the groundbreaking concept of lightweight coverage feedback, the field of fuzzing has seen a vast amount of scientific work proposing new techniques, improving methodological aspects of existing strategies, or porting existing methods to new domains. All such work must demonstrate its merit by showing its applicability to a problem, measuring its performance, and often showing its superiority over existing works in a thorough, empirical evaluation. Yet, fuzzing is highly sensitive to its target, environment, and circumstances, e.g., randomness in the testing process. After all, relying on randomness is one of the core principles of fuzzing, governing many aspects of a fuzzer's behavior. Combined with the often highly difficult to control environment, the reproducibility of experiments is a crucial concern and requires a prudent evaluation setup. To address these threats to validity, several works, most notably Evaluating Fuzz Testing by Klees et al., have outlined how a carefully designed evaluation setup should be implemented, but it remains unknown to what extent their recommendations have been adopted in practice. In this work, we systematically analyze the evaluation of 150 fuzzing papers published at the top venues between 2018 and 2023. We study how existing guidelines are implemented and observe potential shortcomings and pitfalls. We find a surprising disregard of the existing guidelines regarding statistical tests and systematic errors in fuzzing evaluations. For example, when investigating reported bugs, we find that the search for vulnerabilities in real-world software leads to authors requesting and receiving CVEs of questionable quality. Extending our literature analysis to the practical domain, we attempt to reproduce claims of eight fuzzing papers. These case studies allow us to assess the practical reproducibility of fuzzing research and identify archetypal pitfalls in the evaluation design. Unfortunately, our reproduced results reveal several deficiencies in the studied papers, and we are unable to fully support and reproduce the respective claims. To help the field of fuzzing move toward a scientifically reproducible evaluation strategy, we propose updated guidelines for conducting a fuzzing evaluation that future work should follow

MHC class I–specific antibody binding to nonhematopoietic cells drives complement activation to induce transfusion-related acute lung injury in mice

In a manner partially independent of activating Fcγ receptors, antibody-mediated production of complement component C5a and recruitment of macrophages elicit transfusion-related acute lung injury in mice

Multidifferential study of identified charged hadron distributions in ZZ-tagged jets in proton-proton collisions at s=\sqrt{s}=13 TeV

Jet fragmentation functions are measured for the first time in proton-proton collisions for charged pions, kaons, and protons within jets recoiling against a $Z$ boson. The charged-hadron distributions are studied longitudinally and transversely to the jet direction for jets with transverse momentum 20 $< p_{\textrm{T}} < 100$ GeV and in the pseudorapidity range $2.5 < \eta < 4$. The data sample was collected with the LHCb experiment at a center-of-mass energy of 13 TeV, corresponding to an integrated luminosity of 1.64 fb$^{-1}$. Triple differential distributions as a function of the hadron longitudinal momentum fraction, hadron transverse momentum, and jet transverse momentum are also measured for the first time. This helps constrain transverse-momentum-dependent fragmentation functions. Differences in the shapes and magnitudes of the measured distributions for the different hadron species provide insights into the hadronization process for jets predominantly initiated by light quarks.Comment: All figures and tables, along with machine-readable versions and any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2022-013.html (LHCb public pages

Study of the B−→Λc+Λˉc−K−B^{-} \to \Lambda_{c}^{+} \bar{\Lambda}_{c}^{-} K^{-} decay

The decay $B^{-} \to \Lambda_{c}^{+} \bar{\Lambda}_{c}^{-} K^{-}$ is studied in proton-proton collisions at a center-of-mass energy of $\sqrt{s}=13$ TeV using data corresponding to an integrated luminosity of 5 $\mathrm{fb}^{-1}$ collected by the LHCb experiment. In the $\Lambda_{c}^+ K^{-}$ system, the $\Xi_{c}(2930)^{0}$ state observed at the BaBar and Belle experiments is resolved into two narrower states, $\Xi_{c}(2923)^{0}$ and $\Xi_{c}(2939)^{0}$, whose masses and widths are measured to be $$m(\Xi_{c}(2923)^{0}) = 2924.5 \pm 0.4 \pm 1.1 \,\mathrm{MeV}, \\ m(\Xi_{c}(2939)^{0}) = 2938.5 \pm 0.9 \pm 2.3 \,\mathrm{MeV}, \\ \Gamma(\Xi_{c}(2923)^{0}) = \phantom{000}4.8 \pm 0.9 \pm 1.5 \,\mathrm{MeV},\\ \Gamma(\Xi_{c}(2939)^{0}) = \phantom{00}11.0 \pm 1.9 \pm 7.5 \,\mathrm{MeV},$$ where the first uncertainties are statistical and the second systematic. The results are consistent with a previous LHCb measurement using a prompt $\Lambda_{c}^{+} K^{-}$ sample. Evidence of a new $\Xi_{c}(2880)^{0}$ state is found with a local significance of $3.8\,\sigma$, whose mass and width are measured to be $2881.8 \pm 3.1 \pm 8.5\,\mathrm{MeV}$ and $12.4 \pm 5.3 \pm 5.8 \,\mathrm{MeV}$, respectively. In addition, evidence of a new decay mode $\Xi_{c}(2790)^{0} \to \Lambda_{c}^{+} K^{-}$ is found with a significance of $3.7\,\sigma$. The relative branching fraction of $B^{-} \to \Lambda_{c}^{+} \bar{\Lambda}_{c}^{-} K^{-}$ with respect to the $B^{-} \to D^{+} D^{-} K^{-}$ decay is measured to be $2.36 \pm 0.11 \pm 0.22 \pm 0.25$, where the first uncertainty is statistical, the second systematic and the third originates from the branching fractions of charm hadron decays.Comment: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2022-028.html (LHCb public pages

Measurement of the ratios of branching fractions R(D∗)\mathcal{R}(D^{*}) and R(D0)\mathcal{R}(D^{0})

The ratios of branching fractions $\mathcal{R}(D^{*})\equiv\mathcal{B}(\bar{B}\to D^{*}\tau^{-}\bar{\nu}_{\tau})/\mathcal{B}(\bar{B}\to D^{*}\mu^{-}\bar{\nu}_{\mu})$ and $\mathcal{R}(D^{0})\equiv\mathcal{B}(B^{-}\to D^{0}\tau^{-}\bar{\nu}_{\tau})/\mathcal{B}(B^{-}\to D^{0}\mu^{-}\bar{\nu}_{\mu})$ are measured, assuming isospin symmetry, using a sample of proton-proton collision data corresponding to 3.0 fb${ }^{-1}$ of integrated luminosity recorded by the LHCb experiment during 2011 and 2012. The tau lepton is identified in the decay mode $\tau^{-}\to\mu^{-}\nu_{\tau}\bar{\nu}_{\mu}$. The measured values are $\mathcal{R}(D^{*})=0.281\pm0.018\pm0.024$ and $\mathcal{R}(D^{0})=0.441\pm0.060\pm0.066$, where the first uncertainty is statistical and the second is systematic. The correlation between these measurements is $\rho=-0.43$. Results are consistent with the current average of these quantities and are at a combined 1.9 standard deviations from the predictions based on lepton flavor universality in the Standard Model.Comment: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2022-039.html (LHCb public pages

Model potential for the description of metal/organic interface states

We present an analytical one-dimensional model potential for the description of electronic interface states that form at the interface between a metal surface and flat-lying adlayers of π-conjugated organic molecules. The model utilizes graphene as a universal representation of these organic adlayers. It predicts the energy position of the interface state as well as the overlap of its wave function with the bulk metal without free fitting parameters. We show that the energy of the interface state depends systematically on the bond distance between the carbon backbone of the adayers and the metal. The general applicability and robustness of the model is demonstrated by a comparison of the calculated energies with numerous experimental results for a number of flat-lying organic molecules on different closed-packed metal surfaces that cover a large range of bond distances.We gratefully acknowledge funding by the Deutsche Forschungsgemeinschaft through SFB 1083 Structure and Dynamics of Internal Interfaces, Project B6. F. S. acknowledges support by the CSIC through the I-Link 1065 program.Peer Reviewe

Fuzztruction: Using Fault Injection-based Fuzzing to Leverage Implicit Domain Knowledge

Today’s digital communication relies on complex protocols and specifications for exchanging structured messages and data. Communication naturally involves two endpoints: One generating data and one consuming it. Traditional fuzz testing approaches replace one endpoint, the generator, with a fuzzer and rapidly test many mutated inputs on the target program under test. While this fully automated approach works well for loosely structured formats, this does not hold for highly structured formats, especially those that go through complex transformations such as compression or encryption. In this work, we propose a novel perspective on generating inputs in highly complex formats without relying on heavy-weight program analysis techniques, coarse-grained grammar approximation, or a human domain expert. Instead of mutating the inputs for a target program, we inject faults into the data generation program so that this data is almost of the expected format. Such data bypasses the initial parsing stages in the consumer program and exercises deeper program states, where it triggers more interesting program behavior. To realize this concept, we propose a set of compile-time and run-time analyses to mutate the generator in a targeted manner, so that it remains intact and produces semi-valid outputs that satisfy the constraints of the complex format. We have implemented this approach in a prototype called Fuzztruction and show that it outperforms the state-of-the-art fuzzers AFL++, SymCC, and Weizz. Fuzztruction finds significantly more coverage than existing methods, especially on targets that use cryptographic primitives. During our evaluation, Fuzztruction uncovered 151 unique crashes (after automated deduplication). So far, we manually triaged and reported 27 bugs that have been acknowledged and 4 CVEs were assigne

Drone Security and the Mysterious Case of DJI's DroneID

Consumer drones enable high-class aerial video photography, promise to reform the logistics industry, and are already used for humanitarian rescue operations and during armed conflicts. Contrasting their widespread adoption and high popularity, the low entry barrier for air mobility - a traditionally heavily regulated sector - poses many risks to safety, security, and privacy. Malicious parties could, for example, (mis-)use drones for surveillance, transportation of illegal goods, or cause economic damage by intruding the closed airspace over airports. To prevent harm, drone manufacturers employ several countermeasures to enforce safe and secure use of drones, e.g., they impose software limits regarding speed and altitude, or use geofencing to implement no-fly zones around airports or prisons. Complementing traditional countermeasures, drones from the market leader DJI implement a protocol called DroneID, which is designed to transmit the position of both the drone and its operator to authorized entities such as law enforcement or operators of critical infrastructures. In this paper, we analyze security and privacy claims for drones, focusing on the leading manufacturer DJI with a market share of 94%. We first systemize the drone attack surface and investigate an attacker capable of eavesdropping on the drone's over-the-air data traffic. Based on reverse engineering of DJI firmware, we design and implement a decoder for DJI's proprietary tracking protocol DroneID using only cheap COTS hardware. We show that the transmitted data is not encrypted, but accessible to anyone, compromising the drone operator's privacy. Second, we conduct a comprehensive analysis of drone security: Using a combination of reverse engineering, a novel fuzzing approach tailored to DJI's communication protocol, and hardware analysis, we uncover several critical flaws in drone firmware that allow attackers to gain elevated privileges on two different DJI drones and their remote control. Such root access paves the way to disable or bypass countermeasures and abuse drones. These vulnerabilities have the potential to be triggered remotely, causing the drone to crash mid-flight