The Limitations of Deep Learning Methods in Realistic Adversarial Settings

Simple item page
dc.contributor.advisorDumitras, Tudoren_US
dc.contributor.authorKaya, Yigitcanen_US
dc.contributor.departmentComputer Scienceen_US
dc.contributor.publisherDigital Repository at the University of Marylanden_US
dc.contributor.publisherUniversity of Maryland (College Park, Md.)en_US
dc.date.accessioned2023-10-07T05:44:47Z
dc.date.available2023-10-07T05:44:47Z
dc.date.issued2023en_US
dc.description.abstractThe study of adversarial examples has evolved from a niche phenomenon to a well-established branch of machine learning (ML). In the conventional view of an adversarial attack, the adversary takes an input sample, e.g., an image of a dog, and applies a deliberate transformation to this input, e.g., a rotation. This then causes the victim model to abruptly change its prediction, e.g., the rotated image is classified as a cat. Most prior work has adapted this view across different applications and provided powerful attack algorithms as well as defensive strategies to improve robustness. The progress in this domain has been influential for both research and practice and it has produced a perception of better security. Yet, security literature tells us that adversaries often do not follow a specific threat model and adversarial pressure can exist in unprecedented ways. In this dissertation, I will start from the threats studied in security literature to highlight the limitations of the conventional view and extend it to capture realistic adversarial scenarios. First, I will discuss how adversaries can pursue goals other than hurting the predictive performance of the victim. In particular, an adversary can wield adversarial examples to perform denial-of-service against emerging ML systems that rely on input-adaptiveness for efficient predictions. Our attack algorithm, DeepSloth, can transform the inputs to offset the computational benefits of these systems. Moreover, an existing conventional defense is ineffective against DeepSloth and poses a trade-off between efficiency and security. Second, I will show how the conventional view leads to a false sense of security for anomalous input detection methods. These methods build modern statistical tools around deep neural networks and have shown to be successful in detecting conventional adversarial examples. As a general-purpose analogue of blending attacks in security literature, we introduce the Statistical Indistinguishability Attack (SIA). SIA bypasses a range of published detection methods by producing anomalous samples that are statistically similar to normal samples. Third, and finally, I will focus on malware detection with ML, a domain where adversaries gain leverage over ML naturally without deliberately perturbing inputs like in the conventional view. Security vendors often rely on ML for automating malware detection due to the large volume of new malware. A standard approach for detection is collecting runtime behaviors of programs in controlled environments (sandboxes) and feeding them to an ML model. I have first observed that a model trained using this approach performs poorly when it is deployed on program behaviors from realistic, uncontrolled environments, which gives malware authors an advantage in causing harm. We attribute this deterioration to distribution shift and investigate possible improvements by adapting modern ML techniques, such as distributionally robust optimization. Overall, my dissertation work has reinforced the importance of considering comprehensive threat models and applications with well-documented adversaries for properly assessing the security risks of ML.en_US
dc.identifierhttps://doi.org/10.13016/dspace/9fyt-k9dh
dc.identifier.urihttp://hdl.handle.net/1903/30868
dc.language.isoenen_US
dc.subject.pqcontrolledComputer scienceen_US
dc.subject.pqcontrolledArtificial intelligenceen_US
dc.subject.pquncontrolledAdversarial examplesen_US
dc.subject.pquncontrolledComputer securityen_US
dc.subject.pquncontrolledDeep learningen_US
dc.subject.pquncontrolledMachine learningen_US
dc.subject.pquncontrolledMalware detectionen_US
dc.titleThe Limitations of Deep Learning Methods in Realistic Adversarial Settingsen_US
dc.typeDissertationen_US

Files

Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
Kaya_umd_0117E_23725.pdf
Size:
8.8 MB
Format:
Adobe Portable Document Format
Download
(RESTRICTED ACCESS)

Collections

UMD Theses and Dissertations
Computer Science Theses and Dissertations