In most wireless sensor networks (WSNs), sensor nodes generate data packets and send them to the base station (BS) by multi-hop routing paths because of their limited energy and transmission range. The insider packet drop attacks refer to a set of attacks where compromised nodes intentionally drop packets. It is challenging to accurately detect such attacks because packets may also be dropped due to collision, congestion, or other network problems.

Trust mechanism is a promising approach to identify inside packet drop attackers. In such an approach, each node will monitor its neighbor's packet forwarding behavior and use this observation to measure the trustworthiness of its neighbors. Once a neighbor's trust value falls below a threshold, it will be considered as an attacker by the monitoring node and excluded from the routing paths so further damage to the network will not be made.

In this dissertation, we analyze the limitation of the state-of-the-art trust mechanisms and propose several enhancement techniques to better defend against insider packet drop attacks in WSNs.

First, we observe that inside attackers can easily defeat the current trust mechanisms and even if they are caught, normally a lot of damage has already been made to the network. We believe this is caused by current trust models' inefficiency in distinguishing attacking behaviors and normal network transmission failures. We demonstrate that the phenomenon of consecutive packet drops is one fundamental difference between attackers and good sensor nodes and build a hybrid trust model based on it to improve the detection speed and accuracy of current trust models.

Second, trust mechanisms give false alarms when they mis-categorize good nodes as attackers. Aggressive mechanisms like our hybrid approach designed to catch attackers as early as possible normally have high false alarm rate. Removing these nodes from routing paths may significantly reduce the performance of the network. We propose a novel false alarm detection and recovery mechanism that can recover the falsely detected good nodes.

Next, we show that more intelligent packet drop attackers can launch advanced attacks without being detected by introducing a selective forwarding-based denial-of-service attack that drops only packets from specific victim nodes. We develop effective detection and prevention methods against such attack.

We have implemented all the methods we have proposed and conducted extensive simulations with the OPNET network simulator to validate their effectiveness.