While wireless communication has dramatically changed the way people work and interact, the wireless era continues to be plagued by insufficient security. Without necessary countermeasures, even a few attackers can break down the whole network. On the other hand, attacker detection can be extremely challenging in realistic scenarios because misbehavior may also be caused by various other factors, such as noise and uncertainty, and perfect monitoring is either impossible to achieve or too expensive to afford. In this dissertation we have investigated how to secure wireless ad hoc networks against insider attacks in noisy and hostile environments,

based only on local and imperfect monitoring.

In traditional ad hoc network applications, nodes usually belong to the same authority and pursue some common goals. The inherent cooperative nature of such networks makes them extremely vulnerable to insider attacks. For example, by dropping other nodes' packets and/or injecting an overwhelming amount of traffic, insider attackers can easily break down the whole network. In this dissertation we have first studied how to secure such ad hoc networks against insider attacks under noise and imperfect monitoring. Besides devising a set of efficient monitoring and attacker detection mechanisms to defend against routing disruption and injecting traffic attacks, we have also formally analyzed the dynamic interactions between good nodes and attackers under a game theoretic framework, where both the optimal defending strategies and the maximum damage that can be caused by insider attackers have been derived.

In many civilian applications, nodes in ad hoc networks tend to act selfishly. Stimulating selfish nodes to act cooperatively poses one key research challenge, especially in realistic contexts. In this dissertation we have also investigated how to design attack-resistant cooperation mechanisms for such networks. We have first designed an attack-resistant cooperation stimulation strategy for mobile ad hoc networks, then formally analyzed the issue of secure cooperation in ad hoc networks under a game theoretic framework. Finally, we have derived a set of reputation-based attack-resistant and cheat-proof cooperation strategies for such ad hoc networks that can work well in noisy and hostile environments under imperfect monitoring.