In this thesis we propose that the addition of a satellite overlay to large or dense wireless networks will result in improvement in application performance and network reliability, and also enable efficient security solutions that are well-suited for wireless nodes with limited resources. We term the combined network as a hybrid wireless network. Through analysis, network modeling and simulation, we quantify the improvement in end-to-end performance in such networks, compared to flat wireless networks.

We also propose a new analytical method for modeling and estimating the performance of hybrid wireless networks. We create a loss network model for hybrid networks using the hierarchical reduced loss network model, adapted for packet-switched networks. Applying a fixed point approximation method on the set of relations modeling the hierarchical loss network, we derive a solution that converges to a fixed point for the parameter set. We analyze the sensitivity of the performance metric to variations in the network parameters by applying Automatic Differentiation to the performance model. We thus develop a method for parameter optimization and sensitivity analysis of protocols for designing hybrid networks.

We investigate how the satellite overlay can help to implement better solutions for secure group communications in hybrid wireless networks. We propose a source authentication protocol for multicast communications that makes intelligent use of the satellite overlay, by modifying and extending TESLA certificates. We also propose a probabilistic non-repudiation technique that uses the satellite as a proxy node. We describe how the authentication protocol can be integrated with a topology-aware hierarchical multicast routing protocol to design a secure multicast routing protocol that is robust to active attacks.

Lastly, we examine how the end-to-end delay is adversely affected when IP Security protocol (IPSEC) and Secure Socket Layer protocol (SSL) are applied to unicast communications in hybrid networks. For network-layer security with low delay, we propose the use of the Layered IPSEC protocol, with a modified Internet Key Exchange protocol. For secure web browsing with low delay, we propose the Dual-mode SSL protocol. We present simulation results to quantify the performance improvement with our proposed protocols, compared to the traditional solutions.