Quantum photonics provides a powerful toolbox with vast applications ranging from quantum simulation, photonic information processing, all optical universal quantum computation, secure quantum internet as well as quantum enhanced sensing. Many of these applications require the integration of several complex optical elements and material systems which pose a challenge to scalability. It is essential to integrate linear and non-linear photonics on a chip to tackle this issue leading to more compact, high bandwidth devices. In this thesis we demonstrate a pathway to achieving several components in the quantum photonic toolbox on the same integrated photonic platform. We focus particularly on two of the more nontrivial components, a single photon source and an integrated quantum light-matter interface. We address the problem of a scalable, chip integrated, fast single photon source, by using atomically thin layers of 2D materials interfaced with plasmonic waveguides. We further embark on the challenge of creating a new material system by integrating rare earth ions with the emerging commercial platform of thin film lithium niobate on insulator. Rare earth ions have found widespread use in classical and quantum information processing. However, these are traditionally doped in bulk crystals which hinder their scalability. We demonstrate an integrated photonic interface for rare earth ions in thin film lithium niobate that preserves the optical and coherence properties of the ions. This combination of rare earth ions with the chip-scale active interface of thin film lithium niobate opens a plethora of opportunities for compact optoelectronic devices. As an immediate application we demonstrate an integrated optical quantum memory with a rare earth atomic ensemble in the thin film. The new light matter interface in thin film lithium niobate acts as a key enabler in an already rich optical platform representing a significant advancement in the field of integrated quantum photonics.