Episodic memory is a cornerstone ability that allows one to recall past events and the context in which they occur. Many different tasks have been used to assess the development of episodic memory during early childhood. Previous longitudinal work on individual tasks has noted accelerated changes from approximately 5 to 7 years, suggesting non-linear changes in memory ability during early childhood. However, the extent to which tasks relate to one another and are indicative of the latent construct of episodic memory is not known. Further, improvements in memory are thought to relate to underlying changes occurring in the functionally distinct subfields of the hippocampus (i.e., CA2-4/dentate gyrus (DG), CA1, and Subiculum) during this developmental period. This study examined changes in episodic memory ability, hippocampal subfield volume, and the relation between changes in episodic memory and volume of hippocampal subfields during early childhood (4 to 8 years) using longitudinal data and a structural equation modeling framework. Results suggest that episodic memory ability improves substantially during this period, with consistent improvements between 4 to 8 years. Further, there are robust increases in subiculum, CA1, and CA2-4/DG volume between 5 to 6 years of age. Finally, within this sample, there were relations between the development of hippocampal subfields and improvements on a single source memory task commonly used to assess episodic memory. Interestingly, this relation was most robust between subiculum and source memory. Overall, these results highlight the ability to use laboratory tasks to characterize developmental changes in episodic memory, highlight 5- to 6-years as a period of developmental change in hippocampal subfields, and further support a role of the hippocampus in supporting episodic memory.