Passively collected mobile device location (PCMDL) data contains abundant travel behavior information to support travel demand analysis. Compared to traditional travel surveys, PCMDL data have larger spatial, temporal and population coverage while lack of ground truth information. This study proposes a framework to identify trip ends and impute travel modes from the PCMDL data. The proposed framework firstly identify trip ends using the Spatio-temporal Density-based Spatial Clustering of Applications with Noise (ST-DBSCAN) algorithm. Then three types of features are extracted for each trip to impute travel modes using machine learning methods. A PCMDL dataset with ground truth information is used to calibrate and validate the proposed framework, resulting in 95% accuracy in identifying trip ends and 93% accuracy in imputing five travel modes using the Random Forest (RF) classifier. The proposed framework is then applied to two large-scale PCMDL datasets, covering Maryland and the entire U.S. The mode share results are compared against travel surveys at different geographic levels.