Phosphorus (P) loss from agricultural fields to surface water represents a major environmental challenge in agricultural nutrient management. To reduce P loading, areas where both P source and transport conditions are present must be identified and appropriate management practices implemented to reduce the source or break transport connectivity. The Maryland P Site Index (MD-PSI) was modified from a multiplicative structure to a component structure and renamed University of Maryland Phosphorus Management Tool (UM-PMT). In the UM-PMT, each component is the product of source, transport, and management factors specific to a P loss pathway. Our objectives were to evaluate the UM-PMT for accuracy, investigate soil conditions in ditch-drained agricultural systems, compare different methods for degree of P saturation (DPS) calculation, and compare numerical and categorical final scores of the multiple versions of the Maryland P loss risk indices. Agronomic soil samples were collected from fields across Maryland, and analyzed for P, aluminum (Al), and iron (Fe) concentration using multiple extractions, soil texture was determined, and degree of P saturation (DPS) was calculated using five methods. Deep soil samples were collected and analyzed similarly from three sites on Maryland's eastern shore. A poor relationship was identified between UM-PMT and modeled P loss data (R2=0.09), but the relationship improved with modifications to UM-PMT calculation (R2=0.97), which resulted in UM-PMT Version 2 (UM-PMT v.2). Soil Fe concentration was responsible for a large proportion of DPS at one sample location on the Eastern Shore, demonstrated through poor correlation between two methods for DPS calculation, including and excluding Fe concentration. Numerical differences existed between different methods for DPS calculation and these translated to differences in UM-PMT final score, particularly in the Lower Shore region. The UM-PMT v.2 categorized more fields as HIGH risk than MD-PSI but less than UM-PMT. Neither version of the UM-PMT was very sensitive to management factor input variables. Evaluation of tools like the UM-PMT for accuracy, sensitivity, and magnitude of change is necessary to understand potential economic and environmental impacts of implementing new indices as nutrient management tools.