Data literacy is crucial for adolescents to access and navigate data in today’s technology-driven world. Researchers emphasize the need for K-12 students to attain data literacy. However, few available instructions have incorporated validated assessments. Therefore, I developed and implemented the Data literacy Assessment for Middle graders (DLA-M) that can diagnose students’ current stages fairly and support future practices regardless of their language backgrounds. I initiated the study with two research questions: a) How valid is a newly developed assessment to measure middle-grade students’ data literacy? b) How fairly does the new assessment measure data literacy regardless of students’ language backgrounds?A new assessment purported to measure two competencies of data literacy of 6th to 9th graders: a) Interpret data representations and b) Evaluate data and data-based claims. I used the Evidence-Centered Design (ECD) as a methodological framework to increase the validity of the assessment. I followed the five layers of the ECD framework to develop and implement the DLAM. Then I analyzed the data from implementing the assessment and gathered five types of validity evidence for validation. Based on the collected validity evidence, I concluded that the assessment was designed to represent the content domain that is purported to measure. The assessment had internal consistency in measuring data literacy except for nine eliminated items, and the data literacy scores from the overall assessment were reliable as well. Regarding item quality, item discrimination parameters met the quality criteria, but difficulty estimates of some items did not meet the intended design. Empirical cluster analyses revealed two performance levels from the participants. Differential item functioning analyses showed that item discrimination and difficulty were not differentiated between language minority students (LMSs) and their counterparts with the same data literacy level. These results did not reveal the possibility of unfair interpretations and uses of this assessment for LMSs. Lastly, I found significant interaction effects between the DLAM scores and the two variables about students’ English reading proficiency and use of technology. This study delineated how to develop and validate a data literacy assessment that could support students from different linguistic backgrounds. The research also facilitated the application of a data literacy assessment to school settings by scrutinizing and defining target competencies that could benefit adolescents’ data literacy. The findings can inform future research to implement data literacy assessments in broader contexts. This study can serve as a springboard to provide inclusive data literacy assessments for diverse student populations.