One of the more difficult analyses that schools need to perform is comparing internal assessments to external assessments. In statistical terms, we are judging the validity of our classroom assessments. I’m not a big fan of jargon, but just so we are all on the same page, validity refers to whether or not your assessment is measuring what you expect it to measure.
By comparing our internal assessments to our externals (IB or AP scores, SATs, MAP, etc.) we’re actually making the assumption that these external measures are already valid. We are using them as a benchmark for validity (which has its own philosophical complications).
Now – first you need to look at your data and determine if it is quantitative or categorical data. Most standardized test scores are quantitative – they span integer values or decimal values. But school grade books could be percent based (again, this would be quantitative) or standards based – which is categorical.
If you are comparing quantitative to quantitative, 8th-grade linear regression can give you a quick snapshot of your validity; but if you throw in a categorical variable, we now need to do masters level categorical data analysis and logistic regression.
So here’s a quick experiment:
A school has 4 teachers who use MAP scores and standards based grade books. To change the standards grades across 4 standards – they average them and get a decimal value.
ex. A student who scores 3, 3, 2, 2 would get an overall score of 2.5
We now have quantitative vs quantitative and can use much easier techniques! Below is a scatter plot and linear regression lines where each color represents a teacher.
So now it looks like 3 teachers have similar correlations between grades and MAP scores while the 4th has a steeper trend. It’s hard to say at this point why that is, but it should be a catalyst for partner teachers to begin a discussion. We could look at correlation coefficients (Pearson’s) or compute residuals to see how good each line fits the data.
In all cases, there is a positive correlation between grades and MAP. Is it strong enough? That’s for your school and your teams to decide. But this team can pat themselves on the back for three-quarters aligning so well.
Disclaimer: The data and graphics used on this site are simulated re-creations intended to protect the privacy of the original data sources.