When we start the school year, we can have a lot of data goals in mind. Perhaps we have big ideas about using data portfolios or data walls. It can be an exciting and generative time while we still have energy. However, we really have only one reason to use data in the first few months:
The priority for data in the first few months of school, is to get to know your students.
Our goal is to learn who they are as people, and who they are in their learning. Here are 3 strategies to get to know your students this fall:
Use a questionnaire to learn about your students
I ask my students just a few questions on the first day: What do you want me to call you in class? What pronouns do you prefer? What has been your relationship with learning my subject in the past? What is a hope you have for this year? Is there anything else you’d like to share?
Asking names and pronouns signals your room is safe and inclusive. Asking about past experiences can be an enlightening view into how a student approaches your class. I collect my responses digitally with a Microsoft Form, but my goal with surveys is to keep them short for easy completion.
My Microsoft forms survey
Review past file data
You may work on a team where transitions from year-to-year happen as part of your structures. If so, great. If not, you may be like me in needing to review files. I look in two places: the last report card and standardized testing data.
It takes me about a day or so to review the report cards of 80+ students. I’m particularly looking at the narrative pieces for feedback on what each learner is like. This might be the most tedious process, but super valuable.
Collect diagnostic data
Even with the last report card, we may not know if gaps were formed or if there are new areas for extension and growth. Therefore, I collect formal diagnostic data. However, this doesn’t mean that I start every year with a pretest – in fact, I start every year with collaborative tasks and mindset activities.
Instead, I employ a gradual diagnostic. As part of students’ homework or in class tasks, I will ask them to complete 3 questions online targeting specific standards from the previous grade and the current grade. I target known “tricky” standards or use priority standards set by our school.
After a few weeks, I have a good idea of what my students are comfortable on and where we might need to remediate.
After this series of diagnostics, I can see we may want to spend time on 6.EE.1
Again, our priority is to learn how to teach the students we have in front of us. They are different from last year and different from the next year. It’s our job as educators to use the information at our disposal to figure out what they need as learners.
In our last post, Chris Smith wrote about the data pipeline: the process by which we clean, wrangle, and pre-process data so that we can visualize and make proper inferences. Today we give you another necessary step when telling data stories: Know your data.
In my role as a Data Coach, I spend a majority of my time helping people to understand what a certain metric means and what are plausible inferences to make from the data. What I find most often is one of two scenarios: (1) people have little idea how a certain metric might be calculated, and (2) people are overconfident in their ability to make inferences from data and often jump to conclusions. So here are three steps for you and your school to know your data better to avoid those two pitfalls.
Know The Math Behind The Data
Within reason, you must now a little bit of math to understand your data. You need to know how the metric is calculated so avoid making false assertions. Let me tell you a tale of my own experience:
I’ve worked at schools that take annual standardized tests. These tests have a “Growth Projection” metric in the fall for each student and then when we take the test again, we see ow many of our students met this projection. Here was my approximate data over 7 years:
Are seeing what I’m seeing? My first 5 years I hovered in the high 40, low 50 percent range before climbing to the 60-70 percent range in 2016-2017. The first inference to commonly make is that only around 50% of my students are meeting growthtargets. While a perfect 100% is not a fair expectation, this is far too low, right? A previous administrator thought so and I had to set goals to improve my scores. Do you side with that inference and action plan?
Not so fast. If you don’t know how those projections are calculated, then how can you infer what percentage is appropriate? It turns out in this test, that each students’ growth projection is calculated from an average of a huge data set of similar students. You know the thing about averages, right? They’re in the middle – meaning approximately half your students will be above that number and half below. The target percentage to achieve on this metric is therefore to have around 50% of your students meet projections. I’d go as far as to say that when my data started to stretch into the 70% range, that now we have a real problem: I might be teaching to the test to get metrics that high or something in the curriculum might be too standardized test-oriented.
Make sure you know (within reason) how your data is calculated to make accurate inferences.
Know The Limitations Of Your Data
This strategy takes humility. People who like data often point to how numbers are more concrete or a hard science as compared to individual perceptions. However, its so often that the second we get data, we start making inferences well beyond the scope of the data itself.
At its core, data only measures what it measured. For example: if your students take a history quiz, you may use the results to have ideas about how much your students understand about the unit or topic. However, that is not what the data measured, that is meaning we have added to the data, or inference. Concretely, all the data 100% says is how that group of students did at answering those specific questions on that day. Any further conclusions about what students know, or what was taught well, is an inference made by us, not the data.
All data is limited this way. The SAT only measures how a student does on the SAT and colleges have used that data to infer potential college success. The grades we assign in class only measure the sets of data we have collected and we infer indirectly that it shows proficiency. This is why I’m a big proponent of building multiple sets of data for high-stakes conclusions. It’s fine to use a single quiz to recommend a student do review assignment or the teacher re-tool a lesson. It’s one small piece of data, but the stakes are low. However, placement tests to determine course placements? All we’ve measured is how a student does on one test on one day on those particular questions, but we’ve determined potential years of course options.
Humility to understand limitations and willingness to do the work to triangulate data is a necessity in a positive data culture.
Appoint Experts Who Will Do the Research When Necessary
Large data sets are constantly changing and evolving. The NWEA MAP Growth tests have over 150 potential variables/metrics for each student and reports from College Board and IBO are similarly large. It’s impossible to study all such variables, and in my experience with representatives from those organizations, it can be hard for their customer facing representatives to know how those variables are measured. No shade being thrown here, it’s just a lot.
At my school, as the Data Coach, one of my functions is to read documentation. Like all coaches, staying on top of our role requires reading research. For me, sometimes that means studying standardized testing documentation and learning how metrics are calculated. With so much amazing data to dig into, someone needs to be able to clarify how the data is gathered, measured, and how it can be used to generate theories. If your school has the means, designate a point-person to become a specialist in understanding specific data sets. Not a general “data person” who is “good at numbers,” but someone who will become an expert in standardized tests, a person who will become an expert in your database data sets, etc. This person needs to be willing to follow my previous two recommendations of knowing the math and knowing the limitations.
In 2020, in the shadow of terrible events, the trauma faced by students who are not white, male, cis-gender, etc. gained important voices. Let’s be clear, there’s a lot of work to be done, and we wrote about some of the work can take place right now in your school based on data and research.
It’s been hard reading stories about Education lately. The focus seems to be on “Learning Loss,” a nebulous term which may refer to lost class hours or lower test scores due to the pandemic. Simultaneously, I’m reading stories about educators leaving the field in droves – overworked, feeling a lack of competency, being asked to sacrifice even more. My heart hurts. What if instead of this deficit thinking that targets test scores and class time (two things we have little control over), we switched to focusing on our humanity? We do have control over the ways we interact with, and support one another.
It’s no secret that I believe access to data can transform learning. Data illuminate needs and areas for growth as well as successes and areas of celebration. For the past decade I have been using data to track my students academic achievement, their standardized test growth, their trajectories into upper grades, and more. It has helped me shape and improve my practice. (more…)
