Skip to main content

The Myth of the Slipping Math Student?


I've been teaching in Colorado for six years, and there's always been a troubling pattern in our state standardized math scores. As students progress from 3rd to 10th grade, the percentage that score proficient and advanced declines dramatically. Here are the percentages of students scoring proficient and advanced by grade level, averaged over all the years the test has been given (typically 2002-2008):












GradeAvg. % P+A
369
469
561
655
744
843
934
1029


The easiest explanation (and the one I've tended to believe) is that students' abilities are, in fact, slipping as they got older. That would be a good assumption if the test at each grade level was equally difficult. But what if the test questions were, on average (and adjusted for grade level), more difficult as students got older? Is it fair to assume a test with increasingly difficult questions would result in lower scores, even with sophisticated score scaling systems that take question difficulty into account?

Fortunately, the state releases "item maps" that describe the difficulty of each item on every test. Using 4 points for an advanced item, 3 points for a proficient item, 2 points for a partially proficient item, and 1 point for an unsatisfactory item, we can come up with an average difficulty for the CSAP at each grade level. Let's add that column to our table:












GradeAvg. DifficultyAvg. % P+A
32.4369.25
42.4368.5
52.5361.14
62.6955.43
72.9644
83.0442.86
93.1334
102.9628.86


This begs for regression analysis. How strong is the correlation between the difficulty of the questions and the scores?


The correlation is surprisingly strong, and the coefficient of determination (R squared) is 0.88, meaning that the average item difficulty is statistically responsible for 88% of the variance in the test scores. 88%? That's big. Statistics rarely tell the whole story, but 88% raises serious doubts that it's just a matter of slipping math students. Why wouldn't the state want to maintain a steady average difficulty year-to-year? Wouldn't that make year-to-year performance comparisons more reliable?

Comments

Popular posts from this blog

The 15-Year Blogoversary

15 years and 1,213 posts! My first experience with the World Wide Web came in 1995, and by 1997 I had my own web page. The first web authoring tool I remember using was Composer, an HTML editor built into the Netscape Communicator suite. That helped me learn some HTML, and later I used Microsoft Word 97 and then FrontPage 98 and later Macromedia Dreamweaver to design more elaborate pages. Some of my FrontPage-built sites are still on the web. As I learned more about HTML standards and validation I wrote more HTML by hand, but I still wanted a way to make publishing to the web easier.

By 2001 I understood that (a) sites should be updated regularly and (b) FTP'ing sites and pages from my desktop to a server was a bit of a pain. I had heard about some early blogging platforms and chose one, Blogger, to try out. As you can see, I'm still here.
My first post using Blogger came on December 8, 2001. A few months later I paid for Blogger Pro, which offered additional authoring tools, l…

Last.fm and Ten Years of Web 2.0

Ten years ago yesterday I scrobbled my first tracks to last.fm. What's scrobbling? On last.fm, scrobbling refers to automatic music track logging to the internet. For me, uploading a record of my music listening habits was my first real experience with "Web 2.0." Remember Web 2.0? It referred to websites of user-generated content that enabled virtual communities and interoperability. Now such sites are too ubiquitous on the web to warrant a special designation — they're just the web. But that wasn't true in 2006, and even though I'd been putting content on the internet since 1996, at the time it was enough to make me a little nervous. What did these strangers want with my data, and what was in it for me?

Ten years and 24,941 scrobbles later, I have my answer: I have a really cool record of all the music I've listened to the past 10 years! Well, not "all," technically: I've certainly listened to music in places and on devices that didn't …

Why Eleanor Roosevelt Would Have Liked Google+

And why Google+ won't be replacing Twitter or Facebook for most of us anytime soon
"Great minds discuss ideas. Average minds discuss events. Small minds discuss people."
-- Eleanor Roosevelt

I know too much already has been written about Google+ and its place in the world of social networking, but I've recently developed a new perspective which might help some of you who are trying to decide how and when to use Google+ versus Twitter or Facebook.

Eleanor might have said "small minds discuss people," but there's more than one way to discuss people and none of us are consistently small-minded. People are important, and the people who are most important to us are those with which we have mutual friendships or family relationships. This is why Facebook is best at people: it enforces (if we ignore fan pages) a symmetric follower model, ensuring that we are connected to people who want to also be connected to us. Those connections, often with people who we don&…