Skip to main content

The Myth of the Slipping Math Student?


I've been teaching in Colorado for six years, and there's always been a troubling pattern in our state standardized math scores. As students progress from 3rd to 10th grade, the percentage that score proficient and advanced declines dramatically. Here are the percentages of students scoring proficient and advanced by grade level, averaged over all the years the test has been given (typically 2002-2008):












GradeAvg. % P+A
369
469
561
655
744
843
934
1029


The easiest explanation (and the one I've tended to believe) is that students' abilities are, in fact, slipping as they got older. That would be a good assumption if the test at each grade level was equally difficult. But what if the test questions were, on average (and adjusted for grade level), more difficult as students got older? Is it fair to assume a test with increasingly difficult questions would result in lower scores, even with sophisticated score scaling systems that take question difficulty into account?

Fortunately, the state releases "item maps" that describe the difficulty of each item on every test. Using 4 points for an advanced item, 3 points for a proficient item, 2 points for a partially proficient item, and 1 point for an unsatisfactory item, we can come up with an average difficulty for the CSAP at each grade level. Let's add that column to our table:












GradeAvg. DifficultyAvg. % P+A
32.4369.25
42.4368.5
52.5361.14
62.6955.43
72.9644
83.0442.86
93.1334
102.9628.86


This begs for regression analysis. How strong is the correlation between the difficulty of the questions and the scores?


The correlation is surprisingly strong, and the coefficient of determination (R squared) is 0.88, meaning that the average item difficulty is statistically responsible for 88% of the variance in the test scores. 88%? That's big. Statistics rarely tell the whole story, but 88% raises serious doubts that it's just a matter of slipping math students. Why wouldn't the state want to maintain a steady average difficulty year-to-year? Wouldn't that make year-to-year performance comparisons more reliable?

Comments

Popular posts from this blog

Last.fm and Ten Years of Web 2.0

Ten years ago yesterday I scrobbled my first tracks to last.fm. What's scrobbling? On last.fm, scrobbling refers to automatic music track logging to the internet. For me, uploading a record of my music listening habits was my first real experience with "Web 2.0." Remember Web 2.0? It referred to websites of user-generated content that enabled virtual communities and interoperability. Now such sites are too ubiquitous on the web to warrant a special designation — they're just the web. But that wasn't true in 2006, and even though I'd been putting content on the internet since 1996, at the time it was enough to make me a little nervous. What did these strangers want with my data, and what was in it for me?

Ten years and 24,941 scrobbles later, I have my answer: I have a really cool record of all the music I've listened to the past 10 years! Well, not "all," technically: I've certainly listened to music in places and on devices that didn't …

The 15-Year Blogoversary

15 years and 1,213 posts! My first experience with the World Wide Web came in 1995, and by 1997 I had my own web page. The first web authoring tool I remember using was Composer, an HTML editor built into the Netscape Communicator suite. That helped me learn some HTML, and later I used Microsoft Word 97 and then FrontPage 98 and later Macromedia Dreamweaver to design more elaborate pages. Some of my FrontPage-built sites are still on the web. As I learned more about HTML standards and validation I wrote more HTML by hand, but I still wanted a way to make publishing to the web easier.

By 2001 I understood that (a) sites should be updated regularly and (b) FTP'ing sites and pages from my desktop to a server was a bit of a pain. I had heard about some early blogging platforms and chose one, Blogger, to try out. As you can see, I'm still here.
My first post using Blogger came on December 8, 2001. A few months later I paid for Blogger Pro, which offered additional authoring tools, l…

Hibernation

It's the last half-hour of 2017 and I'm pushing out a post to keep my blogging streak alive. I can't imagine this is interesting to anyone else, but that's never really been what this blog is for. I could write more, and maybe I should write more, but the only goal I've had for this blog for a while is to write at least once a year.

The last week of the year has become what I call my period of "hibernation." Between Christmas and the New Year, I stay home, do whatever I want on whatever schedule I want. This year, that meant a lot of nights staying up until 4 am and sleeping until 10 or 11. Other than trips to get food, I pretty much had no social interaction for the week. I don't know that I purposely take this time as a mental vacation, but that's sort of what it is.
I had several goals for this year's hibernation, but the one I've done best to stick to is physical: pedaling my stationary bike 40 miles each day. There is a Strava challe…