Skip to main content

The Myth of the Slipping Math Student?


I've been teaching in Colorado for six years, and there's always been a troubling pattern in our state standardized math scores. As students progress from 3rd to 10th grade, the percentage that score proficient and advanced declines dramatically. Here are the percentages of students scoring proficient and advanced by grade level, averaged over all the years the test has been given (typically 2002-2008):












GradeAvg. % P+A
369
469
561
655
744
843
934
1029


The easiest explanation (and the one I've tended to believe) is that students' abilities are, in fact, slipping as they got older. That would be a good assumption if the test at each grade level was equally difficult. But what if the test questions were, on average (and adjusted for grade level), more difficult as students got older? Is it fair to assume a test with increasingly difficult questions would result in lower scores, even with sophisticated score scaling systems that take question difficulty into account?

Fortunately, the state releases "item maps" that describe the difficulty of each item on every test. Using 4 points for an advanced item, 3 points for a proficient item, 2 points for a partially proficient item, and 1 point for an unsatisfactory item, we can come up with an average difficulty for the CSAP at each grade level. Let's add that column to our table:












GradeAvg. DifficultyAvg. % P+A
32.4369.25
42.4368.5
52.5361.14
62.6955.43
72.9644
83.0442.86
93.1334
102.9628.86


This begs for regression analysis. How strong is the correlation between the difficulty of the questions and the scores?


The correlation is surprisingly strong, and the coefficient of determination (R squared) is 0.88, meaning that the average item difficulty is statistically responsible for 88% of the variance in the test scores. 88%? That's big. Statistics rarely tell the whole story, but 88% raises serious doubts that it's just a matter of slipping math students. Why wouldn't the state want to maintain a steady average difficulty year-to-year? Wouldn't that make year-to-year performance comparisons more reliable?

Comments

Popular posts from this blog

Last.fm and Ten Years of Web 2.0

Ten years ago yesterday I scrobbled my first tracks to last.fm. What's scrobbling? On last.fm, scrobbling refers to automatic music track logging to the internet. For me, uploading a record of my music listening habits was my first real experience with "Web 2.0." Remember Web 2.0? It referred to websites of user-generated content that enabled virtual communities and interoperability. Now such sites are too ubiquitous on the web to warrant a special designation — they're just the web. But that wasn't true in 2006, and even though I'd been putting content on the internet since 1996, at the time it was enough to make me a little nervous. What did these strangers want with my data, and what was in it for me?

Ten years and 24,941 scrobbles later, I have my answer: I have a really cool record of all the music I've listened to the past 10 years! Well, not "all," technically: I've certainly listened to music in places and on devices that didn't …

The 15-Year Blogoversary

15 years and 1,213 posts! My first experience with the World Wide Web came in 1995, and by 1997 I had my own web page. The first web authoring tool I remember using was Composer, an HTML editor built into the Netscape Communicator suite. That helped me learn some HTML, and later I used Microsoft Word 97 and then FrontPage 98 and later Macromedia Dreamweaver to design more elaborate pages. Some of my FrontPage-built sites are still on the web. As I learned more about HTML standards and validation I wrote more HTML by hand, but I still wanted a way to make publishing to the web easier.

By 2001 I understood that (a) sites should be updated regularly and (b) FTP'ing sites and pages from my desktop to a server was a bit of a pain. I had heard about some early blogging platforms and chose one, Blogger, to try out. As you can see, I'm still here.
My first post using Blogger came on December 8, 2001. A few months later I paid for Blogger Pro, which offered additional authoring tools, l…
If you find this page, congratulations. It's not exactly public yet. I've been maintaining a website at www.scholars.uni.edu/~johnson for several years, but my time at the university is coming to an end and I've established downclimb.com as a more permanent home on the web. I'm not yet sure how I want to juggle all the content of the site, but with this page I've decided to give Blogger a try. It provides a convenient, web-based interface for updating web logs, or "blogs". I don't have a lot of time at the moment to give this site the polish it needs, but I figure I'll give this a try and see if I like it. If not, I can always go back to the old-fashioned ways that have served me thus far. Let's hope Blogger doesn't go broke...