I learned something really interesting last week that challenged some of my own practices. I went to a standards-setting workshop in Sacramento to help create cut scores for the Algebra I CMA. While I can’t share a lot of the information we were given, I can share the process we went through for determining cut scores.
First we all looked at the standards and discussed what skills a student at each level should be able to demonstrate. We wrote 5 bullet points that described the Basic, Proficient, and Advanced student. Then we looked at the test that ordered the problems from least difficult to most difficult. This was done with statistical analysis based on how students who took the test last year performed. When doing this in practice, a teacher would not have this data and would have to order the problems in another fashion. Based on the 5 bullet points, each teacher went through and “bookmarked” the problem that would mark the next level of proficiency. You started at number one and went through the problems until you got to the place in the test where you thought the lowest performing basic student should be able to go based on the description of a basic student. Then you flipped through the next section of problems until you got to the problem that you felt marked the shift in skill level from Basic to Proficient, and so on for Advanced.
This was really interesting for me. I get why CST cut scores are where they are now. When we make cut scores on our benchmarks, we typically go with an “acceptable” percentage and don’t really think about the questions on the test. I appreciated that the state and ETS wanted me to describe proficient based on a skill set and not a number. In fact, we were encouraged to ignore what percent a kid would get on the test and instead really think about the skills in which a student should be able to demonstrate proficiency. Thank you, state of California, for pushing my math brain out of numbers for a bit to instead think about the overall skill level of my students. I needed that.