Wednesday, May 19, 2010

Wallpapering Traditional Multiple-Choice Tests

 Wallpapering is preparing, in advance of the test, a mark pattern to be used when students do not have answers they can verify and trust. Students have three options after marking all the questions that can be used to report what is known or can be done:

  1. Turning in the answer sheet yields an accurate, honest, but unfair score unless omit or judgment is given a value equal to, or higher than, knowledge; as is done with Knowledge and Judgment Scoring (KJS) and Confidence Based Learning (CBL).

  1. Randomly marking the remaining questions gives judgment a value of zero. The score is less accurate, honest, and fair the lower it gets until it only reflects answer sheet marking ability. The test is a high anxiety academic casino game at the lowest levels (orders) of thinking.

  1. Wallpapering is a defensive measure. It reduces test anxiety. It increases fairness and test security. It shares the same good luck.

Being prepared reduces test anxiety. This includes how to make a forced-choice mark when you do not have a trusted answer. The age-old advice is to pick one option, such as C. Wallpapering adds one more step: Everyone in the class makes the same mark (with KJS and CBL everyone just omits).

A fair test requires a fair starting score (which exists with KJS and CBL).
The active starting score on traditional multiple-choice tests is about 33%, on average. That is a range of independent starting scores of about two letter grades. Wallpapering reduces this range.

Wallpapering produces a security code. The wallpaper marking-pattern can be made as elaborate as needed. Over half of the marks on an answer sheet can come from wallpapering when test scores drop below 50%. A set of answer sheets marked right and wallpapered, and with no erasures, indicates no tampering.

NCLB raw scores below 40% are now listed as Proficient in several states. The distribution of scores from marginal students with equal abilities follows the normal curve of error. The distribution widens as the test scores descend. It is gambling. Some pass. Some fail. This is not fair.

Wallpapering reduces this unfairness. All students in the group (class) mark the same answer when they cannot trust making a right mark. They do the same thing at the same time rather than individually trust to luck. This does not change their individual test scores, on average.

Wallpaper is an answer sheet created BEFORE seeing the test. Individual variation is markedly reduced. The simplest example is for all in the group to agree to mark the same letter when in doubt. More variable patterns can be created using mnemonics for easy memory. Short patterns can repeat every few questions. The Christmas tree repeats every 4 questions (A, B, C, D) on a 4-option test. Longer patterns can use poetry and music.

Doing the same thing at the same time has evolved in birds as a means of protecting individual members of the flock from predators. The tight formation protects individual members and decreases the energy needed to fly. The same protection and energy savings applies in schools of fish.

Wallpapering has this effect for marginal students taking tests using RMS. It reduces the random lucky score variation in individual test scores. Wallpapering allows students to do the same thing at the same time with equal ease when marking a trusted right answer or marking the equivalent of omit using KJS or CBL. A few minutes of planning equal a few millennia of evolution in protecting marginal students from the vagaries of NCLB testing.


Multiple Choice Bubble Sheet Template:
Teacher-Author: E. Fisher         Price: FREE

No comments:

Post a Comment