Briefing Paper January 2002

Pennsylvania's New Teacher Tests: An Assessment

By
Barbara S. Plake and
Stephen A. Herzenberg

Beginning fiscal year 2002, and in each of the following four years, Pennsylvania plans to spend $4 million per year to implement a new system of "teacher tests" — the Professional Development Assistance Program (PDAP) Assessments.

20 percent of teachers are scheduled to take the assessments each year. The tests will be used to identify schools or school districts that need professional development.

The briefing paper considers whether the new teacher tests are likely to improve teacher effectiveness or student achievement. It concludes that they are not. It therefore recommends ending the PDAP program. Some of the resources saved should be shifted to a Teacher Effectiveness Initiative (TEI) that would include assessment and professional development tied more closely to making teachers more effective in the classroom.

The KRC analysis of the PDAP program consists of two sections. In the first section, nationally recognized testing specialist Barbara Plake evaluates the PDAP assessments. In the second section, we briefly describe the Teacher Effectiveness Initiative and evaluate it against the PDAP assessments.

The PDAP assessments require all teachers to take a reading test geared to the reading skills that students must master by grades 5, 8, or 11. Teachers in elementary school, or in math or science at the junior high or secondary level, must also take a mathematics test. This means, Professor Plake finds, that the PDAP assessments:

Assess the general mathematics and reading literacy of Pennsylvania teachers;

Do not measure how well teachers teach and do not, in most cases, measure teachers' command of the subject and grade level that they teach;

Do not measure, in lay person's terms, the right thing _ how well teachers do their job _ and therefore do not have what test experts term "content validity."

Plake points to other problems with the PDAP assessment program:

The scores on one third of the mathematics topics on the PDAP assessments will be assessed by less than five test questions, making the scores in these areas unreliable.

The high quality of a test is ordinarily documented with evidence showing that scores on the test correlate with scores on other assessment instruments. No such evidence has been published regarding the PDAP assessments. For this reason, the PDAP assessments do not have what is sometimes termed "statistical validity."

There is no objective standard for defining when teachers' PDAP test scores require them to receive additional professional development.

There is no objective standard defined regarding the fraction of teachers in a school or school district who must score low on the PDAP assessments before teachers are required to receive professional development.

Lacking objective standards, the temptation will be to evaluate teachers based on performance compared to their peers, and to point to places where high proportions of teachers score in the bottom 25 percent. But since scoring in the bottom 25 percent _ or the top 25 percent _- tells us little about how effective teachers are, such invidious comparisons serve no positive purpose.. They could simply become another opportunity to criticize teachers and public education.

Following the Plake evaluation, the last part of the paper proposes that Pennsylvania abandon the PDAP in favor of a Teacher Effectiveness Initiative.

The TEI would emphasize assessment of teachers' effectiveness in the classroom and professional development geared to directly improving classroom teaching. It should include demonstration projects that expand

Opportunities for student teachers to gain classroom experience,

Mentoring for new teachers by experienced, master teachers, and

Team teaching and other peer collaboration among experienced teachers.
In Pennsylvania, a TEI could signal a new direction for education reform. It would recognize and nurture teachers' commitment to children and to teaching well. By working with, not against, teachers Pennsylvania can achieve a revolution in teaching practice that unlocks unprecedented learning.

This document is an on-line summary of a Keystone Research Center report. The entire report is available for download as a PDF file at the KRC Web site www.keystoneresearch.org © 2001 Keystone Research Center