"What we found is it's a much more in-depth process, and it requires more time ... on the administrative and teacher part," said Karen Schoonover, chief academic officer at New Hope Academy Charter School. "But the upside is it leads to a much more valuable conversation about improving instruction in the classroom."
Pennsylvania has been working to craft a new evaluation system for educators. The current system has been criticized for rating nearly all teachers and principals as satisfactory.
In 2010-11, a handful of school districts tried out a new observation system. For 2011-12, districts and schools statewide were invited to pilot the new tool in the second phase of the effort, and more than 100 volunteered.
In York County, those participating in the pilot are the Dover Area, Northeastern, Red Lion and York City school districts and New Hope Academy Charter School.
Once the new system is complete, half of a teacher's evaluation will be based on an observation divided into four categories: planning and preparation, classroom environment, instruction, and professional responsibilities.
The other half will be based on student achievement -- 15 percent will be building-level data, 15 percent will be teacher-specific data, and 20 percent will be elective.
The second phase involved having districts pilot the new observation model.
Those participating said the new observation system is more comprehensive than that used in the past.
It involves more back-and-forth between the teacher and evaluator, said Amy Glusco, supervisor of curriculum and instruction for the Red Lion Area School District, where 18 teachers participated.
The system includes meetings between the teacher and evaluator before and after the observation, she said, and both parties use a form that is passed back and forth during the process. It's more work for both the teacher and administrator, she said.
Erica Fabie, second grade teacher at Windsor Manor Elementary School, said the new tool makes teachers think about whether they are teaching the same way every day or using different methods to reach all of the kids in a class.
"I think it's a great way, as a teacher, to reflect and think about what you're doing," she said.
Fabie said the meetings before and after the observation are important, because it lets administrators see a bigger picture and it gives the teacher a chance to think about the lesson afterward.
"I definitely think, overall, it's a great tool," Fabie said. "It's better than what we currently (have)."
Paul Arigo, a social studies teacher at Red Lion Area Junior High School, said the system forces teachers to make sure they look at the standards and align their lessons accordingly.
The old observation system was more black and white -- either a teacher accomplished something or didn't. This one offers more of a range, he said.
He, too, said the system gives teachers and their supervisors more of a chance to talk about what was going on.
"You can kind of debate as to what you were trying to do in class, what they saw and what you were trying to accomplish," he said.
He was a little nervous about the new system at first, but now thinks it seems "very worthwhile," he said.
Sue Kanigsberg, director of curriculum and instruction and pupil personnel services for Dover Area School District, said participants there have been through the first three steps, and are still working on the final unannounced walk-throughs.
Previously, administrators did unannounced walk-throughs in classrooms, she said, but they were meant for supervision purposes and were not linked to teachers' evaluations.
The new system blends announced and unannounced observations, she said.
There were some unexpected insights, Kanigsberg said.
It had sort of been predicted that teachers would rate themselves higher than their actual rating, she said.
"They rated themselves more critically," she said, noting the process is more reflective for teachers.
For some, the system wasn't all that new.
Northeastern's observation system was already based on the model that is the basis for the state system, said Stacey Sidle, assistant superintendent for Northeastern School District, so it wasn't a huge change for them. But the process is more "clinical."
The system is much more evidence-based, she said, and that has added focus for the observations. The teacher gets the evaluator's notes back, and then they have a follow-up discussion.
They come up with a goal to focus on, she said, and another informal observation is held later.
Glusco, too, said the new system removes much of the subjectivity of the old.
There are certain things an evaluator must do -- count how many students raised their hand to answer a question, or count how many higher-level thinking questions were asked -- and record it, she said.
"That's a little bit of a mindshift for the evaluator," she said.
Deborah Wortham, York City School District superintendent, said she was already planning to implement the model the state has adopted in York City when she found out about the pilot.
Though only a dozen or so teachers are participating in the pilot, at the request of the state, she said, the entire district is unofficially embracing the model. Teachers not participating in the pilot were still given the option of having their evaluation done that way.
Wortham said she believes the observation and evaluation process provides a "golden opportunity" to improve student achievement. She was accustomed to having observations include pre- and post-observation meetings, she said.
"That really makes it a collaborative process," she said.
Several educators participating noted that the system is more time consuming.
That's the "perpetual challenge" with many things in education, Glusco said.
"It's the time to implement it all and do it well," she said.
Some also noted some logistical hurdles, such as trying to fit a lot of information in tiny spaces allotted on forms provided.
Glusco also noted that during training, they were advised to get forms back to teachers quickly, which sometimes could pose challenges when administrators also have to deal with other issues that might come up.
Schoonover said that though the process takes more time, an administrator's time is better spent on improving teacher quality. If teacher quality improves, administrators will have to spend less time on discipline.
"It's a balance," she said. "You just have to tilt it in the right direction."
Sidle said the time necessary for the process might become challenging when the system is implemented district-wide.
Several educators emphasized that process piloted only counts for half of a teacher's evaluation in the system Pennsylvania is creating.
The other 50 percent is "the part everyone's going to debate," Kanigsberg said, referring to the portion that will be based on student achievement.
Legislation is also necessary for the state to include student achievement in teachers' evaluations.
Still, several district officials said it was worthwhile to participate in the pilot.
Kanigsberg said she believes it is one of Dover's best moves because the district now has many administrators and a group of teachers trained for the future.
"I feel it put us ahead of the game," she said.
Here's what Carolyn Dumaresq, Pennsylvania's deputy secretary of education, said is coming up as the state works to craft new evaluation systems for educators:
Now: The state will be gathering final evaluation information from those piloting the new observation system during the 2011-12 school year as well as feedback from participants. That will be used to make any necessary tweaks to the system.
2012-13: The adjusted observation system will be used. Those districts that already piloted it can choose to participate again and more districts can come on board if they choose.
The state will also pilot a new observation system for principals and specialists, who are non-classroom professionals like guidance counselors, this year.
The state will also be working on determining how the elective student data portion of the evaluation, which will be 20 percent of the total evaluation, will work. Student achievement information for individual teachers will start being gathered this year, but it won't be used in evaluations until a teacher has three years of growth scores banked.
2013-14: The goal is to have the observation tool for teachers ready and implemented statewide, as well as the building-level student achievement piece, which will be 15 percent of the total.
The principal and specialist evaluation systems will move into phase 3.
2014-15: The state hopes to have the principal and specialist evaluation system in place statewide.
Also: For the teachers' evaluation system, supervisors will be able to participate in an "interrater reliability process," Dumaresq said, designed to make sure that if two principals observed the same classroom, they'd give nearly the same rating.
Teacher ratings from phase two will also be submitted to Mathematica, a company that will use the data plus student test scores associated with those teachers to determine which criteria correlate to better student achievement.
Race to the Top
In late 2011, Pennsylvania was awarded about $41 million from the federal Race to the Top Grant competition. About half of that will be distributed to local education agencies, according to the state.
Part of the grant will be used to implement the new evaluation system for teachers an prinicipals, according to the state. A district must agree to participate in the teacher evaluation project in order to receive funds, according to information online.
Learn more by visiting www.education.state.pa.us and clicking on "Race to the Top" under "PDE Quick Links."