April 2010 Archives

After Governor Charlie Crist of Florida vetoed SB 6, which was designed to overhaul merit pay for teachers, he convened a group to prepare for the second round of the Race to the Top grant. This group included teachers, superintendents, school board members, members of the PTA, and other leaders. However, a vital member of the team was noticeably absent: a professional evaluator.

Professional evaluators help organizations plan and conduct evaluations of programs, products, processes, policies, etc. In the education realm, evaluators help to determine (among other aspects) whether a program is increasing student achievement, decreasing achievement gaps, increasing high school graduation rates, and increasing college enrollment - all of which are components in the Race to the Top grant program.

When you have a professional evaluator on your team, that person (1) helps you plan and implement techniques to gain visibility as to whether your program is meeting the set objectives, (2) advises you on how to allocate resources where they will bring the most benefit, and (3) provides results, recommendations, and information for you to make informed decisions on how to proceed in making that program a success.

Based on a listing by the American Evaluation Association (AEA), "an international professional association of evaluators devoted to the application and exploration of [evaluation]", there are at least ten different evaluation organizations in the state of Florida that specialize in educational evaluation. I'm sure that one or more of them would be happy to assist Florida in the grant application process.

This should be a wake-up call for all states applying for grants of any kind: if there is an evaluation component in the grant (and there definitely IS in the Race to the Top grants), then you need a professional evaluator to assist you in planning and conducting that evaluation.

How you plan and conduct that evaluation can mean the difference between winning and losing the grant - which equates to vital federal funding that states cannot afford to pass up.

1. With every problem comes a solution.
2. Evaluation can help your organization improve existing processes.
3. Select an evaluator who keeps up with the trends and issues in your industry, and with whom you can discuss viable solutions to what ails your organization.
4. Think outside the box with regard to evaluating a program, product, etc. There are a number of evaluation techniques that may get at the heart of what you're looking for.
5. The evaluator should discuss a range of options that provide value to you and to the organization. Be explicit about your information and service needs so the evaluator can readily accommodate you.
6. Before every project, and throughout the process, ensure that both you and your evaluator are clear on what's being evaluated, your needs and goals, and that you understand the data collection methods and other techniques your evaluator will use to help you.
7. A proficient evaluator will document not only the costs of the project, but the value you as the client gain for the endeavor.
8. Before engaging in any project, consider how it will impact your business in the short, intermediate, and long terms.

Topics of teacher effectiveness and merit pay have developed from such programs as the NCLB reform and the Race to the Top grant programs. Individual states such as Florida, Colorado, and California, to name a few, have begun to step up to the plate in designing their own evaluation plans for these programs.

If nothing else, the Race to the Top grant program has become a catalyst for change to a more efficient and effective paradigm: that being a more holistic evaluation plan that takes a host of factors into account and puts power into the hands of school leadership to work together with teachers and parents for improved student achievement, a school culture geared toward learning, and a cadre of young people prepared for the post-high school plans of their choice.

An effective evaluation of teacher effectiveness comes not from one test, nor from testing (or any other one factor) alone, but from a comprehensive array of factors. While the only true measure of teacher effectiveness may come from the results of a student's life ten years hence (my 8th grade math teacher comes to mind, who helped students feel valuable and confident in their abilities), schools can and should design a holistic evaluation for teacher effectiveness and merit pay scenarios. These factors may include, but are not limited to, the following (schools may want to focus on different aspects depending on their school's purpose):

• Student test scores
• Teacher observations and interviews
• Teacher self-reports
• Teacher portfolios of their work
• Parent involvement and interaction

Some argue that there are many factors beyond a teachers' control (such as socio-economic status, broken homes, etc.), and they fear that teachers will be held accountable for those factors. While I am not advocating that teachers be called on the carpet for these situations, there is more to a teacher's effectiveness than simply test scores.

As states and schools wrestle with defining and evaluating teacher effectiveness, they may want to ask themselves the following questions:

1. What difference are teachers making in their students' lives?
2. How are teachers preparing students to succeed in and out of the classroom?
3. Do students develop a love for learning because of their teachers?
4. Is the school culture geared toward learning, or is it run like a business or an assembly line?
5. Are teachers rewarded for their efforts in their students' learning, thus giving teachers the motivation to think beyond the textbook?
6. Are teachers given access to professional development opportunities and other resources to build their skills and increase their ability to help make a difference?
7. Do school administrators make the time to meet with teachers often to encourage efforts and provide guidance?

If a teacher is being paid to simply exist in the classroom, then we as a society are doing our children a disservice.

Designing and executing a comprehensive evaluation of teacher effectiveness and merit pay will not only provide visibility and evidence of effectiveness, but will pinpoint areas for improvement so that schools may allocate resources where they are most needed. An effective evaluation spans the whole realm of a given program, and directly stems from the missions and goals of an organization. Schools and states would do well to step back and formulate a theory of change (also known as a logic model, impact map, etc.), and then use that information to create long-term evaluation plans to help capitalize on strengths, identify areas of improvement, and develop their capacity to evaluate their programs.

1. Utilize the problem-solving abilities of the evaluator you work with, not just expertise, as you work together to make your organization more efficient and effective.
2. Evaluators take their own medicine - they are constantly upgrading their own efficiency, too.
3. Your evaluator will work with you on a project schedule that works for everyone.
4. A professional evaluator strives for both quality and value in assisting you in making your business successful.
5. A professional evaluator should care about your organization enough to at least offer preliminary guidance on your project regardless of whether you are officially contracted with that person for work.
6. A professional evaluator evaluates the quality of his or her own work (called metaevaluation). Be prepared at the end of a project to discuss the outcome and quality with the evaluator.
7. Evaluators often serve as advisors to boards of directors. In that capacity, they are able to use their expertise, learn from others, and give back to the community.
8. Be open to new ideas and possibilities. What may not have worked well in the past may be just the solution to today's needs.