December 2009 Archives

1. Building trust with an evaluator helps that person understand your needs as an organization and can bring greater objectivity, precision, and usefulness to an evaluation.
2. Look for an evaluator who brings enthusiasm and life to an evaluation or other project. This helps infuse new ideas for improvement into the organization.
3. Select an evaluator who is serious about professional development - individually and in collaboration with colleagues.
4. When contracting for evaluation services, keep in mind the quality and value you receive.
5. With proper training from an evaluation consultant, your organization can become self-sufficient with regard to your evaluation needs.
6. Look for an evaluator who listens to your organizational needs, and then makes recommendations as necessary.
7. Look for an evaluator who keeps his or her professional skills current, to include evaluation, consulting, management, and communication.
8. Make meetings meaningful.

1. Work with your evaluator to decide who the stakeholders are and their degree of involvement in the evaluation or project.
2. Anecdotes or stories are an excellent way to get your ideas across with an evaluator and with your staff.
3. Consider subscribing to various consultants' websites for tips on evaluation, organizational development, or other topics.
4. An evaluator makes recommendations; you as the stakeholder and client choose to accept or reject those recommendations to any degree.
5. The evaluation reporting process is flexible to accommodate your organizational needs, so do not hesitate to discuss with the evaluator a particular reporting style or protocol that you prefer.
6. Discuss with the evaluator various consulting experiences as they pertain to your organization to get an idea of philosophy, work style, and usefulness.
7. Look for an evaluator who brings value to your organization - someone who helps translate information into knowledge you can use to be more effective in your business.
8. Look for an evaluator who will provide useful recommendations you can implement immediately to make a difference in your business.

1. Ask "What if" questions about your program to know how to evaluate relevant aspects of it.
2. When evaluating productivity and development, consider professional development as an investment in your organization's future.
3. Make sure that people can identify with your program (evaluate that aspect).
4. Don't be afraid to act on the results of an evaluation regardless of the outcome of that evaluation.
5. Learn to ask "Why" as a problem-solving tool for your program. Asking why is the first step in evaluation.
6. Set realistic expectations when designing an evaluation.
7. Coordinate with the evaluator regarding your reporting protocol or preferences.
8. Keep up with reading in your profession, as well as on evaluation of aspects of your work. An evaluator can recommend books or articles to peak your interest and inform your business.

A recent news article from the Christian Science Monitor, "Eyeing Stimulus Money for Education, States Adopt Reforms (posted Thursday, December 3, 2009 at http://www.eschoolnews.com/funding/funding-news/?i=62081), highlights the need for evaluation in making progress with education reforms.

In particular, the article points out four areas where states could receive grant money: "Establishing data systems that track students' progress from preschool through college, developing and using rigorous standards and assessments, improving teacher effectiveness and the distribution of high-quality teachers, and turning around the lowest-performing schools." The article states that the last two issues have not been addressed much because of the obscure nature of defining teacher effectiveness and developing a system to improve schools.

That's what evaluation is all about. Organizations work with professional evaluators to define topics such as teacher effectiveness, establish criteria and standards for what should be happening, and then take steps to evaluate whether or not those things are happening. Organizations may then choose to implement recommendations generated from the study to enhance and improve the program.

While creating standardized definitions, criteria, and standards for schools across the country will certainly be a more involved process with all levels of education, schools can and should start somewhere by evaluating what is working or not working in school reform (to at least establish a baseline), and then proceeding with strategies for improvement.

National, state, and local organizations must work hand in hand in order for true reform to succeed. Schools would do well to look at this grant offering as an opportunity to evaluate their current situation and then take steps toward improvement.

The Evaluation Baron, LLC can help you evaluate how your school, district, or state is doing in any of these topics. Contact us for a free initial consultation!

1. Include all decision makers when discussing a project.
2. Look for an evaluation consultant who not only consults or completes projects, but who also teaches the skills and significance of evaluation via seminars, speaking engagements, etc.
3. An evaluation consulting contract is typically made with the organization, not with a specific person in that organization.
4. A meeting is as much about people's thoughts, ideas, and attitudes as it is about getting things done.
5. The client should be in charge of meetings due to the decision-making nature of some meetings.
6. Don't decline evaluation services just because you cannot afford to pay due to the economic downturn. With corporate restructuring and refinement, you can't afford NOT to enlist the help of a consultant. Together the two of you can work out something, such as an exchange for in-kind services.
7. An evaluation consultant will help you understand how your programs relate to your organization's internal and external vision.
8. Evaluation is a lot more than finance and numbers.

Recently I attended a one-day conference, during which time I literally heard 14 different terms to denote evaluation. Let me briefly share them with you, along with a simple explanation of each one.

1. Evaluation: Judging the merit or worth of something, such as a program, process, product, etc. Yes, it was only a few times, but I did hear this word...like a sweet melody ringing in my ears.
2. Analysis: An investigation of data pertaining to a situation, or the situation itself.
3. Assessment: This term is usually used within the context of educational testing, but is generally considered a subset of evaluation.
4. Find Out: Although simple, what logically follows from this phrase is the question of how and why are you going to find out...for what purpose and by what means.
5. Observation Checklist: This is one of many tools used with data collection to make sense of what is being observed, such as a class, a performance, a group interacting somehow, etc.
6. Tracking: This term denotes the monitoring side of evaluation, which is an ongoing collection of data on a given topic, usually to determine compliance.
7. Review: To make an appraisal of data or of a situation, such as a review of an existing program, etc.
8. Key Indicators: This phrase is synonymous with criteria and standards (i.e., what should the program be doing, and to what standard should it occur).
9. Measuring Success/Measurement: This term is also considered a subset of evaluation, and often pertains to quantitative data, although measurement itself can be in quantitative or qualitative terms.
10. Performance Improvement: This term is often used in the human resource or healthcare fields to denote such things as employee productivity and wellness.
11. Feedback: Reflective of customer service, feedback can be opinions or other information about your programs. Feedback can come from both internal and external sources.
12. Rate: This term denotes a judgment of quality, and is often seen in surveys (e.g., "On a scale of 1-5, rate the effectiveness of the following...").
13. Accountability: Evaluations help to establish accountability for organizational performance. Evaluations link performance criteria to results to show funders or other stakeholders the efficiency, effectiveness, and return on investment of a program, product, or process.
14. Harvesting: This term denotes the data collection and analysis phase of an evaluation, and can be by various means (e.g., web analytics, documentation review, surveys, interviews, data mining, ethnography).

Let us know how you feel! Tell us what term you affiliate with, and how you use it (or want to use it) in your organization!

No matter what term you use, The Evaluation Baron, LLC can help you increase the effectiveness and efficiency of your programs, products, or business processes by aligning results with your organizational goals.

Contact us for a free initial consultation!