Peer Review of Teaching

The following was modified from the University of Wisconsin (1998) website on Peer Review.

What is a peer review?

Although "peer review" often elicits the impression of one faculty member visiting another's classroom, observing the colleague, and then writing an assessment for the instructor's record. Peer review is more than an assessment (summative review), but also a method to improve teaching (formative review). Controversy exists as to whether formative and summative reviews can be performed simultaneously; it is the opinion of the Center for Teaching, Learning & Scholarship, that formative peer review should be conducted separately from summative peer review.

How can peer reviews be performed?

Peer reviews can be accomplished via observation, collaboration on courses, interviewing one another's students, writing/reviewing colleague's portfolios (administrative, course, teaching), examining each other's course syllabi and materials, and dialoguing about teaching in general. Mentoring can also be used to improve teaching particularly with senior faculty members acting as the mentors to junior faculty. External review is another option.

What tools exist to perform peer reviews?

Some examples of tools that can be used to gain feedback from colleagues are in the appendices of this packet and include:

What evidence can be used to evaluate the quality of a peer's teaching?

Establishing the quality of teaching for any faculty member, particularly when a written assessment will result, should incorporate a range of assessment tools. These tools may include self review, student evaluations, and peer reviews. Each of these assessment tools can address some issues of teaching quality and effectiveness, but not all issues. Triangulation, or viewing from a combination of perspectives, is essential to arrive at an accurate evaluation of teaching quality and effectiveness.

What characteristics form an effective peer review program?

Several characteristics of an effective peer review program include:

Prior to initiating a peer review program intended to show evidence of teaching quality, faculty to be reviewed, reviewers, and subsequent users of the review need to be in substantial agreement on the purposes, focus, breadth, depth, and kinds of results expected from the review. The techniques appropriate to a review will depend on the purpose, focus, breadth, depth, and kinds of results expected.

How should the peer review process be approached?

Those who participate as reviewers in the process of peer review of teaching have an opportunity to contribute to the coherency and assessment of their curricula. For example, when an instructor reviews a colleague's teaching within the same department or college, s/he can more clearly see how her/his own teaching activities contribute to the curriculum and how courses within the curriculum interrelate. The new perspectives instructors gain by participating in peer review can help improve the learning experience across a program by helping faculty transform their approaches and course content, and make explicit the connections between courses.

Serving as a reviewer should be an expected contribution from established faculty in providing leadership in teaching. A thoughtful and carefully conducted review is an invaluable aid to an instructor or a decision-making body, and reviewers' efforts should be rewarded.

There are several general guidelines that apply to all forms of peer review of teaching. First, there are no universal criteria for evaluating teaching. The criteria to be applied in any review depend on several factors, including the discipline, size and type of class (including electronic or distance learning formats), characteristics of the instructor, and characteristics of the learners. Thus, any review is context-specific.

Second, it is essential that the reviewer and the instructor being reviewed agree in advance about the focus of the review and the criteria to be used. Any review of instruction should address the needs of the person being reviewed. Does s/he want to develop/improve teaching? Or does s/he want to produce evidence of teaching quality for a formal review? The person to be reviewed may want feedback about specific aspects of teaching; if so, the review should be focused on, or at least include, these aspects. In any case, advance discussion and agreement are essential. There are dozens of things that one might observe during a lecture, discussion, or clinical teaching experience; obviously, no review can attend to all of these.

Third, it is essential that the reviewer be informed about and open to a variety of approaches to instruction. One of the issues that peer review touches on is academic freedom. The freedom to espouse ideas and to educate in the way one believes is best is paramount to the quality of a major university. Faculty members within a single department may have divergent perspectives on their discipline. It is essential that a review of teaching not infringe on the rights of the person being reviewed. It is important that the reviewer share or be informed about and open to the approach taken by the instructor. Some have suggested that reviewers should not only share the same orientation toward instruction, but that they should be from the same discipline or even sub-discipline. Such similarity may be important in some cases, but it does not seem necessary in all cases. There are times when the fresh eye of someone from another discipline or another campus may provide valuable feedback.

What follows are some general considerations for reviewers to think about. Specific instructions/information for reviewers are provided with each technique.

Considerations

  1. Who should conduct the review?
  2. What is the process?
    • Meet with the colleague you are reviewing prior to conducting the review.
    • Understand the purpose of the review.
    • Understand the aspect(s) of teaching you are reviewing.
    • Select and become familiar with appropriate peer review technique(s).
    • Understand in what way and to whom you should deliver feedback.
  3. What are the advantages to the reviewer?
  4. Are there training opportunities for reviewers?

1. Who should conduct the review?

An important consideration in selecting a reviewer is the purpose of the review. If an individual is seeking to use the review to improve teaching, the status of the reviewer with regard to professorial rank and membership in the same department may not be important. A review focused on assessment of content requires an expert in the same discipline. One focused on effectiveness of teaching methodology requires a reviewer with experience in employing those methods. If the purpose of the review is to provide evidence for a personnel decision, considerations that may be important in selecting reviewers include professorial rank, objectivity (reviewer outside the department, college, or perhaps, the institution), and credentials (recognized expert in the discipline or teaching methodology).

The individual being reviewed should be an integral part of the process and, therefore, should play a role in selecting or providing names of reviewers.

2. What is the process?

Include the individual being reviewed in designing the review process. Implementation of the review will be more effective, and the individual will be more receptive to feedback, if s/he has played an integral role in the process.

3. What are the advantages to the reviewer?

Faculty who serve as reviewers can gain recognition as they assume a leadership role in teaching and contribute to the overall coherency of the curriculum of their department or college. They are exposed to a wide variety of teaching issues and techniques as well as to a wealth of content, which may have an impact on their own teaching.

4. Are there training opportunities for reviewers?

Some reviewers may benefit from participation in conferences, workshops, or seminars that address various aspects of conducting reviews, e.g., how to include the colleague being reviewed in the process, how to deliver feedback effectively, etc. Information regarding faculty development programs and other teaching resources can be reached on the CTLS website at www.samford.edu/groups/ctls.

How should one evaluate the evidence obtained during the peer review?

One purpose of peer review is summative review, a formal assessment of the contributions of a teacher/professor to the teaching program. Summative review may be a broad assessment, looking at some evidence of the person's teaching over a number of years/courses. On the other hand, such review may be an in-depth review, a detailed assessment of the person's work in a small number of classes. The evidence needed to carry out an assessment obviously differs depending upon whether breadth or depth is the focus.

In general, the evidence used for summative review should come from a variety of sources. Information should be obtained from the person being reviewed. In addition, evaluations by the students in the course should be included. Evidence may be provided by the instructor's colleagues, either through peer review by colleagues on campus, or by selected persons elsewhere who review the written record, in a manner similar to the external review of scholarship. Numerical data may be included: the number of students in each course, trends in enrollment in the person's courses over time, distributions of course grades. Information from any one source is necessarily incomplete. The goal should be to have available information from several sources, and to integrate these into a comprehensive review.

It is also important to look at evidence collected over a period of semesters or years. The practice of teaching evolves as the instructor learns more about the types of students who take the course and about classroom and evaluation techniques that do and don't work, and as the knowledge base of the course changes. Everyone should not be expected to be an outstanding teacher; rather, the goal should be that everyone's teaching improves over time. Thus, longitudinal data are very important, especially data about the same course over two or more semesters.

In reviewing evidence, it is important to remember that each type of evidence has certain strengths and certain weaknesses.

How can our department design a peer review program?

This section provides some guidelines for departments interested in instituting a peer review of teaching program. These guidelines are primarily in the form of questions, on which the department needs to come to consensus. The answers to these questions suggest which paths to peer review are likely to be most useful and productive for a department. Note that it is likely that different departments may arrive at somewhat different peer review programs because their perception of their needs and constraints differ.

How can our department design a peer review program?

This section provides some guidelines for departments interested in instituting a peer review of teaching

Background Work

Peer review of teaching can be a useful way to find out some things about student learning and faculty teaching. Effective implementation requires that an entire department discusses and, in some sense, agrees on why and how peer review is to be performed. It is particularly important that faculty who are likely to be reviewed have significant input into how and why such a review is to be performed. We think that peer review of teaching programs that are imposed on a department from outside the department or by a small subset of the department are unlikely to be effective or helpful. In this section, a number of questions are posted that a department should address as a group, and come to some consensus on answers. The questions are listed first, then discussed in a little more detail, including come common responses. The responses of your department may be similar to the range of responses indicated, or they may be quite different.

Questions:

  1. What kinds of information would the group like to get from a peer review process and what will they do with it?
  2. Who should get to see this information?
  3. Is peer review, as the group understands it, a good way to get this information?
  4. What other (non-peer review) information does the group need to make a complete picture?
  5. What are the major impediments to implementing a peer review process in this group?
  6. Is the group, in general, enthusiastic about using peer review for information gathering?
  7. How will the group organize itself to take the next steps toward implementation?

The intent of addressing these issues before beginning a peer review of teaching program is to clarify to all what is sought and what kinds of results are likely to be obtained. Among the many elements that contribute to successful peer review programs at those institutions that have been successful with peer review, is the sense of participation in the design and implementation of the program by all parties. Settling these issues beforehand can alleviate much of the concern (though not all) surrounding a peer review program, and make it more likely that the program will accomplish the goals set out for it.

To design a peer review program, consider answering the following seven questions, with particular emphasis and attention paid to question #1. This question, if fully explored and explicitly addressed, makes many of the other questions relatively easy to respond to.

Question Details

1. What kinds of information would the group like to get from a peer review process?

Common responses include:

There are many more kinds of information which departments could seek from a peer review of teaching program. As later questions suggests, there may be other better ways of finding some of the information in this list. This issue, of what kind of information does a department want is critical in designing a peer review of teaching program which will actually provide reliable information of the kind the department is seeking.

2. Who should get this information?

Common responses:

This issue, who sees the results of a peer review, is often one of the most difficult in developing a peer review of teaching program. If a clear set of objectives for peer review has been established from question #1, there may be less contention on the issue of who sees the results. For personal review and feedback, for example, it may be quite appropriate that only the reviewer and reviewee see the results. For tenure and promotion cases, evidence of peer review of teaching may be required. For other objectives developed by the department, discussion of who will see the results before any peer review has occurred is essential.

This issue gets at the base of what will the results be used for. Will they be for promotion? Merit raises? Firing "lousy" teachers? Helping all faculty improve their teaching? As noted above, these issues should be settled before the program is initiated, and the decisions made need to be adhered to.

3. Is peer review, as the group understands it, the best way to get this information?

Common responses:

While peer review can provide useful information on many teaching and learning issues, it does not provide information on all issues. In some cases, it may not be the best way to find out what a department wants to know. For example, peer review is a great way for a good teacher to find out how she/he could be better. It is probably not a great way to find out what students like best and least about a course or curriculum unless student interviews are included as part of the peer review. Peer review can help obtain information about teaching and learning which is not available by other means, but it should not be the only way in which information about teaching and learning is obtained. Different methods (student questionnaires, focus groups, other "tests") should be employed to obtain as complete a picture of the teaching and learning experience as possible.

4. What other (non-peer review) information does the group need to make a complete picture?

Common responses:

Except for student questionnaires, most of these alternative ways of gathering information are probably not well developed on most parts of the Samford University campus. More departments are beginning to try some of these, particularly interviews of recent and 5 and 10 year graduates. The intent of this question is to focus on the objectives set out in question #1, and consider whether peer review of teaching can provide all or part of the information sought. If peer review can provide only a part of the information sought, what will constitute the other parts? (i.e. program assessment requirements for accreditation)

Reliance on student questionnaires, because of the well-developed process to administer them, is tempting. We encourage departments to think of student questionnaires as a source of a particular kind of information, and to include that information with other sources in evaluating teaching and learning.

5. What are the major impediments to implementing a peer review process in this group?

Common responses:

For any department, there are real impediments to initiating a peer review program. If these impediments are strong enough, any peer review program may be doomed, and those attempting to develop one may be wasting their time. These issues need to be addressed at the beginning of the program, and they need to be dealt with. As each department character and culture is different, we doubt any canonical solution to these impediments exists. The members of the department are in the best position to decide what their major impediments will be, and how to overcome them. Some, like lack of experience, will go away as experience is gained. Others, like lack of time, won't go away and need to be addressed directly for a program to solidify. For example, "lack of time" usually implies that this task (peer review) is added to the range of tasks faculty are already responsible for, and must compete for time with other teaching and research activities. Clearly, this is true. Departments may shift this discussion somewhat by suggesting the issue is as much one of priorities (which is most important) as one of total tasks and total time. Departments have flexibility in establishing and promoting activities that they feel are of very high priority over others of a lower priority.

6. Is the group, in general, enthusiastic about using peer review for gathering information?

Common responses:

This is a question which summarizes, in some sense, several of the previous ones. At this point in a department discussion, the issues of what information might be gained from a peer review of teaching program and what it will cost in time and effort should begin to be evident. Departments need to decide whether the benefits outweigh the costs. If so, a peer review of teaching program could be quite successful; if not, it is unlikely that developing and implementing a program will be beneficial.

7. How will the group organize itself to take the next steps toward implementation?

Common responses:

This is the key step towards implementation-what happens after a department has decided it wants to do peer review of teaching? Are people willing to put in the time and energy to make the program a success? Further, the department has agreed on what its objectives are for peer review, and what the results will be used for. What's next?

There are several issues included in "What's next?" For example, who will complete the description of the department's peer review program to formalize the "Why?, How?, Who?, When?" Who will seek out the resources available? The first issue needs to be settled by the department. We think that the broader range of ideas and opinions used to form the program, the better; so we would encourage the entire department to continue being involved in the development. If you want to create your own resources, feel free, but why not tell others about them so they can benefit from all your work? Your input can be shared on the CTLS website.

Summary

There are probably as many kinds of peer review of teaching programs possible as there are departments at Samford University. A few select example programs are available for your review. The emphasis of this discussion of developing a peer review of teaching program is to avoid prescribing a perfect or model program which is unlikely to suit anybody, but rather to suggest a method by which departments could create their own program. This sounds like a lot more work than adopting someone else's tried and tested model program, but it's not. Most departments I know, would spend at least as much time arguing over which model to adopt and how to modify the one they chose as they would spend creating a program which is tailored to suit their specific department's needs. Spending that time "up front" seems slow and inefficient; it is likely to pay huge dividends later when the group doesn't need to debate small issues which arise, because they've already had extensive discussions about objectives and means, and have come to consensus on them. There will still be disagreements and discussions, but they are likely to be about the substance of the issues, rather than the range of preconceptions faculty bring to peer review. In the current vernacular, faculty are likely to all be "on the same page" if they have had discussions of objectives and means at the beginning.

The CTLS supports the idea that to overcome resistance to change, people involved in the change need to be involved in the development of the process from the beginning, and need to have a real voice in what will happen. "Change" in this case may be changing how some of the information regarding teaching and learning is gathered, or "change" may expand to changing how teachers and students perceive their roles in a learning environment.

Additional Resources