Peer Review of Teaching
The following was modified from the University of Wisconsin (1998) website on Peer Review.
What is a peer review?
Although "peer review" often elicits the impression of one faculty member visiting another's classroom, observing the colleague, and then writing an assessment for the instructor's record. Peer review is more than an assessment (summative review), but also a method to improve teaching (formative review). Controversy exists as to whether formative and summative reviews can be performed simultaneously; it is the opinion of the Center for Teaching, Learning & Scholarship, that formative peer review should be conducted separately from summative peer review.
How can peer reviews be performed?
Peer reviews can be accomplished via observation, collaboration on courses, interviewing one another's students, writing/reviewing colleague's portfolios (administrative, course, teaching), examining each other's course syllabi and materials, and dialoguing about teaching in general. Mentoring can also be used to improve teaching particularly with senior faculty members acting as the mentors to junior faculty. External review is another option.
What tools exist to perform peer reviews?
Some examples of tools that can be used to gain feedback from colleagues are in the appendices of this packet and include:
- Classroom observation
- Collaborative inquiry
- Course portfolios
- Departmental/school requirements
- External review of course materials/content
- Instructional development programs
- Interviewing students about their learning experiences
- Interviewing students about what they are learning
- Research on a current/new course
- Student outcome review
- Supervision of student research
- Teaching circles
- Teaching portfolios
What evidence can be used to evaluate the quality of a peer's teaching?
Establishing the quality of teaching for any faculty member, particularly when a written assessment will result, should incorporate a range of assessment tools. These tools may include self review, student evaluations, and peer reviews. Each of these assessment tools can address some issues of teaching quality and effectiveness, but not all issues. Triangulation, or viewing from a combination of perspectives, is essential to arrive at an accurate evaluation of teaching quality and effectiveness.
What characteristics form an effective peer review program?
Several characteristics of an effective peer review program include:
- a program consisting of a series of reviews over en extended period of time;
- program planning by reviewer and reviewee;
- significant feedback to the faculty being reviewed;
- the opportunity for written comments by reviewee to be appended to the review; and
- discussion of resources available for development based on previous review results.
Prior to initiating a peer review program intended to show evidence of teaching quality, faculty to be reviewed, reviewers, and subsequent users of the review need to be in substantial agreement on the purposes, focus, breadth, depth, and kinds of results expected from the review. The techniques appropriate to a review will depend on the purpose, focus, breadth, depth, and kinds of results expected.
How should the peer review process be approached?
Those who participate as reviewers in the process of peer review of teaching have an opportunity to contribute to the coherency and assessment of their curricula. For example, when an instructor reviews a colleague's teaching within the same department or college, s/he can more clearly see how her/his own teaching activities contribute to the curriculum and how courses within the curriculum interrelate. The new perspectives instructors gain by participating in peer review can help improve the learning experience across a program by helping faculty transform their approaches and course content, and make explicit the connections between courses.
Serving as a reviewer should be an expected contribution from established faculty in providing leadership in teaching. A thoughtful and carefully conducted review is an invaluable aid to an instructor or a decision-making body, and reviewers' efforts should be rewarded.
There are several general guidelines that apply to all forms of peer review of teaching. First, there are no universal criteria for evaluating teaching. The criteria to be applied in any review depend on several factors, including the discipline, size and type of class (including electronic or distance learning formats), characteristics of the instructor, and characteristics of the learners. Thus, any review is context-specific.
Second, it is essential that the reviewer and the instructor being reviewed agree in advance about the focus of the review and the criteria to be used. Any review of instruction should address the needs of the person being reviewed. Does s/he want to develop/improve teaching? Or does s/he want to produce evidence of teaching quality for a formal review? The person to be reviewed may want feedback about specific aspects of teaching; if so, the review should be focused on, or at least include, these aspects. In any case, advance discussion and agreement are essential. There are dozens of things that one might observe during a lecture, discussion, or clinical teaching experience; obviously, no review can attend to all of these.
Third, it is essential that the reviewer be informed about and open to a variety of approaches to instruction. One of the issues that peer review touches on is academic freedom. The freedom to espouse ideas and to educate in the way one believes is best is paramount to the quality of a major university. Faculty members within a single department may have divergent perspectives on their discipline. It is essential that a review of teaching not infringe on the rights of the person being reviewed. It is important that the reviewer share or be informed about and open to the approach taken by the instructor. Some have suggested that reviewers should not only share the same orientation toward instruction, but that they should be from the same discipline or even sub-discipline. Such similarity may be important in some cases, but it does not seem necessary in all cases. There are times when the fresh eye of someone from another discipline or another campus may provide valuable feedback.
What follows are some general considerations for reviewers to think about. Specific instructions/information for reviewers are provided with each technique.
1. Who should conduct the review?
An important consideration in selecting a reviewer is the purpose of the review. If an individual is seeking to use the review to improve teaching, the status of the reviewer with regard to professorial rank and membership in the same department may not be important. A review focused on assessment of content requires an expert in the same discipline. One focused on effectiveness of teaching methodology requires a reviewer with experience in employing those methods. If the purpose of the review is to provide evidence for a personnel decision, considerations that may be important in selecting reviewers include professorial rank, objectivity (reviewer outside the department, college, or perhaps, the institution), and credentials (recognized expert in the discipline or teaching methodology).
The individual being reviewed should be an integral part of the process and, therefore, should play a role in selecting or providing names of reviewers.
2. What is the process?
- Include the individual being reviewed in designing the review process. Implementation of the review will be more effective, and the individual will be more receptive to feedback, if s/he has played an integral role in the process.
- Meet with the colleague you are reviewing prior to conducting the review. During the pre-review meeting, you can discuss: (1) the purpose of the review and the aspect(s) of teaching you will be considering, (2) your colleague's teaching philosophy, course objectives, syllabus (means of meeting the course objectives), and assessment of student learning, (3) the review technique, including in what form and to whom feedback will be given, and (4) other questions/concerns.
- Understand the purpose of the review. Is your colleague seeking to improve his/her teaching and student learning? Has the department chair or mentoring committee requested the review to provide evidence of the quality of teaching in order to rank/compare the individual within the unit/profession for personnel decisions (e.g., appointment, promotion, tenure, teaching award, merit)?
- Understand the aspect(s) of teaching you are reviewing. There are many aspects of teaching that can be reviewed over time. It is essential to understand clearly what aspect is being reviewed currently in order to provide a useful evaluation. Although not an exhaustive list, some examples follow:
- Are you being asked to observe the instructor in the classroom with the students? If so, what aspect are you evaluating: lecture style/presentation, effective use of small group discussions/exercises to achieve course goals?
- Are you being asked to review syllabus materials and assignments to ascertain whether the content is appropriate, current, and properly sequenced?
- Are you be asked to evaluate the student response to the class by: 1) observing what the students are actually doing in the classroom and how they are interacting with the instructor and/or each other, 2) interviewing students, 3) ascertaining if students have achieved certain goals or have enjoyed their experience in the course as a result of the teaching, or 4) obtaining information from students in some other way?
- Are you being asked to review instructional objectives and goals to ascertain if they are sensible and achievable, to observe how the instructor gives feedback to students, to review examinations, to examine the conceptual framework for a course, or to decide if the course material is integrated, representative, and intellectually rigorous?
- Are you being asked to evaluate how the course fits in with the overall curriculum?
- Select and become familiar with a review technique appropriate for the aspect(s) of teaching you are reviewing. Choose a technique based on the purpose of the review and what aspect of teaching is being assessed. The techniques included in this packet contain descriptions of their purpose and implementation, including the time needed to conduct the review. Each technique has advantages and disadvantages. In some cases, you may work with the colleague you are reviewing to select an appropriate technique; in others, you may be asked to use a particular technique, e.g., observing teaching. The frequency of review over time (how many times during the course or during the faculty member's probationary or post-tenure review period will the review be conducted) should be determined. In addition, it is important to establish criteria that are progressive as the instructor's experience and rank increase.
- Understand in what way and to whom you should deliver feedback. Feedback may be delivered verbally or in written form (e.g., letter, standardized form) to the colleague or to the department chair/mentoring committee. The nature of the feedback depends on the relationship of the reviewer and the colleague being reviewed. The colleague will be more receptive to constructive criticism from a reviewer that is trusted and has a positive relationship with the colleague. The CTLS recommends that you begin with areas of strength before engaging in a discussion of areas that require improvement. Feedback should be as specific as possible.
3. What are the advantages to the reviewer?
Faculty who serve as reviewers can gain recognition as they assume a leadership role in teaching and contribute to the overall coherency of the curriculum of their department or college. They are exposed to a wide variety of teaching issues and techniques as well as to a wealth of content, which may have an impact on their own teaching.
4. Are there training opportunities for reviewers?
Some reviewers may benefit from participation in conferences, workshops, or seminars that address various aspects of conducting reviews, e.g., how to include the colleague being reviewed in the process, how to deliver feedback effectively, etc. Information regarding faculty development programs and other teaching resources can be reached on the CTLS website at www.samford.edu/ctls.
How should one evaluate the evidence obtained during the peer review?
One purpose of peer review is summative review, a formal assessment of the contributions of a teacher/professor to the teaching program. Summative review may be a broad assessment, looking at some evidence of the person's teaching over a number of years/courses. On the other hand, such review may be an in-depth review, a detailed assessment of the person's work in a small number of classes. The evidence needed to carry out an assessment obviously differs depending upon whether breadth or depth is the focus.
In general, the evidence used for summative review should come from a variety of sources. Information should be obtained from the person being reviewed. In addition, evaluations by the students in the course should be included. Evidence may be provided by the instructor's colleagues, either through peer review by colleagues on campus, or by selected persons elsewhere who review the written record, in a manner similar to the external review of scholarship. Numerical data may be included: the number of students in each course, trends in enrollment in the person's courses over time, distributions of course grades. Information from any one source is necessarily incomplete. The goal should be to have available information from several sources, and to integrate these into a comprehensive review.
It is also important to look at evidence collected over a period of semesters or years. The practice of teaching evolves as the instructor learns more about the types of students who take the course and about classroom and evaluation techniques that do and don't work, and as the knowledge base of the course changes. Everyone should not be expected to be an outstanding teacher; rather, the goal should be that everyone's teaching improves over time. Thus, longitudinal data are very important, especially data about the same course over two or more semesters.
In reviewing evidence, it is important to remember that each type of evidence has certain strengths and certain weaknesses.
- Evidence from the Instructor. The instructor should provide information about his/her teaching philosophy and approach to the specific course, or a more general reflective statement. Teaching practices make sense only in the context of the instructor's goals for the course. An excellent vehicle for gathering information from the instructor is the course portfolio. If less information is desired, copies of the syllabus, assignments and examinations will usually be the minimum.
- Evidence from Colleagues. Colleagues can provide assessments of the currency and intellectual quality of the material being taught. Colleagues on campus can provide this information based on a review of materials and observing teaching. Colleagues elsewhere can provide the information via external review of the written material described above. On occasion, colleagues will be able to provide evidence based on team teaching or research on a course in progress.
- Evidence from Students. Students can provide information about the impact of the course on them. They can evaluate the organization and clarity of classroom presentations. They can assess the integration of the various activities in the course - lectures, readings, discussions, projects, assignments, exams - and how each of these contributed to their learning. They can provide a self-conscious assessment of how much they have learned, though they may not be aware of some of the outcomes of the course. Students are routinely asked to provide such assessments through student course evaluations. An alternative way to solicit information from students is by interviewing students about their learning experience or interviewing students about what they have learned. These interviews could be conducted by faculty colleagues within or outside a department, or by other professional staff.
How can our department design a peer review program?
This section provides some guidelines for departments interested in instituting a peer review of teaching program. These guidelines are primarily in the form of questions, on which the department needs to come to consensus. The answers to these questions suggest which paths to peer review are likely to be most useful and productive for a department. Note that it is likely that different departments may arrive at somewhat different peer review programs because their perception of their needs and constraints differ.
Peer review of teaching can be a useful way to find out some things about student learning and faculty teaching. Effective implementation requires that an entire department discusses and, in some sense, agrees on why and how peer review is to be performed. It is particularly important that faculty who are likely to be reviewed have significant input into how and why such a review is to be performed. We think that peer review of teaching programs that are imposed on a department from outside the department or by a small subset of the department are unlikely to be effective or helpful. In this section, a number of questions are posted that a department should address as a group, and come to some consensus on answers. The questions are listed first, then discussed in a little more detail, including come common responses. The responses of your department may be similar to the range of responses indicated, or they may be quite different.
The intent of addressing these issues before beginning a peer review of teaching program is to clarify to all what is sought and what kinds of results are likely to be obtained. Among the many elements that contribute to successful peer review programs at those institutions that have been successful with peer review, is the sense of participation in the design and implementation of the program by all parties. Settling these issues beforehand can alleviate much of the concern (though not all) surrounding a peer review program, and make it more likely that the program will accomplish the goals set out for it.
To design a peer review program, consider answering the following seven questions, with particular emphasis and attention paid to question #1. This question, if fully explored and explicitly addressed, makes many of the other questions relatively easy to respond to.
1. What kinds of information would the group like to get from a peer review process?
Common responses include:
- information on "How good is my teaching, and how could I make it better?"
- information for promotion, i.e. a description of teaching and teaching development for an individual
- information on what students have learned in a class/curriculum
- information on relevance/interest/utility of course/curriculum
There are many more kinds of information which departments could seek from a peer review of teaching program. As later questions suggests, there may be other better ways of finding some of the information in this list. This issue, of what kind of information does a department want is critical in designing a peer review of teaching program which will actually provide reliable information of the kind the department is seeking.
2. Who should get this information?
- only the person being reviewed (may be appropriate when "How good is my teaching, and how could I make it better?" is the major departmental objective)
- department chair/administrators
- divisional committee
This issue, who sees the results of a peer review, is often one of the most difficult in developing a peer review of teaching program. If a clear set of objectives for peer review has been established from question #1, there may be less contention on the issue of who sees the results. For personal review and feedback, for example, it may be quite appropriate that only the reviewer and reviewee see the results. For tenure and promotion cases, evidence of peer review of teaching may be required. For other objectives developed by the department, discussion of who will see the results before any peer review has occurred is essential.
This issue gets at the base of what will the results be used for. Will they be for promotion? Merit raises? Firing "lousy" teachers? Helping all faculty improve their teaching? As noted above, these issues should be settled before the program is initiated, and the decisions made need to be adhered to.
3. Is peer review, as the group understands it, the best way to get this information?
- yes (reviewing peers is the best or only way to find out what we'd like to know)
- no (reviewing peers really won't provide useful information on a topic)
While peer review can provide useful information on many teaching and learning issues, it does not provide information on all issues. In some cases, it may not be the best way to find out what a department wants to know. For example, peer review is a great way for a good teacher to find out how she/he could be better. It is probably not a great way to find out what students like best and least about a course or curriculum unless student interviews are included as part of the peer review. Peer review can help obtain information about teaching and learning which is not available by other means, but it should not be the only way in which information about teaching and learning is obtained. Different methods (student questionnaires, focus groups, other "tests") should be employed to obtain as complete a picture of the teaching and learning experience as possible.
4. What other (non-peer review) information does the group need to make a complete picture?
- student questionnaires (end-of-course surveys)
- student interviews
- graduate interviews (5 or 10 years after graduation)
- student performance measures (standardized tests, professional registration,etc.)
Except for student questionnaires, most of these alternative ways of gathering information are probably not well developed on most parts of the Samford University campus. More departments are beginning to try some of these, particularly interviews of recent and 5 and 10 year graduates. The intent of this question is to focus on the objectives set out in question #1, and consider whether peer review of teaching can provide all or part of the information sought. If peer review can provide only a part of the information sought, what will constitute the other parts? (i.e. program assessment requirements for accreditation)
Reliance on student questionnaires, because of the well-developed process to administer them, is tempting. We encourage departments to think of student questionnaires as a source of a particular kind of information, and to include that information with other sources in evaluating teaching and learning.
5. What are the major impediments to implementing a peer review process in this group?
- lack of time
- lack of experience in peer review
- mistrust of the process
For any department, there are real impediments to initiating a peer review program. If these impediments are strong enough, any peer review program may be doomed, and those attempting to develop one may be wasting their time. These issues need to be addressed at the beginning of the program, and they need to be dealt with. As each department character and culture is different, we doubt any canonical solution to these impediments exists. The members of the department are in the best position to decide what their major impediments will be, and how to overcome them. Some, like lack of experience, will go away as experience is gained. Others, like lack of time, won't go away and need to be addressed directly for a program to solidify. For example, "lack of time" usually implies that this task (peer review) is added to the range of tasks faculty are already responsible for, and must compete for time with other teaching and research activities. Clearly, this is true. Departments may shift this discussion somewhat by suggesting the issue is as much one of priorities (which is most important) as one of total tasks and total time. Departments have flexibility in establishing and promoting activities that they feel are of very high priority over others of a lower priority.
6. Is the group, in general, enthusiastic about using peer review for gathering information?
- yes (we really want to get the information peer review can provide and are willing to do what it takes to get it)
- o (we don't want to know this information badly enough to do peer review)
This is a question which summarizes, in some sense, several of the previous ones. At this point in a department discussion, the issues of what information might be gained from a peer review of teaching program and what it will cost in time and effort should begin to be evident. Departments need to decide whether the benefits outweigh the costs. If so, a peer review of teaching program could be quite successful; if not, it is unlikely that developing and implementing a program will be beneficial.
7. How will the group organize itself to take the next steps toward implementation?
- we have no idea
- let's look at websites and literature on peer review of teaching
This is the key step towards implementation-what happens after a department has decided it wants to do peer review of teaching? Are people willing to put in the time and energy to make the program a success? Further, the department has agreed on what its objectives are for peer review, and what the results will be used for. What's next?
There are several issues included in "What's next?" For example, who will complete the description of the department's peer review program to formalize the "Why?, How?, Who?, When?" Who will seek out the resources available? The first issue needs to be settled by the department. We think that the broader range of ideas and opinions used to form the program, the better; so we would encourage the entire department to continue being involved in the development. If you want to create your own resources, feel free, but why not tell others about them so they can benefit from all your work? Your input can be shared on the CTLS website.
There are probably as many kinds of peer review of teaching programs possible as there are departments at Samford University. A few select example programs are available for your review. The emphasis of this discussion of developing a peer review of teaching program is to avoid prescribing a perfect or model program which is unlikely to suit anybody, but rather to suggest a method by which departments could create their own program. This sounds like a lot more work than adopting someone else's tried and tested model program, but it's not. Most departments I know, would spend at least as much time arguing over which model to adopt and how to modify the one they chose as they would spend creating a program which is tailored to suit their specific department's needs. Spending that time "up front" seems slow and inefficient; it is likely to pay huge dividends later when the group doesn't need to debate small issues which arise, because they've already had extensive discussions about objectives and means, and have come to consensus on them. There will still be disagreements and discussions, but they are likely to be about the substance of the issues, rather than the range of preconceptions faculty bring to peer review. In the current vernacular, faculty are likely to all be "on the same page" if they have had discussions of objectives and means at the beginning.
The CTLS supports the idea that to overcome resistance to change, people involved in the change need to be involved in the development of the process from the beginning, and need to have a real voice in what will happen. "Change" in this case may be changing how some of the information regarding teaching and learning is gathered, or "change" may expand to changing how teachers and students perceive their roles in a learning environment.