Contents
Guest Commentary
Peer-Reviewed Articles
Items of Interest
The Integration of Technology into Pharmacy Education and Practice
Most technology users employ less than 20% of the capabilities offered by both their hardware and software. This usually holds true whether one is considering personal computers (PCs), personal digital assistants (PDAs), word processors, or especially pharmacy operations management and documentation software. The appropriate use of technology requires a learning curve (usually ranging from a few hours to a few weeks) and many of us are neither patient enough nor committed enough to explore technology beyond that which is necessary to solve our most immediate problems. This means we will learn how to get a label out of the community pharmacy system or enter an order into a hospital system we purchased and then often stop right there. The inventory package to increase profitability or the key performance indicator reports are never discovered. In an everyday application like e-mail, once we learn how to answer e-mails and open attachments we are satisfied and move on to our next task. Unfortunately, we never learn the features and benefits offered by these and other technologies that will truly maximize our efficiency and effectiveness through their use in our practice.
The authors believe that technology needs to be so integrated into pharmacy education and the curricula of schools and colleges of pharmacy that students would never think of practicing their profession without the support of technology. In general, we realize we are more enthusiastic about this vision than most of our colleagues. One of the authors took his first computer course in 1966 and has been able to continually focus on all aspects of pharmacy-related technology while at Auburn. Remember, technology performs two primary functions which both relate to the work humans do. First, technology can be used to completely replace the work of humans, especially when that work is repetitive and tedious. Thus, there are many robotic dispensing systems that are faster and more accurate than humans doing the same task. Secondly, technology enhances the work humans perform.
We can think of no better example of technology that enhances human performance than that of information technology. Pharmacy informatics can be defined as the specialized application of computers and other information technologies that are used to advance the profession. With thousands of new articles being published every week in the biomedical literature, it is impossible for any practitioner to claim that he or she is "keeping up" with the relevant information generated globally that can impact the provision of pharmaceutical care. We believe that the primary purpose of all information is to reduce our uncertainty when making decisions. Technologies classified as information appliances can deliver evidence-based information at the point of care that has been updated on a daily basis. Wireless technology allows us to review clinical information, such as laboratory results, as soon as they become available. Technology is available to support the provision of pharmaceutical care in the achievement of desirable patient outcomes. Pharmacy educators now need to integrate the support of all available technologies into pharmacy education.
At least one of our clinical colleagues claims to be old enough that he had to learn how to practice pharmacy with only primary literature to guide his decisions. The authors viewed the Facts and Comparisons Web site and saw that their first tertiary publication was printed in 1947. During the 1970s there were many quality tertiary references from which to choose, but they were nearly all in book format. These resources are now being slowly converted into an electronic format. Some book publishers merely displayed their books electronically without change. Other publishing and prospective drug utilization review businesses initially released their tertiary references in an electronic form. Some of these still appeared more like books than electronic products. The format in which these references were presented became more granular and the best products today give our pharmacists and student pharmacists very quick to the exact nugget of information they seek. For example, a user desiring correct dosing on a drug for use in renally impaired patients can go directly to the information needed; therefore, there is no need to read a monograph or even a paragraph to find the answer being sought.
Publishers have progressed from books, to floppy disc-based, to CD ROM-based, to Internet-based, to PDA-based products. Furthermore, the buzzword in healthcare technology is Mobile Care. Technology now allows pervasive computing in pharmacy practice. We are able to acquire information and communicate our knowledge from anywhere to anywhere. In fact, one of the latest definitions of the Internet is every computer, cell phone, PDA, and pager in the world being able to communicate with every other one. The actual hardware that accomplishes this information management continues to weigh less and do more. A SmartPhone/PDA we acquired for less than $200 through our university allows us to simultaneously carry a cell phone, speaker phone, voice recorder, voice dialer, e-mail, Internet browser, fax machine, pager, MP3 player, digital camera, still and motion picture viewer, the Microsoft Office Professional document management application, and over 500MB of clinical references and pharmacy specialized applications. We can now be as connected as we wish. Our students need to be prepared to practice in a world supported by this level of technology. Wireless infrastructure in our program has made a wonderful difference in the level of utilization of student computers throughout our educational process. It is now possible for students to sit at picnic tables several hundred yards from our building and be completely connected to our network resources. Just as children grow up today thinking that the Internet has always been there, accessing the highest quality information to support the provision of pharmaceutical care should be second nature to our student pharmacists.
The clinical workup of the patient starts with appraisal, then moves to intervention, follows with the evaluation of the impact of interventions, and concludes with a plan for monitoring and following up with each patient appropriately. Technology exists to support each of these steps that will make providers more efficient and effective at managing their practice. Moreover, technology exists that is affordable, portable, and more recently connected to a multidisciplinary care team where the five rights (right drug, right strength, right form, etc.) we use in pharmacy can be extended to include the right provider doing the right procedure in the right facility for the right patient at the right time. A wealth of technology exists to assist the delivery of pharmaceutical care, whether the problem being addressed is medication regimen adherence, screening for drug-related problems, patient safety, measurement of therapeutic outcomes, or patients' self-care management. Integration and standardization of these technologies is now one of the highest priorities for all of healthcare.
For students to become equipped to utilize this array of technology, they must first become computer literate. We have placed this requirement in our admissions material at Auburn University. We stipulate that basic computer, Internet, Microsoft Office Suite, and electronic communication skills be obtained in students' two-year pre-pharmacy education prior to starting our four-year professional program. We require students to have or purchase a wireless laptop computer that meets a set of specifications (listed on our web site). When the student arrives on campus we load off line CD-ROM-based drug references, PDA emulators, and a variety of utilities necessary to complete our pharmacy curriculum and connect to our specialized systems. WebCT is our primary syllabus management technology. We utilize GroupWise as our organizational groupware package for managing e-mail, to do lists, calendars, appointments, and special events notification.
When we initially launched the notebook computer requirement, we utilized a lease program but found that allowing students to purchase their own computer at or above our specifications was much more satisfactory. Without fail, when we specified an individual computer and negotiated the lease price for 24 months, a significant price reduction would occur on the specified computer within 30 days following our signing a contract. When technical difficulties would arise during the lease period, students would bring the notebook computer we "made" them purchase back to the school and say, "What are you going to do to fix this problem?" When students purchase their own technology some of them exceed specifications and purchase ultra-portable devices. Since they own the device they tend to work directly with the manufacturer for technical difficulties they encounter and are able to make their computer last beyond the 24 months of the previous lease program.
We have tested students for computer literacy upon arrival, but have found that their computer skills are seriously lacking. Approximately 20% of our admission classes are still relatively computer illiterate when they enter the pharmacy program. This should be self-correcting as public schools expose young students to greater levels of computing in their early education.
We have placed a significant number of electronic resources online for student use. We have a wide array of medical and pharmacy references online that are mounted with restricted access by username and password. We require that students construct their own web page for professional promotion and for posting various projects required in our classes. We are incorporating an electronic portfolio that will further list the educational accomplishments and competencies attained by students. In many of our courses grades are communicated electronically, assignments are collected electronically, and documentation of patient care visits are transmitted as e-mail attachments for use by faculty mentors who provide students with verbal and electronic feedback. We have acquired a full-featured electronic medical record and are modifying the existing 37 specialties to include a customized pharmacotherapy specialty and a hospital/inpatient specialty. This secure record will house all of the documentation of patient care rendered in our educational program, research initiatives, and clinics operated by the school. It will serve to create an enterprise solution for all of our inpatient and ambulatory uses.
In a recent curriculum revision, we are constructing a curriculum-wide skills laboratory that has the potential for 24/7 access by students and eventually by alumni. This laboratory material may account for as much as one third of the educational effort required by a student each semester. Distance education techniques, even within our professional program, will be utilized for many of the skills taught through the laboratory. When students need to manipulate an actual object such as a blood glucose meter, a visit to the laboratory will supply them with needed tools. Several pharmacy programs in the US to include Minnesota and Cincinnati have constructed such a laboratory and seem to be utilizing them successfully in their respective programs. One possible scenario for collaboration is that a "centers of excellence" approach will emerge in the country (and perhaps internationally) so that redundant development of materials will become unnecessary. For example, while Auburn University may have modules in Healthcare Informatics and patient communication available for use by other schools, we may wish to connect to information concerning herbal treatments or oncology from other universities.
Some schools have adopted and required the use of PDAs from the beginning of their curriculum. We have elected at Auburn to train second and third year students in the selection and use of PDAs and have supported them in the discounted purchase of both hardware and software. We encourage the use of PDAs in their interdisciplinary module sequences during their second and third years and in clinical rotations during their fourth year. We are considering the requirement of PDAs for fourth year students to be able to measure the clinical impact (and potential financial benefit) of our students in clinical rotations sites. De-identified patient data could be collected and aggregated to measure both financial and therapeutic outcome evidence. These data should provide information on the benefits being derived by institutions that are providing practice settings for our student pharmacists. Similar to any healthcare setting, colleges and schools of pharmacy need to be able to integrate all aspects of their educational program to ascertain where improvements are both needed and possible.
So far we have discussed the progress made in teaching students how to use technology in the practice of pharmacy. However, much less progress has been made in using technology in the process of teaching. In essence, technology has been used mainly to facilitate traditional teaching methods rather than to enhance or transform them. For example, many lecture halls are electronically assisted with LED projectors and display cameras. Material presented during lecture in PowerPoint, Word or Adobe Acrobat formats is also available for download to the students' computers. Electronic publishing is the next generation of document reproduction that has supplanted mimeographing, dittoing, and Xeroxing of classroom materials. But the written linear nature of the material has not changed. The material is still to be read and understood by the student.
The great challenge in teaching is to use technology to create more dynamic and realistic learning environments that require the student to actively explore material and reach their own conclusions. For example, our colleague Bruce Berger teaches first- year students how Motivational Interviewing can serve as a more effective approach to helping patients address compliance issues. He assigns readings for class, lectures on the material, holds discussions about the material, and conducts limited role playing in class. There is still a major leap remaining if the students are to incorporate Motivational Interviewing into counseling of patients. Many students never make this transition; however, Interactive computer technology offers the potential to help students make the necessary transfer.
Toward this end, we have begun to develop a standardized virtual patient that would allow students to experience using Motivational Interviewing with a patient. The second author of this paper and Brad Barker, a graduate student, have used software for digital video and speech recognition to build an engine for a prototype virtual patient. Digital video is used to present a patient's comment. Currently the student pharmacist is limited to saying one of three to five possible responses to the patient. Once the computer has recognized which alternative the student has used, the computer branches to play one of several different videotaped patient responses. Because the lag time is only one to two seconds, the virtual patient has the feel of spoken interaction with a real person. The interactive script allows students to experiment with a multitude of paths through such a counseling episode. In this manner, students can experience how counseling based on Motivational Interviewing helps patients to increase compliance with their treatment regimen. In addition, the program can allow a virtual coach (ie, a video of the instructor or an animated avatar figure) to pause the flow of interaction and provide feedback and just-in-time instruction. We believe that a library of such standardized will allow students to achieve a level of assessment, repetition, and skills attainment that would have been previously impossible. In addition to simulating general counseling and interviewing skills, a virtual patient can potentially simulate many other kinds of interactions including clinical appraisal, history taking, and training sessions.
Virtual patients can be further incorporated into the case studies often used in problem- based learning. Instead of presenting students with a patient's history, students would be required to take the virtual patient's history. If the student uses poor interviewing strategies, the virtual patient can become resistant and fail to provide the information required to arrive at an acceptable treatment plan for the patient. Instead of communication being a supplemental skill taught on the side, communication would become an integral aspect of the case study method. Students would be able to practice interviewing and counseling patients in a wide range of case studies throughout their courses.
In summary, we envision all potential scenarios for the current and future delivery of pharmaceutical care (and pharmacy education) as employing high levels of technology. The Institute of Medicine is promoting an informatics emphasis in all health curricula. We are enthusiastic about the increased efficiency and effectiveness that can be achieved through the appropriate integration of supporting technology for the student, teacher, and practitioner. While there will always be a required learning curve and rapid obsolescence when using technology, we are confident that the benefits outweigh the barriers for its use.
Please feel free to contact us to discuss this commentary in greater depth.
Implementing On-Line Case Presentation Evaluations
Poster previously presented at the 2002 AACP Annual Meeting, July, 2002, Kansas City, Missouri.
Abstract
To improve and expedite feedback and ensure confidentiality, an on-line assessment form was developed and compared with a traditional paper based assessment form. Thirteen discussion group leaders in two different therapeutics courses evaluated four to six students each week using a password protected on-line assessment form. When the assessment was complete, an automated notification was emailed to students. Subsequently, the student could access a password-protected site to obtain their individual feedback. The 7 discussion group leaders who had used the paper form the previous semester, and the on-line assessment form now, completed a questionnaire to compare the timeliness, the quality of feedback given, and the confidentiality of the on-line and paper systems. Discussion group leaders concluded the on-line system took longer to complete, but the quality, timeliness, and confidentiality of the feedback was improved, making it a better means of administering feedback. An on-line assessment can be employed to provide better feedback in a timely manner and can increase confidentiality for individual students.
Key Words: assessment, technology, web-based, feedback, pharmacy education, on-line evaluations
Introduction
The exploration of effective uses of technology is critical to higher education. It is important that as educators we do our best to capitalize on the capabilities of new technologies to improve operational efficiency and day-to-day practice. Not only can technology improve efficiency, but it can also provide many useful ways to improve our education and assessment methods. In the Pharmacy Practice Department at the St. Louis College of Pharmacy, on-line lecture and course evaluations have been shown to be an effective means to elicit feedback from students. The on-line system provided both quantitatively and qualitatively superior student comments, enhanced student satisfaction, and allowed for more efficient use of time by faculty and staff [1]. Due to the success of the on-line evaluation tool used by students to provide faculty feedback, we implemented an on-line case presentation assessment tool for faculty to provide feedback to students in our therapeutics courses.
Throughout the therapeutics sequence, two-hour case discussions were held on a weekly basis (10 times per semester). Each class of approximately 150 students was divided into discussion sections that consist of approximately 25 students. Within each section there were 5 groups of 4 to 6 students, and 1 faculty discussion leader. Each week, 1 or 2 cases were assigned for the groups to prepare prior to the discussion meeting. During the discussion period, a group was randomly chosen to present 1 of the cases. Each student in the group was then randomly assigned one of the following sections for verbal presentation: assess patient specific medical problems, evaluate current therapy, select and recommend therapy, monitor patient's medical problems and therapy, or educate the patient or the health care professional. Each student's performance was graded on the information presented and the ability to communicate the information, which accounts for 70% of their score. The remaining 30% of their score was determined by an overall group score. Previously in our therapeutics courses, feedback for case presentations had been done on a group assessment form (Appendix A). Many problems existed with this method of providing feedback.
First, the form itself was not conducive to providing formative feedback. The form was designed as a checklist to ensure that students addressed all components of the case performance criteria, but did not allot space for additional comments, which could be directed at helping students improve the following week. Second, the form did not allow for confidential, student specific feedback . Each group received one feedback form with all of the scores, thus negating any attempts to maintain confidentiality for individual group members. Finally, the timeliness with this system was poor in that students did not receive the feedback until the following week in class.
Web-based technology such as Web CT (4.1 Standard Edition, WebCT, Inc, 2003) and Blackboard Learning System (Release 6, Blackboard Inc, 2003) are now available to enhance course work in both distance and on-campus courses. In addition, many health sciences training programs are utilizing other forms of computer-assisted instruction to improve student performance and enhance learning [2-10]. Little, however, has been published on using the Internet solely as a feedback tool for instructors teaching on-campus courses. The importance of the "classroom feedback loop" and the need for students to receive appropriate and focused feedback early and often is stressed by experts in the field of assessment in higher education [11]. An on-line assessment tool has the potential to assist faculty in better closing that loop, by providing improved feedback in a timely manner. The purpose of this study was to describe the implementation and evaluation of an on-line assessment tool for the case presentation component of the therapeutics courses. Specific emphasis was placed on determining if deficiencies identified with an existing evaluation system could be overcome with this new technology.
Methods
An on-line assessment tool was created with the current case presentation criteria using Active Server Pages connected to an Access database (Appendix B). The decision to create an on-line assessment tool came about when third party vendor software could not be found to support the specific application requirements. The tool was created and maintained internally by members of the pharmacy practice department. Prior to the start of the fall semester, a short teaching session was held for the 13 discussion group leaders in the 2 therapeutics courses (Therapeutics 1 and 3) on how to use the on-line tool. The 147 and 156 students in Therapeutics 1 and 3 respectively, were oriented to the system on the first day of class. Following each discussion meeting, faculty group leaders accessed a password-protected site to complete the on-line assessment of the students. Once the instructor was finished, an email was generated that informed the students that their assessment was complete. Students could subsequently access the password-protected site to obtain individual feedback.At the end of the fall semester, discussion leaders who had used both the traditional paper system in previous years and the current on-line system completed a survey created by the course coordinators. Discussion leaders used a five-point Likert scale (strongly agree, agree, no opinion, disagree or strongly disagree) to answer the following questions: 1) the feedback I was able to give was more useful using the on-line system versus the traditional paper system, 2) the feedback was timelier with the on-line system versus the traditional paper system, 3) I believe confidentiality was increased with the on-line system, 4) it is important that confidentiality was increased, and 5) I had easy access to providing feedback with the on-line system. In addition to these questions, discussion leaders were asked which method they preferred and which system took longer to provide feedback.
Results
All 7 discussion leaders who used both systems for assessment completed the survey. Overall, discussion leaders indicated that the quality, timeliness, and confidentiality of the feedback were improved with the on-line feedback system compared to the traditional paper assessment (Table 1). While each discussion group leader stated the on-line system took longer to complete, each rated it as a better means of administering feedback and preferred using the on-line system when compared to the traditional paper system.
While we did not formally poll students regarding their opinion of this on-line system, almost all comments received by the course coordinators were positive. Students appreciated the timeliness and increased confidentiality of the feedback. Several students noted that the quality and quantity of the feedback provided was much improved with the on-line system compared with the old paper form. Student satisfaction with using on-line systems for providing feedback was previously evaluated when students providing feedback to faculty was studied, and was overwhelmingly positive.[1]
Discussion
The on-line assessment system appears to be an effective means of administering student feedback on weekly case presentations. Despite the increase in time required to complete the evaluation, the discussion leaders agreed that the on-line assessment system improved the quality and timeliness of feedback. The flexibility of the on-line system appears to benefit both faculty and students. Staff time was also decreased as the need for copying and distributing paper assessment forms was no longer present.
One of the aims of this study was to determine if the deficiencies of the old paper evaluation system could be overcome with Internet based technology. Based on the results from the discussion leader survey and from course coordinator observation, it appears that it has. First, the on-line system provided adequate space for comments as needed by the instructor. By placing the evaluation tool on the Internet, space on the paper and the length of the form were not an issue. Instructors overwhelmingly felt they were able to provide more useful feedback using the on-line system rather than the paper form. Second, instructors rated the system as one that increased the confidentiality provided to students, which likely allowed them to provide more detailed feedback. The on-line system allowed for confidential, student specific feedback without the instructor being required to fill out multiple forms. Finally, according to the faculty surveyed, the timeliness of the evaluations was greatly improved. Faculty were asked to complete their evaluations within 24 hours of the presentation and were able to do so from any computer with Internet access. Students were able to access the feedback at their leisure and reply by email for rapid clarification or questions. This allowed students to make adjustments in their case work-up prior to the next discussion.
In order for formative assessment to be useful, several obstacles must be overcome. First, feedback must be given in a timely manner. Second, it must be tailored to the individual, and third, feedback must be both diagnostic and prescriptive[12]. While the on-line system helps us overcome the first two obstacles, it is still the quality of the faculty feedback that ultimately drives student improvement.
Limitations
One possible limitation to utilizing an on-line evaluation system could be computer access. In order for students to retrieve and review on-line assessments, computers must be available. At the St. Louis College of Pharmacy, several measures have been taken to provide adequate computer access for the students. There are currently two large computer labs available for student use. In addition to this, a new laptop computer initiative was begun in the fall of 2002 where all entering freshman were equipped with a laptop computer. Coinciding with this program is the installation of a wireless network, eventually providing campus wide Internet access.
Secondly, the increase in time needed to complete the student assessments can be viewed as another limitation. However, providing formative and useful feedback is one that takes time and effort, regardless of the method used. Although time to complete the on-line assessment was increased, faculty agreed that it was a more effective means of providing feedback that allowed them to give written evidence to support their assessment.
A final consideration is our relatively small number of faculty (n=7) that were surveyed. While this may be considered a limitation, it is believed that the 7 that were polled were representative of the entire therapeutics faculty. They were chosen because they had experience providing feedback with both systems and could objectively compare the two. The overwhelming agreement among those polled makes us comfortable in accepting the overall positive responses regarding the system.
Conclusion
Faculty providing diagnostic and prescriptive feedback will continue to be what drives the assessment process. An on-line assessment tool may assist faculty in better closing the feedback loop, by providing better feedback in a timely manner. At the same time, this type of tool allows for feedback to be tailored to the individual student who is working in a group, allowing for more specific and directed feedback that might not otherwise be provided. Based on these findings, it is planned that all courses in the pharmacy practice division that provide formal group feedback will utilize on-line assessment systems.
References
1. Kasiar JB, Schroeder SL, Holstad SG. Comparison of traditional and web-based course evaluation processes in a required, team-taught pharmacotherapy course. Am J Pharm Educ. 2002;66:278-270.
2. Santee J. A Web-Based Practice Examination to Improve Student Performance Concerning the 200 Most Prescribed Drugs. Am J Pharm Educ [serial online]. 2003;67(4); Article 102. Available at: http://www.ajpe.org/view.asp? path=aj6704/aj6704102/aj6704102.xml&pdf=yes. Accessed February 16, 2004.
3. Hilger AE, Hamrick HJ, Denny FW Jr. Computer instruction in learning concepts of streptococcal pharyngitis. Arch Pediatr Adolesc Med. 1996;150:629-31.
4. Bell DS, Fonarow GC, Haws RD, Mangione CM. Self-study from web-based and printed guidelines materials: A randomiozed, controlled trial among resident physicians. Ann Intern Med. 2000;20:938-46.
5. Schwid HA, Rooke GA, Ross BK, Sivarajan M. Use of a computerized advanced cardiac life support simulator improves retention of advanced cardiac life support guidelines better than textbook review. Crit Care Med. 1999;27:821-4.
6. Bayne T, Bindler R. Effectiveness of medication calculation enhancement methods with nurses. J Nurs Staff Dev. 1997;13:293-301.
7. Toth-Cohen S. Computer-assisted instruction as a learning resource for applied anatomy and kinesiology in the occupational therapy curriculum. Am J Occup Ther. 1995;49:821-7.
8. Lyon HC Jr, Healy JC, Bell JR, et al. PlanAlyzer, an interactive computer-assisted program to teach clinical problem solving in diagnosing anemia and coronary artery disease. Acad Med. 1992;67:821-8.
9. Santer DM, Michaelsen VE, Erkonen WE, et al. A comparison of educational interventions. Multimedia textbook, standard lecture, and printed textbook. Arch Pediatr Adolesc Med. 1995;149:297-302.
10. Andrews PV, Schwarz J, Helme RD. Students can learn medicine with computers. Evaluation of an interactive computer learning package in geriatric medicine. Med J Aust. 1992;157:693-5.
11. Angelo TA, Cross KP. Classroom assessment techniques: A handbook for college teachers. San Francisco, CA: Jossey-Bass, Inc; 1993:6.
12. Buchanan T. Using the world wide web for formative assessment. J Educ Tech Systems. 1998-1999;27:71-79.
Appendix A: Paper Assessment Form
Appendix B: On-line Assessment Tool
Developing Search Strategy Guides for Teaching Pharmacy Students How to Find Full-Text Journal Articles
Abstract
In an academic setting, knowing how to find full-text articles, especially articles within an electronic database, can be exasperating. Simple strategies can be developed in accordance with resources available that will greatly assist students toward finding full-text articles. These search strategies are not designed to teach how to search various full-text databases, but merely points the student in an systematic order to those resources (both print and electronic) that are available. Designing full-text guides involves combing 3 factors: identification of available resources; where they are physically or electronically located; and illustrating the ease of accessibility. The order of these three will vary among institutions. The task is to evaluate these 3 factors and design a straightforward guide so students can easily locate full-text articles. Using such strategy guides reinforces the lifelong learning process. Due to the nature of the profession, current information regarding drugs and diseases will always be a necessity.
Introduction
Finding a full-text journal article, especially an electronic journal, is often a very frustrating experience for the pharmacy student. The need may arise from a class assignment or the student's individual research endeavors. Often, pharmacy students do not know how to begin the process of finding full-text journal articles, even if they have the bibliographic citation for the article. Citations with abstracts normally do not provide enough information to make a definitive decision regarding the efficacy of a drug, treatment interventions, dosing regiments, etc; therefore, the full-text article is necessary in most cases.
It is the author's opinion that in the future, assignments to review primary literature will become more prevalent in pharmacy education. First, there appears to be a viable notion that pharmacy education should consider adopting the principles of evidence-based medicine (EBM).[1] EBM is predominantly contained in the primary literature. Second, pharmacy students soon discover that to understand the full scope of drug treatment, both the drug and the disease must be considered; thus, basic medical journals that report on clinical trials, disease reviews, new drug indication studies, etc., must be available and easily accessible. Third, because of its nature, pharmacy education is considered to be a "lifelong learning" process; therefore, finding, acquiring and searching for primary literature is something the pharmacy student and the pharmacist will be doing throughout their pharmacy career. For this reason alone, learning very early in the pharmacy curriculum a structured strategy for obtaining full-text articles would be beneficial, if not essential.
The purpose of this brief article is to offer suggestions to schools of pharmacy in order that they might effectively develop their own strategies toward assisting students for finding full-text journal articles.
Role of the Pharmacy School
The role of the pharmacy school for developing search strategies to teach their students how to find full-text journal articles is critical. This is largely due to the 3 reasons mentioned above, but primarily because developing these full-text search strategies is a learning process in and of itself. The same principle is applicable to learning how to search databases such as PubMed (Medline) or the Iowa Drug Information System (IDIS). These learned strategies are required throughout the pharmacy curriculum and likely, throughout one's professional career. Development of guidelines for full-text search strategies is no different than developing guidelines for searching PubMed or other databases. These guidelines are particularly useful for drug information courses and clinical rotations. When developed, guidelines can be used by all personnel, not just students. Faculty members need to be very cognizant of journals available (and not available) in full-text. How many times does it happened that a faculty member assigns an article for to review that was not available as a full-text via any means for the student? This normally happens when the faculty member simply is not aware of what is available.
Developing Guides for Full-Text Journal Articles
The drug information faculty are most likely the best suited to develop strategy guides for finding full-text journal articles and should collaborate with the school/university health librarians to identify all available medical resources. This aspect will be discussed in more detail later. It is highly recommended that the guides be developed in a Web-based format, that is, that they be placed on the Internet. There are several reasons for this, but primarily because the Internet is generally accessible by all students; and second, because Web-based items are easy to edit. An example of a Web-based format is the McWhorter School of Pharmacy, Samford University, Guide for Finding Full-Text Journal Articles (URL: http://www.samford.edu/schools/pharmacy/dic/fulltxt.htm).[2] In this particular format, the pharmacy student (or user) simply follows the litany order, beginning with the easier strategies and working through to the more time consuming. The guides developed at the McWhorter School of Pharmacy for finding full-text journal articles are not designed to teach the student how to search databases, but merely point the student to those resources (both print and electronic) that are available in a systematic order.
One of the biggest advantages for designing full-text guides is to distinguish the from nonfull-text databases. Nonfull-text databases are those that render only a bibliographic citation and normally an accompanying abstract. A good example of a nonfull-text database is PubMed; however, occasionally, it does link to free full-text articles, but it is generally considered a nonfull-text database. Full-text databases are usually procured via the school library or sometimes by a specific school and for some databases, the full-text capability may be procured separately. Compounding the issue, many databases have only selected journals that are full-text, and even within these full-text databases, journals with full-text capability change frequently because of licensing agreements. There are numerous medical related full-text databases, for example, Elsevier's Science Direct, MD Consul, EBSCOs Academic Search Elite or Health Source:Nursing/ Academic Edition, and InfoTrac OneFile Plus, to name a few. IDIS, a pharmacy specific database, also has options for full-text capability. Obviously, some journals are "free-standing" and are available not full-text in any database.
There are numerous methods of how to organize strategies for full-text searching; however, there is a basic hierarchical structure that should be applied that will enhance the user's ability to follow a logical order beginning with the easiest, then proceeding to the more difficult or time consuming. Obviously, each school of pharmacy will have different full-text resources available and this must be taken into account, but the logical order should remain about the same.
Designing full-text guides involves combing 3 factors: identification of available resources; where they are physically or electronically located; and illustrating the ease of accessibility. The order of these three will vary among institutions. The task is to evaluate these 3 factors and design a straightforward guide so students can easily locate full-text articles.
Pharmacy Specific Resources
The first full-text category listed should be those resources specific to pharmacy. These resources would be those that are immediately available and likely most pertinent for the pharmacy student. A good example of this type of resource would be print or electronic journals specific to the pharmacy student that might be physically available in the school of pharmacy.
A second example of these types of resources would be pharmacy and/or medical specific full-text databases, e.g., IDIS, MD Consult, Cochrane, Ovid, etc.
University Resources
The next category of resources are those immediately outside the realm of the pharmacy school. A good example of this type of resource would be both the print and electronic collection within the on-line catalog of the university library. University libraries catalog their electronic resources different ways, for example, some place a journal title link in the catalog to the Web location of the journal in their on-line database. Others place the full-text electronic journals into a separate listing within the library's Web pages, while others may do a combination of both. The reason for mentioning this critical item is that pharmacy students are often not well versed as to what full-text resources are available or how to access them via the university library, especially full-text capabilities due to the many databases and other resources, which is discussed below in more detail.
Most, if not all, university libraries have an interlibrary loan system (ILL) for obtaining full-text articles. The foremost problem with ILLs is the lack of promptness. Often, receipt of the article may take in excess of 2 weeks. Student assignments will normally require the article sooner than 2 weeks.
University full-text medical databases are one of the best sources for articles. Some have already been mentioned above, others databases are the Cumulative Index to Nursing and Allied Health (CINAHL), Wiley Interscience, etc. One of the major difficulties is that students are often not aware of the databases or cannot easily find them on the university library's Web pages. Full-text databases must be clearly and easily identifiable to the user. It serves no purpose to have very expensive full-text databases, yet students are unaware of them or worse, do not know how to access them.
Other Resources.
The category of other resources is basically a last resort category, i.e., when the above strategies product no results. Considerable trial-and-error attempts will likely occur in this category. Several Web sites have very good resources (links) to free full-text journals, e.g., The Free Medical Journals[3] or PubMed's LinkOut.[4] This search strategy is very time consuming, but the user might be fortunate and locate the desired journal at one of these sites. There are numerous other journal sites created by academic medical libraries or medical associations. Some are free to access, while others require an id and password.
Summary
Schools of pharmacy should develop easily used strategies for finding full-text articles and preferably, structure them in some type of sequential order depending on resources available. The simpler the guides are, the more likely they are to be used and understood. Also, placing them on the Internet will enhance the overall utility and students' ability to find articles, tremendously reducing library personnel's time by having to explain an often complex and lengthy procedure. Full-text guides will assist toward a lifelong learning process. Current information regarding drugs and diseases will always be a necessity.
References
1. Beck DE. Pharmacy educators: can an evidence-based approach make your instruction better tomorrow than today? Am J Pharm Educ. 2002;66:87-8.
2. Guide for Finding Full-Text Journal Articles. Available at: http://www.samford.edu/ schools/pharmacy/dic/fulltxt.htm. Accessed September 26, 2003.
3. The Free Medical Journals. Available at: http://www.freemedicaljournals.com. Accessed September 26, 2003.
4. PubMed LinkOut. Available at: http://www.ncbi.nlm.nih.gov/entrez/ journals/ loftext_noprov.html. Accessed September 26, 2003.
Tracking Pharmacy Students' Use of Personal Digital Assistants (PDAs) in a Clinical Simulation Laboratory Course
Thanks to Gregory Poon, BScPhm, PhD, and David Dubins, PhD, who at the time of this study were graduate students at the Faculty of Pharmacy, University of Toronto.
Abstract
Objectives: To determine the role and value of personal digital assistants (PDAs) with the ePocrates software program in pharmacy education.
Setting: A standardized-patient driven clinical simulation course for 4th year (senior level) pharmacy students, over a two-year period with 2 different classes (n = 237).
Methods: Students were provided with a standard library of pharmacy reference textbooks in addition to a Palm Pilot with ePocrates (freeware version) software. As part of the clinical simulation course, students interviewed standardized patients (actors trained to portray patients with medical conditions), and were expected to consult tertiary references to assist them in their interviewing, problem solving, and patient management activities. Students were asked to self-report utilization of PDAs (Personal Data Assistants) vs. text-based resources. Following completion of the course, focus groups of students were convened to discuss the study, and different methods they had selected to access information during the clinical simulation activities.
Results: Students demonstrated purposeful selective use of PDAs, but more frequently and more consistently preferred using text-based references. The exception to this was in circumstances in which they believed text references would be out of date or they clearly identified their own information need. In situations of greater ambiguity regarding the information need, students preferred to access the textbooks, since these provided an opportunity for visual scanning that appeared to assist them in problem-identification and problem-solving.
Conclusions: PDAs may complement textbooks within a clinical simulation environment, but do not appear to replace them. Since the small screen limits text-scanning ability, students (who are in a high knowledge-uptake stage of their professional development) found greater utility in use of textbooks, despite comfort, familiarity and general liking of PDAs.
Keywords: Pharmacy Education, Personal Digital Assistants, PDAs, Learning Styles, Educational Technology
Background
The use of technology to optimize student learning is ubiquitous in post-secondary education in North America.[1] Advances in technological prowess, coupled with steadily declining costs, have allowed students in many disciplines to benefit from the processing power and memory capacity of a variety of electronic devices, including personal computers (PCs) and personal digital assistants (PDAs). Widespread availability of Internet-based resources, and an increasingly computer-savvy student body have resulted in changes to academic programs and curricula.[2]
Pharmacy education and practice have traditionally been on the leading edge of incorporating technologies.[3] As health care professionals, pharmacists have been among the first to routinely use (and rely upon) computers as part of professional practice. Today, few clinicians could function without access to electronic pharmacy records, databases, and other computer-based applications. As a profession, pharmacy is frequently seen as a leader within health care regarding the introduction of new technologies. Within education, pharmacy was one of the first health professions to integrate computers into academic programming in the 1970s and 1980s. During this time, use of efficiency optimizing database programs (such as dispensing or patient record-keeping software) was emphasized.
Today, within the academic pharmacy literature, there have been numerous case reports detailing varying degrees of success with implementation of computer-assisted or technologically mediated teaching-and-learning tools.[4,5,6,7,8,9,10] For example, use of Web-based course notes to complement or supplement in-class presentations is now a routine expectation for many students and faculty. With growing availability of band-width, increasing ease of transmission through high-speed Internet services, and a more sophisticated end-user, video-streaming and other functionalities have also become more commonplace.
A key area of discussion and debate in the academic computing literature revolves around the balance between technologies which enhance connectivity and those which enable access to content. Proponents of connectivity-enhancing technology describe the value of creating virtual communities of learners and practitioners, so individuals may engage in interactive, social learning environments. From this perspective, the value of technology may be seen as allowing individuals to form relationships despite geographic or temporal distances or barriers. The value of these relationships is to facilitate deep learning, rather than superficial acquisition of facts. Alternatively, proponents of content-accessing technology describe the value of unfettered access to a large (and ever-growing) database of relevant facts and knowledge. Through the use of sophisticated search engines, point-and-click technologies, and memory media, it is possible to access vast amounts of information in a relatively easy manner. Advocates suggest that it is now possible to create entirely virtual libraries, completely free of written text-based references (such as textbooks), and that this is a desirable end-point given the limitations of paper-based materials. Such limitations include cost, the need to constantly update editions, size and bulkiness of texts, the lack of customizable searching facilities, and the need to "wade" through a mass of printed material in order to find a specific, desired piece of information.
Over the past decade, significant advances have been made in creating smaller, more portable content-accessing technologies. As costs for memory have declined significantly over time, the widespread availability of such devices has increased. A key feature of such devices is their ability to capture and store a large amount of data, and provide the end-user with the capacity to customize personal databases and portals of entry. Major advantages of such devices are their relative portability and ease of use.
Among the more popular of such content-accessing technologies, Personal Digital Assistants (PDAs) have become commonplace. First introduced in the late 1980s, PDAs were initially somewhat bulky and awkward, and were limited in portability and utility due to memory constraints. Advances in technology have now resulted in development of sophisticated hand-writing translation platforms (such as Graffiti), larger and expandable memory, user-friendly colour interfaces, and longer battery life to enhance portability.
As content-accessing technologies, PDAs possess remarkable potential within health care education and practice, particularly within an information-intensive profession such as pharmacy.[11] Most pharmacists are familiar with the need to consult references to verify important facts and salient details; indeed, as a professional group, it may be argued that pharmacists demonstrate a unique reliance on tertiary reference sources to support professional practice. With the emergence of personal computers, an increasing number of formerly text-based tertiary references (eg, Physician's Desk Reference and Remingtons: The Science and Practice of Pharmacy) have become more widely available in electronic/digital formats, replacing the need for expensive, space-consuming textbooks. However, personal computers occupy space, require electricity and are generally a non-mobile technology.
Over the past 5 years, there have been significant advances in the number and quality of PDA-based clinical references that have been developed and marketed to pharmacists. These references provide the same high-quality information as tertiary textbooks, but with the advantage of portability and search engines to facilitate information retrieval. Currently, many clinicians rely heavily on their PDAs to provide up-to-date information that formerly would have required greater time and effort to locate using textbooks. The availability of PDA readers, more comfortable and user-friendly colour interfaces, and improved fonts have all contributed to greater acceptability of this technology within professional practice.
As a result of changes to the way pharmacists access information in practice, there have been increasing calls to ensure that pharmacy students are well-prepared for the technologically intensive nature of professional work. As a group, pharmacy students demonstrate a relatively high degree of computer literacy and are generally open to the use of technology. Particularly for those students who have received secondary education in North America over the past decade, the presence of computers in the classroom and at home has resulted in a high degree of acceptance of the role of technology in enhancing the quality of patient care.
Nonetheless, the role and value of content-accessing technology in pharmacy education has not been adequately assessed. While there have been case reports outlining the use of PDAs in courses and development of specific, customized PDA-based platforms and programs for pharmacy students, the way in which PDAs actually improve the efficiency and/or effectiveness of learning and clinical skills development has not been described.
Objective
The objective of this study was to determine the role and value of personal digital assistants with the ePocrates software system in pharmacy education. For the purposes of this study, the PDA was used as a content-accessing technology only; although PDAs may also be used as connectivity-enhancing technologies, this was not a focus of this study because such applications are currently somewhat limited in practice.
Setting
This study was undertaken with 2 cohorts of senior-level (year IV) pharmacy students in the undergraduate (BScPhm, the entry-level degree) program at the University of Toronto. Students in this program are all required to take a professional practice laboratory course in their senior year, Pharmacy Practice Laboratories IV (PPL). The PPL course consists of a 10-week series of clinical simulations involving standardized patients (actors trained to portray patients with simulated medical conditions).[12] Students are divided into laboratory groups of 8-10, and placed under the tutelage of a pharmacist teaching assistant. At the beginning of the course, students are introduced to a fictional, multi-generational "family." Over the 10-weeks of the course, family members (portrayed by standardized patients) visit the simulated pharmacy setting, interacting with the pharmacist. In certain weeks, standardized patients return for follow-up visits, in order to model disease progression and to emphasize continuity of care. In addition, the course has been mapped out to emphasize connectivity among different family members; for example, in week 3, a teenage girl visits the pharmacy to have a prescription for oral contraceptives filled, and to receive counseling on safer sex; in week 7, her irate father appears insisting on speaking with the pharmacist who dispensed these medications without his knowledge.
PPL attempts to recreate the environment of a typical community pharmacy through the innovative use of clinical simulations involving standardized patients. Each case within the course is designed to portray complex clinical and psycho-social issues, and since patients may re-appear in future sessions, students are required to maintain clinical records and notes, engage in monitoring and follow-up, and provide pharmaceutical care. A global assessment system is used in this course, in which integration of students' clinical knowledge and communication skills is required in order to successfully address critical issues in the case.
Students in the program take the PPL course during their final year, along with several other courses, including advanced pharmacotherapeutics. The sequence of cases was designed to complement the course schedule in pharmacotherapeutics and to provide reinforcement and clinical application of material learned in this course. This means that students may attend a case-based lecture in a certain therapeutic area at the end of one week, and then be required to apply this information in a clinical simulation the next week. Thus, students in this course (who were participants in this study) are learners and novices without significant clinical experience or expertise, who have an on-going need for reinforcement of their learning.
Methods
Students in PPL were provided with training in use of a PDA device (Palm Pilot Vx ) and a free-ware program (ePocrates). The ePocrates program is freely available and downloadable from the Internet (www.epocrates.com) and may be used by clinicians, students, or the general public at no cost. An enhanced version of this program is also available for cost. ePocrates was selected for this study since it is also frequently used by pharmacists due to its ease of access, relatively rapid updating of information, ease of use, and lack of subscription or on-going costs. As part of this study, a one-hour tutorial was provided to all students on the use of this program.
Following this training session, students began the PPL (clinical simulation) course described previously. For each simulation, students were provided with both the PDA (loaded with ePocrates) as well as standard pharmacy reference textbooks. The reference textbooks sources were made available to students should they require them as part of their role-playing in the clinical simulation. Students were not required to utilize either the PDA or the textbook based references; these were simply made available to students to more realistically simulate a clinical setting and to provide them with support as learners.
Following each clinical simulation role play, students reported the number of times they utilized either print-based textbooks or the PDA, and the nature of this utilization (ie, reason for "looking up" information). These reports were collected and analyzed. At the end of the course, two focus groups with randomly selected students were convened to discuss results and guide data interpretation.
This study was undertaken over a two-year period with 2 consecutive senior level classes. The class of 2000 consisted of 117 students and the class of 2001 consisted of 120 students.
Results
No statistically significant differences were noted between the class of 2000 and the class of 2001, or between male and female students in their utilization of the PDA or text-based references during clinical simulations. Consequently, for convenience, data are reported and analyzed for a single cohort of 237 students, rather than two different cohorts.
Figure 1 illustrates the frequency of use of text-based references and the PDA across the 10-weeks of the clinical simulation course. Each time a reference book was consulted or the PDA utilized during the simulation, it was counted as an independent event (eg, if a student consulted a reference to verify a dose on a prescription at the beginning of the simulation, then consulted the same [or a different] reference later in the simulation to learn about side effects, this counted as two events). In some weeks the total number of consultations was less than 237, since not all students consulted a reference (PDA or text) during the course of a simulation.
Figures 2a and 2b illustrate the reasons for consulting references and a comparison of type or references used (textbooks or PDA), based on category. Table 1 provides demographic and background information about student-participants in this study, including sex, age, previous experience with computers and PDAs, and previous clinical experience.
Discussion
Demographic information regarding students' familiarity and comfort with use of technology was confirmed through focus group discussions. In general, participants demonstrated a relatively high degree of comfort with PDA technology, and many (40.5%) owned their own PDA (usually a Palm Pilot) prior to involvement in this study. All students surveyed indicated they had used computers in the past within a pharmacy practice setting and the vast majority (91.1%) indicated they owned a personal computer. In focus group discussions, students indicated a high degree of comfort with use of technology in pharmacy education and practice, and an expectation that they would be required to access technology and electronic databases as part of their undergraduate education and post-graduate training and practice. Male students reported greater ownership and familiarity with computers and PDAs.
Analysis of data from this study illustrates differential utilization of textbook references and the PDA-based ePocrates. For this study, commonly occurring textbooks in Canadian pharmacy practice were made available to students, including the Compendium of Pharmaceuticals and Specialties (CPS, a compilation of drug-specific monographs similar to the Physicians' Desk Reference in the US), Therapeutic Choices (a standard reference providing algorithms for treatment of commonly occurring medical conditions), Martindales (The Extra Pharmacopoeia), as well as a commonly-used pharmacotherapeutics textbook, such as DiPiro's Pharmacotherapy).
As illustrated in the data, students generally preferred use of the print-based textbooks over use of the PDA, despite expressing general comfort and familiarity with use of the PDA device. Over the 10 weeks of the course, textbooks were consistently used more frequently than the PDA, with the exception of week 9. The difference in utilization frequency narrowed somewhat over the course of 10 weeks, but not in any statistically significant manner. Interestly, frequency of PDA use also spiked at week 5.
Focus group discussions with students provided insight into the spiked frequencies at weeks 5 and 9. In both of these sessions, clinical simulations involved complex disease states (HIV/AIDs in week 5 and cancer chemotherapy in week 9). Students indicated that they switched to use of PDAs for these sessions simply because they believed that information in the printed textbooks would not be as up-to-date as what they could find in the PDAs, which were updated off the ePocrates website on a monthly basis. However, students also reported first seeking information in the textbook, then moving to the PDA only when they believed the textbook was inadequate for their information needs. This propensity to first access written textbooks, then move to the PDA was confirmed over the course of the 10-weeks, and was consistent between the two cohorts studied.
Data regarding reasons for use of PDAs vs. textbooks similarly illustrates significant differences. In focus group discussions, students were asked to reflect upon their experiences during the study and provide reasons for these findings. There was broad agreement that one key reason for differences in utilization of PDAs vs textbooks related to the way in which data was presented in these different media. Students indicated that with a textbook, it is possible to scan multiple sections of a drug monograph or a chapter in a quick and efficient manner. Students found this to be a particularly useful feature of textbooks in situations in which they were not certain of which specific piece of information they required. For such ambiguous information, the ability to scan a broader array of information was an important problem-solving strategy for students.
One student provided an example related to side effects. During the clinical simulation, the patient reported feeling "tense" and "tight" muscles and wanted advice regarding appropriate analgesia. The student had previously identified that the patient had recently started on a statin for hyperlipidemia and the student suspected that this temporal link was important. Having only recently completed a lecture in pharmacotherapeutics of hyperlipidemia, the student's knowledge base was somewhat weak and he needed to confirm his suspicion:
"That's the problem with the [PDA]. It's great if you know exactly what you need - like the dose of the drug, or a drug interaction. But if you're trying to figure something out - like this side effect - well, it's not going to list tense or tight muscles like that. But when I checked [the reference text] and did a quick read of the side effects section [of the drug monograph], there it was."
As this student's experience suggests, participants' perception of the utility of the PDA appeared to be a function of the specific circumstance in which information was required. In circumstances in which students felt confidence in knowing what piece of information was required, the PDA was accessed preferentially over textbooks. Examples of such situations included verification of dosing, identification of specific drug-drug or drug-food interactions, or identification of contraindications. In other circumstances, where the information need was somewhat less defined, more ambiguous, or where the problem was not entirely clear to the student, preference appears to have been given to use of textbooks. Examples of such situations included providing general counseling information, identifying mechanism of action/pharmacological activity of a medicine (categorized under "Other" in Figures 2a and 2b), or trying to establish a causal link between emergence of a side effect and initiation of a new drug.
In focus group discussions, students identified the value associated with scanning a drug monograph for information, particularly when they were unsure as to how best to proceed. On many occasions, they described how scanning for information in a textbook appeared to provide important cues to them for further information gathering or questioning of the patient and suggested potential solutions to problems. In contrast, the small size of the PDA screen, coupled with its inherent text limitations and somewhat awkward scanning facilities, limited the utility of the PDA in such circumstances.
Interestly, students themselves identified how each type of resource (eg, PDA or textbook) could be most efficiently and effectively used. During clinical simulations, the students acted in a consistent and predictable manner, regardless of their underlying propensity towards use of technology, previous experience with PDAs, or previous clinical experience. Despite lack of formal instruction regarding optimal circumstances for use of textbooks vs. use of PDAs, students appeared to develop and use a series of informal rules governing how to optimize success in clinical simulations and information acquisition.
Limitations
A major limitation of this study is the use of only one PDA-based platform, ePocrates. Currently, there are many different software programs available. ePocrates was selected because it was freely available, had no cost, and was generally well recognized within the pharmacist community. However, its interface may not be as sophisticated as other programs, consequently limiting students' ability to scan. Thus, conclusions of this study may not necessarily apply to PDAs in general, or other PDA-based programs (including the for-pay version of ePocrates) specifically.
Methodologically, this study relied upon self-reporting and self-reflection by students regarding their performance in clinical simulations. Such methods may be imprecise and suffer from time-linked memory deterioration. Alternative methods for tracking utilization on the PDA could be used that do not rely upon self-reporting and observers could also be used to track utilization of textbooks to improve accuracy of reporting.
Limitations
A major limitation of this study is the use of only one PDA-based platform, ePocrates. Currently, there are many different software programs available. ePocrates was selected because it was freely available, had no cost, and was generally well recognized within the pharmacist community. However, its interface may not be as sophisticated as other programs, consequently limiting students' ability to scan. Thus, conclusions of this study may not necessarily apply to PDAs in general, or other PDA-based programs (including the for-pay version of ePocrates) specifically.
Methodologically, this study relied upon self-reporting and self-reflection by students regarding their performance in clinical simulations. Such methods may be imprecise and suffer from time-linked memory deterioration. Alternative methods for tracking utilization on the PDA could be used that do not rely upon self-reporting and observers could also be used to track utilization of textbooks to improve accuracy of reporting.
Conclusions
While there appears to be little doubt that technology is an important facilitative tool for pharmacy education, this study has illustrated both important strengths and limitations of a specific technology and software platform. Students in this study appeared to generate a series of tacit, consistent rules to assist them in identifying circumstances, in which accessing textbooks would be more strategically advantageous than using a PDA and vice-versa. Students' information-seeking behaviours illustrate the importance of visual scanning of information (such as a drug monographs) in helping them to identify, frame and solve certain kinds of clinical problems. The nature of printed textbooks facilitates such scanning and allowed students an opportunity to make connections between clinical cases and drug information resources. Though seemingly time-inefficient, the act of flipping through a textbook appears to be an important step in problem solving for some students. When confronted with an unfamiliar situation, students appear to be able to derive cues from textbooks in a more meaningful way than from the PDA. Conversely, in situations where students are more certain of their information need, the PDA appears to be a more useful and efficient resource. Features such as pull-down menus facilitate access to specific facts without the distraction of scanning text.
Despite comfort with the technology platform, students' learning needs are such that textbooks still appear to be an important tool for development. While it may be argued that some digital media (such as PC Pads) can recreate the scanning facility afforded by textbooks, further research will be required to confirm whether such technology will actually change students' information-seeking behaviours.
It appears that PDAs are an important complement to, but not a replacement for, textbooks in pharmacy education. Students in this study self-adapted to the differing strengths and limitations of each media, and appeared to evolve a self-regulating system for optimizing use of resources within the context of a clinical simulation. Further research in information seeking behaviours of students is required to understand this important skill; in the mean time, it appears that print-based textbooks will continue to have an important role in pharmacy education.
References
1. Zarotsky V, Jaresko GS. Technology in education - where do we go from here? J Pharm Pract. 2000;13:373-381.
2. Ellington TE, Thacker D, Kushner JM. The evaluation of pharmacy students' use, knowledge and attitudes toward computers and proprietary databases. ASHP Midyear Clinical Meeting. 2000; 37(Dec):P-240R.
3. Miller LG. Exploring the potential impact of the electronic revolution on pharmacy education. J Pharm Teach. 1998; 6(3):3-7.
4. Chong E, Balen RM, Jewesson PJ. Design of multimedia educational presentations via streaming media. Am J Health-Syst Pharm. 2003;60(23);2475-2478.
5. Leemans L, Verstraeten A, Zwaenepoel L, Laekeman G. The use of a virtual learning environment during the internship of final year pharmacy students. Pharm Edu. 2003; 3(2):73-76.
6. Brown MC, Lind PR, Sorensen TD. Development and implementation of software to facilitate documentation, reflection, and feedback in a service learning experience. American Association of Colleges of Pharmacy Annual Meeting. 2002;103 (Jul):11.
7. Grilla JA, Stolte SK, Lewis J, Robinson ET. Integration of web-based computer aided instruction into a non-traditional Doctor of Pharmacy program. J Pharm Prac. 2000; 13(Oct):382-391.
8. Fein JL, Colaizzi JL. Implementation of computer technology initiatives to enhance pharmacy education. American Association of Colleges of Pharmacy Annual Meeting. 2000;101 (Jul):292.
9. Triller DM, Bruce SP, Hamilton RA. Reference databases to enhance pharmacy education and communication skills. American Association of Colleges of Pharmacy Annual Meeting. 1999;100 (Jul):121.
10. Kier KL, Mauro LS. Cooperative faculty development of a web-based drug information course for non-traditional Doctor of Pharmacy program. American Association of Colleges of Pharmacy Annual Meeting. 1998;99 (Jul):84.
11. MacKinnon GE. Development of a personal digital assistant application for pharmacy documentation. Pharm Edu. 2003;3(1):11-16.
12. Austin Z, Tabak D. Design of a new professional practice laboratory using standardized patients. Am J Pharm Edu. 1998; 62:271-279.
Comparison of Students' Performance and Perceptions of a Web-based Distance Pharmacy Calculations Course to a Campus-based Course
Abstract
Objectives: This study compared full-time entry-level Web-based pharmacy student performance and perceptions to campus-based student performance in a pharmacy calculations course over 3 years.
Methods: Web-based and campus-based student performance was measured using identical assessments within each year, including an online mastery examination. Student perceptions were assessed using responses to the course evaluations each year.
Results: The study population comprised 172 students who completed the online course and 323 students who completed the on-campus course between 2001 and 2003. Campus-based students performed better than Web-based students in 2001, and both groups performed similarly in 2002 and 2003. Web-based students consistently evaluated the course more positively than campus students.
Conclusions: An entirely Web-based pharmacy calculations course appears to be well accepted by full-time Web-based entry-level students. However, additional strategies may be needed to support Web-based students to ensure similar levels of performance the first time an online course is offered.
Keywords: Pharmaceutical calculations, pharmacy education, Internet education, distance education, Web-based instruction
Introduction
In the fall of 2001, Creighton University School of Pharmacy and Health Professions initiated the first entry-level, Web-based distance doctor of pharmacy degree whereby all didactic courses are delivered via the Internet. Details of the Web-based pathway are available online at http://pharmacy.creighton.edu/spahp/non_traditional/rx/async_overview.asp.
The use of the Internet for delivering course materials to traditional or non-traditional students is not new to pharmacy education, although delivery of entirely online courses with no face-to-face contact during a semester is not common in entry-level pharmacy education. Several manuscripts describe technology-mediated innovations in teaching pharmacy calculations to enhance on-campus student learning, including use of Web-based technologies,[1,2] computer-assisted instruction,[3,4] supplementary materials on multi-platform CD-ROMs,[5] and computer grading of student answers.[6] One study described an online calculations course for foreign-licensed pharmacists seeking licensure in Australia.[7] However, no manuscript was found that evaluates the performance of entry-level students over several years in an entirely online pharmacy calculations course.
One challenge in developing and instructing basic science courses in the entry-level Web-based PharmD curriculum is the lack of documentation of the effectiveness of required courses in a full-time, entry-level, Web-based PharmD program. This manuscript compared the academic performance and perceptions of Web-based and campus-based students in a required pharmacy calculations course. The authors compared Web-based and campus-based student performance and course evaluations in each of the first 3 years of the Web-based pathway.
Course Description
The pharmacy calculations course is a 2 credit hour required course in the first semester of the PharmD program. The same syllabus is used for both campus and Web-based sections of the course. The same instructors teach both campus and online students. Student performance was assessed using the same Web-based examinations and a single Web-based mastery examination. The instructors developed this mastery examination to extract possible questions from a large pool on each content area in the course. Students are allowed a maximum of 4 attempts to pass the mastery exam.
The final percent grade is based on the following weighted components: mastery examination 50%, examinations 40%, peer evaluation of team-based active learning 10%. The final letter grade is assigned based on earning the following minimum weighted overall percent scores: A = 96%, B+ = 91%, B = 86%, C+ = 81%, C = 71%, D = passes the mastery exam with 70% or higher score, and overall percent grade is less than 71%, F = does not pass mastery exam with 70% or higher score, regardless of overall percent score.
Campus-based students received assistance and clarification of content during scheduled class time and during office hours or by appointment. Web-based students received assistance and clarification of topics via e-mail, discussion board postings, and the telephone, including toll-free telephone access for students in the United States.
Both Web-based and campus-based students accessed instructor-generated course material through a course Web site. In 2001, the Web site was developed and maintained using Microsoft FrontPage (Microsoft Corp., Redmond, Washington. http://www.microsoft.com/ frontpage). The instructors redeveloped the course Web site on the Blackboard Learning Management System in 2002 (Blackboard, Inc., Washington, DC, http://www.blackboard.com).
Use of Technology
All entry-level PharmD students receive an intensive orientation by the school's Office of Information Technology and Learning Resources on the use of computer technology and its role in the pharmacy curriculum. All students are issued standard notebook computers immediately prior to the semester, and receive several full days of hands-on training in the use of the computer, Internet use and access, assessment software, discussion board postings, and instant messaging. Students are also introduced to active learning methods used in the curriculum. The use of notebook computers in the campus-based pathway preceded implementation of the Web-based pathway.
Methods
Results
The overall student grades, shown as a percent of total possible points for campus and Web-based students in 2001 - 2003, are listed in Table 2.
Campus-based students in 2001 had a higher overall final percent score compared with Web-based students (p < 0.0001), although both groups demonstrated a high level of overall performance. There was no difference in percent scores in 2002 or 2003 (p = 0.3874 and 0.6444, respectively). A comparison was also performed for final letter grades within each year to determine if there were differences in final letter grades. Final letter grades for campus and Web-based students are listed in Table 3.
Campus-based students in 2001 had higher letter grades on a 4-point scale (p = 0.0002). There was no difference in distribution of final letter grades in 2002 or 2003 (p = 0.0901 and 0.2368, respectively).
Evaluations of the course and instructor by Web-based students were consistently more positive than evaluation by campus students. All student responses to scaled questions on the 2003 course evaluation are summarized in Table 4.
Though evaluations were generally very positive for both campus and Web-based students, most questions showed a statistically significant difference indicating that Web-based students rated questions more positively than campus students.
Students also had the opportunity to provide free-response comments about the course and instructor. Student comments on the evaluations generally included 1 of 3 themes: first, students from both pathways responded favorably to the format of the course that allowed them repeated attempts at examinations, practice mastery exams, and the benefits of group work. Second, students found that the instructor-generated content for each topic was useful and assisted them in learning the content. Third, students indicated that they struggled with the concepts of mastery assessment and critical errors, at least initially.
Discussion
Distance-based education on the Internet is a growing trend in higher education. One can often draw upon published literature to find advantages or disadvantages of new teaching methods. However, a descriptive comparison article on teaching and learning strategies for Web-based and campus-based pharmacy calculations courses has not been published.
The newness of the entry-level, Web-based distance doctor of pharmacy degree necessitates the measurement of outcomes for parity's sake. Documentation and measurement of student performance and perceptions in this required course is an expectation for maintaining the quality of education offered by the school.
Each year, the campus-based students earned higher overall percent scores and overall letter grades, and there was a statistical difference between campus-based and Web-based student performance in the first year. There are a few possible reasons for the differences of these scores. The first possible reason for the campus-based students to have higher overall scores is the tendency for Web-based students to hold full-time jobs while attempting to take full-time course loads in the professional program. The second possible reason is the tendency for Web-based students to procrastinate in completing course requirements, often waiting until the very last hours to complete the mastery exam. Consequently, if the Web-based student did not achieve a desired grade on the mastery exam, it was often too late to study and make corrections to improve overall performance. A third possible reason is that Web-based students may have been more interested in achieving a balance of school, work, family, or other commitments than maximizing an acceptable grade in a course.
The instructors believe that a series of concurrent changes that occurred within the pharmacy program may have negatively affected the perceptions and attitudes of campus-based students on the evaluations, resulting in the more favorable ratings on the evaluation scores by the Web-based students. In particular, the school has identified three core issues that negatively affected campus-based student attitudes. The core issues were the implementation of a laptop computer program one year prior to the implementation of the Web-based pathway, overzealous use of active learning strategies, and structure and process issues occurring in the development of the entire Web-based pathway. The authors felt that the campus-based students tended to be more critical of any course that relied heavily upon technology for implementation or delivery of learning materials.
The implementation of active learning strategies, which is one of the expectations of pharmacy program accreditation, is often not appreciated by campus-based students entering the PharmD program. Many campus-based students enter the program directly after completing pre-pharmacy requirements. Informal discussions with students indicate that many students do not have experience with team-based active learning in their previous academic experience. However, Web-based students do not perceive working with their classmates as a burden, perhaps because the technology provides both the social and academic networking between classmates, or perhaps because they recognize that participating in a team makes learning easier or more enlightening.
The instructors identified a challenging situation regarding to the amount of faculty time needed to prepare teaching and testing materials, monitor discussion boards, answer student e-mails, and design and maintain the course Web site. However, using new technology and new teaching methods allowed the instructors to develop new skills in electronic delivery of course materials. The instructors acknowledge that improvements made to the structure and design of the course Web site for the second and third course offerings, compared with the first offering in 2001, had a positive effect on the students. Web-based students appreciated a well organized and structured Web site that provides the course materials in a logical and intuitive format.
Limitations
Although the first year of the Web-based pathway showed a statistically significant difference in student performance between campus and Web-based students, both groups still showed a high level of performance. A number of changes were made in the school, program, and course level over the first 3 years of the Web-based pathway, so it is not clear if the difference in performance seen in the first year was due to factors external to the course. Further study is needed to determine if these results are generalizable to on-campus entry-level students who enroll in a Web-based pharmacy calculations course. Students select the Web-based pathway at the time of application to the program, so it is not known if the comparison would be applicable to on-campus students.
Conclusions
As pharmacy academia struggles to accommodate the current and predicted shortages of pharmacists, the academy will continue to observe the development of new programs to meet these dire shortages. Newer instructional methods and technologies will have to be utilized to meet the curricular requirements facing a shrinking and aging pool of pharmacy faculty.
An entirely Web-based course appears to be well accepted by full-time entry-level students in the PharmD program. A lower level of performance was noted for Web-based students the first time the course was offered. Therefore, additional support may be needed to ensure that Web-based students achieve similar levels of performance the first time an online course is offered. Following the initiation of the Web-based pathway, the school has increased staff support to facilitate the use of technology in both Web-based and campus-based teaching. Some of these staff resources include 3 instructional designers to support faculty use of technology for teaching, a graphic artist, a web designer, 2 staff members to support Web-based examinations, and 4 full-time information technology staff members. Email-based support and toll-free telephone support is available for students or examination proctors in the event of any technical problems. These staff positions are in addition to an increase in faculty positions over 4 years that is proportional to the increased number of students in the program.
Acknowledgements
The authors would like to thank Dr. Michael S. Monaghan, Creighton University School of Pharmacy and Health Professions for his assistance with the statistical analysis of the outcomes data.
References
1. Ramanathan M, Chau RI, Straubinger RM. Integration of Internet-based technologies as learning tools in a pharmaceutical calculations course. Am J Pharm Educ. 1997; 61(2):141-148.
2. Sause RB, Barcia SM. Computer assisted instruction in pharmaceutical calculations. Am J Pharm Educ. 1981; 45(Feb):41-47.
3. Delafuente JC, Araujo OE, Legg SM. Traditional lecture format compared to computer-assisted instruction in pharmacy calculations. Am J Pharm Educ. 1998; 62(1):62-66.
4. Tapia C, Sapag-Hagar J, Muller M, Valenzuela F, Basualto C, Zunino GF. Development of an interactive CD-ROM for teaching unit operations to pharmacy students. Am J Pharm Educ. 2002; 66(3):280-287.
5. Ramanathan M. Use of multiplatform CD ROMs for disseminating supplementary learning materials in a pharmaceutical calculations course. Am J Pharm Educ. 1999; 63(3):333-338.
6. Zatz JL. Computer grading of open ended pharmaceutical calculations exams. Am J Pharm Educ. 1982; 46(May):35-36.
7. Pappas A, Stewart K, Cesnik B, et al. Evaluation of a computer-based bridging course by overseas-qualified pharmacists seeking Australian registration. Am J Pharm Educ. 1998; 62(3):315-323.
8. SAS Institute Inc. The NPAR1WAY Procedure. In: SAS OnlineDoc, Version 8. Cary, NC: SAS Institute Inc.; 1999.
October 7, 2004
