Georgia State University
Law Review


Volume 20            

number 4

           Summer 2004


Outside the Classroom: Teaching and Evaluating Future Physicians

David Stern, M.D., Ph.D.


Introduction

One might expect the training for professions such as medicine and law to have great similarities. In the United States, both require the same primary education, basic liberal arts education, and a period of apprenticeship. However, the professional education systems, the educational and administrative programs to ensure competence, and the regulations for ensuring public safety are quite different. This Article describes medical education as a foil for legal education by examining the historical, educational, and regulatory perspectives of medical training.

This Article discusses the historical roots of the medical education system and the current requirements for licensure. Because of the differences between legal and medical apprenticeships (medical apprenticeships begin in medical school, rather than in practice), this Article then turns to a description of how students learn to become doctors and how medical teachers evaluate the competency of these students. This Article makes no attempt to compare these systems but leaves that challenge to the reader and the legal scholar with the hope that they will find opportunities to improve legal education and share unique solutions for professional education with medical educators.

I.   Historical Roots of the Medical Education System

Medical education in the 19th century was unorganized and inconsistent, as was the field of medicine itself. There were few


universal principles upon which to base common therapy. In the typical early 19th century scenario, a boy would apprentice himself to


an established physician and remain in his service for two to four years. He would then strike out on his own and find a community that could use his services.[2] There were no licenses or examinations because the government had not created these regulations; the knowledge learned during apprenticeship was not sufficiently universal to permit such evaluations. Unlike today, the job of the physician was more varied, including practices such as dentistry and embalming, and did not automatically carry with it a sense of prestige. Because the profession itself had little distinction, the physician’s clientele and pre-existing social status established his position on the socioeconomic scale. Society defined the wealthy and esteemed physician as the one who served wealthy and revered clients. The “country doctor” served the poorer rural communities.[3]

In the 18th century, formal centers of medical education existed in Europe and began to develop in Philadelphia. By the early 19th century, there were four institutions of medical education in the United States, all loosely associated with institutions of higher learning.[4] As the profit and the need for these institutions became evident, many more developed throughout the country. In the words of Abraham Flexner, a medical education reformer of the early 20th century, “medical colleges have multiplied without restraint, now by fission, now by sheer spontaneous generation.”[5] The schools were proprietary. For example, if a young man had enough money for tuition, he could attend the school. There were no admission criteria, nor were there any standards for the educational program.[6] These institutions created a plentiful supply of physicians, but they were of widely varying quality. The most prestigious doctors traveled to Europe, particularly to Germany, to complete their training. There is no clear evidence that patients were unhappy with this variability in training or expertise. There was no public outcry for a better, more regulated practice of medicine. Because a physician’s power to heal was limited, the public’s expectation of them was low, and nature often “took its course” regardless of the physician’s training.

In this setting, Dr. Nathan Smith Davis organized the American Medical Association (“AMA”) in 1846 with the goals of raising the entrance prerequisites of medical schools and standardizing the requirements for attaining the degree of Medical Doctor (“M.D.”).[7] The Carnegie Foundation for the Advancement of Teaching and the leading academic institutions of the time joined forces to systemically reform medical education in the early 20th century. Harvard University President Charles Eliot was one of many with a desire to make changes in the higher-education system.[8] This desire arose from his association with the “educational elite” who spent time at the European academic centers. In the l8th and early 19th centuries, many of the “best minds” studied in England and France, but by the mid-19th century, they preferred Germany. The “observational school” of French medicine lost popularity in the face of powerful discoveries from the German “experimental school.”[9] The German surge of research emanated from their principles of lehrfreiheit and wissenschaft.[10] As Lawrence Veysey pointed out, Americans converted these ideas from the German “pure learning” and value of investigation over teaching to an American conglomerate of meticulous scientific research.[11] While this formulation of the role of research omitted much of the meaning and value of the concepts used in Germany, it helped bring the research sciences to the foreground in the United States.[12]

In his report to the Harvard University Medical School trustees of 1871 to 1872, Charles Eliot noted, “[t]he ignorance and general incompetency of the average graduate of American Medical Schools, at the time when he receives the degree which turns him loose upon the community, is something horrible to contemplate, considering the nature of a physician’s functions and responsibilities.”[13] To remedy this problem, Eliot suggested entrance examinations (preferably after the bachelor’s degree) and teaching by members of the university faculty, rather than the non-university apprenticeships that existed at the time.[14] By the turn of the century, the University instituted an admission requirement of a full bachelor’s degree.

As universities reorganized their academic structure at the turn of the century, so did the AMA. Faced with inefficiency and a small membership in the late 19th century, the AMA revised its constitution in 1901 to focus on the state and county societies as its units of organization.[15] This gave proportional representation based on the number of members from each state society. The AMA thus consolidated its power through a direct line from county societies through state societies, and ultimately to the national level. In the early 20th century, “[p]rofessional ostracism carried increasingly serious consequences: denial of hospital privileges, loss of referrals, loss of malpractice insurance, and, in extreme cases, loss of a license to practice.”[16] This reorganization and collaboration paralleled the growing force of unionization and trade associations in the pre-war years.

As membership expanded in the early 20th century, the AMA began pushing its original 1846 agenda of medical education reform. To this end, it created the Council on Medical Education (“AMA Council”) in 1904.[17] The AMA Council devised standards for an acceptable curriculum, including a preliminary undergraduate course and five years of medical education. It designed the curriculum to culminate with a one-year internship at a hospital prior to certification by the state medical boards. On the basis of these requirements, the AMA Council surveyed and graded the existing 160 medical schools and found only 82 to be “Class A,” or “acceptable.”[18] The AMA never published the results of this study because of the backlash it would have created against this newly enlivened organization. Instead, the AMA Council invited an outside group, the Carnegie Foundation for the Advancement of Teaching, to conduct a similar investigation.[19]

The Carnegie Foundation enlisted Abraham Flexner, a layman and educator (not a physician), to conduct this study.[20] Flexner reviewed works on the history of medical education. The German text Lehren und Lernen der medizinischen Wissenschaften, which describes the format of German medical education, deeply influenced this literature.[21] Flexner then traveled to the offices of the AMA in Chicago to review the AMA Council’s completed, but unpublished, study.[22] After taking note of these findings, he traveled to Johns Hopkins and Tulane for his first review of medical institutions.[23] After these visits, Flexner formulated the following criteria for appropriate medical education: (1) entrance requirements; (2) “size and training of the faculty”; (3) the sum of endowment and fees, and “what became of it”; (4) the “quality and adequacy of the laboratories . . . and the qualifications of the teachers of the so-called pre-clinical branches”; and (5) the “relations between medical school and the hospitals.”[24]

Flexner then traveled to each of the 155 medical schools operating in the United States and Canada at that time, returning briefly between excursions to write scathing indictments of their policies, facilities, and students.[25] In the final report, he detailed the rationale behind his criteria for incorporating the basic sciences (laboratory sciences) into medical education. Medicine, like science, is based on the experimental method. When a situation confronts either a physician or a researcher, both observe the problem, create a hypothesis (the diagnosis), and act on the basis of this hypothesis. “Investigation and practice are thus one in spirit, method, and object.”[26] Consequently, the future of medical education closely followed research and the scientific method. Flexner identified the subjects of anatomy, physiology, histology, pathology, biochemistry, bacteriology, and pharmacology as the core of the pre-clinical years.[27] Since the publication of Flexner’s report, these courses have become firmly entrenched in the pre-clinical curriculum.

In addition to changes in the pre-clinical years, Flexner also advocated broad structural changes in the clinical year of medical education. He specifically insisted on the following criteria:

(1) [T]he hospital must be of sufficient size; (2) it must be equipped with teaching and working quarters closely interwoven in organization and conduct with the fundamental laboratories of the medical school; (3) the school faculty must be the sole and entire hospital staff, appointment to which follows automatically after appointment to the corresponding school position; (4) the teaching arrangements to be adopted must be left to the discretion and judgment of the teachers, subject only to such oversight as will protect the welfare of the individual patient.[28]

Flexner evaluated each school on the basis of these criteria and listed those which best matched his ideal. In his plan for the reconstruction of medical education, he viewed these schools as the models for all others. Flexner saw Johns Hopkins as the ideal institution, and he encouraged those who wished to remain in business to reform in this direction. It is noteworthy that Flexner, while not a physician, determined the criteria by which to judge the schools, made personal judgments, and had more influence on the structure and direction of the American medical education system than any single individual before or since.

While Flexner clearly articulated these changes, many of them were already underway. As Arthur Bevan discussed 20 years later, after the initial unpublished survey of medical schools by the AMA Council, many schools consented to changes promoted by the AMA.[29] “Fifty schools agree to require by 1910, or before, at least one year of university physics, chemistry and biology and one modern language as a preliminary education before matriculating in medicine. Immediately a number of consolidations were arranged in many cities having several schools.”[30] On the legislative front, there were great advances toward unifying the standards of medical practice across the country and enforcing laws to punish inept practitioners.

These changes led to a reduction in the number of medical schools, but this reduction was not sufficient to meet Flexner’s goal of only 30 schools. In 1910, there were 155 medical schools.  This number fell to 85 in 1920 and reached its lowest point in 1930 with 76 schools.[31] Although no state medical board or national organization forced any medical school to close, many schools combined forces with other institutions and some closed of their own volition.[32] Strong local coalitions of practitioners and politicians ensured the survival of many schools which Flexner had marked for closure; but even those institutions worked to improve along Flexner’s guidelines.

The state boards of medical examiners were another factor that helped unify the structure of medical schools. As relatively new organizations, state licensing boards looked for direction from the profession itself. The AMA was willing to give such direction, especially through the AMA Council. The AMA viewed Flexner’s report as a model for the kind of reform needed throughout the country, and each state board of medical examiners had access to this document. Flexner, the Carnegie Foundation, the AMA, and the university medical schools campaigned to raise requirements for state licensing.[33] Many current licensing applications continue to detail the specific number of credit hours required in each of the specific pre-clinical and clinical areas set forth by Flexner. While these reforms were well underway before the Flexner report, the codification of these ideas made the requirements across state borders more uniform.

Flexner and others who wanted to change medical education used the language of science and efficiency to justify their arguments. While using the language of science to justify itself may seem tautological, science, rather than ethics or philosophy, was gaining credence as the most effective means of argument. The “new science” utilized the experimental method, which sought inherent proof of correctness through pure mathematical calculations and reproducibility. The New York Times editorial following the release of the Flexner report noted that “medicine . . . has for the first time become a science in the sense given to that term in modern usage. Previous to this marvelous development it was, as described in the ancient documents, largely an ‘art and mystery.’”[34] Those who disagreed argued not only with the educational elite, the AMA, and the Carnegie Foundation, but also with scientific “truth.”

The combination and interaction of these factors facilitated change in medical education at the turn of the century. Individually, they would not have had the same impact. As Arthur Bevan claimed in 1928:

[T]he fight for higher standards of medical education [had] been won . . . by the united efforts of the American Medical Association, the Federation of State Medical Licensing Boards, the Association of American Medical Colleges, the American universities and colleges, the great educational Foundations, the Carnegie [F]oundation for the Advancement of Teaching, the Rockefeller Foundation and the General Education Board.[35]

While there has been a series of recent innovations in basic medical education, the foundational structure set in place by this coalition has remained constant during the past century. The most significant recent changes, stemming largely from the post-World War II period, have arisen in postgraduate subspecialty training. With the growing body of medical and scientific knowledge has come a growing specialization among physicians. A century ago, most practitioners in the United States continued in “general practice” after only a few years of postgraduate training, although one may have found an individual specializing in pediatrics or surgery. In today’s medical environment, there are over 50 recognized specialties of medicine, each requiring three to seven years of additional training for certification eligibility. Though these additional certifications can make obtaining and maintaining a position at a hospital easier, they have little to do with the medical license that the individual states grant after one year of supervised medical practice following medical school.

II.   Requirements for Medical Licensure

The U.S. medical education system is the model from which most Western medical education has emerged. However, there are many unique characteristics of the U.S. system. The most obvious characteristic is the general requirement of four years of college education prior to medical school. Most international programs in medicine begin after the equivalent of our high school education and proceed for six to seven years, culminating in the M.D. degree. Of the 126 U.S. medical schools, only a few dozen follow this combined B.A. and M.D. degree format.

The U.S. requirement of a premedical college degree derives not only from the historical developments outlined above, but also from the American belief that a broad, liberal arts education will be of some benefit to all individuals, regardless of their profession. While this theory remains unproven, the educational structure of college prior to graduate school is heavily ingrained in our culture and is not likely to change in the near future. The additional benefit of undergraduate training is that medical schools can insist on a series of premedical requirements for matriculation. Thus, while formal medical education does not begin until medical school, foundational training in science begins many years earlier.

While admission to U.S. medical schools does not guarantee successful completion, an average of 98% of all students admitted to medical school in the United States graduate with an M.D. degree. In spite of the recent small decline in the number of students applying to medical school, U.S. medical schools offer positions to only around 50% of applicants.[36] However, graduation with an M.D. degree does not confer an M.D. license. The licensing process occurs at the state (not national) level. It involves a combination of graduation from a school accredited by the national certification board, successful completion of a national exam, and a minimum of one year of supervised medical training beyond medical school.[37] While the certification of schools, residency programs, and the examination process are identical across the United States, each state medical board provides the actual license to practice medicine. In the past this led to substantial differences in state-level course and exam requirements. However, with the uniformity of expectations for healthcare nationally, the cooperation of the state medical boards, and the national medical licensing examination board, these differences have become negligible in recent years.

The process of becoming a physician begins with acceptance to one of the 126 medical schools accredited by the Liaison Committee on Medical Education (“L.C.M.E.”).[38] The L.C.M.E. is an organization comprised of members nominated by the AMA, the Association of American Medical Colleges (“A.A.M.C.”), along with public and medical student members nominated by these two organizations.[39] This self-regulatory structure is typical of the medical profession because it has long been recognized that only physicians are qualified to regulate other physicians’ training and practice quality. The U.S. Department of Education has designated the L.C.M.E. as the sole source for certifying medical school quality in the United States.[40] This certification not only allows students to apply for government-sponsored educational loans, but also brings with it the recognition by state licensing boards that graduation from one of these certified schools ensures a minimum level of competence.[41]

Graduation from one of these schools is necessary but not sufficient for application to a state medical board. All states now require successful completion of the United States Medical Licensing Examination (“U.S.M.L.E.”). The Federation of State Medical Boards (“F.S.M.B.”) and the National Board of Medical Examiners (“N.B.M.E.”), the sponsors of the U.S.M.L.E., administer this examination in three parts: the first part usually takes place after the second year of medical school; the second part occurs during the fourth year of medical school; and the third part takes place during the first year of residency. Part I of the U.S.M.L.E. determines whether students have a firm understanding of the basic sciences, as they are foundational to the practice of medicine. Part II uses supervisors to evaluate students’ ability to manage patients. Part III determines whether resident physicians can manage a wide array of medical conditions independent of supervision.[42] Results from this examination consist in both a numerical score and a pass/fail judgment. Residency programs often use numerical scores to select the students with the largest knowledge base, but a student only needs a passing score to obtain a medical license.

Students who enroll in medical schools outside the United States must submit their credentials to the Educational Commission for Foreign Medical Graduates (“E.C.F.M.G.”).[43] State medical boards developed the E.C.F.M.G. because they lacked the capacity to identify and evaluate the quality of medical graduates arriving in the United States from around the globe. As with the L.C.M.E., the E.C.F.M.G. has a set of directors who are nominated by physician organizations.[44] For non-U.S. graduates to become practicing doctors in the United States, they must have attended a medical school recognized by the World Health Organization (“W.H.O.”) and the E.C.F.M.G. This is not meant to imply that the W.H.O. or the E.C.F.M.G. actually certify schools. Rather, they rely primarily on lists of schools and their curricula, which either the schools themselves or the governmental Ministry of Health submit. Upon verification that the student actually graduated from one of these schools, the student may apply for examination with the E.C.F.M.G. This process requires two steps uniquely different from the U.S. medical student examination process. First, these individuals must pass an English language proficiency examination (“T.O.E.F.L.”). Second, they must pass a clinical skills examination in which they interview and examine a series of standardized patients while supervisors evaluate them for such competencies as communication skills, diagnostic skills, and physical examination skills. Beginning in July 2004, U.S. medical students must also successfully complete this clinical skills examination as part of the U.S.M.L.E. (Part II.B.).

Upon successful completion of the E.C.F.M.G. process, or graduation from a U.S. medical school and passing the U.S.M.L.E., all students must successfully complete at least one year of postgraduate training. As with medical schools, a physician-run supervisory organization, the Accreditation Council for Graduate Medical Education (“A.C.G.M.E.”), monitors postgraduate training sites.[45] The mandate for this organization comes from both the states, which recognize only those programs certified by the A.C.G.M.E. for new physician license applications, and from the federal government. Federal Medicare and Medicaid legislation allow for a supplemental payment to educational programs certified by the A.C.G.M.E. to account for the added cost of caring for patients in educational settings.[46] This federal funding is a chief impetus for the increase in the amount of postgraduate training received by U.S. physicians. Federal funding has also assured that high quality academic institutions will provide elder and indigent care despite the financial disincentives for charity care in the private practice setting.

After four years of college, four years of medical school, one year of post-graduate training, and a three-stage national examination, individuals may apply to the states for licensure. While most countries have a national physician license, the United States retains a state-level process. This has allowed, in the past, some unscrupulous physicians whose licenses had been revoked in one state to apply for licensure in another state. Such activity has led the United States Congress to pass the Health Care Quality Improvement Act of 1986 (“H.C.Q.I.A.”), which led to the establishment of the National Practitioner Data Bank (“N.P.D.B.”).[47] States submit licensure actions to this national data bank so that prior to licensure and license renewal state medical boards can query the system to ensure that physicians have not received sanctions elsewhere. After verifying examinations, credentials, a query to the N.P.D.B., and the usual licensing fees, a physician receives a license to practice medicine.

Although many physicians still practice with only an M.D. degree and one or two years of postgraduate training, very few graduates in the past 20 years have stopped training at this point. Most proceed to further training in their specialty for three to seven years to become eligible to take a specialty examination. There are currently about 50 different specialties recognized in the United States. Specialty boards work under the umbrella of the A.C.G.M.E., which accredits each residency program. Graduates of these programs are “board-eligible.” If they subsequently complete a series of specialty examinations, these individuals become “board certified,” the highest degree of recognition for medical expertise in the United States. Physicians today complete this certification to garner the confidence of patients and to secure hospital privileges. They also complete the certification because, in recent years, managed health care organizations have used this certification to indicate physician quality and, in some cases, refuse to include physicians’ names in preferred provider listings without such certification. In addition, most specialty certificates are now time-limited, requiring state medical boards to periodically re-examine physicians who wish to retain the specialty certification.

This rich network of self-regulation, state certification, national examination, national oversight, and commercial incentives has led to the assurance of high quality medical care in the United States. Although the costs in terms of time spent by young physicians in training, the expense of education (the average U.S. medical school graduate’s educational student debt is over $100,000), and the cost of care are quite high, few people question the overall skill of physicians and the ability of the profession to perform those skills.[48]

III.   How Physicians Learn Medicine

A.   Pre-Clinical Education

Most four-year U.S. medical schools require a college degree and the successful completion of premedical courses including biology, chemistry, physics, biochemistry, and calculus. While many applicants will have majors in the sciences, medical schools often encourage applications from students with degrees in the social sciences and liberal arts. Admissions committees believe well-rounded students with a broad range of knowledge and experience, as well as excellent communication skills, are ideal candidates. Of course, medical schools expect all students to excel on the Medical College Admissions Test (“M.C.A.T.”), and studies have shown that scores from this examination predict successful completion of school examinations and the national U.S.M.L.E.[49]

The A.A.M.C. has outlined the goals of medical education in a series of reports.[50] Because the nature of disease and the evidence for effective treatment are so widely recognized, these educational goals are applicable to medical schools around the globe.[51] These domains of competence usually include excellence in knowledge, communication skills, technical skills (for example, examination and procedures), professional behavior, self-improvement, and information management.

Despite the similarity in the content of medical education, the process of education varies widely from school to school. The “classic” form of medical education, which derives from the Flexnerian view of medical training, is the “2+2” curriculum; this curriculum consists of two years of basic education in science followed by two years of clinical education. The basic education in science comes in the form of lectures given five or six days a week in the mornings, and laboratory work performed each afternoon. Discipline-specific faculty members teach the courses, and each member is responsible for a final grade in his or her area of expertise. Typical courses include anatomy, biochemistry, physiology, pathology, and pharmacology. While no medical school still follows this rigid structure, remnants of this framework remain in most modern institutions.

One of the earliest variations on this teaching structure is the move from discipline-based to organ-based courses. In an attempt to improve the application of the basic sciences to the practice of medicine, Case Western Reserve University Medical School structured courses around a single organ with many disciplines teaching an element of their curriculum relevant to that organ. For example, students might spend an entire month studying the heart, its anatomy, physiology, pathology, and treatments for conditions of cardiac dysfunction.

In the mid-1970s, referencing the process of legal education, McMaster University, the University of New Mexico, and Michigan State University converted their educational process to a case-based education known as “problem-based learning.” In this approach, faculty members give student groups a patient case to solve. During the process of developing a solution, students generate a series of questions which they must answer. For example, in a case of heart failure the students realize they must learn the heart’s anatomy. With faculty assistance, they divide up assignments in the group, disperse to investigate answers, and return to the group to present their responses. This “pure” form of problem-based education exists at very few medical schools, but many have used adaptations of this format to enrich small group experiences and engage students in self-directed learning.

A recent variation on this theme of student-directed learning comes from the University of Dundee, Scotland where the faculty offers students a large set of learning objectives. The role of the faculty is to provide students with opportunities to learn, but students can educate themselves using many available means, including textbooks, online education, small groups, and lectures. Faculty members measure student performance using examinations, and they base student promotion upon successful completion of (and evidence of competence in) the prescribed learning objectives.

Very few medical schools work strictly within one of these pedagogical domains. Most are hybrids and include lectures, small group experiences, and case or organ-based curricula. With no evidence available to prove a difference in student quality based on curricular format, there is currently no dominant method. Market forces, student feedback, and some pressure from the L.C.M.E. have ensured that most schools keep pure lecture instruction at a minimum.

B.   The Introduction of Clinical Experience

Although there is great variety in the form of pre-clinical medical education, recent years have seen greater uniformity in the introduction of clinical (patient care) experiences. Most schools now begin, either on the first day or within the first few months, with an experience involving patients in their homes, at work, or in community settings. At this point, students usually engage in “health fair” type activities, including blood pressure monitoring and basic health promotion advice. After a brief introduction on how to interview patients, some schools send students to senior centers and other sites to practice speaking with individuals about health problems.[52]

Because communication skills are fundamental to the sound practice of medicine, medical schools are now dedicating an increased amount of time to developing communication skills. Starting with relatively unstructured “conversations” with individuals in the community, these experiences rapidly progress to teaching students how to obtain complete medical histories from patients in acute care settings.

Practice sessions for “real-world” experiences that use “standardized patients” are becoming increasingly more common. Medical schools developed the technology of “standardized patients” in the mid-1970s in response to an assessment problem. Faculty often used the oral examination as a method for evaluating student performance. A student would interview and examine a patient in the presence of a faculty physician. The faculty would then query the student about their reasons for asking the patient certain questions, their findings upon examination, and the nature of the disease diagnosed in the case. The flaws in this testing method generally fall within the following two categories: variability in both the patient case and in the faculty examination. To minimize one source of error, medical schools trained individuals to “portray” a case in a very standardized fashion—always responding in the same manner to the same question, having the same physical complaints, and exhibiting the same body language throughout the interview. This technology has not only made the assessment of clinical skills much more reliable, it has also provided an excellent opportunity for students to practice communication and examination skills in a controlled setting prior to examining real patients with real conditions. Most medical schools have some standardized-patient experiences for students, either as an educational or assessment program.

Throughout the first two years of medical school, faculty members gradually require students to practice interviews and examinations on an increasing number of patients, and in more realistic contexts. Faculty members either observe each examination or review it in written form for accuracy and level of detail. Ultimately, as students prepare to complete the pre-clinical years of medical training, schools and hospitals expect them to make a relatively smooth transition from the classroom experience to mostly hospital and clinic-based work.

C.   Clinical Medical Education

While some medical schools have condensed the pre-clinical phase of medical education to 18 or even 12 months, most continue to follow the “2+2” curricular structure. At the beginning of the students’ third year they make the transition to full-time clinical care under the supervision of individual clinical departments. Students rotate through a series of required “clerkships” in internal medicine, surgery, pediatrics, obstetrics and gynecology, family medicine, and psychiatry. Most schools also require clerkships in neurology and emergency medicine. These rotations usually last one to three months; they may involve experiences at a single hospital or outpatient clinic for many weeks at a time. Students usually participate in clinical care as the most junior member of a health care team. Unlike pre-clinical education, the process of clinical education is nearly identical at every Western-style medical school around the world. Students usually spend the first year of these experiences in a series of required clerkships and the second year in a series of electives or “selectives,” where they must select from a limited set of options (for example, choose one intensive care unit rotation, or one outpatient care rotation).

The daily life of an inpatient clinical clerk begins with an assignment to a specific patient or set of patients, usually limited to no more than two to four patients at a time. The student obtains the patient’s full medical history, performs a complete examination, and reports their findings, possible diagnoses, and treatment plans to the healthcare team. The hospital team itself consists of an attending (faculty) physician, a senior resident, a junior resident, and additional health-care personnel (for example, nurses and respiratory therapists) depending on the setting. The faculty member expects the student to obtain the most detailed and complete information from the patient possible, and the student is the primary contact between the patient and the healthcare team. While students are not ultimately accountable for the patients’ healthcare, they spend more time with the patients than any other healthcare team member and often form a close bond with these individuals.

After presenting the patient’s case to the faculty or resident physicians, the student proposes a set of diagnoses and treatment options. In this setting, the faculty and residents often quiz students in Socratic fashion about the rationale for their ideas, their understanding of the underlying disease, and their choices for treatment. When many students are present, any student in the group is “fair game” to these questions. It is common for faculty to take a single case presented by one student and use it as the medium for a broader inquiry for the entire group about a specific medical condition. At times, this conversation happens at the bedside of the patient or in the hallway. Some faculty physicians prefer to move the conversation to small ward-based conference rooms where they have prepared short, structured didactic lectures.

Ultimately, the resident and faculty physicians decide the patient’s plan of care.  However, occasionally the team simply agrees with the plan laid out by the students in their presentations. While students have no authority to order hospital activity or prescriptions, faculty and residents encourage them to write these orders and prescriptions. Residents and faculty then review the students’ orders, and a system of co-signature assures that an accountable individual agrees with the plan of action. Students in these settings physically perform all the activities of patient care (examination, diagnosis, treatment) but do so under very strict supervision. No action of a medical student leads to changes in patient care without the signature and authorization of a licensed physician.

Each clerkship has both clinical care activity and didactic components. Advanced lectures on disciplinary topics relevant to patient care are often a daily occurrence and take from one to four hours each day. Along with patient care expectations, these lectures constitute the “formal curriculum” of clinical training. Much of the experience, however, is informal in nature and does not happen during any predetermined curricular time. Instead, it occurs during meals, in the hallways, and late in the evenings when students and resident physicians are living and working together for extended periods of time.[53] While students obtain much of the knowledge and the skills of medicine during the formal time of the curriculum, members of the faculty predominantly teach the professional behaviors (or lack thereof) through this informal, or “hidden” curriculum.[54] Through jokes, anecdotes, and informal conversation, the faculty members evoke and reinforce the values of being a physician or a particular specialist.[55] In a form of cultural transmission, students learn not only to think like doctors, but also (for better or worse) to act like doctors.[56]

 

 

D.   Clinical Education for Residents

Residency education begins upon graduation from medical school. While residents are often licensed physicians, they have not yet completed their training and remain under faculty supervision. Residents can order and prescribe treatments for patients without the co-signature of faculty; therefore, their decisions may go unsupervised for a period of hours before a faculty member reviews the case. In most settings, residents make all critical decisions (for example, intensive care unit, emergency department, operating rooms) with faculty present, so there is no delay between the resident’s decision and faculty review. In other less acute settings, or with minor decisions, faculty physicians frequently delay the review for several hours. As residents gain more experience and prove their abilities, the faculty members provide them with more flexibility and more responsibility.

The content of residency education varies widely because medical school graduates enter specialty training in areas as divergent as radiology, pediatrics, and surgery. However, regardless of their specialty, there are some core competencies that the A.C.G.M.E. has outlined which all residents must demonstrate.[57] In addition to these general competencies, each of the approximately 50 approved residency program types (for example, internal medicine, pediatrics, psychiatry) expects specific outcomes. Most also require residents to successfully complete a set of performance requirements, ranging from minor (for example, placement of feeding tubes) to major (for example, hip replacement surgery) procedures. Residency programs range from three to seven years depending on the time required to achieve competence in each specialty. Upon completion of postgraduate training, these individuals (after 23 to 30 years of education) may practice independently.

 

 

E.   Education in the Setting of Patient Care

As evidenced by the above discussion, an increase in responsibility for patient care corresponds to a decrease in direct oversight. Medical students begin their careers by interviewing patients who are healthy and for whom no intervention is necessary. In subsequent years, they examine and plan treatments for ill patients, but they make no decisions without authorization from licensed physicians. In the residency years, faculty physicians give students increasing latitude in their care of patients, but they are still ultimately under faculty supervision. First-year residents usually report to both a senior resident and a faculty member; more senior residents supervise teams of junior doctors and report only to the faculty physician.

This practice of having junior doctors-in-training work with ill patients leads to potential conflicts between the desire to provide optimal care and the desire to train high-quality students. Faculty and residents perform this balancing act daily with clear limits for risky procedures (only faculty and the most senior residents perform certain procedures). Less risky procedures (for example, drawing blood, or placing an intravenous catheter) are often the subject of negotiations between patients and student doctors. Most patients allow students to engage in care with limits on participation—for example, letting a medical student try one or two times to draw blood before insisting on a more experienced phlebotomist.

So why do patients choose to participate in this education process? For some, there is little choice. Indigent care is often only available at academic healthcare centers, and students are an integral part of these centers. Student doctors initially meet patients, whose participation they request, and the faculty encourages patients to see the students first and then the more senior physicians. Similarly, these academic healthcare centers are often the only place for highly specialized services (for example, transplantation, experimental chemotherapy), and students again are necessarily part of that environment.

In recent years, there has been growing evidence that academic medical centers actually provide higher quality care.[58] Arguments for participation in academic healthcare include the fact that each patient has three to five physicians, rather than one, which decreases the risk of error and increases the likelihood that a physician will make an accurate diagnosis. In addition, students and residents have more available time, so patients feel that the system as a whole is more open and responsive. Many individuals believe in medical education itself and view themselves as participants in a process.[59] This theme is particularly common among patients with terminal illness, who often agree to work with medical students as an altruistic gift to future patients who might suffer from a similar disease.[60]

The Veterans Affairs (“V.A.”) medical centers play a central role in U.S. medical education.[61] Many of them are closely affiliated (and physically proximate) to university medical centers. In cost-sharing agreements, the V.A. medical centers support the salaries of some academic faculty members and residents in return for having these individuals provide care in the V.A. system. A common arrangement is for an expert academic physician to have clinics at both the university and V.A. hospitals, thus providing a uniformly high quality of care at both sites. Along with the provision of care, medical students and residents learn from faculty and patients in these hospitals. As the largest health provision and health insurance organization in the United States, the V.A. health care system provides free or low cost care to eligible veterans. These individuals are, in general, enormously generous with a personal experience of sacrifice for the good of others. They freely engage in the educational process as another benefit to society at large.

IV.   Evaluation of Clinical Competence

For the purposes of evaluation, medical educators divide competency to practice medicine into knowledge, skills, and professional behaviors. At the same time that medical educators have developed a clear set of expected outcomes in education, they have also developed a clear set of measures with which to document those outcomes.[62] Knowledge of the science of medicine is fundamental to the practice of medicine and educators have perfected reliable and valid measures of knowledge over the past 30 years. Reliability, in this context, refers to the degree to which scientists can duplicate one measurement of knowledge exactly upon the next measure of knowledge. Validity refers to the degree to which the assessment actually measures the domain of interest. One can easily imagine a highly reliable measure (“what is the sum of 3 plus 3?”) that is an invalid indicator of one’s ability to solve a problem in geometry.

Testing for knowledge in medicine has become both reliable and valid through an intensive program that the N.B.M.E. primarily directs.[63] The program generates questions from clinical cases and is without many of the biasing effects of specific question types and formats. Research has demonstrated the reliability of these experiments. Their use in evaluating students in disciplinary courses, conferring the M.D. degree, and granting state medical licenses lends further validity.[64] Medical educators have a high degree of confidence that those students who perform well on these written examinations have the knowledge base necessary to practice medicine.

A foundation of knowledge, however, is not sufficient for the high quality practice of medicine. For this reason, educators have developed clinical skills to measure students’ abilities in communication, moral reasoning, counseling, and technical skills (relating to both minor and major procedures). In addition to the standardized patient assessment programs outlined above, there are also developing programs of surgical skills assessment, which ensure that students and residents who perform surgery for the first time have had ample opportunities to practice and prove their competence.[65]

The assessment issues with clinical skills exams have little to do with validity because faculty members pattern the cases presented precisely after real clinical scenarios. It is more challenging to ensure that the sample of cases is representative of the domain of interest and that the measures have a high degree of reliability. Because of differences in student familiarity and experience with specific topics, clinical skills examinations are highly case-dependent. Subsequently, high stakes examinations use no fewer than 12 to 15 clinical skills content cases.[66]

The measurement of professional behavior is one of the greatest challenges in medical education today.  Professional behaviors are very difficult to measure with paper-and-pencil tests because of the likelihood that students will respond with socially desirable, as opposed to personally realistic, choices. As a result, the best measures of professional behavior lie in the context of clinical activity and involve a conflict that the student or resident must resolve under supervision.[67] These events do not naturally occur with great frequency, so educators have developed systems of evaluation to describe the professional behaviors of medical students.

The most common form of professional evaluation used in clinical education is the faculty performance evaluation. During clinical experiences, faculty physicians observe students’ performance and rate them on dimensions of professionalism, including compassion, respect, interprofessional relationships, and conscientiousness. While valid, these ratings lack reliability for three reasons. First, any single faculty physician may see only a fraction of all student behavior in a limited set of contexts. For this reason, most clinical performance evaluations use a combination of at least six to eight faculty ratings for a more reliable and stable measure.[68] Supervisory faculty can achieve additional reliability by convening faculty meetings to rate student performance.[69] Second, faculty physicians have a tendency to rate students with greater stringency on knowledge and skills domains, thus providing higher average scores in domains of professional behavior. The use of high-quality rating scales, as well as education of faculty on how to rate student performance, can minimize this “ceiling effect.”[70] Third, the performance of an individual student before a faculty member may be markedly different from his behavior outside of direct supervision. The addition of peer, resident, and nursing ratings of professionalism to the overall picture may provide added validity.[71] Some have used documentation of critical incidents of either outstanding or extremely poor judgment as an additional method of indicating levels of student performance on behavioral domains.[72]

Ultimately, performance grading for clinical medical students usually includes data from a number of sources. Knowledge-based multiple choice examinations, pattern-recognition examinations of radiographs or electrocardiograms, clinical skills examinations for communication and technical skills, and faculty and resident reported knowledge, skills, and behaviors all culminate in an overall grade for these students in each clerkship. The multidimensionality of these assessments makes it challenging to provide a specific ranking of students because one student may excel in one dimension and struggle in others. However, one rarely needs this process of ranking in a field where competence, rather than comparative excellence, is the essential characteristic. By using multiple evaluations in various settings, with a diverse array of measurement instruments, educators have a great deal of confidence in students’ competence.

Conclusion

Professions share common characteristics, including expertise in a body of knowledge, the expectation of educating others in the profession, and the responsibility to self-regulate. While these tenets have remained constant, professions are coming under increasing scrutiny as the public becomes more demanding of excellence. Medical educators, hearing the call of public accountability, are adapting educational programs to teach apprentice practitioners in a way that ensures competent practice. In some cases, this means a change in curricular content and, in other cases, a change in pedagogic style or even the site of education. However, medical school and residency educators can only ensure competence until the time of graduation. Ultimately, the network of accreditation organizations, specialty organizations, hospitals, licensing boards, and educators work synergistically to maintain the quality of medical care through a lifetime of professional practice.


Figure 1. The Paths to Medical Licensure

 

 


        Graduation from L.C.M.E.                       Graduation from W.H.O./E.C.F.M.G.

          Accredited U.S. School                      Recognized International Medical School

 

 


                                                                     T.O.E.F.L. Examination

 

 


                                                         E.C.F.M.G. Clinical Skills Examination

 

 


                           U.S. Medical Licensing Examination

                                                    Parts I, II, and III *

 


               

                              Completion of 1-2 years of A.C.G.M.E.

                                 Accredited Postgraduate (Residency)

                                                         Education

 

 


                                 Application for License and Review of

                                   National Practitioner Data Bank

 

 


                                    State Medical License

 

Legend:

L.C.M.E.: Liaison Committee on Medical Education.

W.H.O.: World Health Organization.

E.C.F.M.G.: Educational Commission for Foreign Medical Graduates.

A.C.G.M.E.: Accreditation Council for Graduate Medical Education.

T.O.E.F.L.: Test of English as a Foreign Language.

 

* Beginning in June 2004, all U.S. medical school graduates must pass the E.C.F.M.G. Clinical Skills Examination with a new designation as the U.S. Medical Licensing Examination Part II.B. See United States Medical Licensing Examination, http://www.usmle.org/bulletin/2004/overview.htm (last visited Feb. 9, 2004).


Table 1. Certification Bodies in U.S. Medical Education[73]

 

Organization

Authority

Membership

L.C.M.E. (Liaison Committee on Medical Education)

Department of Education.

Required for federal student loan programs.

Most state medical boards require graduation from L.C.M.E. certified school to allow individual student licensure.

q    6 members from the Association of American Medical Colleges

q    6 members from the American Medical Association

q    2 student members (one appointed from each organization)

q    2 public members (one appointed from each organization)

E.C.F.M.G. (Educational Commission for Foreign Medical Graduates)

Recognized by state medical boards as the sole source for certification of international medical graduates.

q    2 members from the American Board of Medical Specialties

q    2 members from the Association of American Medical Colleges

q    2 members from the American Medical Association

q    2 members from the Association for Hospital Medical Education

q    2 members from the Federation of State Medical Boards

q    2 members from the National Medical Association

q    1 president of the ECFMG

q    7 At-Large members

A.C.G.M.E. (Accreditation Council for Graduate Medical Education)

Private organization recognized by Medicare/Medicaid in the payment of educational supplements for academic centers providing Medicare/Medicaid service. Recognized by states as the authority for providing guarantee of quality graduate medical education.

q    4 members from the American Board of Medical Specialties

q    4 members from the Association of American Medical Colleges

q    4 members from the Council of Medical Specialty Societies

q    4 members from the American Hospital Association

q    3 members from the American Medical Association

q    3 members from the public

q    1 member from the Federal Government

q    1 residency program director

q    1 Chair of the Residency Review Committee Council


Table 2. A.A.M.C. Medical Schools Objectives Project—Characteristics of Medical School Graduates[74]

Physicians Must be Altruistic

o        knowledgeable of theories and principles that govern ethical decision making and major ethical dilemmas in medicine

o        compassionate and respectful of patients’ privacy and dignity

o        honesty and integrity in all interactions (patients, families, and others)

o        understanding and respect for other health care professionals

o        understanding of the threats to medical professionalism posed by the conflicts of interest in the actual practice of medicine

o        capacity to accept limitations and commitment to improve knowledge and abilities

Physicians Must be Knowledgeable

o        normal structure and function of the body and each of its major organ systems

o        molecular, biochemical, and cellular mechanisms that maintain the body’s homeostasis

o        the various causes of maladies and the ways in which they operate in the body (pathogenesis)

o        knowledge of structure and function (pathology and pathophysiology) of the body and its major organ systems

o        an understanding of the need to engage in lifelong learning to stay abreast of relevant scientific advances, especially in the disciplines of genetics and molecular biology

Physicians Must be Skillful

o        obtain an accurate medical history.

o        perform both a complete and an organ- specific examination

o        perform routine technical procedures

o        ability to interpret results of commonly used diagnostics procedures

o        ability to reason deductively in solving clinical problems

o        ability to construct appropriate management strategies for acute and chronic conditions

o        ability to recognize emergencies and institute appropriate therapy

o        knowledgeable about relieving pain and ameliorating the suffering of patients

o        ability to communicate effectively, orally and in writing, with patients, relatives, colleagues, and others

Physicians Must be Dutiful

o        knowledge of the important non-biological determinants of poor health and of the economic, psychosocial, social, and cultural factors that contribute to the development and/or continuation of maladies

o        ability to identify factors that place individuals at risk for disease or injury and to select appropriate tests for early detection of disease

o        ability to retrieve, manage, and utilize biomedical information for solving problems and making decisions that are relevant to the care of individuals and populations

o        commitment to provide care to patients who are unable to pay and to advocate for access to health for members of underserved populations

 

 

 


Table 3. Accreditation Council for Graduate Medical Education Competencies[75]

 

1. Patient Care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health.

2. Medical Knowledge about established and evolving biomedical, clinical, and cognate (e.g., epidemiological and social-behavioral) sciences and the application of this knowledge to patient care.

3. Practice-Based Learning and Improvement that involves investigation and evaluation of their own patient care, appraisal, and assimilation of scientific evidence and improvements in patient care.

4. Interpersonal and Communication Skills that result in effective information exchange and teaming with patients, their families, and other health professionals.

5. Professionalism as manifested through a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population.

6. Systems-Based Practice as manifested by actions that demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value.

 



      [*]   Dr. Stern is an Associate Professor of Internal Medicine and Medical Education at the University of Michigan Medical School and the Veterans Affairs Ann Arbor Healthcare System. The Author would like to give special thanks to M. Brownell Anderson for her thoughtful and detailed critique of an earlier draft of this Article.

     [2].   Abraham Flexner, Medical Education in the United States and Canada 3 (1910) [hereinafter Flexner, Medical Education].

     [3].   Paul Starr, The Social Transformation of American Medicine 81 (1982).

     [4].   Flexner, Medical Education, supra note 1, at 5.

     [5].   Id. at 6.

     [6].   Id. at 7.

     [7].   Id. at 10.

     [8].   Frederick C. Shattuck, M.D. & J. Lewis Bremer, M.D., The Development of Harvard University 557 (Samuel Eliot Morison ed., 1930).

     [9].   Kenneth M. Ludmerer, Learning to Heal 30 (1985).

     [10].   Id.

   [11].   Id.

   [12].   Id. at 33.

   [13].   Shattuck & Bremer, supra note 7, at 558 (quoting Charles Eliot, Harvard University, Annual Report of the President and Treasurer of Harvard College 1871-1872, at 25-26 (1873)).

   [14].   Id. at 559.

   [15].   See Arthur Dean Bevan, M.D., Cooperation in Medical Education and Medical Service, 90 JAMA 1173, 1173 (1928).

   [16].   Starr, supra note 2, at 111.

   [17].   Bevan, supra note 14, at 1173.

   [18].   Id. at 1175.

   [19].   Id.

   [20].   Id.

   [21].   Abraham Flexner, I Remember 114 (1940) [hereinafter Flexner, I Remember].

   [22].   Id.

   [23].   Id. at 115, 120.

   [24].   Id. at 120.

   [25].   Id. at 115.

   [26].   Flexner, Medical Education, supra note 1, at 56.

   [27].   Id. at 57.

   [28].   Id. at 106.

   [29].   Bevan, supra note 14, at 1175.

   [30].   Id.

   [31].   Id. at 1176.

   [32].   See id. at 1175.

   [33].   See id. at 1173.

   [34].   The Making of Doctors, N.Y. Times, June 12, 1910, at 12.

   [35].   Bevan, supra note 14, at 1176.

   [36].   Barbara Barzansky, Ph.D. & Sylvia I. Etzel, Educational Programs in US Medical Schools, 2002-2003, 290 JAMA 1190, 1192 (2003) (noting that 52% of applicants were accepted in 2002; there were 3.5% fewer applicants than the previous year).

   [37].   See infra Figure 1.

   [38].   See infra Table 1.

   [39].   Liaison Committee on Medical Education, Overview: Accreditation and the LCME, http://www.lcme.org/overview.htm (last visited Jan. 4, 2004).

   [40].   Id.

   [41].   Id.

   [42].   United States Medical Licensing Examination, 2004 USMLE Bulletin: Overview, http://www.usmle.org/bulletin/2004/Overview.htm (last visited Feb. 9, 2004).

   [43].   Educational Commission for Foreign Medical Graduates, ECFMG Certification—General Information, http://www.ecfmg.org/cert/index.html (last visited Feb. 9, 2004).

   [44].   See infra Table 1.

   [45].   See infra Table 1.

   [46].   Richard M. Knapp, Ph.D., Complexity and Uncertainty in Financing Graduate Medical Education, 77 Acad. Med. 1076, 1077 (2002).

   [47].   Health Care Quality Improvement Act, 42 U.S.C. § 11151 (West Supp. 2003).

   [48].   Barzansky & Etzel, supra note 35, at 1193.

   [49].   R.H. Glew et al., Relationship Between Students’ Performances on the NBME Comprehensive Basic Science Examination and the USMLE Step 1: A Longitudinal Investigation at One School, 72 Acad. Med. 1097, 1098 (1997).

   [50].   See infra Table 2.

   [51].   See Institute for International Medical Education, Global Minimum Essential Requirements in Medical Education, 24 Med. Tchr. 130 (2002).

   [52].   The February 2004 issue of Academic Medicine contains a series of articles on this topic.

   [53].   David T. Stern, M.D., Ph.D., Practicing What We Preach? An Analysis of the Curriculum of Values in Medical Education, 104 Am. J. Med. 569, 574 (1998).

   [54].   Frederick W. Hafferty, Ph.D. & Ronald Franks, M.D., The Hidden Curriculum, Ethics Teaching, and the Structure of Medical Education, 69 Acad. Med. 861, 865 (1994).

   [55].   Id. at 865.

   [56].   See id. 

   [57].   See infra Table 3.

   [58].   See Jeroan J. Alison, M.D., M.S. et al., Relationship of Hospital Teaching Status with Quality of Care and Mortality for Medicare Patients with Acute MI, 284 JAMA 1256 (2000); Gary E. Rosenthal, M.D. et al., Severity-Adjusted Mortality and Length of Stay in Teaching and Nonteaching Hospitals, 278 JAMA 485, 490 (1997).

   [59].   D. King, M.D., Attitudes of Elderly Patients to Medical Students, 26 Med. Educ. 360 (1992).

   [60].   K.E. Fletcher & David T. Stern, M.D., Ph.D., Patients Speak: What’s Really Important about Bedside Interactions with Physician Teams, (forthcoming 2004) (unpublished manuscript, on file with Social Science in Medicine).

   [61].   See Jennifer Leeman, M. Div., M.P.H., Dr. P.H. & Kerry Kilpatrick, MBA, Ph.D., Inter-Organizational Relationships of Seven Veteran Affairs Medical Centers and Their Affiliated Medical Schools, 75 Acad. Med. 1015 (2000).

   [62].   See Ronald M. Epstein, M.D. & Edward M. Hundert, M.D., Defining and Assessing Professional Competence, 287 JAMA 226, 227-29 (2002).

   [63].   Susan Case & David Swanson, Constructing Written Test Questions for the Basic and Clinical Sciences (National Board of Medical Examiners, 3d ed. 1998).

   [64].   See Eta S. Berner et al., Use of the USMLE to Select Residents, 68 Acad. Med. 753 (1995). See generally Judy A. Shea et al., Relationships of Ratings of Clinical Competence and ABIM Scores to Certification Status, 68 Acad. Med. S22 (reporting that examinees who failed to pass a specialty examination also tended to be evaluated as less competent by their program directors).

   [65].   See Univ. of Toronto Surgical Skills Ctr., http://www.utoronto.ca/ssc/vision.htm. (last modified June 3, 2002).

   [66].   See Glenn Regehr, Ph.D. et al., Assessing the Generalizability of OSCE Measures Across Content Domains, 74 Acad. Med. 1320, 1321 (1999); C.P.M. van der Vleuten & David B. Swanson, Assessment of Clinical Skills with Standardized Patients: State of the Art, 2 Teaching & Learning in Med. 58 (1990).

   [67].   See Shiphra Ginsburg, M.D. et al., Context, Conflict, and Resolution: A New Conceptual Framework for Evaluating Professionalism, 75 Acad. Med. 56 (2000).

   [68].   See Steven J. Durning, M.D. et al., Assessing the Reliability and Validity of the Mini-Clinical Evaluation Exercise for Internal Medicine Residency Training, 77 Acad. Med. 900, 904 (2002) (finding a correlation between the number of evaluations and reliability). See generally William C. McGaghie, Ph.D. et al., Development of a Measure of Medical Faculty Attitudes Toward Clinical Evaluation of Students, 70 Acad. Med. 47 (1995) (suggesting that faculty attitudes may affect their judgment in performing clinical evaluations of students).

   [69].   Paul A. Hemmer, M.D. & Louis Pangaro, M.D., Using Formal Education Sessions for Case-Based Faculty Development During Clinical Clerkships, 75 Acad. Med. 1216 (2000).

   [70].   A. Frohna et al., Psychometric Properties of Professionalism Scales Improve with the Use of Percentage Anchors and Broader Scales, Oral Abstract Presentation at the AAMC/RIME Meeting (Oct. 2000).

   [71].   D. Stern et al., A Multimodal Assessment of Behavioral Competence, 72 Acad. Med. 429-30 (1997).

   [72].   M.A. Papadakis et al., Early Detection and Evaluation of Professionalism Deficiencies in Medical Students: One School’s Approach, 76 Acad. Med. 1100-06 (2001).

   [73].   Accreditation Council for Graduate Medical Education, at http://www.acgme.org; Educational Commission for Foreign Medical Graduates, at http://www.ecfmg.org; Liaison Committee on Medical Education, at http://www.lcme.org.

   [74].   American Association of Medical Colleges, Learning Objectives for Medical Student Education: Guidelines for Medical Schools, 1998, http://www.aamc.org/meded/msop/msop1.pdf.

   [75].   Accreditation Council for Graduate Medical Education, Minimum Program Requirements Language: Educational Program, Sept. 28, 1999, http://www.acgme.org/outcome/comp/compmin.asp.