MEASURING EFFECTIVE COMMUNICATION AS A
BENCHMARK FOR QUALITY LEGAL SERVICES

Clark D. Cunningham
Professor of Law & Israel Treiman Research Fellow
Washington University School of Law
Campus Box 1120
One Brookings Drive
St. Louis, Missouri 63130-4899
(314) 935-6413 (office)
(314) 935-6493 (fax)
email: cunningc@wulaw.wustl.edu

A Research Proposal from
The Effective Lawyer-Client Communication (ELCC) Project(1)
For more information about ELCC, see our web site:
http://ls.wustl.edu/Communication/
and Cunningham, "Evaluating Effective Lawyer-Client Communication:
An International Project Moving From Research to Reform,"
67 Fordham Law Review 1959 (1999)

In Measuring and Managing Patient Satisfaction, published by the American Hospital Association, the authors confidently assert:
Health services providers today are confronted with two principal challenges. The first is to gain insight into what is important to the patients they serve. The second is to "move the needle," or make measurable changes in the patients' experience of the health care encounter. Krowinski & Steiber 1996: ix.

They link these health care priorities with the evolution of customer-centered service throughout the business world: "customer satisfaction is considered by most to be at the core of good business practices." Id. at 2. Although the American legal profession has been concerned for years with opinion research and anecdotal data indicating client dissatisfaction with lawyers, only recently has the organized bar begun to attribute much of the dissatisfaction to lawyer behavior rather than unrealistic client expectations, according to William Felstiner, former director of the American Bar Foundation. Felstiner 1997: 122. However, in striking contrast to the health care industry (or indeed any other service industry), American lawyers generally do not use even the most rudimentary methods for finding out how their clients experience the services they provide and thus have no way of measuring "how the needle moves" even if they seek to improve client satisfaction. See Cunningham 1999: 1803-4, 1809-10. The failure of the legal profession to gather data about client satisfaction in a systematic way may be attributable to a widespread assumption among lawyers that clients care primarily about outcomes not process, an assumption inconsistent with growing evidence from social science research that the quality of lawyer-client communication is an extremely important determinant of client satisfaction.

In a 1988 article in Trial magazine, social psychologist Tom Tyler (then a research fellow at the American Bar Foundation and currently a professor at New York University) asked the following question:

"Imagine a typical civil case for a lawyer--a divorce. Think of three things a client could care about. The first is winning and securing as many assets as possible. The second is getting a fair settlement (i.e., "having things come out right.") The third is having the dispute resolved fairly. This includes being allowed to participate in the process and being allowed to present one's side of the story. Although all of these things matter somewhat, what do clients care about most?" Tyler 1998: 40.

Tyler then reports on how lawyers have answered this question: "[T]he answer is obvious: Clients want to win." Id. He goes on to say, "Lawyers typically believe that clients judge them according to the size of the outcome ...[and] do not think that clients are particularly concerned with how the problem is solved." Id. He then provides a very different answer, based on the results of extensive interviews and surveys:

"Clients care most about the process by which their problems or disputes are resolved. In particular, they place great weight on having their problems or disputes settled in a way that they view as fair. The second most important issue to clients is achieving a fair or equitable settlement. The least important factor is the number of assets they end up winning." Id. (emphasis added).

Tyler's claim about what matters most to clients is based on a very large body of social science research about what is termed "procedural justice," going back to the mid 1970s.(2)

Recent research in Australia has reached similar conclusions about the key determinants of client satisfaction. A survey by the Civil Justice Research Centre of 430 personal injury plaintiffs indicated that perceived control over the case, ability to participate in the legal process, and fairness of the process was more strongly correlated with plaintiff satisfaction than case outcome, duration or cost. Matruglio 1994: viii Although a majority (65%) were satisfied with the actual case outcome, a larger majority (68%) were not satisfied with the operation of the legal system. In particular, 51% were dissatisfied with the level of information provided by their lawyers, 64% did not think the settlement negotiation process was fair, 67% did not feel that they had control of over the outcome of the case, and 80% would have liked to participate more during settlement negotiations. Id. at 16-17. In 1994 a study commissioned by LawCover, Australia's largest indemnity insurer for lawyers, indicated that by far the most significant cause of professional negligence claims was not dissatisfaction with outcome but instead related to the handling of the client relationship; the most frequent problems were failure to listen to the client, ask appropriate questions and explain relevant aspects of the matter. North & North 1994: 11, 21-26.(3)

The Effective Lawyer-Client Communication (ELCC) project was initiated in 1998 by Washington University and the Centre for Legal Education in Australia and now includes other participants from Australia, England, India, Israel, Scotland, South Africa, and the United States and from a wide variety of disciplines. (See ELCC Advisory Board, attached.) The long-term goal of the project is to determine whether international and interdisciplinary collaboration on the issue of lawyer-client communication can actually change basic institutional practices and beliefs in the legal profession. We are guided to a significant degree by the example of the medical profession, where a greatly increased emphasis on patient satisfaction is both a cause and an effect of extensive social science research on doctor-patient communication. The analogous experience in the health care field indicates that the critical first step is to develop a practical and cost-effective method to assess the effectiveness of lawyer-client communication that correlates that assessment with the degree of client satisfaction.

We have selected the first meeting between lawyer and client for our pilot study. The initial interview is, of course, the one unit of service that is constant across all forms of legal service delivery. It is also one of the most critical units of service. The initial interview: (1) shapes client perception of the lawyer; (2) defines the service to be provided in terms of both problem and goal; and (3) is an important opportunity for client education, e.g. confidentiality, substantive legal rights, what the client can do for himself or herself, and the need to preserve evidence. It may be the only contact between client and lawyer, if the service is one-time information or referral to another service provider, or if it results in a decision by either lawyer or client not to enter into a relationship. In many cases the initial interview may in fact be the most significant communication before outcome determinative events such as hearing or settlement. By assessing effectiveness at the outset of the case, this approach provides feedback to the lawyer during provision of service, thus creating possibility for improved service and increasing the relevance of the assessment both lawyers and clients.

Our first pilot project sites will be law school clinics. (To date law clinics at Case Western Reserve University and the University of Wisconsin have volunteered to be pilot sites.(4)) In addition to sites that will participate in the full pilot, we are also looking for a larger number of sites willing to use the client survey questionnaire so that we can both continue improving the form and develop a large data base for comparative purposes.(5) We would analyze and report questionnaire results for free for such sites also.) Volunteers to participate in the pilot programs will be solicited from new clients of law school clinics. A brochure and consent form explaining the project and their rights as participants will be prepared in accordance with standards both of legal ethics and human subject research. (Consent will also be obtained from the student lawyer participants.) The initial interview will be recorded, either by video or audiotaping. Clients will be given the option to stop the recording at any point during the interview and also to decide at the conclusion of the interview (or any subsequent point during representation) to revoke consent and direct that the recording (including copies and transcriptions) be destroyed.

At the conclusion of the interview the client will be given a questionnaire to be filled out privately before leaving.(6) The questionnaire will use specific, simple questions to test the client's comprehension of basic information sought to be conveyed at the initial interview, to record the client's impressions of communicative effectiveness, and to assess client satisfaction. Our current version (attached) has been designed with input from experts in the health care and customer satisfaction fields, as well as lawyers and social scientists.(7) We are using "Likert-type" questions, which ask the client to indicate agreement or disagreement with various statements on a variable scale reflecting degree of agreement or disagreement. To guard against the risk of the client "agreeing" with everything, and as a cross-check on question comprehension, we insert a number of questions that require disagreement to produce a positive evaluation of the interviewer, e.g. the interviewer "said things I didn't understand" or "asked confusing questions." The form is designed to be applicable to any interview, regardless of subject matter. It only takes up one side of a sheet of paper, giving the option of additional questions on the back, particularly questions designed to test whether specific information a particular clinic wants to convey in initial interviews has been in fact been communicated and comprehended by the client.(8)

The form contains thirteen short statements to which the client is asked to agree or disagree.

Five items pertain to how the client generally felt about the intern(9): (1) made me feel comfortable, (2) treated me with respect, (3) listened to me, (4) was interested in me as a person, and (5) was someone I could trust. Six items pertain to the quality of communication: (1) I didn't say everything that I wanted to say and the intern (2) said things I didn't understand, (3) didn't understand what was most important to me, (4) didn't explain what the intern would do next, (5) asked confusing questions, and (6) understood why I needed legal help. One item asks whether the client knows "what I need to do next," and the final item seeks a general satisfaction judgment, "If I came back to this clinic with a different need for legal help, I would want the same intern to help me." The client questionnaire will be identified only with a code corresponding to the label on the recording and will contain a box to check if the client does not want the interviewer and/or the supervising lawyer to see the responses.

We have also prepared a second questionnaire (attached), identified with a matching code number, to be filled out by the interviewer, "Assessment of Client Interview," which generally parallels the client survey by asking the interviewer to guess how the client will respond to each item.(10) The assessment has four additional items: the client (1) seemed confused, (2) told me the whole story, (3) had unrealistic goals, and (4) didn't tell me the truth.

The client and the interviewer will each place their completed forms in sealed envelopes, which will be forwarded to a central location for data compilation and analysis.(11)

Selected recordings will be transcribed by a court reporting firm and analyzed by a sociolinguist.(12) Our legal research indicates that client consent to the transcription and linguistic analysis of the interview would not prevent the attorney-client privilege from arising (or waive the privilege) as long as all persons transcribing and analyzing the interview are acting as agents of the lawyer in order to assist that lawyer to represent the client well, just as the privilege is not lost if a lawyer uses a interpreter or shows client communications to a consulting expert such as a psychologist, accountant or engineer.(13) For the initial pilot sites, Professor Cunningham will agree to serve as a consulting attorney for each client for the limited purpose of assisting clinic students and attorneys to improve communication with clients during the course of representation, thus subjecting himself to professional discipline and malpractice liability if he violated the confidentiality rights of the clients. He will require the court reporting firm and the linguist who will assist him in providing this consulting service to be bound by the same privilege and duty. The linguist has agreed to review the taped interview upon receipt, even before transcription, and will provide preliminary analysis and suggestions for the clinic within seven working days. A more detailed analysis by the linguist and Cunningham will follow, making use of the court reporter's transcription, to identify specific linguistic features that correspond to the desired objectives of effective initial interviewing. For example, the goal of obtaining a full narrative account from the client could be correlated with interruptions, change of topic, form of question, length and complexity of sentences, use of past and present tense, pauses and silence. Topics that might arise in such an analysis might include:

-How does the form of a lawyer's opening question (e.g. "What brings you here today? or "What can I do for you?") affect the way a client tells her story?

-If a client pauses during her narrative, how does the lawyer's response (e.g. silence, repetition of the client's last words, or a prompting follow-up question) affect resumption of the narrative?

The analysis will also identify potential communication problems specific to each interview, such as apparently incomplete or evasive answers and client questions and concerns that did not receive full responses from the interviewer.

If the client has not checked the "do not show" box on the questionnaire, the results of the questionnaire will be shared with the interviewer and supervisor and correlated with the interview analysis, thus augmenting the feedback to improve ongoing representation.(14) If the client has checked the box, the questionnaire data will be used only as part of the larger data set, where anonymity will be preserved, for three purposes: (1) to provide cumulative information to a clinic about client satisfaction (that eventually can be compared both over time and against other clinics) (2) to test hypotheses developed from the transcript analysis, and (3) to prioritize issues to be addressed in future transcript analysis. In turn the transcript analysis will serve to test hypotheses arising from the questionnaire data and to guide future questionnaire design.

Unlike prior social science studies of lawyer-client communication, the purpose of this project is to actively intervene in the way lawyers communicate with their clients by creating a feedback loop between lawyers and social scientists. The hypothesis to be tested is that lawyers will value this feedback and use it to experiment with their methods of communication. Thus the validity of both the linguistic analysis and questionnaire data must be assessed not only in terms of the respective social science disciplines but also from the perspective of the lawyers. We believe that such social science research can affect the behavior of lawyers if designed, implemented and explained in ways that lawyers themselves find relevant in terms of client satisfaction and more effective representation. Since law school clinics are explicitly designed to shape future legal practice by allowing students to represent clients under controlled conditions that combine external critique with self reflection, they should be particularly receptive settings for developing the methodology to test this hypothesis.

The pilot project is not expected to generate sufficient data to form generally applicable benchmarks for quality lawyer-client communication. The pilot will, however, develop a research methodology that can be transferred to a wide range of legal practice settings. The pilot project will define linguistic features that can be readily identified by lawyers who can review interview tapes themselves for quality control and learning goals. Further, the client questionnaire, when refined through correlations with linguistic analysis in the pilot, can be a valuable and very cost-effective assessment method even when used alone. If a standard client questionnaire becomes adopted widely, then lawyers will be able to compare both individual interviewer and program wide data with a large data set from a number of other lawyers, thus receiving guidance about the levels of client comprehension and satisfaction that can reasonably be attained. (The American health care system is being transformed by the use of such comparative data bases on patient satisfaction.(15)) Such standardized assessment could be particularly useful for high volume providers of legal services, such as legal aid and public defender programs, by creating an empirical data base for making cost-benefit decisions when experimenting with modes of service delivery. This pilot project will also lay the groundwork to test current interviewing practices for effects of gender, age, education, class, ethnicity and language and to develop new and more appropriate methods of communication across cultural lines.

 

REFERENCES:

Casper, Jonathan D., Tom Tyler, & Bonnie Fisher. 1988. "Procedural Justice in Felony Cases," 22 Law & Society Review 483.

Cunningham, Clark D. 1999. "Evaluating Effective Lawyer-Client Communication: an International Project Moving From Research to Reform," 67 Fordham Law Review 1959.

Cunningham, Clark D. & Bonnie McElhinny. 1995. "Taking It to the Streets: Putting Discourse Analysis to the Service of a Public Defender's Office," 2 Clinical Law Review 285.

Felstiner, W.L.F. 1997. "Professional Inattention: Origins and Consequences," in The Human Face of Law: Essays in Honour of Donald Harris 121 (Keith Hawkins, ed., 1997).

Handley, Robin & Damien Considine. 1996. "Introducing a Client-Centred Focus into the Law School Curriculum," 7 Legal Education Review (Australia) 193.

Hosticka, Carl J. 1979. "We Don't Care About What Happened, We Only Care About What is Going to Happen: Lawyer-Client Negotiations of Reality,'" 26 Social Problems 599.

Krowinski, William J. and Steven R. Steiber. 1996. Measuring and Managing Patient Satisfaction. Chicago: American Hospital Publishing.

Lind, E. Allan and Tom R. Tyler. 1988. The Social Psychology of Procedural Justice, New York: Plenum Press.

Mather, Lynn; Richard J. Maiman & Craig A. McEwen. 1995. " 'The Passenger Decides on the Destination and I Decide on the Route': Are Divorce Lawyers 'Expensive Cab Drivers?'," 9 International Journal of Law and the Family 286.

Matruglio, Tania. 1994. Plaintiffs and the Process of Litigation (Sydney: Civil Justice Research Centre)

North, Ronywyn & Peter North. 1994. Managing Client Expectations and Professional Risk (Sydney: Streeton Consulting)

Tyler, Tom. 1988. "Client Perceptions of Litigation," Trial July 1988, p 40.

1. Louise McKinney, Judith Lipton, Bryan Adamson and Kenneth Margolis at Case Western Reserve University School of Law played a major role in the design of the client survey form and other aspects of the pilot project. Valuable advice was also received from Colin Adamson, Melvin Hall, Alan Lambert, Les McCrimmon, Mary Barnard Ray, Lynne Robins, Meredith Ross, Avrom Sherr, and Rodney Uphoff. Support from the Israel Treiman Research Fund is gratefully acknowledged.

2. See Lind & Tyler 1988 (reviewing theory and research on procedural justice); Casper, Tyler & Fisher 1988 (showing that even for felony defendants procedural justice factors, like the amount of time attorney spent with client, have an independent --and perhaps greater--effect on overall satisfaction than actual outcome, such as no jail time or length of prison sentence).

3. LawCover was so impressed by these findings that it began to offer premium reductions to lawyers who participate in a series of workshops on lawyer-client communication. Handley & Considine 1996: 197-8.

4. Law teachers in Australia, England and South Africa have also indicated interest in involving their clinics in the pilot project.

5. This form can be downloaded in either Microsoft Word or WordPerfect format from the ELCC website: http://ls.wustl.edu/Communication/

6. Case Western offers clients the option of being assisted by a support staff person, who could, for example, explain the format and read questions for persons having difficulty with written English. So far no client has asked for such assistance. Case Western also added two sample questions to the top of the form: (A) The Cleveland Indians are the best team in baseball (B) Nobody would want to live in Cleveland. (The client should learn from these samples that a positive opinion sometimes requires agreeing and sometimes disagreeing with the proposed statement.) There is space on the standard form for each clinic to add two short customized sample questions of this sort.

7. We also relied considerably on Chapter Eight, "Quantitative Research," in Krowinski & Steiber: 1996

8. For example, in a criminal defense clinic, questions might check whether the client understands the extent of confidentiality, has been warned about discussing the case with others, knows what and when the next court action will be, and knows how to contact the clinic. Other useful information, particularly for monitoring quality of public defender services, could be gathered, e.g. a report from the client about the location, privacy and duration of the interview.

9. The generic form is based on the specific form used by Case Western, which decided to use the term "intern" to refer the law clinic student who conducts the interview. Other clinics might choose to use the term "interviewer" or "student attorney" and obviously "lawyer" can be substituted if the form is used in a legal aid, public defender or private practice setting.

10. The assessment form is also on the ELCC web site.

11. Professor Alan Lambert from the Washington University Psychology Department has agreed to help design the questionnaire and to compile and analyze the answers; Professor Tom Tyler (New York University Psychology Department) has agreed to consult with Lambert on questionnaire design.

12. Lynne Robins, a nationally recognized expert on professional discourse currently on the faculty of the University of Washington Department of Medical Education, has agreed to provide the linguistic analysis. Although most of Robins' work has been on doctor-patient communication, she participated in a study of client interviewing at one of the clinics of the District of Columbia School of Law. Gellhorn, Roth & Robins (1994). Robins plans to work with ELCC to develop a major long-term funding proposal to apply this methodology to both legal and medical professional discourse

13. Cunningham & McElhinny 1995:294-96.

14. In the initial pre-test of the questionnaire at Case-Western in April 2000, none of the clients checked the "do not show" box.

15. Attached is an excerpt from marketing materials distributed by Press, Ganey Associates, a major firm in the patient satisfaction field, that shows how a hypothetical hospital, "Central General" can compare its patient satisfaction scores on various items: (1) among its own departments, (2) against the previous year, and (3) against the mean score of 172 health care providers using Press Ganey's services. The report also gives "Central General" its percentile rank on each item among the 172 health care providers. It is more difficult to ignore negative satisfaction responses if comparable service providers get much better responses. Likewise, even relatively high satisfaction scores do not result in complacency if competing providers are getter even higher scores. Data such as provided by Press Ganey is routinely used by health care providers to evaluate staff, including doctors (see Cunningham 1999: 1959-60), and to award staff bonuses (conversation with Dr. Melvin Hall, Chief Operating Officer of Press Ganey).