Why Is There Such Reluctance to Improve Review Practices?
On this blog we made several postings over the past months concerning challenges with review practices and some solutions to remedy these problems. The discussions posted here mirror the discussions I have with clients that have brought M/C in to their organizations to address review practices. The normal approach we take to these client presentations is to characterize the problems, seek feedback that we have offered a characterization everyone understands, and get tacit feedback that the characterizations are the types of “problems” observed within the organization. We then offer solutions that apply at the individual and the organizational level.
All this is rather straight forward. What is not straight forward though, are the objections offered to why the solutions are inappropriate. Why do people willingly admit to the observed weaknesses and then steadfastly reject or resist solutions that are proven to mitigate or eliminate the problems? Why is there such reluctance to improve review practices?
I do not have good answers to these questions. I am not even sure I have bad answers. What I do have are theories, which I will share in my next posting. In turn I invite you to share your thoughts on the question: why is there such reluctance to improve review practices?
Who Are We Writing For Anyhow?
A common rule applied in many forms of technical writing is: “before a writer starts to write, he/she should think about who they are writing for.” That is, design your work to satisfy the needs of your audience. A rule we find to be well-justified and one that should be “common knowledge.” Unfortunately, we find this tongue in-cheek pejorative coined by John Kenneth Galbraith is aptly applied to much of the writing in the life sciences; where writing for one’s audience is indeed an uncommon occurrence.
In our work over the years with numerous organizations and individuals we find several constants. One being that authors and reviewers of technical and scientific reports used directly and indirectly to support regulatory submissions generally have little insight into what their audience wishes “to do” and how their audience intends “to use” the documents they produce.
Each year I teach 50+ workshops related to aspects of writing and reviewing technical and scientific documents. In every session I ask participants to describe for me who they are writing for (audience) as well as how (how is the document read) and what (tasks) do they do with their documents. The responses regarding audience ranges from: “I do not know” to “I write for my supervisor” to “I write for the health agency reviewer.” The response to the question regarding usage of documents is highly uniform—there is no response. No one (I literally mean no one) to date has offered even a suggestion as to how their audience reads their documents or what their audience wants to do with the presented documents. Unfortunately, I am not surprised by the lack of response. It is emblematic of the status writing holds in many organizations involved in life science R&D (you can find a discussion on this point in an earlier post).
We suggest the approach taken by many writers in the sciences follows the “readers addressed” theory that suggests readers of technical and scientific reports are passive recipients of information. In this theory the writers emphasize what information they want to write about and how they would like to represent and organize the information rather than analyzing their audience expectations, skills, and reading needs and how to make it easy for them to receive and use the information. Often we find authors and reviewers assume they are writing for a “homogeneous audience”, with similar backgrounds, skills, and preferences to their own. That is, these writers and reviewers see themselves as a member of an “interpretive community” of scientists with shared knowledge, experiences, and points of view. Thus, they assume their audience will find relevance and irrelevance of information as they do.
The reader addressed theory is further encouraged by the widespread following in the life science community of the “logical positivism theory” that data can speak for themselves. This model says: all the writer has to do is to organize and present the data in a clear manner and everyone technically competent will correctly understand the underlying meaning of the presented information.
Our interviews over the years with operatives using documents in a variety of health regulatory agencies suggests that in practice most enterprises presenting documents to these agencies indeed see the health authority reader as passive recipients of information. A case in point: we had a client prepare a briefing document for the FDA as a prelude to a 90-minute meeting to address eight questions. The briefing document is to inform the regulators on data and factors directly connected to the questions to be addressed in the meeting. Routinely these documents are 60-80 pages in length. Our client prepared a document that was over 525 pages. I suggest such a dense document defies the operative meaning of the word “briefing.” In this case, they had in excess of 65 pages of information for each submitted question. An assumption made by the authoring team was: “Oh, we cannot make it shorter—it is all very important information and they [FDA] will want to read all of this information.”
Regrettably, it is not. Also regrettably, the common rule urging a writer to think about their audience before writing is far too uncommon when it comes to many forms of writing in the life sciences.
Why Training Fails to Move the Needle on the Work Performance Meter
For those who are involved in managing a group and concerned with enhancing individual and team skill sets, you may want to read this article by David Maister. Maister suggests that much of the money poured into training endeavors yields little return because companies treat training as an isolated event that is not effectively supported in the workplace. Here’s a quote from Maister:
However, I now believe that the majority of business training, by me and by everyone else, is a waste of money and time, because only a microscopic fraction of training is ever put into practice and the hoped-for benefits obtained.
Maister goes on to state that training is often treated as an isolated activity where the skills and concepts are never introduced back in the work place.
Companies train people in new areas but then send them back to their operating groups, subject to the same measures and management approaches as before.
Unfortunately, we see this from time to time in our work as well where training is organized as a stand-alone event, with a life of its own that is disconnected from the day-to-day reality of how work is “done” at the company. This is one of the reasons we only do in-house training and have extensive interface with the management team before and after training engagements.
To improve writing outcomes–be it in terms of pre-writing planning, drafting, or reviewing–it really requires applying different measures and management approaches. Unfortunately we do find situations where middle and upper management are the “absentee landlords” well-removed from committing time and energy to enable the ideas and concepts to be embedded in the organization. We have feedback in some of our workshops as described by Maister in this quote from his article:
Too often junior people are sent off to be trained and they continue to speculate whether their seniors or leaders are really committed and serious about the topics being discussed. As previously noted, they often are not.
Our clients who have bought into this notion have indeed moved the needle on the meter. Maister adds:
If, however, we really want to help people develop skills, we must view “training” the way an exercise instructor would use that word — designing a planned set of activities that engage the right “muscles” and slowly build them up through the experience of doing.
We see this as an essential approach in changing individual and organizational work performance when it comes to the task of creating high quality research reports. To be effective, training in the areas related to individual or group authoring performance must be context rich and ensure that the participants not only intellectualize the ideas, but more importantly that they attempt to apply the ideas in measured steps and with the full backing of management. As the old saying goes: “training ain’t just telling.” As Maister suggests in his article, if you treat training as an activity, that is all you will get–activity. Much of training in the pharmaceutical world is principally in this mode. When an enterprise trains people on an SOP, it is mostly because they have to do this “activity” to be compliant with GXP guidance, not because they are looking to change how people think and perform. I know this is what needs to be done, I just wish companies would not associate the word “training” with the activity of “informing” people about operating procedures.
Bottom line: treat training as a critical ingredient in success and then you indeed see that the needle moves on the performance meter.