Not knowing when good is good enough in writing regulatory documentation has a huge cost

We do not talk much on this blog regarding the use of language or the application of terms in science writing. Principal reason is that much of what we see in regulatory submission documents is genuinely “good enough.” However, others do not necessarily see it that way. I want to share with you how discussions in review roundtables can end up getting focused at really absurd levels of detail with a misapplied sense of establishing quality communication. 

In our consulting work, we try to be disciplined during our document reviews and only comment on language when it truly obscures or alters meaning. Being grammatically perfect in regulatory submission documents is a nice notion, but in practice consumes way too much time and organizational energy and will yield little in terms of outcomes.

We share this point with people all the time...but at times the advice goes unheeded and even worse…at times people just do not know when to move on and address real big concerns in their documents.

A case in point is a situation I observed regarding a long winded discussion in a review meeting over the use of the term “very critical”. The term “critical” in a medical sense means: of a patient's condition having unstable and abnormal vital signs and other unfavorable indicators. In theory, the meaning of critical is a black-or-white proposition without qualifications regarding gradation. Something is either critical or it is not. Therefore there should be no adverbs, like “very” in front of the term “critical” to connote a measureable degree of criticality. In this roundtable review the team got caught up in a 30 minute discussion that involved only two people arguing whether to use the term “very critical” or change it to “critical.”

Being pragmatic, I’d have to say: “Guys what are you thinking? You hold a team hostage for 30 minutes to argue over grammatical accuracy? To argue over something that will not matter when and if read by a regulatory reviewer. There were 10 professionals sitting in the room and 8 did nothing for 30 minutes. Cost of salaries alone is enough argument to say “Forget about it, let’s move on…we cannot afford to argue over such insignificant detail.” When we add in the opportunity cost (what these 10 people collectively could have been doing with their time), then for sure you have to make the argument.

This above episode gets played out time and time again in review sessions all over the pharma and medical device industries and is the reason why I am steadfast in my position that the vast majority of people involved in authorship and review do not know the answer to the question “How do you know when is good, good enough?” The end result is inordinate amounts of time can be applied at the wrong level of detail in reports and submission documents.

Change, change everywhere...yet it all remains the same

Change, change everywhere....yet as far as I can tell, most writing practices used to day to create research reports in the life sciences are the same as those deployed in 1928.


I did a workshop the other day for a group of medical writers looking at aspects of managing documentation projects. I started the session with a photograph taken in a newspaper office in 1928. I suggested to the group that other than the typewriters on the desk, the ways they work are essentially the same as the people posing in that 1928 photo. My comment got everyone’s attention, though I am sure some did not receive well the message. I am quite comfortable with my statement as traditional documentation development practices in the realm of medical writing have remained more or less unchanged for decades.


I explained to the group that much of the medical writing work they engage in is played out exactly the same way it was in 1928. Their current writing world remains one that is largely a work platform where they produce drafts in isolation and then send the drafts out to team members (electronically now unlike paper of the past) who then review in isolation and ultimately return the marked up documents to the authors (exactly like they did in 1928.)


My point to the group was if you really want to change outcomes, then you better think about changing your work practices. Doing things largely the same way will continue to yield the same outcomes. Just because you now have the reviewers grab your document from a server file, or you do round table reviews via WebEx, or you author your report in an enhanced electronic template does not mean you have made a radical departure from how people authored research reports or other documents 20-30 years ago. And I do know that the same weaknesses found 50+ years ago in document products and practices remain today.


You want different outcomes? Then change the practices associated with planning, authoring, and reviewing documents, not merely the tools and the processes you lay on top of practice.

More thoughts on the limited sophistication of documentation practices in the life sciences

I mentioned in this post that I consider the documentation practices for creation of regulatory submission documents in most pharma and medical device enterprises to rather unsophisticated. My position is largely driven by comparing observations of documentation practices to descriptions of varying levels of documentation maturity we have developed. Our descriptors have their roots in the work presented by Joann Hackos in her book: Managing Your Documentation Projects.

Like Hackos, our documentation practices-maturity model is a six point scale ranging from: Level 0 — Oblivious to Level 5 — Optimizing. My observations suggest that the vast majority of regulatory writing falls into the 2.5 range, which is between Rudimentary and Organized/Repeatable.
The Rudimentary documentation organization is one where the vast majority of effort is placed to ensure documentation consistency. All documents are generated utilizing well characterized templates. Document strategic review is largely absent as energy is applied to ensuring structural (that is, grammar and format) accuracy and consistency. Work practices are highly individualized and there is little application of meaningful estimating of document size or time lines.

The belief statements of organizations working at the rudimentary level include the following:

• All writers manage their own projects
• We would like to know more about our reading audience, but nobody takes the time to learn more….so we “suppose” what they want in our documents
• Our users are just like us…….I am a subject matter expert, so every other person educated like me will read documents just like I do
• We talk about what the data means, but we rarely talk about how to represent this meaning in our documents until we are into round table reviews
• We have little concern with how many rounds of review go into creating the final version of a document….we strongly endorse the credo “the end justifies the means”
• Planning document content before actually writing a report is just busy work
• We routinely reverse engineer document development timelines from the stated publication deadline and not from the scale and scope of the intended document
• We care what the customer thinks of our documents, but we do not use any organizationally applied quality standards. Standards are principally driven by teams and their senior reviewers
• File and forget–we do not take time to collectively reflect on documentation work practices practices

The belief statements of organizations working at the organized and repeatable level include the following:

• We have begun to study our document users but see little value in a concerted effort to collect information on how well our documents “satisfice” their needs
• We are surprised or even indignant when we get questions from our regulatory user looking for information that we included in our submission package
• The quality of our document project management is inconsistent, but we are okay with that because that is reality and each project is unique
• We do not see meaningful metrics beyond time for documentation projections…if we meet the deadline, then ways of working had to be good
• Nobody knows what others are doing in the process of review…the only way they know is via what may be discussed during a round table review
• We believe in our described “ways of working” until faced with unexpected situations, then we panic and call for “all hands on deck”
• We can easily get caught up in “process” at the expense of “product”

Now let’s contrast the above with belief statements of organizations working at the optimized level which include the following:

• We always engage in collaborative pre-writing planning of documents to make sure we fulfill strategic purpose of any given document
• We are thoroughly committed to understanding our document users and we work to systematically collect information from them
• We know how big a document will be even before we write it
• We always do end-of-project analysis to collect lessons learned and then disseminate this information across the organization
• We maintain a database of critical documentation work practice parameters and benchmark all documentation projects
• We recognize that accuracy and consistency are just the start of ensuring quality….we have expanded focus to quality of argument and document usability
• We are learning how to be innovative and not let the process control us

Originally published on our Knowledge Management blog

Designing the architecture of the argument in development reports

Kirk Livingston, a teacher and a medical writer working largely in the medical device industry, as well as a fellow blogger at LivingstonContent, shared this comment on my previous post regarding poor rhetorical shaping of arguments in research reports.

There’s a lot of work involved with producing solid, well-reasoned conclusions. Can it even be accomplished as an “authoring team” or is it the work of an individual? Recent research about medical device companies in Minnesota suggests communication teams are chronically understaffed. So–who has time to come to the right conclusions? Thanks for the thoughtful post.

I agree there is considerable work that goes into producing solid, well-reasoned conclusions. I am certain the work can indeed be accomplished by an authoring team. The caveat here is, it can be accomplished as long as the team engages in truly collaborative authorship work practices and makes use of pre-writing planning tools to help shape the argument.

I am not so sure that writing teams are chronically understaffed. I think the real issue here is the limits of interest and skill that team members may have towards the task of writing. As I reflect on 17 years of work associated with the authorship of regulatory documentation, I am convinced adding numbers to the equation will have little bearing on the rhetorical qualities of any given document. Larger writing teams will likely yield only emotional comfort--the notion of safety in numbers.

Producing high quality documents is a function of knowing what you want the document to do for you, a sense of where arguments must be played out in a document, and what writing tools to use in order to get true collaboration and sharpen everyone's focus to achieve the objectives you want to document to support.

Producing high quality documents in the forum of pharmaceutical and medical device research requires understanding how to build out the red thread of logic in a research report. In pre-writing planning it starts with something as simple as building a table that is to be filled in by the authoring team. The table has three columns to be completed by the team:

Primary & Secondary Objectives | Conclusions | Key Data

You then have one row in the table for each objective.

The team's task is then is to build out conclusions about achievement of each objective and what data warrants that conclusion.  A simple but powerful writing tool that helps a team to lend considerable shape to the architecture of the argument that must be represented within a report.


Originally published on our Knowledge Management blog

Effective argumentation often missing from clinical research reports

Much of clinical research reporting associated with regulatory submission strategies must be persuasive writing. In particular, discussions about the significance of study findings and how these findings directly support development claims. To win the point of persuasion, such writing must follow conventions of well-framed arguments that avoid rhetorical flaws.

I've been doing quite a bit of work over the past two months with groups who are responsible for producing clinical research reports. Reflecting back on my work, I find that many involved in authoring or reviewing these documents do not recognize the elements that contribute to effective arguments. They often confuse summaries of study findings as being meaningful conclusions about the significance of research findings.

One of the common rhetorical flaws is that authoring teams are comfortable making broad generalizations and see such generalizations as being solid conclusions and part of well-framed arguments. Broad generalizations seldom stand scrutiny of regulatory review and the generalizations of truly difficult and controversial topics never withstand close scrutiny. Faulty generalizations are easy to spot. For instance terms like......"generally well tolerated;" "suggests relevant differences;" and "may be associated with" appear throughout discussion and conclusion sections of many clinical research reports.

Originally published on our Knowledge Managment blog

4 Books Every Technical/Scientific Writer Should Considering Reading

The following books are ones that "new" technical and scientific communicators should consider reading. The books look at technical writing from different perspectives and will prove useful to the experienced communicator as well.

Metaphor and Knowledge: The Challenges of Writing Science. Ken Baake

Summary: “Metaphor and Knowledge offers a sweeping history of rhetoric and metaphor in science, delving into questions about how language constitutes knowledge. Weaving together insights from a group of scientists at the Santa Fe Institute as they shape the new interdisciplinary field of complexity science, Ken Baake shows the difficulty of writing science when word meanings are unsettled, and he analyzes the power of metaphor in science.” (partial text available from Google Books)

What Writing Does and How It Does It: An Introduction to Analyzing Texts and Textual Practices. Charles Bazerman, Paul Prior, editors

Summary: “In What Writing Does and How It Does It, editors Charles Bazerman and Paul Prior offer a sophisticated introduction to methods for understanding, studying, and analyzing texts and writing practices. This volume addresses a variety of approaches to analyzing texts, and considers the processes of writing, exploring textual practices and their contexts, and examining what texts do and how texts mean rather than what they mean. Included are traditional modes of analysis (rhetorical, literary, linguistic), as well as newer modes, such as text and talk, genre and activity analysis, and intertextual analysis.” (full text)

Technical Writing: Process and Product. Sharon Gerson and Steven Gerson

Summary: “Technical Writing: Process and Product guides readers through the entire writing process—prewriting, writing, and rewriting—developing an easy-to-use, step-by-step technique for writing the types of documents they will encounter on the job. The authors' reader-friendly style engages readers in the writing process and encourages hands-on application. Discusses prewriting, writing, and rewriting in relation to ethics, audience identification, electronic communication, and the role of technical writing in the workplace. For anyone looking to utilize more effective written communication in their jobs.” (summary and e-learning resources available at publisher's website)

Professional and Technical Writing Strategies: Communicating in Technology and Science. Judith VanAlstyne and Merrill D. Tritt

Summary: “Comprehensive and easy-to-read, this award-winning reference for those in technical, engineering, and scientific fields emphasizes practical writing. Its presentation and applications offer simple guides that users can easily emulate. It combines instruction, sample papers, exercises and writing projects for manuals, correspondence, research and publication articles, and oral technical communications.” (summary)

Originally published on our Knowledge Management blog

Why documentation work practices in pharma and medical device industries need enhancement

I recently taught a workshop for a client where I gave the participants a good news/bad news scenario. The good news, I suggested, is the documentation work practices I observed in their company were like what I saw in other pharma companies. I then suggested the bad news is their work practices were like what I saw in other companies. My point to them is that the bar for sophisticated work practices is set pretty low in the pharmaceutical industry.  I used as my line of reasoning our McCulley/Cuppan modification of JoAnn Hackos’ Document Capability Maturity Model (DCMM) she presents in her book: “Managing Your Documentation Projects.”

To this group I suggested their documentation work practices would come out at about a 2 on our 6 point scale, which suggests also the point on the scale I will place most of the documentation groups in the pharma industry. Most were surprised at the low ranking and a few were even offended I think. By the end of our workshop, all were feeling different about what it takes to create high quality technical documentation in the life sciences.

I’ll characterize here a documentation project undertaken by this group I had in the workshop and suggest such work is emblematic as to why I give them (and the industry) a low DCMM score.
The group generated a briefing document to be submitted to a regulatory health agency in advance of a request (a standard practice across the industry.) The briefing document is intended to inform the health agency as to what topics the company wanted to receive feedback from the health agency regarding an ongoing drug development project.
The company had an hour meeting with the agency and ended up wanting to address 8 topics within the hour (an ambitious agenda.)

The documentation team went through a three draft process to create the briefing package where the number of questions and the actual questions changed w/ each draft. I mean they were truly using a “fire; aim; now we are ready” approach to create this briefing package. The approach of: "activity is everything, so we are way too busy to plan."

Each draft looked dramatically different than the previous version and required massive investment of time and energy from the team. In the end the briefing package was 83 pages—that is a little over 10 pages per question. The first question did not appear until page 25 (which means there were 24 pages of introductions and throat clearing before we get to the meat. Curiously to me, none of the questions appeared in the Table of Contents of the briefing package and the document did not have an Executive Summary (we want you to work as hard in reading this document as we did in crafting it.)

Then I asked the authoring team what they did as a documentation project post-mortem for lessons learned, such as solicit usability feedback from the customer using the document or to identify how the documentation work practices impacted the process and the product. The answer was, “Oh we did not do anything like what you suggested. The document must have worked because we had our meeting with the agency. We just moved on to the next document.” I asked if they got what they wanted out of the meeting. The answer was a categorical no. “We did not get an answer to our most important question as we had placed it as 8 of 8 in the queue. We ran out of time to address that question.”

I asked how uniform their documentation practices are for the creation for briefing packages (they will generate such packages numerous times in a given year for various development projects). Response was “We have no set approach, so each briefing package will be generated utilizing variable methods and processes.”

How they could be surprised with a low rating for the sophistication of their documentation practices is the interesting question for me. Where comes the comfort with such outcomes? In part, I think, it is due to erroneously assigning the sophistication in conduct and analysis of science as carrying over to documentation work practices.

Originally published on our Knowledge Management blog

Critical Thinking Skills

I am doing some work on how to use visual thinking in the collaborative work environment and got into an interesting discussion with the client regarding critical thinking skills,  the subtle differences for application of these skills in the business place of drug and device development versus the academic place, and how well prepared people may be for the demands of problem-solving in the workplace.

As a result of the conversation, I got looking for some reference materials regarding critical thinking and methods to improve these skills.  I came across this blog by Tim van Gelder: "Bringing Visual Clarity to Complex Issues" and his post regarding how critical thinking skills are acquired. Check it out.........a very interesting read. I am quite interested in van Gelder's comments regarding "situated cognition" where he states:

CT [critical thinking] is deeply tied to particular domains and can only be acquired through properly “situated” activity in each domain.  Extreme versions deny outright that there are any generic CT skills (e.g. McPeck).  Moderate versions claim, more plausibly, that increasingly general skills are acquired through engaging in domain-specific CT activities.

Originally published on our Knowledge Management blog

The importance of understanding the reader and the need to be informed by reading theory

In thinking about the ExLPharma conference where I spoke two weeks ago on review, I talked for some time about the importance of reviewers understanding their readers and the need to be informed by reading theory so as to become a truly good reviewer. My comments were largely well received by the group attending the conference and, for most of the points I raised regarding reading theory, were truly novel for them. I came away from the conference with the notion that few people actively engaged in creating business documents have invested time in thinking about what regulatory readers actually “do” with their documents and how they actually read these mission-critical documents.

Given that most people inspect a document for accuracy tells me that the prevalent view regarding reading is “if we get the numbers and the words right, then we are good to go and surely everyone will understand what we mean.”

I remain perplexed as to why this view is so uniformly applied. I am even more perplexed why some people I cross paths with in my client settings struggle to accept the notion that readers construct a meaning that they personally create from a text, so that "what a text means" can differ from reader to reader. Readers construct meaning based not only on the visual cues in the text (the words and format of the page itself) but also based on the knowledge readers already have stored in their memory. This pre-existing knowledge readers bring with them as they encounter a text is very potent and not much appreciated by many I work with across the life sciences.


Originally published on our Knowledge Management blog

6 Questions Regarding Medical Writing as a Profession

A follow-up to some earlier posts regarding medical writing as a profession. I have six questions I am pondering about the notion that medical writing is a profession. I welcome your thoughts on these questions.

1. Why should medical writers attend to professionalization issues? Why not?
2. Why is professional status necessary or desirable? Why not?
3. If you want to use the term professional, then who should set the standards and minimum qualifications for use of the term "professional"? How should the standards be established? Does an AMWA or EMWA certification make you a professional?
4. What can be learned from other professions that have achieved a professional identity?
5. Some observers will suggest that seeking professional status is merely a form of elitism where desire is to control knowledge and restrict access?
6. What theories/practices inform medical writers as authorities and thus warrant professional status?

Originally published on our Knowledge Management blog

Proving Document Quality

You say you produce high-quality document products? Great, now prove it! Although many can recognize quality when they read it, the prospect of measuring documentation quality seems dubious to many.

I work with a great many clients where authors say they generate high quality documents, but they can only show that their documents are accurate. That is, accurate in terms of template, source data, and grammar conformance. Beyond these three inspection parameters, many regard documentation quality as inherently immeasurable. I argue that these three parameters by themselves are wholly insufficient measures of document quality and that important aspects of how documents convey meaning can be measured.

Surely they must see other metrics of value, especially when considering factors like the number of regulatory questions elicited by errors of omission and commission in their very accurate submission documents or the number of protocol amendments generated over the life of their very accurate research protocol.

Some see additional metrics as meaningful, but not for their type of documentation. In the realm of clinical development documentation, many consider their output unique and beyond measurement. I hear more than a few medical writers say, “We know our own field intimately, and everywhere we look we see shades of gray and unique situations, so measuring is not useful.” and others in clinical development say “We are unique, and your notions of quality do not apply to us.”

Still others find the prospect of measuring output impractical or suggest that even if they had data, “Against what standards could we make comparisons?”

Even easily measured attributes like accuracy are not effectively tracked. Documents are routinely verified/corrected by quality control operators, yet the output is rarely tabulated and statistically analyzed to provide a measure of writer performance. The intention is merely to ensure the document is archived as accurate in terms of template, source data, and grammar conformance.

I do not know of any pharmaceutical or medical device company that makes use of any statistical quality control techniques to track their document quality. It would be easy to apply similar principles as used in the auto industry or pharmaceutical manufacturing. The auto manufacturers turn out millions of units every year, and each company tracks manufacturing defects, but not by examining every car. Instead, they pull a few cars off the assembly line and tear them apart. Pharmaceutical manufacturers test an appropriate sample from each manufacturing lot. In both cases the products are measured against a set of well-defined acceptance criteria.

Such acceptance criteria exist for technical and scientific documents (in our work at McCulley/Cuppan we have developed such a set of criteria that we use during our training and consulting work for clients) so such quality control methods could easily be deployed.


Originally published on our Knowledge Management blog

More Regarding Merit of Peer Review

Last week the New York Times had a major article devoted to web alternatives to peer review. Actually it does not describe an alternative to peer review but rather an alternative way of doing peer review via the web. The concept is not new, but by getting major coverage by the NYT, perhaps web-mediated review is starting to get some real traction.

Way back in 2004, Gerry McKiernan, Science and Technology Librarian at Iowa State University, posted a great paper suggesting internet-based peer review alternatives: "Peer Review in the Internet Age: Five Easy Pieces."

Last point for this post, here is an interesting editorial paper: "Is Peer Review Censorship?"  written by Arturo Casadevall and Ferric C. Fang that appeared in the American Society for Microbiology. As a point of emphasis to echo comments we have made previously in this blog, I note their comment that:

The current system persists despite abundant evidence of imperfections in the peer review process. Most scientists would agree that peer review improves manuscripts and prevents some errors in publication. However, although there is widespread consensus among scientists that peer review is a good thing, there are remarkably little data that the system works as intended.

Once again, why is a community so driven by evidence, so willing to cast a closed eye to the paucity of support for the practice of peer review?


Originally published on our Knowledge Management blog

What's wrong with Peer Review? Quite a bit, actually.

I continue to follow conversations on other blogs and web sites where people are talking about peer review.  Here's the link to a very interesting paper by Michael Nielsen: “Three myths about scientific peer review” that is definitely worth the read. Another article I suggest you check out is by Alison McCook and published in The Scientist: Is Peer Review Broken? In particular, please read the section titled The Religion of Peer Review where McCook expands on our argument that an abundance of data from a range of journals suggests peer review does little to improve papers.

So the question I have to pose then is "given the lack of evidence that peer review works, then why do so many scientists in the life sciences remained joined at the hip to the notion that review really makes for high quality publications?".


Originally published on our Knowledge Management blog

More on Topic: Is Medical Writing Really a Profession?

I posed this question on a medical writing affinity group I belong to on Linked In (Professional Medical / Scientific Writers). The shared commentary, all from medical writers, has been quite interesting. In this post I want to share a few of the commentaries and my responses.

It was stated by one commentator: “I do feel, because of the way this question is worded, there is a need to defend my profession.” Another said: “It seems to me that whether or not one describes medical writing as a profession is partly down to semantics, and what any individual person understands by the term profession.”  In response, I am not asking you to defend your work, but I am asking you to defend that the work of medical writers as a class is indeed a profession. I shared some thoughts regarding how the scholars characterize the concept of a profession, as I think much more than semantics is involved here.

Another commented “Is medical writing a profession? Well, some parts of it are more professionalized than others, but I think.... as a whole? No, currently it's a job description.” I wholly concur with this position for reasons I have already expressed and additional reasons I will add in this discussion entry. I modified slightly another comment made in the Linked In discussion to help shape a sense of the needed attributes for applying the moniker of professional to medical writing: “That practitioners are expected to understand the complexities of language and communication we've seen in this discussion so far and medical writers use workplace research and academic theory to push the practice of medical writing forward.”

Another interesting comment posed as a question is the following: “On another level, the question is: are you hiring a profession or an individual?” My observations find writing approaches within organizations and across the industry are idiosyncratic and outcomes unpredictable, so therefore the answer is clearly you are hiring individuals. By the way—this variability contributes to why so many enterprises struggle to identify “markers” to use to judge capabilities of medical writing candidates.
This same person said: “which part do we emphasize: the medical or the writer?” The answer I suggest is neither—the emphasis is on reporting first and foremost (and reporting in a manner that is accurate and compliant), then medical insight, and a distant third is writing (in terms of sharing information and shaping knowledge).

Lastly, I want to highlight the following comments: “Currently, I don't know that medical writers are seen as the experts on Medical Writing -- most of the public discourse seems to think we are unreflective cheerleaders for whoever pays our salaries....” and “ We do a lot of the applied science stuff already. But we have not come together to share that as professionals, and I think that's the biggest barrier to professionalism .”  Harsh commentaries both, but I suggest each is valid based on my observations of the ways of working in this industry. Unfortunately, I will suggest medical writers who do “applied science stuff” in terms of writing practice and document products are far and few between.

Yes, I want medical writing to be considered a profession. I did not start this post to raise a beef with the notion of medical writing. I opened in the discussion in the hopes of fomenting a thought-filled argument about what it takes to call oneself a professional medical writer. An aspect of professionalism is intelligent practice as defined by scholars like Schon. Intelligent practice is the application of knowledge to the body of work at hand. In this case, documents. I am suggesting that such intelligent practice is not a common attribute within or across communities of medical writing. I say this because of a significant amount of data that suggests the vast majority of documents produced by medical writers fail. These documents fail in terms of purpose, conveyance of logic, and meeting reader’s needs.

In defense of my claim I submit to you that there is considerable data in the public record regarding target-audience usability for the output of medical writing. A survey of keen interest to me shows that FDA reviewers cite poor document quality as a significant impediment to effective, efficient reviews; a comprehension study of clinical study protocols found that over 50% of test subjects (clinical investigators and study sight managers) failed to achieve a passing score in identifying roles and decision making responsibilities in the conduct of a clinical trial. I also suggest that much has been and continues to be written about the poor quality of manuscripts submitted to journals for publications. Additionally, analyses of the writing output in various clinical research genre suggests the documents are written for a persona that has a high command of the English language, enjoys reading long, dense narrative passages, and has an exquisite memory (I am pretty sure this persona does not match the broad audience these document products are intended to serve.)

When I raise these above points with medical writers, which I do all the time, I can say nearly all are unaware of any elements of the work I generalized above. Further, the vast majority of medical writers I encounter are unaware of any aspects of reading theory. Also I suggest to you that few medical writers can distinguish the differences between the concepts of data, versus information, versus knowledge; few of the hundreds of medical writers I have spoken to in the past 10 years have heard of the concept of plain language; most have not heard of the concept of document usability; and even fewer attempt to apply such concepts to their work (why would you when you see your work is to report versus transfer knowledge.) There are certainly individuals and even groups of medical writers who operate in a manner that warrant use of the term professional, but I am unable to find as appropriate, the broad application of the term to the various communities within medical writing.

I am suggesting that the intelligent practice of seeing clinical research documents as tools to be used by others outside of an enterprise is largely absent from the “ways of working” mindset of most medical writers. I am also suggesting that the task of document production—at all levels and for all forms of documentation in the clinical research community—is largely inefficient. Few know what it really costs the organization per page to produce a final version document (many are afraid to know that number); few consider techniques and tools to really transform the document production process (much of what I see done to improve work streams is at best incremental, but is largely “feel good” activity that has no impact whatsoever); few engage in meaningful post work reflection where performance is analyzed consistently across document projects by well-described standards. I could go on, but I think this is enough detail to support my position.

A hallmark of professionals as defined by Donald Schon and others is a continual reflection on practice, both during work and post-mortem and then application of what is learned to transform ways of working and product. If reflection is occurring, then I suggest little fruit is born by the activity for I hear all of the same criticisms of medical writers and clinical documents today as I did in 1994.


Originally published on our Knowledge Management blog

Is Medical Writing Really a Profession in Practice?

I’ve been reading a book by Donald Schön, The Reflective Practitioner: How Professionals Think in Action, that has got me thinking about the working definition of the term “profession” and whether this term is appropriately applied to the field of medical writing. I am not sure it is for the following reasons.

When considering medical writing as a profession then one must ask whether the knowledge base of medical writing has the requisite properties and whether it is regularly applied to everyday problems of practice. Many who work to define professions state that the systematic knowledge base of any profession is thought to have 4 essential properties:

1. Specialized
2. Firmly bounded
3. Scientific
4. Standardized

Some would suggest the above parameters make Medical Writing a good fit under the moniker of profession. However, I call into question how many medical writers really look closely at the disciplines of writing or knowledge management. Much output I see and many conversations I have with medical writers suggest many write to standards bounded by myths and personal preference and not driven by the evidence of science.
Edgar Schein suggests there are three well-defined components to professional knowledge:

1. Underlying discipline or basic science
2. Skills and attitudinal component
3. Applied science component (yields diagnostic problem-solving techniques to the actual delivery of the services)

I would find that Medical Writing fails to be a true profession as suggested by Schein’s criteria. I see the central gap being the failure to develop a general body of scientific knowledge bearing precisely on the problem of effective and efficient medical writing practices. I say this because of my McCulley/Cuppan work where I have many observations of how medical writing is considered and applied in the pharmaceutical and medical device industries.

For Schein, basic and applied sciences are convergent whereas practice is divergent. This suggests that the hallmark of the true practitioner is the ability to take the convergent knowledge base and convert it into skills and services that are tailored to the unique requirements of the client system—a process that demands divergent thinking skills. My observations suggest that many who operate under the aegis of medical writing are poorly placed to engage in effective, meaningful divergent thinking. The preferred thought pattern is at best the convergent model of “what did I/we do last time?” and at worst “just let me populate the tables and make sure this document is compliant with the template and ICH E3 guidance."

I think I am largely in the camp of those who would call Medical Writing something besides a profession. But I am certainly not with those who would suggest it is but an avocation. Frankly, such a description is quite derisive.

I find myself considering applicable to medical writing the Grady McGonagill position that those who rely on intuition and reflect on their work only episodically risk accumulating unintegrated clusters of habitual practice. They may develop to a level of competence but not beyond. I’d say this is a reasonable description that fits the largest cohort of those who work under the mantel of medical writing.

So what do you think—is Medical Writing truly a profession in practice?


Originally published on our Knowledge Management blog

7 Rules for Great PPT Presentations

Having an understanding of what makes for an awful PowerPoint presentation can make it intimidating to create your own. Who wants to be the creator of one of those presentations?
In our consulting work at McCulley/Cuppan, we've sat through, created, and helped others create many presentations (including ones to FDA and other regulatory agencies), so we have some experience with good and bad PPT. Here's a list of rules we've developed over the years for designing effective presentations:

1. Adopt your audience's mind--it's always about them, not you
When you are the subject matter expert and have done lots of research, you may feel compelled to share it all in the PPT slides. Avoid this. Instead, focus on what your audience is most interested in. (Your knowledge will come through in the way you discuss the topic and answer questions.)
2. Clarify your intentions
How will you proceed with the presentation? Will there be a Q&A session? If so, when? Let your audience know upfront how much time you will take and the basic outline of how the time will be spent.
3. Simplify, simplify, simplify
Simpler is always better. Simple design, simple text. As Hans Hofmann said nearly 100 yers ago: "To simplify, eliminate the unnecessary so the necessary can speak."
4. Embrace limitations and practice restraint--know when to stop
Once again, all of your knowledge does not need to be demonstrated in this one presentation. Respect the time and concentration limits of your audience.
5. Do like the pros do--script, storyboard, create
Planning is the key to effective presentations. Work with the end in mind even before you build the first slide and the content of your presentation will be more compelling.
6. Think communication not decoration
PowerPoint is a tool to help you convey knowledge; keep the slide design simple so the focus stays on the content.
7. All presentations are storytelling, so become a master storyteller
Learn from the master storytellers around you, people whose presentations you've enjoyed attending. If you don't have any nearby, learn from well-respected presenters such as Garr Reynolds, Steve Jobs and Guy Kawasaki. Tell a tale that engages the audience and makes them want to listen and learn from you.  

 

Originally published on our Knowledge Management blog

Top 4 Don'ts for PPT Presentations

We've all been there, sitting through endless PowerPoint presentations where the speaker drones on and on and the slides are so full of text you squint no matter how close you are to the screen. We each have our least favorite PPT foibles, but here's a list of the four worst offenders:

I never met an animation technique that I didn't like.

Slide decoration doesn't equal slide design. Animation distracts from the content of your presentation and the purpose of the presentation is to convey information, so limit distractions and keep it simple.

Why can't everyone just read what I have on the slide?

Slides make up one portion of the presentation; they should not "stand alone" (even when you are offering copies of slides for notes). If the slides can stand alone, then what is the purpose of you being there? Just write a report instead.

Charts and tables are "islands that speak by themselves".

Charts and tables are never "islands that speak by themselves", whether in presentation or report. Some explanation is required, on the purpose of the table or the conclusions drawn from the data. Once again, a table on a slide augments your spoken presentation and should not be treated as a stand alone.

Everything I know on this topic must go on these presentation slides.

Also known as "cram everything in obsession". All of your knowledge on a topic doesn't need to appear on every page of the slide. Slides should be easy to read, and as brief as possible, otherwise people will spend their time reading your slides instead of listening to and being engaged in the presentation.

And remember, PowerPoint (or Keynote for fellow Mac users) isn't right for all meetings, briefings, and presentations as shown in the article "We Have Met the Enemy and He Is PowerPoint" from the NY Times about the usage of PPT in the US Military. The "bowl of spaghetti" map shown in the article is a great example of why everything shouldn't be crammed into one slide.


Originally published on our Knowledge Management blog

More On Why Do So Many Feel Complex Language Is Needed To Have Good Scientific Writing

I made post a few weeks back addressing the topic of why some authors in the life sciences feel compelled to construct documents that are dense and difficult to read. Here's a link to that post.  The post generated quite a bit of discussion in a medical writing discussion group I belong to on LinkedIn.  The discussion there has motivated me to do some further investigation on this topic. I want to share with you some interesting pieces of information that I hope impacts how you think about what constitutes "good writing."

I came across a series of papers by J. Scott Armstrong, who is at the Wharton School of Business at the University of Pennsylvania. In these papers, Armstrong considers the broad question that academic communications should enhance knowledge and, therefore, researchers should invest energy in developing understandable ways to present their findings.

In his paper, Unintelligible Management Research and Academic Prestige, Armstrong explores the notion that if the goal of successful communication is to share information with others and, if science places a premium on successful communication, then, all other things being equal, journals should prefer articles that are clearly written to those that are not. Armstrong concluded from the studies presented in this paper that clear communication of one's research is not the norm in the prestigious journals he examined, nor is it widely appreciated in his small test of the academic community at three universities.

In a another paper, Research on Scientific Journals:Implications for Editors and Authors Armstrong states the following: "A review of editorial policies of leading journals and of research relevant to scientific journals revealed conflicts between 'science' and 'scientists'. Owing to these conflicts, papers are often weak on objectivity and replicability. Furthermore, papers often fall short on importance, competence, intelligibility, or efficiency."

In yet another paper, Barriers to Scientific Contributions: The Author’s Formula, published in the journal Behavioral and Brain Sciences, Armstrong, with tongue in cheek, describes a set of rules that authors can use to increase the likelihood and speed of acceptance of their manuscripts. Authors should: (1) not pick an important problem, (2) not challenge existing beliefs, (3) not obtain surprising results, (4) not use simple methods, (5) not provide full disclosure, and (6) not write clearly.


Originally published on our Knowledge Management blog

What Skills Do You Need To Be A Medical Writer?

I am preparing a workshop for the medical writing group at one of my McCulley/Cuppan clients. The focus of the workshop is on enhancing writing skills. I know some reading this post and many of the people who will attend this workshop are thinking, “Why do workshops on writing for people who make their living writing?” So I am thinking that I will open each workshop session by posing and then addressing this unspoken question.

My answer will likely wrinkle most brows and even irritate a few, not the primary objective, but perhaps a necessary adverse event to get my point across.

Why have such a workshop? Because most medical writers that I know HAVE NOT been effectively exposed to the skill sets I see as critical to success as a writer charged with creating protocols, reports, and summary documents that describe clinical research.

My point is this—for most of the people I will have in the workshops (largely with life science backgrounds) the training received as “writers” occurred principally during their academic pursuit of a scientific discipline. It largely ended, in any formal sense, with their undergraduate composition classes. However, for some the task of writing is further cultivated through mentoring during postgraduate training. Beyond the academic experience, most of the exposure they receive is just a refinement of the “writing skills” they received in the academic arena. Any training of the type I plan to talk about was likely haphazard at best.

To help back my argument, I did a web search on the term “medical writing skills.” Here is the most common definition I found on the web:

A medical writer must have excellent writing skills, a strong command of English and grammatical rules, attention to detail, good organizational and analytical skills, and an ability to meet deadlines.

This definition is what the American Medical Writers Association (AMWA) and European Medical Writers Association use to describe the skill sets of a medical writer and these attributes are what are commonly listed in job postings for medical writers.

I am suggesting this commonly applied definition is wholly inadequate. It is missing what I consider the first-order skills one needs to succeed today as a medical writer. I’d even say that a strong command of the English language and grammatical rules is not a first-order skill to be a good medical writer—a statement many will call antithetical.

The definition of the term skill is: the ability, coming from one's knowledge, practice, aptitude, etc., to do something well. If all you bring to the table is writing skills, a strong command of English and grammar, attention to detail, good organizational and analytical skills, and an ability to meet deadlines, then I suggest you will not do well with the task of a medical writer.

Consider what the AMWA website lists as training sessions they offer to make you a good medical writer: workshops on basic editing, writing, communication, and bibliographic skills as well as specialized editorial and publication skills and in-depth consideration of issues in writing.
Or look at what the University of Chicago has as the core curriculum for their Medical Writing Certificate Program:

• Introduction to Medical Editing
• Advanced Medical Editing
• Writing in the Medical Sciences
• The Internal Logic of Medical Articles
• Interpreting and Reporting Biostatistics
• Designing and Editing Tables and Graphs
• Understanding and Mastering the Clinical Study Report

There is little from any of these institutions regarding the skills we see critical for medical writers. Skills like:

• Fostering collaborative writing versus cooperative writing
• Project management of intellectually focused projects
• Conflict resolution during document reviews
• Acting as a document consultant to help devise methods to capture knowledge
• Skills to support the capture, propagation, and sharing of knowledge
• How to leverage the collective wisdom of a drug development team

We argue that to succeed, a medical writer must see their role as a knowledge manager, not a scribe of clinical narratives in various types of documents. To see more of our arguments on this notion of writer as knowledge manager you can check out these other posts on our blog:

The Role of the Professional Writer in the World of Drug and Medical Device Research
The Value of Broadening the Role of the Professional Writer in Life Science Research Organizations
DIA Musings: How Medical Writers View Themselves
Difficulties Assessing the "Value Added" of Professional Communicators Writing in the Life Sciences


Originally published on our Knowledge Management blog

Improved Document Quality Does Pay

Quality control and improvement in document content and design can provide a significant financial benefit to  pharmaceutical and medical device organizations. It is our observation that companies must learn that writing is a process open to continuous improvement.

However, I find in my consulting work that many people working in the life sciences question the merits of attempts to improve the communication quality of their technical and scientific documents.  Quite a few of these people subscribe to the notion that as long as I get the study design right or have the correct data, then that is good enough. This may be the case for a small subset of documents, but the assumption clearly does not apply to the vast majority of documents produced in the life sciences. It is our observation that an effort to improve document quality generally does pay off for the individual and the organization. Though these pay-offs may not be easily seen unless there is an attempt to capture data regarding the impact of better document quality.

Here's a few examples where organizations instituted quality control programs and measured change in selected outcomes in order to answer questions about their document products and processes. These organizations found document quality by design does pay.

The Motorola Corporation substantially improved its operation after instituting a document quality program. The company made changes in R&D and Finance operation guides. These changes were focused on providing clearer directions and rationales in the guides and an easy-to-use format. These changes have helped streamline processes, and reduced delays from review and rework. The company calculated savings at US$ 20,000,000 per year. Source – Business Week Oct. 25, 1992
The United Kingdom Department of Health and Social Security spent the equivalent of US$ 50,000 to develop and test a series of new forms. They have reported annual savings of over US$ 2.9 million in reduced staff time from the forms being properly completed the first time.
Source – Plain Language Principles and Practice by Erwin R. Steinberg

The United Kingdom Department of Customs and Excise cut a 55% error rate to 3% by revising lost-baggage claim forms used by airline passengers.  Source – American Institutes for Research
An  aerospace contractor lost 15 consecutive contract proposals over a three year time period before a quality proposal writing group was called in to review the quality of the proposals and to help the proposal development teams institute new work practices and improve writing skills for developing large proposals. After instituting changes in documentation work practices and processes, the contractor “won” 11 of 12 proposals.
Source – Shipley Associates private communication


Originally published on our Knowledge Management blog

Why Do So Many in the Life Sciences Feel Document Language Has to Be Complicated to Be Good?

Not too long ago I was working on a client project related to improving the communication quality of clinical study protocols. At project initiation we did a document assessment of communication quality on six recently completed protocols. One of my consulting colleagues framed a very high-level assessment of the communication quality of these documents. Her assessment can be summarized as follows—it appears the authors of these protocols assume their reading audience will start reading the document in serial progression from page one to page n, have a high command of the English language, enjoy reading long, dense narrative passages, and have exquisite memories.


In reality, the targeted reading audience for these documents—clinical study site investigators and managers, as well as members of Institutional Review Boards—clearly do not prefer to read documents structured in such a manner. Who would?

We took one of the protocols and restructured the document to optimize communication quality. We applied many of tenets espoused by the “plain language” movement that has received considerable traction within the many offices of the United States Federal Government. Here’s the link: http://www.plainlanguage.gov

The model protocol was reviewed by our client sponsor and she found the document to be “not scientific enough.” It looked too different than what she was used to. Of course, what she was used to was protocols containing long dense narratives, protocols making ineffectual assignment of agency to various parties participating in the conduct of clinical research; protocols that treat words like “should” and “must” as synonyms; and protocols that assume all users of the document have English as their mother tongue.

Why do so many authors in the life sciences remain firm believers in the notion that scientific discourse has to be complicated to be good? The arguments against plain language in scientific documents do not hold water—that plain language cannot be used with a clinical and technical reading audience, that plain language oversimplifies intentions and meaning, and that plain language is not precise. Check out this list of articles supporting the use of plain language on the plainlanguage.gov website.


Originally published on our Knowledge Management blog