Regulatory Writing Must Be "Fit for Function" Not Perfect

Documents for the medical regulatory audience do not have to be perfect. Rather, they have to be "fit for the intended function."

I suggest to you that the majority of people associated with drug and device development do not either understand or appreciate this concept. I also suggest to you that the majority do not understand that you apply different document designs to different document genres. Which in turn means you have different standards for different genre. Fit for function means you do not work to one standard.

Fit for function means a document is built and judged by standards that are determined by audience and the task or role the document will take when in the hands or on the computer screen of the specified audience.

A few weeks back I was working with a development team that judges all of their work by one standard. No matter what they are writing, the team judges their handiwork by their personal standards for a research manuscript. This one standard is applied to clinical study reports, protocols, and investigator brochures. I know this to be the case because review comments are often predicated with statements like: "Well, when I was writing manuscripts we always did.............in the Discussion Section. We need to do that in this report." or "I think the reviewers at the agency would find this very interesting to read. So let's expand the discussion here." and "You have a hyphen break at the end of the line in this paragraph. We cannot have mistakes like this in our documents."

Fit for function places focus squarely on the elements that matter, that is, the underlying logic of what the document is to convey or enable the reader to do. So in the world of drug and device development, this means authoring teams have to move away from the notions that when creating documents for regulatory readers, they are either simply reporting data or writing for an audience interested in knowledge acquisition. Working to either of those standards guarantees that your documents will not be fit for function. Rather, documents must be built following the underlying logic of what the reader is trying to do with the document.

More on how we characterize the logic of document genres in my next post.

Something to consider: using intranets as Knowledge Management tool to support collaborative pre-writing planning

Jakob Nielsen is a name some of you may recognize. He is a self-appointed guru on all matters regarding usability of web sites and web-mediated work tools. You can find much of his work summarized on his web site titled Useit.

Last week Nielsen came out with his list of Top 10 designed business Intranets for 2011. Here's the link.

I noted with interest Nielsen's comment "If there's anything that has been overused, abused, and hyped beyond the level of cliché, it's "knowledge management." Thus, it might be better to say that many of this year's winners were strong in "managing knowledge" on their intranets." 

Managing knowledge. This is a concept where I see drug and medical device teams struggle all the time. One reason for the struggle is the slow uptake of tools that can help foster an effective work environment and change deeply ingrained cultural practices.

I believe intranets, such as WIKIs are great tools that should be deployed at the project team level. These platforms become incredibly valuable from the moment a clinical development plan is written until well after a dossier is submitted to the regulatory authorities. 

While it is true that knowledge management is not a technology issue, effort must still be spent in providing a suitable environment to facilitate knowledge capture and sharing. 

I am suggesting the use of team specific intranets as a way to promote cultural change in an organization, at the level of knowledge-sharing activities and also for shaping broader work behaviors. In most companies where I train or consult, little has really changed since the 1960s in how people approach the planning and authoring of documents.  Using intranets, like a WIKI, can quickly get a team applying best practices in terms of pre-writing planning.

Minimal Time and Effort Should be Applied to the Creation of the Clinical Study Report Synopsis

How much time and effort to apply to the creation of the clinical study report synopsis?


This is another question I am asked on a regular basis and a line of discussion that repeatedly comes up when I am working with clients to help streamline work practices. I usually draw slack jaws accompanied by an incredulous stare as I give my answer: 


"The amount of time should be minimal, involve no more than three people, the level of effort better be next to nothing, and the time should be no more than an hour to create and a whole lot less time to review."


My reasoning is very simple and straightforward.........."You apply time and effort to the development of a product in relationship to the product's strategic value. The value of the CSR Synopsis to the regulatory reader is virtually zero."


Think about it. The CSR Synopsis affords little utility to what the reviewers are looking to accomplish when they choose to enter the framework (that is, the document) of an individual clinical study. If a regulatory reviewer wants a “snapshot” of a study, they will likely take a contextualized snapshot at a higher level of a drug submission dossier. That is, the documents in Module 2. They do not enter the framework of the study report to get generalized or summarized information. They are at the Module 5 level and embedding themselves in a clinical study report because they are seeking answers to narrowly defined questions.


At McCulley/Cuppan we have queried regulatory reviewers about how they "use" a study report synopsis. Their responses support the premise I have laid out for you in the above paragraph.


So this gets us back to the question on time and effort. Why generate a study report synopsis with every draft? Why allow the full team to look at the document? Talk about wasting time and energy. 


In our assessment of review practices at pharmaceutical and medical device companies we see the same review pattern played out time and time again. The study report synopsis is generated with the first draft and in the review process it consistently garners the attention of the full review team (many of whom never make it all the way through the results sections during the course of their reviews.) Same thing happens on each subsequent draft.


Given the synopsis is but a summary of the body of work presented in the study report, it should not be generated until that body of work is completed and signed off as "good to go." The synopsis does not even warrant review. It should be critiqued. Critique is a comparative read. A reading to ensure that the synopsis accurately and appropriately portrays the sum total of key details of the study. The critique process requires at best two people and certainly no more than three. All being subject matter experts drawn from the key clinical disciplines represented in the research study.

Importance of language and writing style in a clinical study report

How important is language and writing style in a clinical study report?  I was recently asked this question by a medical writer working for one of my McCulley/Cuppan clients. The writer is dealing with a team that seems to obsess over every word in every draft and the writer is looking for some help in how to address the situation.


Here is my response to the question:


You are asking about lexical and syntactical elements of writing (the third element of writing is grammatical.) 


Lexical pertains to the words (vocabulary) of a language. In the context of clinical research we need to talk about several applied lexicons of scientific phraseology that apply broadly to science and then narrowly to a specific therapeutic area. The admittedly most distinctive feature of any clinical study report is the application of specific scientific and technical prose. So, language is very important in a CSR to avoid lexical ambiguity (why I so love statisticians and their demands for careful use of language when describing statistical observations) in order to allow the reader to derive the intended meaning.


My experience suggests that many people in Pharma think attention to syntactical elements (style) means they are either eliminating ambiguity or improving clarity of message. Rarely is this the case.


You have heard me say before that style does not matter in the type of writing represented in clinical study reports submitted to regulatory authorities in the US and elsewhere.

My position is supported by current discourse theory. Discourse theory states that, as a rule in scientific writing, meaning is largely derived from the precise use of key scientific words, not how these words are strung together. It is the key words that create the meta-level knowledge of the report. Varying style does little to aid or impede comprehension.


What happens is people often chase and play around with the style of document. Largely they are looking to manipulate an advanced set of discourse markers specific for clinical science writing or some subset specific to a therapeutic discipline. Discourse markers are the word elements that string together the key scientific words and help signal transitions within and across sentences. These discourse markers are the elements that provide for style. There are macro markers (those indicating overall organization) and micro markers (functioning as fillers, indicating links between sentences, etc.) Comprehension studies show that manipulating discourse markers--that is, messing with style--in most instances does not influence reader comprehension. It is worth noting that manipulation of macro markers appears to have some impact on comprehension for non-native speakers of English (why it is worth using textual advanced organizers to help with document readability.)


So the net-net is: there is little fruit to be picked from messing with style in a clinical study report. Put review focus on the use and placement of key terms.


This is a bit of a non-sequitur to the question, but a concept I’d like to share. To derive meaning from scientific text, readers will rely on their prior knowledge, and cues provided by the key terms and data they encounter or fail to find in a sentence, paragraph, table, or section of a clinical study report. So what I’d really prefer to get people thinking about is the semantical elements of their documents. Semantics is fundamentally about encoding knowledge and how you as an author enable the reader to process your representation of knowledge in a meaningful way. Semantics is about how much interpretive space you provide to the reader in a document by what you say and equally important, by what you do not say. Of course you cannot get to the point of thinking about semantics unless you see clinical study reports as something more than just a warehouse for data.

Not knowing when good is good enough in writing regulatory documentation has a huge cost

We do not talk much on this blog regarding the use of language or the application of terms in science writing. Principal reason is that much of what we see in regulatory submission documents is genuinely “good enough.” However, others do not necessarily see it that way. I want to share with you how discussions in review roundtables can end up getting focused at really absurd levels of detail with a misapplied sense of establishing quality communication. 

In our consulting work, we try to be disciplined during our document reviews and only comment on language when it truly obscures or alters meaning. Being grammatically perfect in regulatory submission documents is a nice notion, but in practice consumes way too much time and organizational energy and will yield little in terms of outcomes.

We share this point with people all the time...but at times the advice goes unheeded and even worse…at times people just do not know when to move on and address real big concerns in their documents.

A case in point is a situation I observed regarding a long winded discussion in a review meeting over the use of the term “very critical”. The term “critical” in a medical sense means: of a patient's condition having unstable and abnormal vital signs and other unfavorable indicators. In theory, the meaning of critical is a black-or-white proposition without qualifications regarding gradation. Something is either critical or it is not. Therefore there should be no adverbs, like “very” in front of the term “critical” to connote a measureable degree of criticality. In this roundtable review the team got caught up in a 30 minute discussion that involved only two people arguing whether to use the term “very critical” or change it to “critical.”

Being pragmatic, I’d have to say: “Guys what are you thinking? You hold a team hostage for 30 minutes to argue over grammatical accuracy? To argue over something that will not matter when and if read by a regulatory reviewer. There were 10 professionals sitting in the room and 8 did nothing for 30 minutes. Cost of salaries alone is enough argument to say “Forget about it, let’s move on…we cannot afford to argue over such insignificant detail.” When we add in the opportunity cost (what these 10 people collectively could have been doing with their time), then for sure you have to make the argument.

This above episode gets played out time and time again in review sessions all over the pharma and medical device industries and is the reason why I am steadfast in my position that the vast majority of people involved in authorship and review do not know the answer to the question “How do you know when is good, good enough?” The end result is inordinate amounts of time can be applied at the wrong level of detail in reports and submission documents.

Change, change everywhere...yet it all remains the same

Change, change everywhere....yet as far as I can tell, most writing practices used to day to create research reports in the life sciences are the same as those deployed in 1928.


I did a workshop the other day for a group of medical writers looking at aspects of managing documentation projects. I started the session with a photograph taken in a newspaper office in 1928. I suggested to the group that other than the typewriters on the desk, the ways they work are essentially the same as the people posing in that 1928 photo. My comment got everyone’s attention, though I am sure some did not receive well the message. I am quite comfortable with my statement as traditional documentation development practices in the realm of medical writing have remained more or less unchanged for decades.


I explained to the group that much of the medical writing work they engage in is played out exactly the same way it was in 1928. Their current writing world remains one that is largely a work platform where they produce drafts in isolation and then send the drafts out to team members (electronically now unlike paper of the past) who then review in isolation and ultimately return the marked up documents to the authors (exactly like they did in 1928.)


My point to the group was if you really want to change outcomes, then you better think about changing your work practices. Doing things largely the same way will continue to yield the same outcomes. Just because you now have the reviewers grab your document from a server file, or you do round table reviews via WebEx, or you author your report in an enhanced electronic template does not mean you have made a radical departure from how people authored research reports or other documents 20-30 years ago. And I do know that the same weaknesses found 50+ years ago in document products and practices remain today.


You want different outcomes? Then change the practices associated with planning, authoring, and reviewing documents, not merely the tools and the processes you lay on top of practice.

More thoughts on the limited sophistication of documentation practices in the life sciences

I mentioned in this post that I consider the documentation practices for creation of regulatory submission documents in most pharma and medical device enterprises to rather unsophisticated. My position is largely driven by comparing observations of documentation practices to descriptions of varying levels of documentation maturity we have developed. Our descriptors have their roots in the work presented by Joann Hackos in her book: Managing Your Documentation Projects.

Like Hackos, our documentation practices-maturity model is a six point scale ranging from: Level 0 — Oblivious to Level 5 — Optimizing. My observations suggest that the vast majority of regulatory writing falls into the 2.5 range, which is between Rudimentary and Organized/Repeatable.
The Rudimentary documentation organization is one where the vast majority of effort is placed to ensure documentation consistency. All documents are generated utilizing well characterized templates. Document strategic review is largely absent as energy is applied to ensuring structural (that is, grammar and format) accuracy and consistency. Work practices are highly individualized and there is little application of meaningful estimating of document size or time lines.

The belief statements of organizations working at the rudimentary level include the following:

• All writers manage their own projects
• We would like to know more about our reading audience, but nobody takes the time to learn more….so we “suppose” what they want in our documents
• Our users are just like us…….I am a subject matter expert, so every other person educated like me will read documents just like I do
• We talk about what the data means, but we rarely talk about how to represent this meaning in our documents until we are into round table reviews
• We have little concern with how many rounds of review go into creating the final version of a document….we strongly endorse the credo “the end justifies the means”
• Planning document content before actually writing a report is just busy work
• We routinely reverse engineer document development timelines from the stated publication deadline and not from the scale and scope of the intended document
• We care what the customer thinks of our documents, but we do not use any organizationally applied quality standards. Standards are principally driven by teams and their senior reviewers
• File and forget–we do not take time to collectively reflect on documentation work practices practices

The belief statements of organizations working at the organized and repeatable level include the following:

• We have begun to study our document users but see little value in a concerted effort to collect information on how well our documents “satisfice” their needs
• We are surprised or even indignant when we get questions from our regulatory user looking for information that we included in our submission package
• The quality of our document project management is inconsistent, but we are okay with that because that is reality and each project is unique
• We do not see meaningful metrics beyond time for documentation projections…if we meet the deadline, then ways of working had to be good
• Nobody knows what others are doing in the process of review…the only way they know is via what may be discussed during a round table review
• We believe in our described “ways of working” until faced with unexpected situations, then we panic and call for “all hands on deck”
• We can easily get caught up in “process” at the expense of “product”

Now let’s contrast the above with belief statements of organizations working at the optimized level which include the following:

• We always engage in collaborative pre-writing planning of documents to make sure we fulfill strategic purpose of any given document
• We are thoroughly committed to understanding our document users and we work to systematically collect information from them
• We know how big a document will be even before we write it
• We always do end-of-project analysis to collect lessons learned and then disseminate this information across the organization
• We maintain a database of critical documentation work practice parameters and benchmark all documentation projects
• We recognize that accuracy and consistency are just the start of ensuring quality….we have expanded focus to quality of argument and document usability
• We are learning how to be innovative and not let the process control us

Originally published on our Knowledge Management blog