Why do we do some of the things we do in clinical study reports? Part 1

Over the past three months I have been doing a series of workshops focused on the authorship of clinical research reports for a couple clients. This work has me posing questions about two very specific notions

1. Why do people still demand that a clinical research report intended for incorporation into regulatory submission packages must be written as “stand-alone” documentation?
2. Why do we still organize clinical research reports by the long-standing convention of Introduction, Objectives, Methods, Results, Discussion, and Conclusion?

In this post let’s consider the notion of the “stand-alone” clinical study report.

First question I have is why must these documents be stand alone? The term “stand alone” generally refers to a document or device that is self-contained and does not require any other document or device to fully function. I argue that a clinical study report submitted to a drug regulatory agency is part of a larger corpus and as such the report should never be construed as a stand-alone document.

It is a given that most submissions made to drug regulatory agencies contain multiple research studies. So you will never have a “stand-alone” research study. So an individual study is part of a corpus of clinical research that best operates collectively. Also regulatory submissions have multiple sections as mandated by guidance that are topoi (places of argument) for different and often integrated attributes of clinical research drawn from this corpus. So clearly the clinical study report is not a stand-alone document in the instance of interpretation and argumentation.

Another point, a clinical study report incorporated into a regulatory submission will have numerous appendices including the final version research protocol and the statistical analysis plan. So I struggle to understand the premise that says you must incorporate extensive elements of the protocol and stats plan into the body of the clinical study report in order to assure the report is a stand-alone document. I truly fail to see the merit of this premise.

My last point is why do so many feel it is necessary to write the Introduction Sections to clinical study reports in the classic inductive form of rhetoric that moves from describing the disease condition, to the therapeutic void, to the chemical or biological construct of the drug under study and how this drug’s pharmacological action addresses a therapeutic void. The standard response is, “well we want to make this a stand-alone document.” Why? All the rhetorical moves you are trying to make in the Introduction will happen for sure elsewhere in the drug submission package in the appropriate topos. So why waste your time here, at the low level of the clinical study report? In my way of thinking the study report Introduction is intended to convey only two points:

1. Why are you doing the study (the answers to what questions do you want at the end of the study that you do not have now?)
2. What previous work has informed the design of this study?

Build Documents Following Logic of What the Reader is Trying to "Do"

I ended my previous post stating that documentation teams have to recognize their documents must be built following the underlying logic of what the reader is trying to "do" with the document. By this I mean there are different design choices for document content depending on what the reader is intending to accomplish through accessing the document. 


If your reader is making decisions versus merely being informed about a body of work, then this greatly impacts document content and a major difference between regulatory medical writing and the medical writing associated with the development of manuscripts.


For instance, if you are creating a summary-level regulatory submission document, the logic of what this document is to "do" for the user will dictate design decisions. A summary-level document is not intended to report results; it is intended to synthesize study findings in a way so as to build the information base for the therapeutic activity and safety of a drug or device in various patient populations. The summary documents are meant to address issues associated with development work and candidly represent limits of current understanding. It is in the summary documents that one must argue that observed differences in the data (or lack of differences) are determined reliably and provide a low probability of alternative results upon further study. In creating summary documents authoring teams must recognize that successful arguments need to establish warranted claims that respond to the known or theorized questions or concerns of the reader. 


A second example, if you are writing a risk management plan, like a RiskMAP for FDA, then the document must be designed to clearly demonstrate the logic underlying the risk assessment and the quantitative and qualitative assignment of risk. The document must clearly lay out the strategy to monitor and characterize the nature, frequency, and severity of the risks associated with the use of a product. Ultimately the document has to characterize why the proposed plan and the applied tools are the most appropriate to minimize a health outcome risk potential on a go forward basis. 


My final example, if you are writing a clinical research protocol, then you must recognize the logic of this document is not only to demonstrate study design and when assessments will be made in or out of the clinic. It is essential the document also be seen as a vehicle to represent potential benefit in relation to risk (what an institutional review board is looking to assess when they consider the merits of a research study.) Also a protocol must clearly establish the agents and actors engaged in the conduct of the research trial and the assignment of decision-making to the actors (all users of the document need to know this information.) In my review of protocols, I routinely find these documents generally obscure actors (who will do what) to varying degrees of opacity and often leave assignment of decision-making as implied versus explicit. In such instances, the document is clearly not written reflecting the underlying logic of what the document is intended to “do” in the hands of the users of the document.

Regulatory Writing Must Be "Fit for Function" Not Perfect

Documents for the medical regulatory audience do not have to be perfect. Rather, they have to be "fit for the intended function."

I suggest to you that the majority of people associated with drug and device development do not either understand or appreciate this concept. I also suggest to you that the majority do not understand that you apply different document designs to different document genres. Which in turn means you have different standards for different genre. Fit for function means you do not work to one standard.

Fit for function means a document is built and judged by standards that are determined by audience and the task or role the document will take when in the hands or on the computer screen of the specified audience.

A few weeks back I was working with a development team that judges all of their work by one standard. No matter what they are writing, the team judges their handiwork by their personal standards for a research manuscript. This one standard is applied to clinical study reports, protocols, and investigator brochures. I know this to be the case because review comments are often predicated with statements like: "Well, when I was writing manuscripts we always did.............in the Discussion Section. We need to do that in this report." or "I think the reviewers at the agency would find this very interesting to read. So let's expand the discussion here." and "You have a hyphen break at the end of the line in this paragraph. We cannot have mistakes like this in our documents."

Fit for function places focus squarely on the elements that matter, that is, the underlying logic of what the document is to convey or enable the reader to do. So in the world of drug and device development, this means authoring teams have to move away from the notions that when creating documents for regulatory readers, they are either simply reporting data or writing for an audience interested in knowledge acquisition. Working to either of those standards guarantees that your documents will not be fit for function. Rather, documents must be built following the underlying logic of what the reader is trying to do with the document.

More on how we characterize the logic of document genres in my next post.

The McCulley/Cuppan Standards Development Process We Use with Our Clients

As I mentioned in a previous post, in our McCulley/Cuppan consulting work we find the prevalent standards used to determine the “success” of a document are largely driven by simple measures of accuracy and then a passel of “home brewed” concepts for characteristics of document that are largely idiosyncratic ideas about what matters to the reader.

When you have 10 people reviewing a document you will end up with at least 12 opinions about the quality of the document (incongruous number is intentional as sometimes you have a reviewer offering more than one opinion that often conflict) and ways of describing quality that are all over the map. People use different terms to describe quality and if they actually use the same term, then it is highly unlikely that they will use the same definition for the term. So the first problem faced in the review process is the vocabulary used to describe quality attributes in a document.

When writers and reviewers compose or edit text, they continually make decisions that concern semantics—the meaning their words convey—and syntax—the way the words are arranged and other structural elements of the document. However, writers and reviewers often base these decisions on assumptions that have not been tested with technically-oriented adult readers or complex, data-rich, technical documents. Worse yet is many assumptions have never been tested to determine validity. Thus, there are actions ordered by writers and reviewers that may not in fact have the expected effects on a readers' performance (we know this is certainly true with one very important reading audience for pharmaceutical and medical device companies—the regulatory agency, like FDA). So the second problem faced in the review process is to understand what document elements have a meaningful impact on semantics and where to focus time and attention on the syntactical elements of a document.

The first thing we do with a client is an examination of the terms used formally (such as in guidance documents) and informally (such as review comments in documents) to describe quality. This will give us a sense of how the organization views quality and how sophisticated they may be in trying to create a common platform that describes document quality for the organization.

The second thing we do is to provide clients with the terms McCulley/Cuppan uses to describe document quality and why the concepts underlying these terms are extremely important to help determine document communication quality. A very important consideration is that the terms should speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms.

We spend a huge amount of time talking about semantics. The term "semantics" refers to the study of how language conveys meaning. Using a broad definition of semantics, we help our clients learn how to focus on different features of a document. Features like word choice, the position of information in document sections, paragraphs and in documents as a whole, idea importance, and the visual representation of data.

Then we work with a client team to create and vet working definitions for the various quality standards.

We then roll out the standards in a workshop setting and show people how the standards are applied to the types of documents they have and will produce.


Originally published on our Knowledge Management blog