When I first started working at McCulley/Cuppan, Greg gave me the task of reviewing a clinical study report, not only as a way to gauge my reviewing skills, but as a starting place for lessons on judging document quality using the McCulley/Cuppan Document Standards. The first instruction he gave me was "Read before you edit."
Read before you edit.
Sounds simple, but when asked to review a document, most people start off by looking for mistakes. I remember sitting there, trying to pay attention to the document when all I could see was a misspelled word. All too often we see ourselves as that teacher of grade school with the red pen in hand, correcting verb tense and punctuation errors. We don't see the document at the meta-level so we miss the message and logic of the document.
Think about how you read a book, an article, or this blog. You read for meaning. If I leave a TyPo or too, you still understand that this post is about editing and reviewing documents. Now consider your habits when someone asks you to look at a document. Even if they say they want you to make sure that a paragraph makes logical sense, your eye probably catches all the typos and then proceeds to analyze the logic.
The truth is that inspection is easier than reviewing and is a quick way to show your contribution to a document. A way to say Look, see all these good edits I made? The reality is that lots of time is wasted, especially on first or second drafts, fixing nit-picky mistakes in grammar and punctuation on sentences that will be cut out of the document before the third draft. If you read a document (or section of a document) once all the way through, you will avoid re-reading sentences again and again just to get the meaning and avoid the need to revert your changes back to the original sentence.
During the last few months, I've been assessing a number of documents to judge reviewer performance as part of my work at McCulley/Cuppan. What I see all too often is that a reviewer will correct a sentence, changing a word or two, only to go back and change the sentence again within that same review session or in a subsequent draft. It is apparent many reviewers read to fix grammatical and stylistic errors. They are reading and analyzing each word individually versus parsing the sentence or paragraph for meaning. So a given word might seem to be the wrong choice at first glance, but if the reviewer were to read the entire paragraph, then the author's chosen word may be seen as appropriate. Style is a much overrated, overwrought topic during reviews. This is especially important when you read documents written by multiple people, where writing style and word usage is likely to vary widely.
But whether you are inspecting (editing for punctuation and grammar) or reviewing (assessing for purpose, logic, and content), reading a document in its entirety before making revisions places you closer to how the reader is going to engage with the document. In the assessments I have done, I have seen so many reviewers miss the forest because they were busily engaged inspecting branches on trees.
22 February 2011
15 February 2011
Editing When You Should Be Reviewing Costs Serious Money
The idealized model of document project management describes an iterative process of information planning, content specification, authorship, review, and publishing. Working to develop best-of-craft work practices aligned with this model can help organizations meet their goals efficiently and at lower costs.
Most organizations in the life sciences do not carefully consider their true costs to produce a document. When all the hidden costs associated with review are added in, the cost-per-page to produce a final version document becomes significant. If you add in opportunity costs for staff time dedicated to additional rounds of review (even if a document meets a specified deadline for the final version) then the cost-per-page skyrockets. Many organizations choose to ignore this point, which is why in my presentations on strategic review I have a slide with the Great Pyramids of Giza and the caption: “We can do anything we want as long as we have a 24/7 schedule and an expendable supply of labor.”
The goal for review is to improve document communication quality—not grammatical accuracy or data integrity, which is done via inspection. To be most effective and efficient, reviews need to be strategic and orchestrated. This means the primary focus of review by subject matter experts should be on the intended messages and arguments and testing the document to ensure it minimizes the prospects for a qualified reader to construct alternative meanings. Tasks not accomplished if you are editing a document for grammar and style.
Our systematic analysis of review practices at numerous companies strongly suggests that while significant time and energy is expended on document review, the collective effort is generally over-sized in relation to the improvements made in the communication quality of the documents. That is, there are more people and more rounds of review than should be needed to get to a final version document. Another way to look at it is that reviews do not really move the communication quality meter very far given how much time and resources are thrown at the task.
Our analysis suggests that review performance is hampered by poorly defined expectations for what the team wants to accomplish in a document, that reviewers do not apply systematic means of analysis to their review (meaning team reviews are largely ad-hoc feeding frenzies), and that reviewers stray from attending to the messages and arguments of a document, instead attending to matters truly related to publishing standards. In other words, reviewers stop being reviewers and become editors. Editing when you should be reviewing comes with serious costs.
04 February 2011
Build Documents Following Logic of What the Reader is Trying to "Do"
I ended my previous post stating that documentation teams have to recognize their documents must be built following the underlying logic of what the reader is trying to "do" with the document. By this I mean there are different design choices for document content depending on what the reader is intending to accomplish through accessing the document.
If your reader is making decisions versus merely being informed about a body of work, then this greatly impacts document content and a major difference between regulatory medical writing and the medical writing associated with the development of manuscripts.
For instance, if you are creating a summary-level regulatory submission document, the logic of what this document is to "do" for the user will dictate design decisions. A summary-level document is not intended to report results; it is intended to synthesize study findings in a way so as to build the information base for the therapeutic activity and safety of a drug or device in various patient populations. The summary documents are meant to address issues associated with development work and candidly represent limits of current understanding. It is in the summary documents that one must argue that observed differences in the data (or lack of differences) are determined reliably and provide a low probability of alternative results upon further study. In creating summary documents authoring teams must recognize that successful arguments need to establish warranted claims that respond to the known or theorized questions or concerns of the reader.
A second example, if you are writing a risk management plan, like a RiskMAP for FDA, then the document must be designed to clearly demonstrate the logic underlying the risk assessment and the quantitative and qualitative assignment of risk. The document must clearly lay out the strategy to monitor and characterize the nature, frequency, and severity of the risks associated with the use of a product. Ultimately the document has to characterize why the proposed plan and the applied tools are the most appropriate to minimize a health outcome risk potential on a go forward basis.
My final example, if you are writing a clinical research protocol, then you must recognize the logic of this document is not only to demonstrate study design and when assessments will be made in or out of the clinic. It is essential the document also be seen as a vehicle to represent potential benefit in relation to risk (what an institutional review board is looking to assess when they consider the merits of a research study.) Also a protocol must clearly establish the agents and actors engaged in the conduct of the research trial and the assignment of decision-making to the actors (all users of the document need to know this information.) In my review of protocols, I routinely find these documents generally obscure actors (who will do what) to varying degrees of opacity and often leave assignment of decision-making as implied versus explicit. In such instances, the document is clearly not written reflecting the underlying logic of what the document is intended to “do” in the hands of the users of the document.
If your reader is making decisions versus merely being informed about a body of work, then this greatly impacts document content and a major difference between regulatory medical writing and the medical writing associated with the development of manuscripts.
For instance, if you are creating a summary-level regulatory submission document, the logic of what this document is to "do" for the user will dictate design decisions. A summary-level document is not intended to report results; it is intended to synthesize study findings in a way so as to build the information base for the therapeutic activity and safety of a drug or device in various patient populations. The summary documents are meant to address issues associated with development work and candidly represent limits of current understanding. It is in the summary documents that one must argue that observed differences in the data (or lack of differences) are determined reliably and provide a low probability of alternative results upon further study. In creating summary documents authoring teams must recognize that successful arguments need to establish warranted claims that respond to the known or theorized questions or concerns of the reader.
A second example, if you are writing a risk management plan, like a RiskMAP for FDA, then the document must be designed to clearly demonstrate the logic underlying the risk assessment and the quantitative and qualitative assignment of risk. The document must clearly lay out the strategy to monitor and characterize the nature, frequency, and severity of the risks associated with the use of a product. Ultimately the document has to characterize why the proposed plan and the applied tools are the most appropriate to minimize a health outcome risk potential on a go forward basis.
My final example, if you are writing a clinical research protocol, then you must recognize the logic of this document is not only to demonstrate study design and when assessments will be made in or out of the clinic. It is essential the document also be seen as a vehicle to represent potential benefit in relation to risk (what an institutional review board is looking to assess when they consider the merits of a research study.) Also a protocol must clearly establish the agents and actors engaged in the conduct of the research trial and the assignment of decision-making to the actors (all users of the document need to know this information.) In my review of protocols, I routinely find these documents generally obscure actors (who will do what) to varying degrees of opacity and often leave assignment of decision-making as implied versus explicit. In such instances, the document is clearly not written reflecting the underlying logic of what the document is intended to “do” in the hands of the users of the document.
02 February 2011
Regulatory Writing Must Be "Fit for Function" Not Perfect
Documents for the medical regulatory audience do not have to be perfect. Rather, they have to be "fit for the intended function."
I suggest to you that the majority of people associated with drug and device development do not either understand or appreciate this concept. I also suggest to you that the majority do not understand that you apply different document designs to different document genres. Which in turn means you have different standards for different genre. Fit for function means you do not work to one standard.
Fit for function means a document is built and judged by standards that are determined by audience and the task or role the document will take when in the hands or on the computer screen of the specified audience.
A few weeks back I was working with a development team that judges all of their work by one standard. No matter what they are writing, the team judges their handiwork by their personal standards for a research manuscript. This one standard is applied to clinical study reports, protocols, and investigator brochures. I know this to be the case because review comments are often predicated with statements like: "Well, when I was writing manuscripts we always did.............in the Discussion Section. We need to do that in this report." or "I think the reviewers at the agency would find this very interesting to read. So let's expand the discussion here." and "You have a hyphen break at the end of the line in this paragraph. We cannot have mistakes like this in our documents."
Fit for function places focus squarely on the elements that matter, that is, the underlying logic of what the document is to convey or enable the reader to do. So in the world of drug and device development, this means authoring teams have to move away from the notions that when creating documents for regulatory readers, they are either simply reporting data or writing for an audience interested in knowledge acquisition. Working to either of those standards guarantees that your documents will not be fit for function. Rather, documents must be built following the underlying logic of what the reader is trying to do with the document.
More on how we characterize the logic of document genres in my next post.
I suggest to you that the majority of people associated with drug and device development do not either understand or appreciate this concept. I also suggest to you that the majority do not understand that you apply different document designs to different document genres. Which in turn means you have different standards for different genre. Fit for function means you do not work to one standard.
Fit for function means a document is built and judged by standards that are determined by audience and the task or role the document will take when in the hands or on the computer screen of the specified audience.
A few weeks back I was working with a development team that judges all of their work by one standard. No matter what they are writing, the team judges their handiwork by their personal standards for a research manuscript. This one standard is applied to clinical study reports, protocols, and investigator brochures. I know this to be the case because review comments are often predicated with statements like: "Well, when I was writing manuscripts we always did.............in the Discussion Section. We need to do that in this report." or "I think the reviewers at the agency would find this very interesting to read. So let's expand the discussion here." and "You have a hyphen break at the end of the line in this paragraph. We cannot have mistakes like this in our documents."
Fit for function places focus squarely on the elements that matter, that is, the underlying logic of what the document is to convey or enable the reader to do. So in the world of drug and device development, this means authoring teams have to move away from the notions that when creating documents for regulatory readers, they are either simply reporting data or writing for an audience interested in knowledge acquisition. Working to either of those standards guarantees that your documents will not be fit for function. Rather, documents must be built following the underlying logic of what the reader is trying to do with the document.
More on how we characterize the logic of document genres in my next post.
Subscribe to:
Posts (Atom)