Improved Document Quality Does Pay

Quality control and improvement in document content and design can provide a significant financial benefit to  pharmaceutical and medical device organizations. It is our observation that companies must learn that writing is a process open to continuous improvement.

However, I find in my consulting work that many people working in the life sciences question the merits of attempts to improve the communication quality of their technical and scientific documents.  Quite a few of these people subscribe to the notion that as long as I get the study design right or have the correct data, then that is good enough. This may be the case for a small subset of documents, but the assumption clearly does not apply to the vast majority of documents produced in the life sciences. It is our observation that an effort to improve document quality generally does pay off for the individual and the organization. Though these pay-offs may not be easily seen unless there is an attempt to capture data regarding the impact of better document quality.

Here's a few examples where organizations instituted quality control programs and measured change in selected outcomes in order to answer questions about their document products and processes. These organizations found document quality by design does pay.

The Motorola Corporation substantially improved its operation after instituting a document quality program. The company made changes in R&D and Finance operation guides. These changes were focused on providing clearer directions and rationales in the guides and an easy-to-use format. These changes have helped streamline processes, and reduced delays from review and rework. The company calculated savings at US$ 20,000,000 per year. Source – Business Week Oct. 25, 1992
The United Kingdom Department of Health and Social Security spent the equivalent of US$ 50,000 to develop and test a series of new forms. They have reported annual savings of over US$ 2.9 million in reduced staff time from the forms being properly completed the first time.
Source – Plain Language Principles and Practice by Erwin R. Steinberg

The United Kingdom Department of Customs and Excise cut a 55% error rate to 3% by revising lost-baggage claim forms used by airline passengers.  Source – American Institutes for Research
An  aerospace contractor lost 15 consecutive contract proposals over a three year time period before a quality proposal writing group was called in to review the quality of the proposals and to help the proposal development teams institute new work practices and improve writing skills for developing large proposals. After instituting changes in documentation work practices and processes, the contractor “won” 11 of 12 proposals.
Source – Shipley Associates private communication


Originally published on our Knowledge Management blog

Why Do So Many in the Life Sciences Feel Document Language Has to Be Complicated to Be Good?

Not too long ago I was working on a client project related to improving the communication quality of clinical study protocols. At project initiation we did a document assessment of communication quality on six recently completed protocols. One of my consulting colleagues framed a very high-level assessment of the communication quality of these documents. Her assessment can be summarized as follows—it appears the authors of these protocols assume their reading audience will start reading the document in serial progression from page one to page n, have a high command of the English language, enjoy reading long, dense narrative passages, and have exquisite memories.


In reality, the targeted reading audience for these documents—clinical study site investigators and managers, as well as members of Institutional Review Boards—clearly do not prefer to read documents structured in such a manner. Who would?

We took one of the protocols and restructured the document to optimize communication quality. We applied many of tenets espoused by the “plain language” movement that has received considerable traction within the many offices of the United States Federal Government. Here’s the link: http://www.plainlanguage.gov

The model protocol was reviewed by our client sponsor and she found the document to be “not scientific enough.” It looked too different than what she was used to. Of course, what she was used to was protocols containing long dense narratives, protocols making ineffectual assignment of agency to various parties participating in the conduct of clinical research; protocols that treat words like “should” and “must” as synonyms; and protocols that assume all users of the document have English as their mother tongue.

Why do so many authors in the life sciences remain firm believers in the notion that scientific discourse has to be complicated to be good? The arguments against plain language in scientific documents do not hold water—that plain language cannot be used with a clinical and technical reading audience, that plain language oversimplifies intentions and meaning, and that plain language is not precise. Check out this list of articles supporting the use of plain language on the plainlanguage.gov website.


Originally published on our Knowledge Management blog

Designing Documents for the Regulatory Reader

Over the years I have heard numerous presentations or read articles by regulators at FDA that suggest the documents submitted by pharmaceutical and medical device companies are not designed to support the reader tasks of the regulatory reviewer. When considering these comments, I’d suggest the core problem is that these documents are written from the perspective of the writers, with little consideration given over to what a reader is trying “to do” with these documents.

Here are some generalizations about regulatory readers and corresponding document design actions:

1. Regulatory readers work under a spotlight, in agencies subject to governmental scrutiny, with high levels of public criticism. Therefore it is important to write discussions to bring in focus the troublesome study issues of the development program. Highlight the issues as questions the regulators are likely to ask and provide responses to the questions, appropriate rationales for the responses, and then explicitly point to the corresponding data that underlies or supports your response.
   
   
2. Regulatory readers may be in the same broad field as the industry scientists who developed the drug or device, but they likely will not have the same level of specialization. Therefore, explain key decisions regarding development programs, study designs and methods of analysis. It is important to provide scientific and regulatory context to help support scientific expectations and reasoning.
   
   
3. Each regulatory reader brings specialized expertise to a team evaluation, where multiple reviewers work with different sections of the marketing request application for a drug or a device and arrive at various perspectives on the filing. Therefore authors must assume global use of their documents with multiple readers taking multiple perspectives. Authors must provide entry context at the beginning of main sections. That is, orient readers with purpose and main messages at the beginning of sections.
   
   
4. Regulatory readers are swimming in a sea of documentation. Make it easy for them to navigate your document. Carefully consider how discussions will be sequenced within sections and across sections of the reports as well as summary documents. Assume your regulatory reader applies a coordinate review to your document. Design pages for easy skimming and scanning. Use informative headers and deep subsection structuring.
   
   
5. Each regulatory reader has individual ways of working—individualized practices, work habits, and standards that have evolved as they have become expert reviewers. A common point is that they all read “against the text” by looking for weaknesses in the conduct of science, logic of arguments, and unsupported positions. This means authors must be aware that the regulatory reader is largely looking for what is not there in the document. It is therefore important to design documents assuming multiple points of entry for the reader. Critical information may need to be summarized in several locations to ensure the reader does not miss critical arguments. Repetition of information must be carefully managed so as not to become redundant.

Originally published on our Knowledge Management blog

Make workplace training effective

I’ve been facilitating quite a few workshops lately related to the concept of strategic review. At each session I’ve had numerous people comment to me that they found the workshop to be amongst the best training experiences of their professional lives. One person commented, “too bad the rest of the training I experience cannot be as practical and applicable to my work world.” I’ve heard comments like this time and again over the years. It is unfortunate that so much money and time is invested in “training” to yield little to nothing in terms of outcomes. An outcome that is readily avoidable when you design training using assumptions about how adults learn.

Research shows that adults in the workplace setting learn best in interactive, problem-based, applied situations that incorporate learning through simulations, case studies, and participant presentations. Adult learners need to see that the professional development learning and their day-to-day activities are related and relevant. New skills and concepts must demonstrate practical value and allow people to alter established work practices. In our training sessions we also center discussions on client documents, workplace situations, and real-life case studies. Adults want to be the origin of their own learning and will resist learning activities they believe are an attack on their competence. Thus, professional development needs to give participants some control over the what, who, how, why, when, and where of their learning.


Originally published on our Knowledge Management blog

What’s Wrong with PowerPoint as a Document Authoring Tool?

In our McCulley/Cuppan consulting work we recently had a new client invite us to work with an authoring team on applying best practice to the planning, writing, and reviewing of a regulatory submission document. The document was going to be a significant piece of work, requiring over 500 pages and involving multiple authors across several scientific disciplines.

At the first meeting, the project leader announced she wants to continue the use of PowerPoint (PPT) as the document planning tool. Reasoning for this approach was in part because she and other team members already invested considerable time and effort in generating a 540+ slide deck representing data and messages to be in the regulatory document and PPT was a very familiar tool from extensive use in developing presentations.

I have to say we were mildly surprised by the demand to use PPT as the primary tool to plan and outline such a large, complex document. We have encountered other organizations using PPT for document planning purposes, but never on such a large scale. On the surface, the choice of PPT as the tool to produce initial draft documents seems reasonable. It is familiar to many, provides an authoring environment that produces output that can appear on screen as an outline, can be commented on in oral or remote review, and can be easily augmented and updated. All of these apparent benefits would support an argument for using PPT as an outlining tool to plan any and all documents. However, the use of this tool does not readily scale to developing large, complex technical documents.

Christine Haas, Karen Schriver, Thomas Huckin, Edward Tufte, and others tell us much about how readers interact with and read texts. From this collective body of work we have learned some things that can help us produce texts in an effective manner that, equally, is perceived as of very high quality. The prime method is to use tools that enable the design and review of texts as you expect your readers to engage in the reading and analysis of the text.

Successful collaborative authoring is significantly rooted in careful and thorough front-end planning. Choices of authoring tools are among the critical aspects of the document planning process, as tool choices impact (enabling or constraining) every other aspect of the planning and documentation processes. As authors and managers of authors, it is incumbent upon all of us to choose tools that accommodate our desired set of outcomes. Authors and managers must be cognizant of authoring tools that accommodate not only themselves and their ways of working, but others as well.

It is our position that use of PPT for document planning negatively impacts all potential collaborative authoring and review outcomes. Our claim assumes that the goal of the work is to generate an effective document, economically produced, that meets or exceeds end-user expectations.

I have outlined here key advantages and disadvantages of using PPT to plan and facilitate documentation of a multi-year, complex pharmaceutical development work.

PPT Advantages

• use is a habituated format; it’s familiar.

PPT Disadvantages

• presentations constrain data reporting rather than facilitate collaborative/interpretive processes (see my previous blog post on PowerPoint presentations).
• format creates/maintains a huge but fragmented vision of the process and product, impacting output (see Schriver and Tufte).
• PPT does not scale well to large documents as it limits information organization and searching is cumbersome, impacting review and the authors' ability to migrate material from PPT into a document-based format.
• PPT presentations do not accommodate major revisions/reorganization; impacting logic, content, and organization of the ultimate document product (see Schriver).
• the output decreases in clarity as the number of slides increases, impacting author/reviewer interpretation (see Tufte).
• PPT output is not a document; conversion to MS Word format is inefficient, time-consuming, and expensive.

The point seems clear in the choice of using PPT as an authoring tool is that familiarity wins, despite that, from a business, work, or quality perspective, the disadvantages of PPT clearly outweigh this single advantage. Eschewing proven document authoring platforms for the familiar may have unintended consequences and bear a high tariff.

Originally published on our Knowledge Management blog

How Do You Know When Good is Good Enough?

How do you know when good is good for any document you may produce?

I ask this question in every workshop I facilitate. Generally the response is a head nod followed by the comment "Yes, that is the question.....I wish I had the answer." There is the rub. We rarely sit back and consider the notion of what attributes we need in place to have a high quality communication product. 

Often we will work on a document until we run out of time (I am convinced the only marker used by the majority of people authoring documents in the pharmaceutical or medical device world is time.) We do this because we have not defined what document quality "is."

Defining document communication quality means developing expectations or standards of quality. Standards can be applied at the level of an individual, team, or an organization. Defined standards or definitions of quality are prerequisites for measuring quality. If standards don’t exist, they must be designed.

Standards are explicit statements of expected quality. They may apply to writing and reviewing practice as well as to the document product. In terms of writing, document standards communicate expectations for how a particular document will communicate to the user what is to be known or to be done. In essence the standards establish the parameters that ensure a document achieves the desired results.

Originally published on our Knowledge Management blog

The McCulley/Cuppan Standards Development Process We Use with Our Clients

As I mentioned in a previous post, in our McCulley/Cuppan consulting work we find the prevalent standards used to determine the “success” of a document are largely driven by simple measures of accuracy and then a passel of “home brewed” concepts for characteristics of document that are largely idiosyncratic ideas about what matters to the reader.

When you have 10 people reviewing a document you will end up with at least 12 opinions about the quality of the document (incongruous number is intentional as sometimes you have a reviewer offering more than one opinion that often conflict) and ways of describing quality that are all over the map. People use different terms to describe quality and if they actually use the same term, then it is highly unlikely that they will use the same definition for the term. So the first problem faced in the review process is the vocabulary used to describe quality attributes in a document.

When writers and reviewers compose or edit text, they continually make decisions that concern semantics—the meaning their words convey—and syntax—the way the words are arranged and other structural elements of the document. However, writers and reviewers often base these decisions on assumptions that have not been tested with technically-oriented adult readers or complex, data-rich, technical documents. Worse yet is many assumptions have never been tested to determine validity. Thus, there are actions ordered by writers and reviewers that may not in fact have the expected effects on a readers' performance (we know this is certainly true with one very important reading audience for pharmaceutical and medical device companies—the regulatory agency, like FDA). So the second problem faced in the review process is to understand what document elements have a meaningful impact on semantics and where to focus time and attention on the syntactical elements of a document.

The first thing we do with a client is an examination of the terms used formally (such as in guidance documents) and informally (such as review comments in documents) to describe quality. This will give us a sense of how the organization views quality and how sophisticated they may be in trying to create a common platform that describes document quality for the organization.

The second thing we do is to provide clients with the terms McCulley/Cuppan uses to describe document quality and why the concepts underlying these terms are extremely important to help determine document communication quality. A very important consideration is that the terms should speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms.

We spend a huge amount of time talking about semantics. The term "semantics" refers to the study of how language conveys meaning. Using a broad definition of semantics, we help our clients learn how to focus on different features of a document. Features like word choice, the position of information in document sections, paragraphs and in documents as a whole, idea importance, and the visual representation of data.

Then we work with a client team to create and vet working definitions for the various quality standards.

We then roll out the standards in a workshop setting and show people how the standards are applied to the types of documents they have and will produce.


Originally published on our Knowledge Management blog