Difficulties Assessing the Value-Added of Professional Communicators

Here is a post originally published in 2008, but the topic is worth repeating.

As mentioned in previous installments of this discussion, we at McCulley/Cuppan believe that the role of the professional communicator can add considerable value to the research and development process. But we recognize that in order to justify the need (and the added expense) for this expanded role, managers have the daunting task of trying to quantify the value added of such professionals to the organization.

The professional communicator must add value to a company’s information processes and products in order to justify their presence within the pharmaceutical organization. Mead (1998) defines the concept of value quite simply: “Value can be defined as the benefit of an activity minus its cost.” However, to apply that concept of value to the role of a communicator or science writer is not so simple. The value of the communicator in the life science research enterprise is not easy to determine for one principal reason: what value does one place upon a timely, efficient, and effective regulatory submission documents?

The problem is that communication in the life science industries does not lend itself to easy analysis against the traditional measures of professionalism that are routinely applied in other aspects of research or in other writing settings for that matter.

To measure the benefits of the professional communicator’s activity within the life science research organization, and in particular within a research project team, it is necessary to turn to the existing body of research illustrating the ways in which communicators provide project teams with valuable input and experience that enhance the overall quality, timeliness, and labor allocation to the tasks of research and reporting research. Quantitative and qualitative research methods, including case studies and surveys, offer data to demonstrate the significant effect professional communicators have on both organizational processes and products. However, little research and few publications directly address the roles and value of the professional writer within the life science research industries.

It is thus necessary for savvy managers to cast a wide net and look to other fields for relevance to the context of life science research. Managers should consider the task of writing in the work of engineering, aerospace, and computer industries, industries that share an intense document development environment similar to what we see in the life science research environment. These industries are similar to the life sciences in that the document product is used by regulators or buyers to create an informed opinion of the company’s proposed product or service. The caveat we must offer is that there is a thin volume of literature assessing the value added by communicators to these organizations, so the existing body of case studies may not be sufficient. Managers may need to seek out others in their organization or industry that have made use of communicators on their project teams.

Professional communicators can contribute to the understanding of the value added by documenting their work and comparing their tasks and targets with company benchmarks. Communicators must document their tasks over the life of the project because it is not possible to assess value added simply by looking at the documents that writers produce. Jong (1997) points out the problems in the “inspection model of quality control,” a model that when applied to documentation, focuses merely on the cost of writing and occassionaly the cost of review. This model is “inherently vulnerable to error,” as significant costs and errors may be missed. Jong claims, “The best way to improve the quality of the output is to improve the quality of the input” (40). Improving quality of input suggests that researchers carefully consider how they present the logic of their interpretations, how they design information to satisfy readers’ needs, and how they represent the resolution of issues within the framework of their document or sets of documents. Communicators can facilitate this consideration and ensure that the documents are logical, complete, and meet readers’ needs. One way to demonstrate these skills to managers is by comparing “before-and-after” documents that show how the communicator improved the logic and readability of initial drafts or how a communicator involved from the beginning of a project can better convey the resolution of issues than a writer brought in at the end of the project.

The variety of roles for writing in the life science research environment makes it difficult to employ a simple model for calculating the value added of writing specialists where the principal output is an informational product. As Fisher (1998) states, “The profession of technical communication is difficult to define in scope” (186). Within the pharmaceutical and life science research industries, writers and communicators have very different roles in various enterprises. Some are primarily writers, others function principally as editors, some coordinate the compiling of documents for registration filings, some facilitate team-based document development, and some concentrate on knowledge management. The challenge is to understand how professional communicators can contribute to efficiently and effectively producing the desired outcome: a high quality document product that helps in the conveyance of knowledge or the advancement of work on drugs and medical devices. Professional communicators must be able to defend their roles to management. By providing “before-and-after” versions of the documents and recording their tasks and timelines (thus enabling managers to compare those with previous projects), professional communicators may be able to reverse the current trend of having writing as a wholly separate task from research and may be able to start a new trend of utilizing professional communicators.

Works Cited

Fisher, J. “Defining the Role of a Technical Communicator in the Development of Information 

Systems.” IEEE Transactions on Professional Communication 41 (1998): 186-199.

Jong, S. “The Quality Revolution and Technical Communication.” Intercom 44 (1997): 39-41.

Mead, J. “Measuring the Value Added by Technical Documentation: A Review of Research and Practice.” Technical Communication, Third Quarter (1998): 353-379.


Originally published on our Knowledge Management blog

A New Outlook is Needed Regarding Document Review

Companies need to become more evaluative and methodical regarding their own work practices.

The costs associated with planning, authoring and reviewing research reports and regulatory submission documents are difficult to determine. If we consider the direct time and costs invested in authoring, reviewing, and publication preparation, then a conservative estimate on the cost of the final report will range from $50,000 to $200,000. If we add in the opportunity costs for time spent on endless draft versions versus other more productive professional work, then the costs start to skyrocket. When you add up the collective costs across a volume of documents generated in a year, well, now you are talking about some really big numbers.

Why in the life sciences are the gross inefficiencies of review practices tolerated? Is it the old "Out of sight, out of mind" approach? Then again, it may be comfort in a bias for action: "Hey we hit the deadline, so no worries, the end justifies the means."

Much can be done in terms of specific actions to shine a spotlight on the inefficiencies of review and encourage effective work practices:

•  Articulate and rely upon meaningful document quality standards and and best-of-craft guidance for executing effective reviews. Emphasize shared standards over individual preferences.

• Define the scope, purpose, audiences, and argumentative strategy for the document before drafting. In turn, use this early document planning to guide reviews. In other words, practice "Aim, ready, fire!" as the standard documentation method.

• Define reviewer focus and responsibilities, acknowledging unique and strategic expertise. Involve specific reviewers for specific purposes. Inform all reviewers regarding roles and points of review focus and the differences between reviewer and editor. Problems of word choice, style preferences, transcription accuracy, and format should only be handled by the writer/editors and not made the objective, intentional or otherwise, of review.

We do not believe that changing non-productive practices is an easy matter, or companies would have already done so. We do believe that recognizing non-productive review practices should be an object of focus for more organizations. We understand that collaborating to develop complex documents with sound arguments involves difficult cognitive and social practices. If a company establishes the goal of producing quality documentation through efficient and effective review practices, it will find that it must do a lot of work to counter the ingrained tendencies of people to focus on low-level stylistic edits as opposed to high-level strategic and rhetorical concerns.

How the Most Sophisticated Documentation Groups Operate

At the apex of our version of the Documentation Capability Maturity Model are the Level 6 "Optimizing" groups. These are very sophisticated writing groups that are continually looking at ways to enhance work practices and processes so as to better serve their customer needs.

At this level, the job descriptions for all the subject matter experts contain extensive descriptions regarding their roles and responsibilities in the development of high quality document products. Work performance is not merely judged on how well they execute design and conduct of studies, but also on how effective they documents are in supporting organizational strategies and economic objectives.

At this level, the writing groups rely upon carefully defined document quality standards that reach well beyond style guides and template preferences. These groups articulate detailed guidance for executing effective strategic reviews. All understand the importance of working to shared standards over individual preferences. Reviewers authenticate and sign off for documents meeting strategic intentions and communication quality. If problems arise downstream, then the reviewers are held as culpable for the problems.

Documentation project management inside a Level 6 writing group tracks the amount of time along with other parameters applied by individuals to planning, authoring, and reviewing documents. Performance is always reviewed for "lessons-learned" at the end of all major writing projects.

At this level, there is a clear commitment to the assessment of document usability for the target audience and even testing of  document designs for certain types of documents (such as clinical study protocols)  early in the document life cycle.

These writing groups take full advantage of authoring tools to assure information is effectively generated once and then repurposed to other documents as a drug or device asset moves forward in the development life cycle.

Lastly, these very sophisticated groups make time in their busy schedules for innovation both in terms of work practice and work tools.

I do not know of any groups in the pharma or med device industries who have the above mentioned attributes. Do you?

Still More on What Sophisticated Writing Groups Do

Continuing on the discussion of the last two posts regarding how we categorize the level of sophistication of writing groups. The next stop on the documentation capability sophistication chain is Level 5--Managed and sustainable.

The Level 5 writing organization applies a broad range of sustainable best-of-craft work practices.

At this level, the planning of business critical documents occurs in parallel with the planning of clinical research studies. Writing teams always deploy document prototyping techniques, such as populating sections of a clinical study report after the study protocol has been completed and planning the report results sections once the statistical analysis plan is finalized. Level 5 writing teams can tell you how many pages will be in their study report even before LPLV because there has been so much planning. Level 5 writing teams always map arguments across the sections of a clinical study.

Level 5 writing teams are aware of agile authoring techniques, but have not yet deployed these work practices. Level 5 writing teams clearly understand that repurposing information involves a whole lot more than merely cut and paste.

Level 5 writing groups clearly understand what strategic review means. The teams articulate and rely upon defined document quality standards and guidance for executing effective reviews. They always define reviewer roles and responsibilities acknowledging unique and strategic expertise.They recognize that the problems of word choice, style preferences, transcription accuracy, and format should be passed onto the writer/editors and not made a focus of review.

Level 5 writing teams routinely solicit document user information and maintain databases to help them track and understand usability and readability statistics on all of their documents. At Level 5, teams engage in root cause analysis to ascertain why questions were received from regulatory agencies. Level 5 writing teams apply standards and measures to the task of document authorship and review that are well down the highway from the simple metrics of time and draft numbers. Level 5 writing teams always engage in a lessons learned session at the end of each documentation project and such sessions are not seen as merely an activity to be filed and forgotten. Process and practice is tweaked and refined for the next time.

More on Sophistication of Writing Groups

In our McCulley/Cuppan version of a Documentation Capability Maturity Model, the fourth level is called Organized and Repeatable as suggested by JoAnn Hackos in her various books. But perhaps a better term to use in place of repeatable is consistent, as at the fourth level, the application of well-defined work practices is much more consistent across documentation projects. In these organizations, the majority of team members operate by the credo: we recognize some of our processes and work practices represent "best of craft" and we know they will get us through any crisis.

At this level, the writing group does keep project tracking data in a simple database. Unfortunately,  most of the data only tracks time and draft iterations. These remain the only parameters used to create project milestones.

At this level there is some recognition that the role of the medical writer involves more than "just writing" and on some teams writers are seen as knowledge managers and they are actively involved in team meetings well before data base lock. However there remains credibility issues for the writing group in the broader organization where writers are often seen as necessary evil and only "just write" the reports.

A fourth level writing organization routinely uses pre-writing planning and project kick-off meetings to shape team expectations. At this level, the writers are more aware of document design considerations that impact usability, but little effort goes into mounting discussions with teams about document design during the pre-writing planning. This remains a discussion item for draft review.

Little attempt is made to formally collect information from document end-users about readability and usability of the documents submitted to them. Any information collected happens on a casual basis and is largely applied in ineffective ways.

Some of the belief statements found in the Level 4 Writing Group are as follows:

• We are surprised and even sometimes mad at our document end-users when we get questions from them regarding information that was incorporated into submitted documents.
• We recognize that we cannot just have meetings where we talk about the data, that we need to have meetings where we plan how and what we are going to say about the data in the reports, but this does not always happen.
• We have good pre-writing planning and review tools, but it is a struggle to get the subject matter experts to actually use them.
• Reviewers still spend too much time editing and not reviewing because they believe editing style and word choice help to make a document significantly better.
• During the review process many reviewers still feel compelled to revisit sections already reviewed in an earlier draft version of the document.
• Team members recognize best practice review calls for different roles and points of focus during the review process, but many still do not follow the guidance.
• Sometimes we get stuck in our processes and still like to make all documents "look just like the last one that got approved."

So How Sophisticated is Your Writing Group?

From time to time I have talked about the sophistication of writing groups in the pharma and medical device industries. My position is that for the most part, writing work practices in the life sciences are well removed from "best of craft" work practices.

In my authoring workshops, I offer the portrayal that most writing groups in pharma and medical device companies rate only a 2 or 3 for sophisticated work practices on a six point scale. I argue that most are rudimentary at best in terms of sophistication.

This usually gets me a couple of the desired guffaws from the people in the room. I remind them, that just because you are really, really sophisticated in the conduct of science does not mean you are equally sophisticated in the tasks associated with reporting on this science.

The six levels in our writing sophistication system are based on the parameters as created by JoAnn Hackos, The scale of sophistication is as follows:

1. Oblivious
2. Ad-hoc
3. Rudimentary
4. Organized and repeatable
5. Managed and sustainable
6. Optimizing

We have created criteria for each level that differs from what Hackos did for the software world. Our criteria for Rudimentary, where we think most writing groups fall, is characterized as follows:

• We use style guides and templates for all of our documents and routinely make decisions on what to do based upon previous documents "approved" by senior management.
• We always coordinate on design and basic messages and worry about writing style across documents in a development program so that we can assure consistency in terms of appearance, style, and common messages.
• We make use of documentation project management to  assign resources and ensure documentation projects meet timelines and budgets.
• We recognize that documentation team performance varies across teams and we DO NOT know the performance factors having the greatest influence.
• We DO NOT systematically track user feedback regarding readability and usability of our documents. 

The credo for rudimentary groups is: "We always follow our routines except when we panic."

The belief statements at this level would include the following:

• We are supposed to develop information strategies for our reports before we write them, but we can never get the Subject Matter Experts to take the process seriously.
• We have lots of meetings to talk about what the data means, but we rarely have a meeting to talk about how we will represent the data in our report and never talk ahead of time on how we will represent the implications for what we see or fail to see in the data.
• We don't have time to talk about how we want to design arguments in our reports. We have more important things to do.
• We have no idea how big a document will be until after it is written.
• Just write everything you have to say and we'll fix it during review.
• Anybody with the similar professional training as I have will want to read a report in exactly the manner I choose to read it. 

So where does your group stack up?

More on Review--Surveys Show Performance Has a Long Ways to Go

In a poll on the McCulley/Cuppan website  we posed the question: "How satisfied are you with the review performance in your organization?"

The response revealed what we've consistently seen in our consulting work at McCulley/Cuppan:  
87% of respondents saw room for improvement in the review performance of their organization.

Though this was an informal poll with 38 respondents,  23 of the respondents (60%) were either Unsatisfied or Very Unsatisfied and only 3 respondents were Very Satisfied. 

Here's the breakdown:

• 3 (8%) Very Satisfied
• 2 (5%) Satisfied
• 10 (26%) Somewhat Satisfied
• 11 (29%) Unsatisfied
• 12 (32%) Very Unsatisfied

In a survey at a pharma company we asked 139 individuals to identify their personal successes and frustrations with review performance as experienced in that company. The results were quite telling:

• Listed frustrations outnumbered successes 2.5 to 1
• 13.5% of respondents cited poor reviewer discipline as their greatest frustration
• 19.5% of respondents cited collaborative pre-writinng planning of key messages as the root cause of successful document review projects.