Showing posts with label review practice. Show all posts
Showing posts with label review practice. Show all posts

08 April 2012

Article Published: Missed Opportunities in the Review and Revision of Clinical Study Reports

It has been some time since the last blog post. I am not taking time off from posts. Rather, my workload and travel schedule have created a hectic environment where I have found it difficult to carve out time to create and upload useful posts.

This post is an unabashed personal plug for an article I coauthored with my colleague Stephen Bernhardt. The paper: Missed Opportunities in theReview and Revision of Clinical Study Reports appears in the April issue of the Journal of Business and Technical Communication

In the paper we look to further the understanding of review practices as applied to large and complex technical documentation. Specifically, the paper describes several case studies where we examined closely the review efforts of teams in the process of finalizing clinical study reports.  Our title suggests we see considerable room for improvement of review practices.

We see our work as an extension of other studies of review practices in professional settings. 
  • Paradis, Dobrin, and Miller (1985) first established document review as a critical site of contentious interaction in a research and development environment (Exxon). These authors contrasted document review from the opposing perspectives of managers and employees, detailing the ways that individuals often worked without shared purpose or expectations, resulting in frustrating differences of perception, feelings of resentment, and the need to substantially rework documents during successive reviews. 
  • Van der Geest and van Gemert (1997) pinpointed a general sense of frustration with review processes in several Dutch companies, noting that “both writers and reviewers find reviewing the most cumbersome stage in the process of text production, given that many parties are involved in it and that it is loaded with different expectations” (p. 445).
  • Henry (2000), working with data gathered by many interns at various professional sites, found that reviews were “fraught with second guessing” and required “interpretations of organizational culture to the ends of adequately and appropriately delivering discursive products” (p. 65).
  • And several studies (Henry, 2000; Katz, 1998; Paradis et al., 1985) discuss the activity of “document cycling,” an activity in which documents pass through multiple and sometimes conflicting reviews, as various reviewers weigh in with commentary.

A theme throughout the literature is that document review frequently brings into play conflicting or competing purposes. In addition to improving a document, review sometimes functions to evaluate worker performance (Couture & Rymer, 1991) or to discipline individuals (Henry, 2000, p. 81). Thus, interpersonal dynamics are frequently in play. Katz (1998), in particular, highlighted how review processes can function positively as a way to socialize new workers, helping them learn how to both write and work successfully within the local culture. Henry (2000) shared this concern for how interns come to understand the ways that organizations perform work.

The position we take in this paper is that changing nonproductive, conditioned, inefficient practices is NOT an easy matter, or companies would have already done so. We suggest that recognizing nonproductive review practices and understanding the causes for such practices should be an object of focus for more organizations. We understand that collaborating to develop complex documents with sound arguments involves difficult cognitive and social practices. But if a company establishes the goal of producing quality documentation through efficient and effective review practices, it will find that it must do a lot of work to counter the ingrained tendencies of review teams to focus on low-level stylistic edits as opposed to high-level rhetorical concerns.

03 May 2011

A New Outlook is Needed Regarding Document Review

Companies need to become more evaluative and methodical regarding their own work practices.
The costs associated with planning, authoring and reviewing research reports and regulatory submission documents are difficult to determine. If we consider the direct time and costs invested in authoring, reviewing, and publication preparation, then a conservative estimate on the cost of the final report will range from $50,000 to $200,000. If we add in the opportunity costs for time spent on endless draft versions versus other more productive professional work, then the costs start to skyrocket. When you add up the collective costs across a volume of documents generated in a year, well, now you are talking about some really big numbers.

Why in the life sciences are the gross inefficiencies of review practices tolerated? Is it the old "Out of sight, out of mind" approach? Then again, it may be comfort in a bias for action: "Hey we hit the deadline, so no worries, the end justifies the means."

Much can be done in terms of specific actions to shine a spotlight on the inefficiencies of review and encourage effective work practices:
  •  Articulate and rely upon meaningful document quality standards and and best-of-craft guidance for executing effective reviews. Emphasize shared standards over individual preferences.
  • Define the scope, purpose, audiences, and argumentative strategy for the document before drafting. In turn, use this early document planning to guide reviews. In other words, practice "Aim, ready, fire!" as the standard documentation method.
  • Define reviewer focus and responsibilities, acknowledging unique and strategic expertise. Involve specific reviewers for specific purposes. Inform all reviewers regarding roles and points of review focus and the differences between reviewer and editor. Problems of word choice, style preferences, transcription accuracy, and format should only be handled by the writer/editors and not made the objective, intentional or otherwise, of review.

We do not believe that changing non-productive practices is an easy matter, or companies would have already done so. We do believe that recognizing non-productive review practices should be an object of focus for more organizations. We understand that collaborating to develop complex documents with sound arguments involves difficult cognitive and social practices. If a company establishes the goal of producing quality documentation through efficient and effective review practices, it will find that it must do a lot of work to counter the ingrained tendencies of people to focus on low-level stylistic edits as opposed to high-level strategic and rhetorical concerns.

11 March 2011

More on Review--Surveys Show Performance Has a Long Ways to Go

In a poll on the McCulley/Cuppan website  we posed the question: "How satisfied are you with the review performance in your organization?"

The response revealed what we've consistently seen in our consulting work at McCulley/Cuppan:  
87% of respondents saw room for improvement in the review performance of their organization.

Though this was an informal poll with 38 respondents,  23 of the respondents (60%) were either Unsatisfied or Very Unsatisfied and only 3 respondents were Very Satisfied. 

Here's the breakdown:
  • 3 (8%) Very Satisfied
  • 2 (5%) Satisfied
  • 10 (26%) Somewhat Satisfied
  • 11 (29%) Unsatisfied
  • 12 (32%) Very Unsatisfied
In a survey at a pharma company we asked 139 individuals to identify their personal successes and frustrations with review performance as experienced in that company. The results were quite telling:
    • Listed frustrations outnumbered successes 2.5 to 1
    • 13.5% of respondents cited poor reviewer discipline as their greatest frustration
    • 19.5% of respondents cited collaborative pre-writinng planning of key messages as the root cause of successful document review projects.