Inefficient Meeting Practices Cost Money

As I read the meeting manifesto by Al Pittampalli, Read This Before Our Next Meeting (now available to read for free from Amazon), I noticed many similarities between what Pittampalli writes and what Greg has been advising clients for years. Mainly, that there is a high cost to inefficient and ineffective work practices.

Anyone working in corporate America will be able to relate to the book, and, hopefully, learn from it. For us at McCulley/Cuppan, the focus is, as always, on how those in biomedical R&D can improve work practices.

Below are a few highlights from the book that relate to what we consider best practices (bolded text is quoted from the book).


The Modern Meeting moves fast and ends on schedule.
"Traditional meetings seem to go on forever, with no end in sight." How often have you felt this way? And how often do these meetings accomplish the intended purpose of the meeting, if there was a clearly stated purpose?

What we've seen over the years is that review meetings, those scheduled to discuss a document in person, last for hours, going from page 1 to page n through a document, with hours wasted on word choice and often leaving the author feeling overwhelmed. When the time limit is up on this type of meeting, more meetings are scheduled that will follow the same pattern.

The Modern Meeting limits the number of attendees.
More often than not, executives are involved in these document review meetings, even on meetings focused on editing and not strategic review, along with anyone who ever had anything to do with the project. How many man-hours are wasted sitting in a meeting instead of working on discovery or development? As Greg mentioned in his post "Editing When You Should be Reviewing Costs Serious Money", "when all the hidden costs associated with review are added in, the cost-per-page to produce a final version document becomes significant."

Meetings should be scheduled for a set amount of time, a time limit that is short enough to prevent repetitive attacks on one word or phrase (nitpicking), but long enough to actually allow for the goal of the meeting to be accomplished. Only the people who are directly responsible for a decision or must act on that decision should be in the meeting. Just because a person provided a line of text to the document does not mean that they should attend the meeting. If a person feels they must be included, provide them with a meeting outline and/or a meeting recap.

The Modern Meeting rejects the unprepared. 
Often meetings are scheduled to review a draft of a document, but the meeting ends with only a few pages of that 100 page document having been marked up. Then more meetings are scheduled and the cycle continues.

Meetings should have a clearly defined purposed. If a meeting has a clearly declared purpose, the leader of the meeting should be able to provide a list of items to accomplish for the meeting and the time allotted for each action item. This helps the scheduler of the meeting weed out people whose presence is unnecessary. This also ensures everyone is prepared for the meeting. According to Pittampalli, if you aren't prepared, you shouldn't attend.

The Modern Meeting produces committed action plans.
At the end of a well-organized meeting, there should be a committed action plan, not just a deadline. Too often with our clients we've seen meetings that produce reams of notes for the authors of the document with a deadline that keeps moving as more notes are piled into the author's inbox.

If meetings, and work practices in general, are efficient and focused only on the contributions of those directly involved, there is less opportunity for circular arguments and nitpicking and less contradictory comments for the author to wade through. Plus there is more time to actually work.


For more tips on improving meetings, follow Al Pittampalli on Twitter at @Pittampalli or view his blog.

More on What is a Document?

So what is a document?

In response to my last blog post, I have been asked by several individuals—"so then what is a document?"

My short answer—"I do not know for sure."

Now for the long answer.

The widely accepted definition for a document is as a textual record. This definition served us well in the past. But now with digital records, semiotics, and information retrieval tools; I am not sure the definition meets the needs of how we communicate in 2012.

As early as the 1930s Paul Otlet, an Information Scientist of considerable renown, suggested that the definition of documents also include digital images and even three dimensional objects. I am not prepared to toss all the elements Otlet describes into the mix. But I am prepared to suggest that documents are organized physical evidence and as such the organization transcends the classic definition for a document as this vehicle is a less relevant communication medium in 2012 than it was is 1982. I do not have a preferred term, I wish I did, but I do suggest we attempt to move away from the term document as it suggests a domain for organized physical evidence that does not match the reality of the digital age.

Suzanne Briet suggested a definition some time ago that a document is evidence in support of a fact. I rather like this notion. She makes the point that documents should not be viewed as being concerned with texts, but with access to the evidence. I suggest this is the essence of all regulatory writing that I talk about often in this Blog. If one considers the models in place for electronic drug submissions, thinking in the classic terms of 8.5 x 11 and A4 is really not very useful.

Rather it is better to be thinking in terms of taxonomies of information or perhaps even semiotics. Semiotics is the study of signs, indication, designation, signification, and communication. Semiotics is closely related to the field of linguistics. I look at semiotics as a valid attribute for this discussion because the life sciences are driven by numbers and what are numbers, but signs and the significance of those signs.

Then there is Michael Buckland who talks about how a key characteristic of “information-as-knowledge” is that it is intangible: one cannot touch it or measure it in any direct way. Knowledge, belief, and opinion are personal, subjective, and conceptual. Therefore, to communicate them, they have to be expressed, described, or represented in some physical way, as a signal or communication.

What we are really talking about happening in regulatory submission packages is the conveyance of knowledge.  This conveyance often transcends the boundaries of a traditional text, that is, a document as it is generally defined. The Briet notion of "evidence in support of a fact" works well as a definition of a document especially if we change the quote to read "evidence is support of a claim."

Need a New Mental Model for Regulatory Documents

Wow……I have been away from this blog a whole lot longer than intended. Those competing interests….like you all understand….are the bane of my existence.

For the past couple months I have been looking closely at how people think about the “vehicles” used to communicate with regulatory health agencies. I am using the word vehicle here because I am trying to divorce myself from the notion of document, in particular, the notion of “a document.” In the modern times of on-screen reading and linked files, what is a document anyhow? To me it is the entire corpus somebody may be able to access, not just one slice of that body.

In my consulting/training interactions at McCulley/Cuppan, I find that the majority of people I interface with in the client setting operate within the mind set of individual documents (some have even smaller boundaries and operate by document sections) that are stand alone with well defined boundaries (pages and page counts.)

I want to argue that the vehicle of communication for regulatory submissions is not a document. It is the full and complete dossier submitted by the sponsor. Documents are just placeholders where I go to get a piece or pieces of information that help answer my questions. I want to argue that the regulatory reader does not see a dossier as a set of documents. Rather they see a dossier as a corpus of information that they will use to answer question and make decisions. The contents are just vehicles they peruse to get what they want.

Applying my working model means you stop seeing documents as “stand alone” and stop saying “this document has to tell a story.” I’d also like you to stop using the word document. That word has baggage I am trying to jettison. Instead I want people to view their work at least as “modules” and preferably as vehicles that help a user to answer very specific questions. Bottom line, a research report is just a part of the constellation that tells the stories. Note the plural form as we have many stories to tell in a dossier, not just one.

Applying my working model means you stop seeing your work as being like a novella—something to be read from page 1 to page n. Applying my working model means you see your body of work as something that is read in a coordinate manner that is defined by very narrowly defined aspect rules of inclusion/exclusion. Applying my model means you stop seeing pages and sections and you start seeing concepts and topics.

My argument is that the selective professional reader at regulatory health agencies cares little about documents, sections, pages, and data tables. I am suggesting such readers care solely about making informed decisions and where in the submission dossier they find vehicles that can get answers to their concept and topic questions.

Why do we still organize clinical research reports by the IMRAD convention?

Here is Part 2 of the questions I posed in my previous blog post.

I have been thinking about this question for some time now. I remain curious as to why we still organize clinical research reports by the long-standing convention of Introduction, Objectives, Methods, Results, Discussion, and Conclusion? The IMRAD form of organization. The year 1665 is often cited as the origin of the platform commonly referred to as scientific papers. It was not until the second half of the 1800s that these documents moved in the direction of “theory – experiment – discussion.” Then starting in the early 1950s the IMRAD structure became the prominent norm for the structure of a scientific paper. 

The supposed reason for the IMRAD structure is that this organizational approach facilitates literature review, allowing readers to navigate articles more quickly to locate material relevant to their purpose. It has been suggested that the IMRAD structure effectively supports a reordering that eliminates unnecessary detail, and allow the reader to assess a well-ordered and noise free presentation of the relevant and significant information. I can see how this argument may apply to research manuscripts published in Journals, but it certainly does not apply to the kinds of study reports submitted to regulatory agency health authorities.  

Unfortunately, the supposedly neat order of the IMRAD arrayed report rarely corresponds to how regulatory readers consider using documents to help them make decisions. That is, decisions about how study results support or fail to support broad or narrowly defined questions/arguments. People are surprised to learn that health authority review agents rarely read study reports. They certainly engage with study reports and make use of the content. But read these documents? No, that is an approach not normally taken by the health authority review agent. 

Frankly, I cannot blame them. After all, the IMRAD form of organization does little to help them get answers to their questions or support their approach to making decisions.

Over the past 50 years, the idealized sequence of the IMRAD structure has on occasion been criticized for being too rigid and simplistic. Perhaps the most famous castigation was made by Peter Medawar in the early 60s. He criticized the IMRAD design for not giving a realistic representation of the thought processes of the writing scientist. And then in the mid 80s A. G. Gross wrote a paper titled: “The form of an experimental paper: A realization of the myth of induction” that challenged the notion of the IMRAD structure as well. 

I am right there too in questioning the merits of this approach. At least when it comes to study reports submitted to health authorities.

I believe the reason for rigid adherence to the IMRAD approach for research document organization is the precedence of the past. Not that it affords the most effective stucture for the reader to get what they want from a document.

As for me—I’d prefer a Question & Answer approach—something along the lines of organizing the report by study objectives with the study methods appropriately subordinated at the back of the report. Give me a report organized as follows: Objective—Results—Discussion—Assignment of Significance. Each section of the report deals with only one study objective and information is presented in the sequence I have suggested. Then the last section of the report would be Conclusions where you integrate and contextualize the study findings to answer the really big question of so what do all these findings mean?

So what do you think is the most effective form of organization for a regulatory clinical study report?

Why do we do some of the things we do in clinical study reports? Part 1

Over the past three months I have been doing a series of workshops focused on the authorship of clinical research reports for a couple clients. This work has me posing questions about two very specific notions

1. Why do people still demand that a clinical research report intended for incorporation into regulatory submission packages must be written as “stand-alone” documentation?
2. Why do we still organize clinical research reports by the long-standing convention of Introduction, Objectives, Methods, Results, Discussion, and Conclusion?

In this post let’s consider the notion of the “stand-alone” clinical study report.

First question I have is why must these documents be stand alone? The term “stand alone” generally refers to a document or device that is self-contained and does not require any other document or device to fully function. I argue that a clinical study report submitted to a drug regulatory agency is part of a larger corpus and as such the report should never be construed as a stand-alone document.

It is a given that most submissions made to drug regulatory agencies contain multiple research studies. So you will never have a “stand-alone” research study. So an individual study is part of a corpus of clinical research that best operates collectively. Also regulatory submissions have multiple sections as mandated by guidance that are topoi (places of argument) for different and often integrated attributes of clinical research drawn from this corpus. So clearly the clinical study report is not a stand-alone document in the instance of interpretation and argumentation.

Another point, a clinical study report incorporated into a regulatory submission will have numerous appendices including the final version research protocol and the statistical analysis plan. So I struggle to understand the premise that says you must incorporate extensive elements of the protocol and stats plan into the body of the clinical study report in order to assure the report is a stand-alone document. I truly fail to see the merit of this premise.

My last point is why do so many feel it is necessary to write the Introduction Sections to clinical study reports in the classic inductive form of rhetoric that moves from describing the disease condition, to the therapeutic void, to the chemical or biological construct of the drug under study and how this drug’s pharmacological action addresses a therapeutic void. The standard response is, “well we want to make this a stand-alone document.” Why? All the rhetorical moves you are trying to make in the Introduction will happen for sure elsewhere in the drug submission package in the appropriate topos. So why waste your time here, at the low level of the clinical study report? In my way of thinking the study report Introduction is intended to convey only two points:

1. Why are you doing the study (the answers to what questions do you want at the end of the study that you do not have now?)
2. What previous work has informed the design of this study?

Designing Regulatory Submission Documents for Decision Making

I have made blog posts from time to time talking about users of various document genre represented in communication about life science research. I believe that for many, my portrayal of how people engage with documents presents a different picture than what they hold in their imagination. At least I hope it is the case that they are designing documents to a meaningful mental model, as that helps to explain why so many documents I examine are rather ineffectual communication vehicles. Unfortunately I believe most documents are generated according to the model of precedence--"what did we do last time?" I think this is generally the only question consistently asked in the life sciences regarding document design.

In this post I want to talk about designing documents to satisfy readers who must make decisions, like a regulatory health agency reviewer. These reviewers must decide whether a drug, biologic, or device can be marketed as desired by the sponsoring company.

There is nice body of work in the cognitive sciences that describes the mind set and reading style of the selective professional reader who is reading documents in order to make a decision or a set of decisions. It is clear from this research that these readers are not empty cups waiting to be filled up with whatever you want to send their way in your documents. Not only is the professional reader very selective in what they will read, they are also very critical. That is, they read against the information you have submitted in your documents looking for insufficiencies and weaknesses.

Remember that the selective professional reader working in the the health regulatory agencies is a very sophisticated reader and in many instances they know what type of information they need right after they review the requested marketing claims for the drug or the medical device.

Here is the part that many people miss in creating their documents. No matter how logical and detailed your information may be, if you have not correctly anticipated what the selective professional reader needs in terms of information and the necessary level of detail, then you have failed. It is that simple.

Research shows clearly that decision-making is a function of schema-based cognitive processes deployed by the selective professional reader. The process is really well understood now and suggests that all documents  to be used in a decision-making environment should be designed to support a schema-based reading style versus the approach of reporting information or summarizing findings.

Schema activation is at the heart of decision making. Schema provide the interpretive framework for a reader to pass judgement on data and written discourse. In the process of reviewing drug and device submissions, the selective professional regulatory reader will make use of many, many schemata during the course of the decision-making process. Each schema they deploy will have multiple slots for information. They will want to fill every slot and they search documents and data bases looking for the details associated with the various slots in a particular schema. For example, a specific schema can be characterized by this question: "will the indicated drug dose lead to an undesirable toxicity profile in the elder, under weight, and hepatically compromised patient?" They will then examine the documents presented to find all the information they desire in making the decision whether the drug label language regarding dosing is acceptable or needs to be modified. And they read only looking for what they believe are the necessary details. In essence, they read at the virtual exclusion of anything within the document(s).

This is why we talk about  the selective professional reader scanning documents looking for very specific information and now using some sophisticated search tools to navigate through documents looking for specific pieces of information and data.

Much of what the selective professional reader at regulatory agencies do is further characterized as an attribute-based search. The attributes are often compared against known standards, but may also be subjective standards. The attributes they are considering are largely tied into the product label claims the sponsor wants to make.

So the net-net here is that life science research professionals need to reconsider what they believe to be acceptable document design parameters. It is essential to build out effective working models for how the selective professional reader using your documents makes decisions and what schemata are being deployed in this process and then design documents that support these specific reading methods.

Presentation: Why Most Document Reviews are Not Really Reviews

View Greg's presentation slide set from the recent DIA Clinical Forum:

The Good, The Bad, and The Ugly of Review Practices in The Clinical Sciences

View more presentations from McCulley/Cuppan LLC