05 June 2010

What's wrong with Peer Review? Quite a bit, actually.

I continue to follow conversations on other blogs and web sites where people are talking about peer review.  Here's the link to a very interesting paper by Michael Nielsen: “Three myths about scientific peer review” that is definitely worth the read. Another article I suggest you check out is by Alison McCook and published in The Scientist: Is Peer Review Broken? In particular, please read the section titled The Religion of Peer Review where McCook expands on our argument that an abundance of data from a range of journals suggests peer review does little to improve papers.

So the question I have to pose then is "given the lack of evidence that peer review works, then why do so many scientists in the life sciences remained joined at the hip to the notion that review really makes for high quality publications?".


Originally published on our Knowledge Management blog

23 May 2010

More on Topic: Is Medical Writing Really a Profession?

I posed this question on a medical writing affinity group I belong to on Linked In (Professional Medical / Scientific Writers). The shared commentary, all from medical writers, has been quite interesting. In this post I want to share a few of the commentaries and my responses.

It was stated by one commentator: “I do feel, because of the way this question is worded, there is a need to defend my profession.” Another said: “It seems to me that whether or not one describes medical writing as a profession is partly down to semantics, and what any individual person understands by the term profession.”  In response, I am not asking you to defend your work, but I am asking you to defend that the work of medical writers as a class is indeed a profession. I shared some thoughts regarding how the scholars characterize the concept of a profession, as I think much more than semantics is involved here.

Another commented “Is medical writing a profession? Well, some parts of it are more professionalized than others, but I think.... as a whole? No, currently it's a job description.” I wholly concur with this position for reasons I have already expressed and additional reasons I will add in this discussion entry. I modified slightly another comment made in the Linked In discussion to help shape a sense of the needed attributes for applying the moniker of professional to medical writing: “That practitioners are expected to understand the complexities of language and communication we've seen in this discussion so far and medical writers use workplace research and academic theory to push the practice of medical writing forward.”

Another interesting comment posed as a question is the following: “On another level, the question is: are you hiring a profession or an individual?” My observations find writing approaches within organizations and across the industry are idiosyncratic and outcomes unpredictable, so therefore the answer is clearly you are hiring individuals. By the way—this variability contributes to why so many enterprises struggle to identify “markers” to use to judge capabilities of medical writing candidates.
This same person said: “which part do we emphasize: the medical or the writer?” The answer I suggest is neither—the emphasis is on reporting first and foremost (and reporting in a manner that is accurate and compliant), then medical insight, and a distant third is writing (in terms of sharing information and shaping knowledge).

Lastly, I want to highlight the following comments: “Currently, I don't know that medical writers are seen as the experts on Medical Writing -- most of the public discourse seems to think we are unreflective cheerleaders for whoever pays our salaries....” and “ We do a lot of the applied science stuff already. But we have not come together to share that as professionals, and I think that's the biggest barrier to professionalism .”  Harsh commentaries both, but I suggest each is valid based on my observations of the ways of working in this industry. Unfortunately, I will suggest medical writers who do “applied science stuff” in terms of writing practice and document products are far and few between.

Yes, I want medical writing to be considered a profession. I did not start this post to raise a beef with the notion of medical writing. I opened in the discussion in the hopes of fomenting a thought-filled argument about what it takes to call oneself a professional medical writer. An aspect of professionalism is intelligent practice as defined by scholars like Schon. Intelligent practice is the application of knowledge to the body of work at hand. In this case, documents. I am suggesting that such intelligent practice is not a common attribute within or across communities of medical writing. I say this because of a significant amount of data that suggests the vast majority of documents produced by medical writers fail. These documents fail in terms of purpose, conveyance of logic, and meeting reader’s needs.

In defense of my claim I submit to you that there is considerable data in the public record regarding target-audience usability for the output of medical writing. A survey of keen interest to me shows that FDA reviewers cite poor document quality as a significant impediment to effective, efficient reviews; a comprehension study of clinical study protocols found that over 50% of test subjects (clinical investigators and study sight managers) failed to achieve a passing score in identifying roles and decision making responsibilities in the conduct of a clinical trial. I also suggest that much has been and continues to be written about the poor quality of manuscripts submitted to journals for publications. Additionally, analyses of the writing output in various clinical research genre suggests the documents are written for a persona that has a high command of the English language, enjoys reading long, dense narrative passages, and has an exquisite memory (I am pretty sure this persona does not match the broad audience these document products are intended to serve.)

When I raise these above points with medical writers, which I do all the time, I can say nearly all are unaware of any elements of the work I generalized above. Further, the vast majority of medical writers I encounter are unaware of any aspects of reading theory. Also I suggest to you that few medical writers can distinguish the differences between the concepts of data, versus information, versus knowledge; few of the hundreds of medical writers I have spoken to in the past 10 years have heard of the concept of plain language; most have not heard of the concept of document usability; and even fewer attempt to apply such concepts to their work (why would you when you see your work is to report versus transfer knowledge.) There are certainly individuals and even groups of medical writers who operate in a manner that warrant use of the term professional, but I am unable to find as appropriate, the broad application of the term to the various communities within medical writing.

I am suggesting that the intelligent practice of seeing clinical research documents as tools to be used by others outside of an enterprise is largely absent from the “ways of working” mindset of most medical writers. I am also suggesting that the task of document production—at all levels and for all forms of documentation in the clinical research community—is largely inefficient. Few know what it really costs the organization per page to produce a final version document (many are afraid to know that number); few consider techniques and tools to really transform the document production process (much of what I see done to improve work streams is at best incremental, but is largely “feel good” activity that has no impact whatsoever); few engage in meaningful post work reflection where performance is analyzed consistently across document projects by well-described standards. I could go on, but I think this is enough detail to support my position.

A hallmark of professionals as defined by Donald Schon and others is a continual reflection on practice, both during work and post-mortem and then application of what is learned to transform ways of working and product. If reflection is occurring, then I suggest little fruit is born by the activity for I hear all of the same criticisms of medical writers and clinical documents today as I did in 1994.


Originally published on our Knowledge Management blog

09 May 2010

Is Medical Writing Really a Profession in Practice?

I’ve been reading a book by Donald Schön, The Reflective Practitioner: How Professionals Think in Action, that has got me thinking about the working definition of the term “profession” and whether this term is appropriately applied to the field of medical writing. I am not sure it is for the following reasons.

When considering medical writing as a profession then one must ask whether the knowledge base of medical writing has the requisite properties and whether it is regularly applied to everyday problems of practice. Many who work to define professions state that the systematic knowledge base of any profession is thought to have 4 essential properties:
  1. Specialized
  2. Firmly bounded
  3. Scientific
  4. Standardized
Some would suggest the above parameters make Medical Writing a good fit under the moniker of profession. However, I call into question how many medical writers really look closely at the disciplines of writing or knowledge management. Much output I see and many conversations I have with medical writers suggest many write to standards bounded by myths and personal preference and not driven by the evidence of science.
Edgar Schein suggests there are three well-defined components to professional knowledge:
  1. Underlying discipline or basic science
  2. Skills and attitudinal component
  3. Applied science component (yields diagnostic problem-solving techniques to the actual delivery of the services)
I would find that Medical Writing fails to be a true profession as suggested by Schein’s criteria. I see the central gap being the failure to develop a general body of scientific knowledge bearing precisely on the problem of effective and efficient medical writing practices. I say this because of my McCulley/Cuppan work where I have many observations of how medical writing is considered and applied in the pharmaceutical and medical device industries.

For Schein, basic and applied sciences are convergent whereas practice is divergent. This suggests that the hallmark of the true practitioner is the ability to take the convergent knowledge base and convert it into skills and services that are tailored to the unique requirements of the client system—a process that demands divergent thinking skills. My observations suggest that many who operate under the aegis of medical writing are poorly placed to engage in effective, meaningful divergent thinking. The preferred thought pattern is at best the convergent model of “what did I/we do last time?” and at worst “just let me populate the tables and make sure this document is compliant with the template and ICH E3 guidance."

I think I am largely in the camp of those who would call Medical Writing something besides a profession. But I am certainly not with those who would suggest it is but an avocation. Frankly, such a description is quite derisive.

I find myself considering applicable to medical writing the Grady McGonagill position that those who rely on intuition and reflect on their work only episodically risk accumulating unintegrated clusters of habitual practice. They may develop to a level of competence but not beyond. I’d say this is a reasonable description that fits the largest cohort of those who work under the mantel of medical writing.

So what do you think—is Medical Writing truly a profession in practice?


Originally published on our Knowledge Management blog

07 May 2010

7 Rules for Great PPT Presentations

Having an understanding of what makes for an awful PowerPoint presentation can make it intimidating to create your own. Who wants to be the creator of one of those presentations?
In our consulting work at McCulley/Cuppan, we've sat through, created, and helped others create many presentations (including ones to FDA and other regulatory agencies), so we have some experience with good and bad PPT. Here's a list of rules we've developed over the years for designing effective presentations:
  1. Adopt your audience's mind--it's always about them, not you
  2. When you are the subject matter expert and have done lots of research, you may feel compelled to share it all in the PPT slides. Avoid this. Instead, focus on what your audience is most interested in. (Your knowledge will come through in the way you discuss the topic and answer questions.)
  3. Clarify your intentions
  4. How will you proceed with the presentation? Will there be a Q&A session? If so, when? Let your audience know upfront how much time you will take and the basic outline of how the time will be spent.
  5. Simplify, simplify, simplify
  6. Simpler is always better. Simple design, simple text. As Hans Hofmann said nearly 100 yers ago: "To simplify, eliminate the unnecessary so the necessary can speak."
  7. Embrace limitations and practice restraint--know when to stop
  8. Once again, all of your knowledge does not need to be demonstrated in this one presentation. Respect the time and concentration limits of your audience.
  9. Do like the pros do--script, storyboard, create
  10. Planning is the key to effective presentations. Work with the end in mind even before you build the first slide and the content of your presentation will be more compelling.
  11. Think communication not decoration
  12. PowerPoint is a tool to help you convey knowledge; keep the slide design simple so the focus stays on the content.
  13. All presentations are storytelling, so become a master storyteller
  14. Learn from the master storytellers around you, people whose presentations you've enjoyed attending. If you don't have any nearby, learn from well-respected presenters such as Garr Reynolds, Steve Jobs and Guy Kawasaki. Tell a tale that engages the audience and makes them want to listen and learn from you.  
     
    Originally published on our Knowledge Management blog

04 May 2010

Top 4 Don'ts for PPT Presentations

We've all been there, sitting through endless PowerPoint presentations where the speaker drones on and on and the slides are so full of text you squint no matter how close you are to the screen. We each have our least favorite PPT foibles, but here's a list of the four worst offenders:

I never met an animation technique that I didn't like.
Slide decoration doesn't equal slide design. Animation distracts from the content of your presentation and the purpose of the presentation is to convey information, so limit distractions and keep it simple.
Why can't everyone just read what I have on the slide?
Slides make up one portion of the presentation; they should not "stand alone" (even when you are offering copies of slides for notes). If the slides can stand alone, then what is the purpose of you being there? Just write a report instead.
Charts and tables are "islands that speak by themselves".
Charts and tables are never "islands that speak by themselves", whether in presentation or report. Some explanation is required, on the purpose of the table or the conclusions drawn from the data. Once again, a table on a slide augments your spoken presentation and should not be treated as a stand alone.
Everything I know on this topic must go on these presentation slides.
Also known as "cram everything in obsession". All of your knowledge on a topic doesn't need to appear on every page of the slide. Slides should be easy to read, and as brief as possible, otherwise people will spend their time reading your slides instead of listening to and being engaged in the presentation.
And remember, PowerPoint (or Keynote for fellow Mac users) isn't right for all meetings, briefings, and presentations as shown in the article "We Have Met the Enemy and He Is PowerPoint" from the NY Times about the usage of PPT in the US Military. The "bowl of spaghetti" map shown in the article is a great example of why everything shouldn't be crammed into one slide.


Originally published on our Knowledge Management blog

31 March 2010

More On Why Do So Many Feel Complex Language Is Needed To Have Good Scientific Writing

I made post a few weeks back addressing the topic of why some authors in the life sciences feel compelled to construct documents that are dense and difficult to read. Here's a link to that post.  The post generated quite a bit of discussion in a medical writing discussion group I belong to on LinkedIn.  The discussion there has motivated me to do some further investigation on this topic. I want to share with you some interesting pieces of information that I hope impacts how you think about what constitutes "good writing."

I came across a series of papers by J. Scott Armstrong, who is at the Wharton School of Business at the University of Pennsylvania. In these papers, Armstrong considers the broad question that academic communications should enhance knowledge and, therefore, researchers should invest energy in developing understandable ways to present their findings.

In his paper, Unintelligible Management Research and Academic Prestige, Armstrong explores the notion that if the goal of successful communication is to share information with others and, if science places a premium on successful communication, then, all other things being equal, journals should prefer articles that are clearly written to those that are not. Armstrong concluded from the studies presented in this paper that clear communication of one's research is not the norm in the prestigious journals he examined, nor is it widely appreciated in his small test of the academic community at three universities.

In a another paper, Research on Scientific Journals:Implications for Editors and Authors Armstrong states the following: "A review of editorial policies of leading journals and of research relevant to scientific journals revealed conflicts between 'science' and 'scientists'. Owing to these conflicts, papers are often weak on objectivity and replicability. Furthermore, papers often fall short on importance, competence, intelligibility, or efficiency."

In yet another paper, Barriers to Scientific Contributions: The Author’s Formula, published in the journal Behavioral and Brain Sciences, Armstrong, with tongue in cheek, describes a set of rules that authors can use to increase the likelihood and speed of acceptance of their manuscripts. Authors should: (1) not pick an important problem, (2) not challenge existing beliefs, (3) not obtain surprising results, (4) not use simple methods, (5) not provide full disclosure, and (6) not write clearly.


Originally published on our Knowledge Management blog

20 March 2010

What Skills Do You Need To Be A Medical Writer?

I am preparing a workshop for the medical writing group at one of my McCulley/Cuppan clients. The focus of the workshop is on enhancing writing skills. I know some reading this post and many of the people who will attend this workshop are thinking, “Why do workshops on writing for people who make their living writing?” So I am thinking that I will open each workshop session by posing and then addressing this unspoken question.

My answer will likely wrinkle most brows and even irritate a few, not the primary objective, but perhaps a necessary adverse event to get my point across.

Why have such a workshop? Because most medical writers that I know HAVE NOT been effectively exposed to the skill sets I see as critical to success as a writer charged with creating protocols, reports, and summary documents that describe clinical research.

My point is this—for most of the people I will have in the workshops (largely with life science backgrounds) the training received as “writers” occurred principally during their academic pursuit of a scientific discipline. It largely ended, in any formal sense, with their undergraduate composition classes. However, for some the task of writing is further cultivated through mentoring during postgraduate training. Beyond the academic experience, most of the exposure they receive is just a refinement of the “writing skills” they received in the academic arena. Any training of the type I plan to talk about was likely haphazard at best.

To help back my argument, I did a web search on the term “medical writing skills.” Here is the most common definition I found on the web:
A medical writer must have excellent writing skills, a strong command of English and grammatical rules, attention to detail, good organizational and analytical skills, and an ability to meet deadlines.
This definition is what the American Medical Writers Association (AMWA) and European Medical Writers Association use to describe the skill sets of a medical writer and these attributes are what are commonly listed in job postings for medical writers.

I am suggesting this commonly applied definition is wholly inadequate. It is missing what I consider the first-order skills one needs to succeed today as a medical writer. I’d even say that a strong command of the English language and grammatical rules is not a first-order skill to be a good medical writer—a statement many will call antithetical.

The definition of the term skill is: the ability, coming from one's knowledge, practice, aptitude, etc., to do something well. If all you bring to the table is writing skills, a strong command of English and grammar, attention to detail, good organizational and analytical skills, and an ability to meet deadlines, then I suggest you will not do well with the task of a medical writer.

Consider what the AMWA website lists as training sessions they offer to make you a good medical writer: workshops on basic editing, writing, communication, and bibliographic skills as well as specialized editorial and publication skills and in-depth consideration of issues in writing.
Or look at what the University of Chicago has as the core curriculum for their Medical Writing Certificate Program:
  • Introduction to Medical Editing
  • Advanced Medical Editing
  • Writing in the Medical Sciences
  • The Internal Logic of Medical Articles
  • Interpreting and Reporting Biostatistics
  • Designing and Editing Tables and Graphs
  • Understanding and Mastering the Clinical Study Report
There is little from any of these institutions regarding the skills we see critical for medical writers. Skills like:
  • Fostering collaborative writing versus cooperative writing
  • Project management of intellectually focused projects
  • Conflict resolution during document reviews
  • Acting as a document consultant to help devise methods to capture knowledge
  • Skills to support the capture, propagation, and sharing of knowledge
  • How to leverage the collective wisdom of a drug development team
We argue that to succeed, a medical writer must see their role as a knowledge manager, not a scribe of clinical narratives in various types of documents. To see more of our arguments on this notion of writer as knowledge manager you can check out these other posts on our blog:

The Role of the Professional Writer in the World of Drug and Medical Device Research
The Value of Broadening the Role of the Professional Writer in Life Science Research Organizations
DIA Musings: How Medical Writers View Themselves
Difficulties Assessing the "Value Added" of Professional Communicators Writing in the Life Sciences


Originally published on our Knowledge Management blog