There are no whole truths; all truths are half-truths. It is trying to treat them as whole truths that plays the devil.�b

      “Truth is rarely pure, and never simple.�c

      The news is again a-twitter with stories of another fallen hero: Avandia. While I have no need for the drug, it seems to echo the Vioxx and Celebrex stories of the recent past. I have arthritis, so those stories DID affect me. Having had to switch medications in mid-stream, as it were, I decided to take a closer look at the numbers being used. [Any of my readers who know my background know I have some interest in statistics and Chemometrics from my work in near-infrared (NIR) spectroscopy.]

As with any news story (e.g., “Man bites dog!�), seldom do the pundits read beyond the first paragraph.

The headline on the Today Show this morning was “Avandia causes 30% increase in heart attacks.� Let’s look at that first by going to the FDA website where the information was posted “for medical professionals,� as per the news. [Under the law, ALL citizens have access to the data, so don’t be put off.]

It seems that “8,604 patients on rosiglitazone (Avandia) [too bad “litazone� sounds so much like litigate] were compared with 5,633 patients randomized to a variety of alternative therapeutic regimens, including placebo.� The population seemed well dispersed among various lengths of disease occurrence, history of other problems and surgeries, etc. All but four of the 42 studies were of six months duration. The overall incidence of myocardial ischemia in the Avendia versus comparators was 1.99% versus 1.51%. This was seen as a 30% excess risk factor.

Now, I seemed to remember something similar in the 55 mile-per-hour national speed limit controversy, so I went back and looked at some notes. When Maine switched from 55 mph to 65 mph, the death rate doubled! That’s right, it doubled! It went from one to two for the year! Hence the title of this piece! So, let’s read farther…often newspeople and lawyers [there are already…same day as newscast…lawyers posting for patients taking Avandia] hope we have short attention spans and will not read further (it is, after all four large pages of print with no picture).

A Balanced Cohort Study used 33,363 patients in a managed care database for its study. “Propensity matching� was used to match risk factors (how novel) for cardiovascular disease and other considerations. The “composite cardiovascular endpoint� was hospitalization for heart attacks (myocardial infarctions) and bypasses (revascularizations). Patients in the study began use of the (dreaded) drug in question between 2002 and 2004. The groups were only rosiglitazone, metformin, or sulfonylurea; oral dual therapy combinations, and insulin combinations. The result?

 With a follow-up time of 1.2 years, the incidence of the composite cardiovascular endpoint was 1.75 events per 100 patient years for the rosiglitazone (Avandia)-containing regimen and 1.76 events per 100 patient years for the other treatments. Now, this is NOT a headline-grabber. Further studies also do not support the first stated study. This could be why the FDA voted 22-1 to continue the sales of the product and to allow doctors and patients to decide whether the risk is acceptable.

 This is all very interesting you say, but what does it have to do with process analytical technologies and Quality by Design (PAT/QbD)? Well, it seems the study that took ALL the factors into consideration was approaching Chemometrics…or, as we say, “Design of Experiment.�

When you use a portion of your data to support a conclusion (e.g., there is a better than 0.9 correlation between the rise and fall of the Mississippi River and the birth rate in Paris, France), you take a chance of using PAT to make your product worse.

When we attempt our first (or tenth for that matter) PAT project, it is critical to have enough inputs and use correct math to glean the causes of “good� v. “bad� product. If we merely look at, say, hardness versus dissolution rate, we totally ignore the distribution of lubricant or particle size of API or moisture distribution or etc., etc. etc.

Vis á vis the topic of this column, I could point out that 100% of the people who go bald drank water prior to baldness and anyone who breathes air will die. These are true, but what conclusions are you drawing from these statements?

The discipline needed for good science is not limited to the lab or the clinic or the production line. Good science and engineering are prerequisites at all stages of health care. But, since I have a limited charter, let it suffice to say, use all information when building a better product. May the force (a.k.a. Chemometrics) be with you.

References and Quoted Dignitaries

1. Benjamin Disraeli, 1804-81
2. A. N. Whitehead, 1861-1947, Dialogues, Prologue
3. Oscar Wilde, 1854-1900, The Importance of Being Earnest, I

By Emil Ciurczak, NIR/PAT expert and Contributing Editor, PharmaManufacturing.com. 

Comments, as always, invited.

IT validation and compliance expert Michael Gregor wrote the following comment yesterday, which needs to be emphasized---particularly the point he makes about software vendor claims.

"...Data integrity can only be accomplished by deploying the required safeguards outlined in 21 CFR Part 11. Although many companies make a good faith effort to implement the majority of Part 11 controls, they lack the necessary procedures required to govern the implementation and maintenance of these controls. Furthermore, organizations often lack the validation documentation required to prove their said application was tested against Part 11 requirements. Therefore, resulting in noncompliance.

In addition to a lack of controls and validation documentation, many companies seem to throw technology at 21 CFR Part 11 in hopes that 21 CFR Part 11 will be satisfied. Unfortunately, it is not that easy. As I stated previously, implementing technology to address 21 CFR Part 11 must be accompanied by the required procedures and validation documentation.

On a side note, a very important point I would like to caution the industry with is the idea of many software vendors claiming 21 CFR Part 11 compliance. I have performed several audits of software vendors over the years and not once did I ever find a software vendor in compliance with 21 CFR Part 11. In fact, many will claim Part 11 compliance because they have an audit trail functionality. Well, we all know that Part 11 is much more than than an audit trail. In the name of patient safety and data authenticity, be very careful when verifying a software vendor’s ability to comply with 21 CFR Part 11.

Lastly, it is important for the industry to understand that as technology evolves, so will the applicability of Part 11. Remember, if you are in doubt as to the applicability of Part 11, do not forget to ask yourself, Is my application’s data used for regulatory purposes?

Michael J. Gregor
CEO, Compliance Gurus, Inc.

21 CFR Part 11 noncompliance is a whole lot less sexy than the colorful sales and marketing pecadillos the industry has seen so far this year, but it is a main theme in Mr. Olagunju's complaints against Novartis and a recurring theme in 483s and regulatory problems. 

 "IT is still the weak link in the chain of compliance," defense attorney Mitch Lazris said at an industry event recently, "And, in the courtroom it's the industry's Achilles Heel." (IT consultant John Avellanet, CEO of Cerulean, LLC quoted Lazris in a presentation he made in Washington this week.  He also commented on data integrity a few days ago for "On Pharma." Here is a slightly longer article, in which he offers best practices.)

The essence of Part 11 is data integrity, and pharma doesn't always get it right. (In the medical devices area,  even news of Baxter's infusion pump recall involved data integrity---it appears that repair, test and inspection data had all been falsified.)

 "FDA's mistake with Part 11 was focusing on the technology hows and not the required outcomes for information integrity," said George Smith, head of CDER's office of compliance, also quoted by Avellanet. Experts agree that managing data is at the root of many of the pharmaceutical industry's most pressing problems today. 

When our magazine started up in 2002, "Part 11," the regulation that pharma loved to hate, was on every healthcare trade magazine headline and magazine cover (including ours).   We don't hear about the regulation all that frequently anymore

But rumors of Part 11's demise have been highly exaggerated. Just because we're not hearing about it again and again doesn't mean it shouldn't be a priority. Only now, it's integrated into the overall fabric of compliance.  The following slide from Mr. Avellanet's presentation, sums up the situation nicely.

 Consultant Michael Gregor, CEO of Compliance Gurus, says many pharma companies fail to:

• follow documented standard operating procedures
• maintain accurate and complete records.
• deploy audit trial functionality in automated applications used for regulatory purposes
• validate computerized systems for their intent of use
• train employees on company SOPs and/or Policies

Mr. Gregor, who will be a guest commentator on our web site and blog, and who is developing an FDA Compliance Index" for pharmamanufacturing.com offered the following responses to the same questions about clinical data validation we'd posed to Mr. Avellanet. Again, his answers refer only to pharma in general and have absolutely no relation to the Novartis legal case, the company of any of its affiliates.

PM - What are the typical mistakes companies make when trying to manage clinical data?

MG - Here are the top ones:

1. Failure to validate their computerized system used for AERs, Statistical Analysis, or Electronic Data Capture. Furthermore, companies typically lack the deviation procedures that govern the deviations from their established procedures (VERY COMMON).

2. Failure to document all changes to a database containing clinical data. Failure to document, assess, OR validate any changes to the software application used to manage clinical data (VERY COMMON).

3. Lack of procedures governing the data validation and data cleaning process.

4. Failure to abide by 21 CFR Part 11, Electronic Records and Electronic Signatures when creating, processing, archiving, or storing data.

5. Failing to create 21 CFR Part 11 requirements during their computer validation efforts for the data management application.

6. Insufficient resources available to follow company procedures or to assure quality of clinical data management.* I have typically seen company’s QA resources stretched so thin they do not have the bandwidth to support the clinical data management function.

*Hmmm. Makes one wonder whether AstraZeneca, in focusing its job cuts (just announced today) on Operations and IT, made a move it may soon regret.

-AMS

It's raining whistleblowers this month.  Reuters just reported this morning (click here for full news brief) that a medical reviewer has testified to senators that he or she was pulled off the review of GSK's Avandia after bringing up potential safety issues.  This suggests that the FDA (like the drug companies it supervise) may still have a way to go before it reaches its professed goal of establishing an atmosphere of open scientific debate. 

A senior Food and Drug Administration scientist has told congressional investigators that the FDA removed him or her from work on GlaxoSmithKline Plc's drug Avandia after voicing concerns about the safety of the diabetes pill, two senators said on Tuesday.

The unnamed FDA medical officer at one time was the primary reviewer for Avandia, according to a letter sent to the FDA by Senate Finance Committee Chairman Max Baucus, a Democrat, and Sen. Charles Grassley, the panel's top Republican.

The scientist has believed since 2005 that there was enough evidence for a strong "black box" warning on Avandia about a risk of congestive heart failure, the senators said.

In a statement titled "Senators reveal effort by the FDA to suppress scientific dissent and downplay safety concerns" released with their letter, they said the reviewer was "sidelined after voicing safety concerns" about Avandia.

The FDA reviewer "was told to stop participation in the review of potential cardiovascular safety problems associated with Avandia," the senators' letter said.

The senators said the scientist was interviewed by committee investigators.

"This new allegation is especially significant and raises our level of concern about FDA interference in safety decisions regarding Avandia," the letter said.

FDA spokeswoman Julie Zawisza said the FDA had received the senators' letter and would respond.

Not to harp on this point, but the drug industry should thank certain people for bringing a certain legal complaint involving a Swiss multinational whose name starts with N to light, because the problems listed in that document (whatever happens with that particular case) are difficult ones and a great many drug companies are wrestling with them today.

IT consultant John Avellanet, head of Cerulean Associates, LLC,  shared his insights on drug safety data management in pharma. I asked him specifically (and generically) about two allegations that were raised in David Olagunju legal complaint: best practices for validating data to comply with 21 CFR Part 11 and whether hard coding of randomization codes is really a problem.

His responses are enlightening.  

Mr. Avallanet is off on Wednesday to give an expert briefing on Data Integrity and 21 CFR Part 11 to FDA.  We'll publish a more extensive interview and case history on our web site and in our next issue, and we're very pleased that John will be writing an article for an upcoming edition of Pharmaceutical Manufacturing.

Please note- John's comments are generic and describe some of what he has seen in pharma and have absolutely no connection whatsoever with the legal complaint, the company involved in that complaint or any of its affiliates.

Here's a sound bite. 

PM -  What mistakes have you seen that pharma companies typically when validating data to comply with Part 11?

JA - There are six core steps to ensuring Part 11 compliance for clinical data (preclinical toxicology reports come from the public literature).  The mistakes run the gamut from simple oversights to multi-million dollar snafus.

The Top three mistakes that I almost always consistently see:

• Lack of clarity and accountability for data integrity. To date, in every company I’ve dealt with, "someone else" is always held accountable for data integrity.  Records Management says it’s an IT issue because IT holds the keys to the computer systems the data is stored on; IT says it’s a Records Management issue because they support systems, not information; the scientists claim that they are responsible for it and so they want to burn it on CD and stick it in their desk drawer because they just know IT won’t be able to find it down the road when the scientist might need it; management assumes that just like quality is accountable for the paper quality systems files (SOPs and the like), they must be accountable for the electronic stuff too…and so on, and so on, and so on.
• A division between R&D/preclinical data and the clinical and production data.  From the FDA’s perspective, this makes no sense (think Quality by Design).  From a cynical IT perspective and from a big consulting firm perspective, this makes it easy.  Either you deal in the R&D world or you don’t.  If you don’t deal with R&D/preclinical, it’s not your problem. 

There’s a Big Pharma company right now that is just beginning to grasp this nightmare; they hired a big consulting firm to handle all their data integrity issues globally, only to find out that the contract specifically excludes anything not already in clinical or in production (the big consulting firm doesn’t deal in R&D and preclinical).  You can just imagine where this is heading.

• Inability to "translate" and achieve mutual understanding between functional units. More and more, I’m convinced this is one of the root causes of the problem.  IT can’t understand Compliance, who can’t understand the Scientists, who can’t understand Purchasing, who can’t understand Senior Management…you get the picture. It’s not that they all need to be in mutual agreement -  they won’t be – but they do need to figure out how to be in mutual understanding; all marching to the same beat of the drum.

PM - Is the "hard coding" of randomization codes a bad thing? What influence would this practice have on results? Are there instances where it is appropriate

JA - The short answer?  Risk of data manipulation rises; decline of proof of patient safety and product efficacy. At its core, automated randomization tests multiple different variables (patients, individual results, genetic makeup, etc.) to make comparisons and draw conclusions from.  Because you are randomly assessing the whole group, when something you don’t expect occurs, then you know that is statistically important.

As a simple example, if you look at all the patients and 15% had heart attacks 30 minutes after taking the drug, but no one else had any heart attacks at all, then – assuming the patients were all healthy – you could say your drug has a 15% of causing a heart attack within 30 minutes.  You’d then want to do further studies to find out why, what is the risk after 30 minutes, etc., etc.

Now, let’s say that I got really good results from an initial clinical trial where I randomized my results, but discovered afterward that if I give my new drug to people who wear glasses, two months after the trial is over, they go blind.  What would the data show if I went back to hard-code my randomization codes to only be those that excluded people who wore glasses?  Results would be great and it would appear that all is good in the world.

Even if it’s not so cynically after-the-fact, once you hard code random sampling, it’s no longer random and now a number of things happen that reduce the credibility of the results:

1. You could cherry-pick either test cases, patients or other variables to get the results you want.

2. Even if you don’t cherry-pick, you end up with a set of comparisons, the results of which, if they are not the same,   can’t help you determine if that difference is important or not (was it due to the drug? some other variable?) because you only tested a set number.  With randomization, you draw better correlations, analyses and conclusions.

3. You introduce two human risks:  human error (when the guy is originally hard-coding) and then human temptation (such as when a company’s internal reviewer is under pressure to make sure that new drug has good results…his or her job might depend on such a thing).

4. You also make it difficult for someone to be able to reproduce your results unless they not only use the same methods, but now the same variables – for example, the same patient genetic makeup.

Hope that some readers find this information useful and that this makes up for Sunday's silly Potter post.

In the meantime, please write in and share your insights and experience.

-AMS

The FDA Alliance, an organization whose 100+ members include six former FDA Commissioners,  issued a statement today applauding the Senate Appropriations Subcommittee on Agriculture's allocation of $1.755 billion to the FDA's 2008 Fiscal year budget.  The figure represents a 12%,  $186-million increase over last year's budget and the largest increase for the Agency in five years.

"FDA has long suffered from inadequate staffing and resources," commented former Commissioner Don Kennedy. "It is clear from our meetings with members of Congress that they recognize FDA's important roles as the nation's premier consumer protection agency, and as a vital gateway for innovative technologies that advance patient cures and meet the many needs of consumers." 

And presumably, everyone at FDA will get to keep their bonuses.

Interesting how Michigan Democrat Art Stupak criticized those FDA bonuses yesterday.  Especially in light of the fact that lawmakers recently voted to increase U.S. Representatives' salaries by $4,400 to almost $170,000.

As FDA spokesman Rob Ali explained in an interview with NPR yesterday, the Agency employs physicians, pharmacists, scientists, biostatisticians and Ph.D.'s. 

What is $5,000 in the face of a circa- $150,000 salary that's less than half  of what many of them would earn in industry?  How many U.S. legislators have M.D.'s or Ph.D.'s?

-AMS

An article in the  Associated Press today reported that the Agency has paid workers more than $8 million in bonuses to keep them from defecting to industries, at a time when it is being pressed to spend more on food and drug safety.

The bonuses come to about $5,000 or more per employee and are three times higher than what was paid five years ago, but even then FDA accounted for over 40% of the $21.6 million that the government paid in retention bonuses, the article says.

The fact is that quite a few FDA employees can earn considerably more in industry than they do at FDA.  That's why so many of them eventually defect.  It takes a certain amount of idealism for the best scientists to stay on. Here's an interview with one of them.

Drug development is becoming more complex than ever, and FDA needs to hire, not tired hacks in search of a pension (the typical and extremely unfair stereotype of the government employee), but excellent scientists capable of asking the tough questions of the industry's best scientists when deciding whether or not to approve a new drug.

FDA's reviewers and scientists can't afford to be two steps behind their peers in industry if the Agency is to ensure that new drugs are truly safe and if science, rather than politics, is to drive FDA's decisions. And excellence costs money.

Morale at FDA is reportedly low, and has been for the past few years. These bonuses are a small price to pay if they help convince more of FDA's best employees to stay on. FDA also needs to fund more education and training, something that Dr. Janet Woodcock has said she plans to do. 

But what do you think? 

-AMS

Several years ago, FDA announced that drug firms that disagreed with its decisions could file a formal dispute resolution (FDR) and request to have the Agency's decision reviewed. 

The prospect was so attractive that about two companies took the Agency up on this offer over the next few years.  We surveyed companies at the time and their consensus was "we'd never do it. It's not worth the effort."

However, Eye on FDA notes that a number of companies are now formally filing FDR's. They're filing these FDR's for nonapproval letters, rather than plant inspections, but Mark Senak reaches the same conclusion about FDR's that our survey respondents did: non vale la pena. 

What does help is proactive communication with the Agency.

More from Eye on FDA.

-AMS

What happens when political agendas are superimposed on questions of science, or risk vs. benefit analysis? At NASA , recall how political appointee and public relations manager George Deutsch threatened scientist James Hansen if he spoke out about climate change or the "unproven Big Bang theory."

We also saw what happened within FDA with approval of the controversial "Plan B" contraceptive. Former surgeon general Richard Carmona's comments in his House testimony yesterday, and the CV of the man who would be his successor, suggest that the U.S. has not yet figured out how to keep politics and religion, or rather certain aspects of some religions, from influencing scientific decisions. 

No one would question the fact that ethics should drive all scientific choices. Ethics may be at the core of all religions, but they're removed from much of the political and religious rhetoric now being advanced.

Mark Senak makes excellent points today in his Eye on FDA blog about Dr. Carmona's testimony, nominee Dr. James Holsinger's background and interesting details on the background of Dr. W. David Hager, formerly on the FDA's Reproductive Health Advisory Committee, in  "The Importance of Credibility in Public Health." Click here to read, if you haven't already.

Politicized science eventually trickles down, not only in the form of bad laws, but decreased science literacy.  In a world where most decisions are based on risk versus benefit, a less informed public votes (or more likely doesn't vote) on proposals advanced by less informed politicians, and a vicious cycle continues. For perspectives from SUNY professor David Triggle, published last year, read on.

-AMS

With somewhat alarming speed, China executed Zheng Ziaoyu, the former head of its SFDA, yesterday.  I thought that the appeal process would take much longer than it did, and that the sentence would be commuted to life in prison but, apparently not, given the current political climate. More on this story from China's press agency. 

-AMS