Validation Studies Find ACC/AHA Calculation Overestimates CV Risk

[andersenpa]

As you probably know the ACC/AHA recently released new statin prescribing guidelines based on 10 yr cardiovascular risk

Validation with well known study groups like Framingham and Women's Health shows significant overestimation of risk- and thus perhaps overprescribing of statins:

 

LOUISVILLE, KY — Four out of five cardiovascular risk-prediction algorithms, including the new American College of Cardiology (ACC)/American Heart Association (AHA) risk calculator, overestimate the risk of cardiovascular events, according to the results of a new study^[1].

The latest analysis, which included patients from the Multi-Ethnic Study of Atherosclerosis (MESA), showed that the 2013 ACC/AHA risk calculator, which is designed to assess the 10-year risk of cardiovascular disease and stroke, overestimated the risk of cardiovascular end points by 86% in men and 67% in women. Overall, the ACC/AHA risk score overestimated risk by a net of 78%, report investigators.

"In terms of calibration, the results aren't surprising," senior investigator Dr Michael Blaha (Johns Hopkins University, Baltimore, MD) told heartwire . "Working with MESA and working with other data sets, many people have noted that these risk scores tend to overestimate risk."

 

The report was published in the February 17, 2015 issue of the Annals of Internal Medicine. Blaha said the calibration of the ACC/AHA risk model—the accuracy of the model when predicting risk—is "moderate" at best. In terms of discrimination, which uses the C statistic to determine how well the model discriminates between high- and low-risk patients, there was little difference between the five risk scores. "We found, overall, the discrimination was moderate again," said Blaha. "It was no better than prior risk scores."

Risk Calculator Challenged from Day One

Based on cholesterol guidelines that were published along with the risk-assessment guidelines, individuals without evidence of cardiovascular disease or diabetes but who have LDL-cholesterol levels between 70 and 189 mg/dL and a 10-year risk of atherosclerotic cardiovascular disease >7.5% are candidates for statin therapy.

Claims that the risk calculator overestimates the risk of cardiovascular events are not new. In fact, just a day after they were launched at the 2013 AHA Scientific Sessions, Drs Paul Ridker and Nancy Cook (Brigham and Women's Hospital, Boston, MA) spoke out, saying that when they tested the ACC/AHA risk-assessment algorithm in three large-scale primary-prevention cohorts—the Women's Health Study(WHS), the Physicians' Health Study (PHS), and the Women's Health Initiative Observational Study(WHI-OS)— cardiovascular risk was overestimated by 75% to 150%.

It also seemed to overestimate risk in another validation cohort from MESA when tested by the guideline authors themselves. At the time, however, only 5-year data was available from MESA, limiting the analysis.

With the 10-year data now available, the researchers tested three Framingham-based scores, including the Framingham Risk Score (FRS) for the prediction of coronary heart disease events (FRS-CHD), the FRS for the prediction of cardiovascular disease events (FRS-CVD), and an FRS modified by the 2001 Adult Treatment Panel cholesterol guidelines (ATP3-FRS-CHD). In addition, the Reynold's risk score, which includes a family history of premature CHD and C-reactive protein (CRP), and the 2013 ACC/AHA risk score were tested. The researchers tested the calibration and discrimination of the five risk algorithms in 4227 MESA participants aged 50 to 74 years old who were followed for 10.2 years.

Overall, the FRS-CHD, FRS-CVD, ATP3-FRS-CHD, and the 2013 AHA/ACC risk scores overestimated the risk of cardiovascular events by 53%, 37%, 154%, and 86%, respectively, in men, and by 48%, 8%, 46%, and 67%, respectively, in women. The Reynold's risk score was the best calibrated model, with investigators reporting the lowest discordance between actual and predicted events (-3%) when tested in MESA. The overestimation of risk could not be explained by differences in the rates of aspirin, statin, or antihypertensive medication use or by differences in coronary revascularization rates, say the investigators.

 

Predicted and Observed Events with the Risk Scores

Risk Score Predicted events (%) Observed events (%) Absolute difference Discordance (%) C statistic FRS-CHD 9.41 6.22 3.18 51 0.68 FRS-CVD 13.28 10.60 2.68 25 0.71 ATPIII-FRS-CHD 6.83 3.17 3.66 115 0.71 Reynold's risk score 7.43 7.64 -0.21 -3 0.72 ACC/AHA 9.16 5.16 4.00 78 0.71

Given the findings, Blaha told heartwire that risk scores are very inherently limited because patient populations differ and that there are limitations with simply measuring a cardiovascular risk factor, like blood pressure or cholesterol, at single point in time. The risk calculator, he said, should only be used as a "starting point" for doctors to talk to their patients about treatment.

"I tend to think that the risk score is as useful to patients as it is to physicians," he said. "It's useful for the average person to plug in their numbers and get a rough idea of their risk. When it comes to actually making a decision, you need much better information than just what the risk score provides."

Balancing Overprescription With Drug Risks

As part of the study, the researchers also focused attention on individuals with a 10-year risk of cardiovascular events or stroke >7.5%, as these would be patients who qualify for statin therapy based on the new cholesterol guidelines. For men with a 10-year risk ranging from 7.5% to 10% using the 2013 ACC/AHA risk calculator, the actual event was 3.0%. For women, the actual event rate was 5.1%.

"This is that sensitive population where treatment might be uncertain," said Blaha. "We noticed similar overestimation of risk compared with the rest of the group. That's important because it affects decision making. Now, a patient might say it overestimates the risk a little bit, but I'm high enough risk that I want to take [a statin], or it could change the equation. It needs to be individualized."

In an accompanying perspective written by Ridker and Cook^[2], the editorialists point out that a 7.5% risk of cardiovascular events at 10 years does not automatically mandate the start of statin therapy, but instead, like the guideline authors themselves suggested, is the beginning of a conversation about how best to manage risk.

Ridker and Cook say the "overestimation" issue with the ACC/AHA risk calculator is important, as some patients might end up on lifelong statin therapy. "This is a relevant issue because statins are associated with a small increase in the risk for diabetes," they write. Countering this, statin use might not be such a concern to patients or physicians because studies have shown statins reduce the risk even in patients with low absolute risk of cardiovascular events, "and the benefits of vascular-event reduction seem to exceed the risk for diabetes," they note.

[cbrsmurf]

^ link requires password

 

Interesting shift with the whole new diet/fats/eggs recommendation about to be changed.

[andersenpa]

Medscape registration is free!

[cbrsmurf]

^ I never remember my password :p

[ventana]

I forgot andersonpa password  :-)

 

Seriously - it brings you to a sign in and andersonpa

[andersenpa]

I forgot andersonpa password  :-)

 

Seriously - it brings you to a sign in and andersonpa

Sweet

You could sign in as me and sign me up for peds and GYN and all sorts of stuff that I hate

[andersenpa]

There fixed it