Validation of Risk Prediction Models for Atherosclerotic Cardiovascular Disease in a Prospective Korean Community-Based Cohort

Article information

Diabetes Metab J. 2020;44(3):458-469

Publication date (electronic) : 2020 January 13

doi : https://doi.org/10.4093/dmj.2019.0061

Jae Hyun Bae ¹^,^*

, Min Kyong Moon ²^,^*

, Sohee Oh ³, Bo Kyung Koo ², Nam Han Cho ⁴

, Moon-Kyu Lee ⁵

¹Department of Internal Medicine, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Korea.

²Department of Internal Medicine, Seoul National University Boramae Medical Center, Seoul National University College of Medicine, Seoul, Korea.

³Department of Biostatistics, Seoul National University Boramae Medical Center, Seoul, Korea.

⁴Department of Preventive Medicine, Ajou University School of Medicine, Suwon, Korea.

⁵Division of Endocrinology and Metabolism, Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.

Corresponding author: Moon-Kyu Lee. Division of Endocrinology and Metabolism, Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-ro, Gangnam-gu, Seoul 06351, Korea. leemk@skku.edu

*Jae Hyun Bae and Min Kyong Moon contributed equally to this study as first authors.

Received 2019 April 01; Accepted 2019 June 13.

Abstract

Background

To investigate the performance of the 2013 American College of Cardiology/American Heart Association Pooled Cohort Equations (PCE) in a large, prospective, community-based cohort in Korea and to compare it with that of the Framingham Global Cardiovascular Disease Risk Score (FRS-CVD) and the Korean Risk Prediction Model (KRPM).

Methods

In the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study, we evaluated calibration and discrimination of the PCE for non-Hispanic whites (PCE-WH) and for African Americans (PCE-AA) and compared their predictive abilities with the FRS-CVD and the KRPM.

Results

The present study included 7,932 individuals (3,778 men and 4,154 women). The PCE-WH and PCE-AA moderately overestimated the risk of atherosclerotic cardiovascular disease (ASCVD) for men (6% and 13%, respectively) but underestimated the risk for women (−49% and −25%, respectively). The FRS-CVD overestimated ASCVD risk for men (91%) but provided a good risk prediction for women (3%). The KRPM underestimated ASCVD risk for men (−31%) and women (−31%). All the risk prediction models showed good discrimination in both men (C-statistic 0.730 to 0.735) and women (C-statistic 0.726 to 0.732). Recalibration of the PCE using data from the KOGES-Ansan and Ansung study substantially improved the predictive accuracy in men.

Conclusion

In the KOGES-Ansan and Ansung study, the PCE overestimated ASCVD risk for men and underestimated the risk for women. The PCE-WH and the FRS-CVD provided an accurate prediction of ASCVD in men and women, respectively.

Keywords: Atherosclerosis; Calibration; Cardiovascular diseases; Cohort studies; Epidemiology; Primary prevention; Risk assessment; Risk factors

INTRODUCTION

Atherosclerotic cardiovascular disease (ASCVD) is a leading cause of death worldwide accounting for one-third of all deaths per year [1]. The development and progression of atherosclerosis is influenced by cardiovascular risk factors, such as age, sex, smoking, physical inactivity, obesity, diabetes mellitus (DM), hypertension, dyslipidemia, and systemic inflammation [2]. Cardiovascular disease (CVD) including coronary heart disease (CHD) and ischemic stroke are largely preventable by the control of modifiable risk factors [3]. In addition, the benefits from CVD risk reduction are greater in individuals with a higher baseline risk [4]. Therefore, risk prediction and stratification of ASCVD through the clustering of cardiovascular risk factors is a useful strategy for determining preventive interventions.

In 2013, the American College of Cardiology (ACC) and the American Heart Association (AHA) developed the Pooled Cohort Equations (PCE), sex- and race-specific equations to predict 10-year risk for the first hard ASCVD events defined as nonfatal myocardial infarction (MI) or fatal CHD, or fatal or nonfatal stroke [5]. The ACC/AHA recommends use of the PCE in the treatment of blood cholesterol [6 7] or high blood pressure (BP) [8] for the primary prevention of ASCVD in adults aged 40 to 79 years. After the release of the 2013 ACC/AHA guidelines, there has been an issue that the PCE may overestimate or underestimate the risk in several populations [5 7]. Indeed, risk overestimation by the PCE has been reported in East Asians [9], Europeans [10], and a contemporary United States (US) population [11]. On the other hand, in South Asians [12] and Native Americans [13], the PCE underestimated actual ASCVD risk. The PCE was developed based on data from cohorts representative of the US population including the Atherosclerosis Risk in Communities study [14], Cardiovascular Health Study [15], Coronary Artery Risk Development in Young Adults [16], and Framingham Original and Offspring cohorts [17 18]. Accordingly, the PCE calculates 10-year ASCVD risk for non-Hispanic whites and African Americans. The equations for non-Hispanic whites can be used for Hispanics, Asians, and Native Americans, but their accuracy has not been evaluated sufficiently in these populations. The performance of risk prediction models for ASCVD depends not only on country-specific race/ethnicity, but also on the prevalence of CVD or risk factors, available interventions to reduce ASCVD events, and socioeconomic status [7]. Therefore, the PCE should be validated in each population to be used for the primary prevention of ASCVD.

In this regard, we conducted the present study (1) to investigate the performance of the PCE for predicting ASCVD risk with 10-year cumulative incidence of ASCVD and (2) to compare it with that of the Framingham Global Cardiovascular Disease Risk Score (FRS-CVD) and the Korean Risk Prediction Model (KRPM) using a large, prospective, community-based cohort in Korea.

METHODS

Study population and ethical statement

The Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study is an ongoing, prospective, community-based cohort study that includes urban and rural residents in Korea. The study design and methods have been detailed elsewhere [19]. Briefly, 10,030 individuals aged 40 to 69 years were voluntarily enrolled in the baseline study between 2001 and 2002, and underwent biennial examinations including a questionnaire, physical examination, and clinical measurements through 2016. Among these participants, we excluded individuals who had a history of ASCVD at the baseline study (n=299), had insufficient data to calculate ASCVD risk (n=866), and were lost to follow-up after the baseline study (n=1,113). Finally, 7,932 individuals (3,778 men and 4,154 women) were included in the present study (Supplementary Fig. 1). The study protocol was approved by the Institutional Review Board of the Seoul National University Boramae Medical Center (IRB No. 07-2016-18). All participants provided written informed consent for the baseline and follow-up data.

Outcome measurements

We defined an ASCVD event as the first occurrence of nonfatal MI, unstable angina, stable angina pectoris, nonfatal ischemic stroke, transient ischemic attack (TIA), and death from ASCVD. The procedures for determining incident ASCVD have been previously described [20]. Briefly, individuals identified by self-reported ASCVD were examined through in-depth interviews and a review of their medical records. Nonfatal MI, unstable angina, and stable angina pectoris were confirmed by changes in the electrocardiogram or cardiac enzymes, coronary angiographic findings, a history of coronary intervention or coronary artery bypass grafting, and medications. Nonfatal ischemic stroke and TIA were confirmed by symptoms and/or signs, findings on computed tomography or magnetic resonance imaging, a history of procedure or surgery, and medications. Deaths from ASCVD were verified by family interviews or death certificates.

ASCVD risk prediction models

We calculated 10-year ASCVD risk by the PCE, FRS-CVD, and KRPM. The FRS-CVD [21] was developed to assess 10-year risk of CVD including a composite of CHD (angina pectoris, coronary insufficiency, MI, and coronary death), cerebrovascular events (ischemic stroke, TIA, and hemorrhagic stroke), peripheral artery disease, and heart failure based on data from original and offspring cohorts of the Framingham Heart Study [22 23]. The KRPM [24] was developed to estimate 10-year risk of ASCVD comprised of nonfatal or fatal MI, and nonfatal or fatal stroke using data from the Korean Heart Study (KHS) [25]. For the PCE, we used sex-specific equations for both non-Hispanic whites (PCE-WH) and African Americans (PCE-AA) because there was no specific equation for Koreans.

Statistical analysis

Continuous variables are presented as mean±standard deviation. Categorical variables are reported as frequencies and proportions. We analyzed men and women separately because of sex differences in cardiovascular risk factors [26] and the burden of CVD [27], and sex-specific prediction models for ASCVD. From the KOGES-Ansan and Ansung study, Cox regression analyses were performed to assess 10-year cumulative incidence of ASCVD after confirming that the assumption of proportionality of hazards was met. The performance of ASCVD risk prediction models was evaluated by calibration and discrimination. Calibration, agreement between predicted and observed risks, was performed by comparing predictive risk with observed events in each decile of model-based probabilities using the Hosmer-Lemeshow chi-square test [28]. A lower chi-square value (χ²) and statistical insignificance (P≥0.05) indicates the goodness of fit of the models. We also calculated discordance between predicted and observed risks. Discrimination, capability to classify individuals with and without events, was evaluated by the C-statistic or the area under the receiving operating characteristic curve [29]. The C-statistic >0.7 indicates good discrimination. Additionally, we recalibrated the PCE for individuals of the KOGES-Ansan and Ansung study to match the predicted and observed risk using a previously proposed method [30]. In the recalibration, we took the regression coefficients from the ACC/AHA PCE's Cox models, but the mean values were used from the KOGES-Ansan and Ansung study for risk factors (Supplementary Table 1). We also developed new equations of the PCE using equation parameters of the PCE-WH and PCE-AA (Supplementary Table 1). Two-fold cross validation (1/2 training set and 1/2 test set) was repeated 100 times to estimate and evaluate the performance of candidate models. We selected the model with the highest average-test area under the receiving operating characteristic curve as the best model. A P value <0.05 was regarded as statistically significant. All statistical analyses were performed using R version 3.4.2 (R Foundation for Statistical Computing, Vienna, Austria) and GraphPad Prism 5 (GraphPad Software Inc., San Diego, CA, USA).

RESULTS

Characteristics of the study participants

Baseline characteristics of the study participants are shown in Table 1 and Supplementary Table 2. In men, the mean age was 51.3±8.6 years and body mass index (BMI) was 24.3±2.9 kg/m². In women, the mean age was 52.4±9.0 years and BMI was 24.9±3.3 kg/m². Total cholesterol levels, high density lipoprotein cholesterol levels, systolic BP, and the proportion of individuals with DM were comparable between men and women. However, the proportions of individuals with hypertension and BP treatment were higher in women (17.6% and 13.1%, respectively) than in men (13.6% and 8.5%, respectively). Current smokers were highly prevalent in men (48.1%) compared with women (3.4%).

Table 1

Baseline characteristics of the study participants

The prevalence of risk factors in the KOGES cohort and study cohorts of the PCE, FRS-CVD [21], and KRPM [24] is shown in Table 1 and Supplementary Table 2. In men, the proportion of BP treatment was lower in the KOGES cohort (8.5%) than in the PCE-WH (16.9%), PCE-AA (31.2%), and KRPM (10.1%) cohorts. The prevalence of DM was lower in the KOGES cohort (7.8%) than in the PCE-WH (8.8%), PCE-AA (15.9%), and KRPM (10.5%) cohorts. Current smokers were highly prevalent in the KOGES cohort (48.1%) compared with the PCE-WH (25.5%) and PCE-AA (35.5%) cohorts, but similar to the KRPM (49.4%) cohort. In women, the proportion of BP treatment was lower in the KOGES cohort (13.1%) than in the PCE-WH (18.5%) and PCE-AA (40.8%) cohorts. The prevalence of DM in the KOGES cohort (6.0%) was comparable with that in the PCE-WH (6.3%) cohort but lower than that in the PCE-AA (17.4%) cohort. Current smokers were less prevalent in the KOGES cohort (3.4%) compared with the PCE-WH (24.9%), PCE-AA (22.7%), and FRS-CVD (34.2%) cohorts.

Cumulative incidence and observed risk of ASCVD

During the mean follow-up of 8.4±2.7 years, 598 ASCVD events (305 in men and 293 in women) occurred; 317 nonfatal CHD including nonfatal MI, unstable angina, or stable angina pectoris, 216 nonfatal stroke or TIA, and 89 deaths from ASCVD (Table 2). Twenty-four individuals had both CHD and ischemic stroke or TIA between the examination cycles. The cumulative incidence of ASCVD per 100,000 person-years was 968.8 for men and 839.3 for women (Supplementary Table 3). The observed 10-year ASCVD risk was 8.05% for men and 7.05% for women (Table 2).

Table 2

Predicted and observed risks of atherosclerotic cardiovascular disease among participants from the KOGES-Ansan and Ansung study

Performance of the ASCVD risk prediction models

The PCE-WH and PCE-AA moderately overestimated ASCVD risk for men by 6% and 13%, respectively, with good calibration (PCE-WH: χ²=10.59, P=0.226; PCE-AA: χ²=17.13, P=0.029) (Table 2, Fig. 1A and B). However, for women, the PCE-WH and PCE-AA underestimated ASCVD risk by −49% and −25%, respectively, with poor calibration (PCE-WH: χ²=258.62, P<0.001; PCE-AA: χ²=107.08, P<0.001) (Table 2, Fig. 2A and B). The FRS-CVD overestimated ASCVD risk for men (91%) with poor calibration (χ²=177.71, P<0.001) (Table 2, Fig. 1C). On the other hand, the FRS-CVD predicted ASCVD risk for women accurately (3%) but showed poor calibration (χ²=24.70, P=0.002) due to the disagreement between the predicted and observed risks in the 5th to 7th deciles (Table 2, Fig. 2C). The KRPM underestimated ASCVD risk for both men (−31%) and women (−31%) with poor calibration (men: χ²=58.39, P<0.001; women: χ²=67.27, P<0.001) (Table 2, Figs. 1D and 2D). In discrimination, the C-statistic showed good discrimination for all models in men (PCE-WH, 0.731; PCE-AA, 0.730; FRS-CVD, 0.730; KRPM, 0.735) and women (PCE-WH, 0.726; PCE-AA, 0.732; FRS-CVD, 0.726; KRPM, 0.729) (Table 2), which showed a reliable classification of individuals with and without ASCVD events.

Fig. 1

Ten-year probabilities of atherosclerotic cardiovascular disease (ASCVD) in men from the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study. (A) The Pooled Cohort Equations for non-Hispanic whites. (B) The Pooled Cohort Equations for African Americans. (C) The Framingham Global Cardiovascular Risk Score. (D) The Korean Risk Prediction Model.

Fig. 2

Ten-year probabilities of atherosclerotic cardiovascular disease (ASCVD) in women from the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study. (A) The Pooled Cohort Equations for non-Hispanic whites. (B) The Pooled Cohort Equations for African Americans. (C) The Framingham Global Cardiovascular Risk Score. (D) The Korean Risk Prediction Model.

Recalibration of the PCE

The PCE-WH and PCE-AA were recalibrated for individuals of the KOGES-Ansan and Ansung study. The recalibrated PCE improved calibration substantially in men (PCE-WH: χ²=18.94, P=0.015; PCE-AA: χ²=15.24, P=0.055) (Fig. 3A and B) but not in women (PCE-WH: χ²=103.56, P<0.001; PCE-AA: χ²=133.57, P<0.001) (Fig. 3C and D). The recalibrated PCE did not affect discrimination.

Fig. 3

Ten-year probabilities of atherosclerotic cardiovascular disease (ASCVD) in men and women from the Korean Genome and Epidemiology Study (KOGES)-Ansan and Ansung study using the recalibrated and new Pooled Cohort Equations. (A) The recalibrated Pooled Cohort Equations for non-Hispanic whites in men. (B) The recalibrated Pooled Cohort Equations for African Americans in men. (C) The recalibrated Pooled Cohort Equations for non-Hispanic whites in women. (D) The recalibrated Pooled Cohort Equations for African Americans in women. (E) The new Pooled Cohort Equations in men. (F) The new Pooled Cohort Equations in women.

New equations of the PCE

We developed new equations of the PCE for individuals of the KOGES-Ansan and Ansung study. The new equations improved the calibration substantially in men (χ²=11.95, P=0.153) (Fig. 3E) but not in women (χ²=16.47, P=0.036) (Fig. 3F). The new equations did not affect discrimination (Supplementary Table 1).

DISCUSSION

In the present study, the PCE overestimated 10-year ASCVD risk for men but underestimated the risk for women in the KOGES-Ansan and Ansung study. The PCE had a better predictive ability for incident ASCVD than the FRS-CVD and KRPM in men. We also found that the recalibration or new equations of the PCE using data from the KOGES-Ansan and Ansung study substantially improved the estimates in men. However, in women, the FRS-CVD predicted ASCVD risk most accurately compared with the PCE and KRPM. The PCE, FRS-CVD, and KRPM had good discrimination in the KOGES-Ansan and Ansung study.

The PCE-WH and PCE-AA moderately overestimated ASCVD risk for men in the KOGES-Ansan and Ansung study. Risk overestimation by the PCE has been reported in external validation studies including Asian populations [9 31]. In the Multi-Ethnic Study of Atherosclerosis (MESA), a prospective community-based cohort of US adults including white, black, Chinese, and Hispanic Americans, overestimation of ASCVD risk was observed in all ethnic groups and it was highest in Chinese [9]. In the KHS, a prospective cohort study in Korea, the PCE also overestimated ASCVD risk for men [24]. The discordance between the predicted and observed risks may result from racial/ethnic differences in genetic predisposition, environment, and cardiovascular risk factors between the development and validation cohorts. Compared with Western populations, East Asians have lower rates of traditional cardiovascular risk factors including obesity [32] and hypercholesterolemia [33]. In our study, the participants also had lower levels of risk factors than the US population [34], except for a greater proportion of current smokers in men. On the other hand, the PCE-WH and PCE-AA showed good calibration for men in the KOGES-Ansan and Ansung study. The predictive ability of the PCE could be affected by changes in the CVD epidemiology with an improvement in the control risk factors. In Korea, total CVD mortality has significantly decreased over the last 30 years [35], which could be largely due to improvement in the management of hypertension [36]. However, in a sensitivity analysis of the MESA, treatment including anti-hypertensive agents, lipid-lowering agents, aspirin, and coronary revascularization did not explain risk overestimation by the PCE [31]. Therefore, additional validation is needed to evaluate the performance of the PCE in Korean men.

The PCE-WH and PCE-AA underestimated ASCVD risk for women in the KOGES-Ansan and Ansung study, which is different from the risk overestimation in men. In line with our findings, ASCVD risk was underestimated for women in the KHS [24] and was less overestimated for women than for men in the MESA [31]. These results suggest that there are sex differences in cardiovascular risk factors that lead to a higher ASCVD risk in women than in men [37 38]. In the INTERHEART study, risk factors for CHD were similar between men and women, but their impact including hypertension and DM were greater in women than in men [39]. However, in the present study, the prevalence of DM and current smoking in women was lower than that of the PCE cohorts, which could not explain risk underestimation by the PCE. Nontraditional risk factors specific to women, such as gestational DM, hypertensive disorders of pregnancy, and menopause [37], might be associated with poor calibration, although they were not evaluated in our study. Moreover, the pathophysiology of ischemic heart disease (IHD) could be different between men and women. Women are more likely to have stable angina pectoris than men because microvascular dysfunction [40], plaque erosion with distal embolization [41], and impaired vasoreactivity [42] are more prominent than obstructive CHD. For this reason, it is possible that more ASCVD events were ascertained by identifying stable IHD in the present study. Further investigation of cardiovascular risk factors and pathophysiology is needed to apply the PCE to Korean women.

In the present study, recalibration and new equations of the PCE substantially improved the predictive accuracy in men but not in women. In the recalibration and new equations, we used the coefficients and mean values for risk factors from the KOGES-Ansan and Ansung study. Previous studies have suggested that using the recalibrated PCE with data from cohorts can improve ASCVD risk estimates in a population [9 11 24 43]. However, a recent study using six US cohorts failed to improve the accuracy of the PCE by simply applying the derivation process to a newer cohort data [44]. Moreover, recalibration could not be applied to the PCE when the degree of miscalibration varied in different risk groups [45]. In the present study, the participants of the KOGES-Ansan and Ansung study showed varying degrees of miscalibration across the predicted risks in women. Consequently, recalibration of the PCE should be performed considering characteristics of the cohort data, the degree of miscalibration, and statistical methods in further studies.

The FRS-CVD overestimated ASCVD risk for men but showed accurate risk prediction for women in the KOGES-Ansan and Ansung study. Previously, the FRS-CVD overestimated ASCVD risk in US [31], European [46], and Asian populations [47]. However, its performance has not been evaluated in the Korean population. In men, risk overestimation may be associated with lower levels of total cholesterol and systolic BP in the KOGES cohort than in the Framingham Heart Study and the Framingham Offspring Study [22 23]. In women, the KOGES cohort had lower proportions of current smokers but a higher prevalence of DM and lower levels of high density lipoprotein cholesterol than the Framingham Heart Study and the Framingham Offspring Study [22 23]. Given the probably higher ASCVD risk in women than in men, as inferred from the PCE, it is plausible that the FRS-CVD showed an accurate prediction in women. In the MESA and Hong Kong Chinese, the FRS-CVD was also useful for predicting ASCVD risk for women [31 45]. Taken these together, the FRS-CVD should be reevaluated in a modern cohort with multi-ethnic groups including East Asian populations.

Finally, the KRPM underestimated ASCVD risk for both men and women in the KOGES-Ansan and Ansung study. The KRPM [24] was developed from the KHS [25], which was composed of individuals aged 30 to 74 years who visited health promotion centers for a medical checkup. Individuals in the KHS had fewer cardiovascular risk factors than the general Korean population [25]. Thus, it is not surprising that the KRPM underestimated ASCVD risk in the community-based cohort. Moreover, the KRPM provided accurate risk prediction in the KHS cohort [24], but it was not validated in other populations. Therefore, further studies are required before the KRPM can be used in place of the PCE or other ASCVD risk prediction models in Koreans.

The strength of our study is that we assessed 10-year cumulative incidence of ASCVD in a large-scale, prospective, community-based cohort in Korea. To ascertain ASCVD events comprehensively, we excluded heart or vascular diseases not associated with atherosclerosis through in-depth interviews and a review of medical records. However, the present study has some limitations. First, some risk factors, such as smoking status and prior diagnosis of hypertension or DM, were self-reported rather than directly measured, which could raise concerns about imprecision. Second, we did not evaluate the effects of interventions on cardiovascular risk factors or ASCVD during the follow-up period.

In conclusion, the PCE overestimated ASCVD risk for men and underestimated the risk for women in a large, prospective, community-based cohort in Korea. The PCE-WH and the FRS-CVD provided an accurate prediction of ASCVD in men and women, respectively, in the absence of a reliable risk prediction model specific to Koreans.

ACKNOWLEDGMENTS

Data in this study were from the Korean Genome and Epidemiology Study (KOGES: 4851-302), National Research Institute of Health, Centers for Disease Control and Prevention, Ministry for Health and Welfare, Republic of Korea. We acknowledge Ji Soo Lee for providing preliminary results about the validation of cardiovascular risk prediction models.

Notes

CONFLICTS OF INTEREST: No potential conflict of interest relevant to this article was reported.

AUTHOR CONTRIBUTIONS:

Conception or design: J.H.B., M.K.M.
PAcquisition, analysis or interpretation of data: J.H.B., M.K.M., S.O., B.K.K., N.H.C., M.K.L.
Drafting the work of revising: J.H.B., M.K.M., S.O., B.K.K.
Final approval of the manuscript: J.H.B., M.K.M., S.O., B.K.K., N.H.C., M.K.L.

References

1. World Health Organization. World health statistics 2016: monitoring health for the sustainable development goals (SDGs) Geneva: World Health Organization; 2016.

2. Lusis AJ. Atherosclerosis. Nature 2000;407:233–241. 11001066.

3. Wong ND. Epidemiological studies of CHD and the evolution of preventive cardiology. Nat Rev Cardiol 2014;11:276–289. 24663092.

4. Blood Pressure Lowering Treatment Trialists Collaboration. Blood pressure-lowering treatment based on cardiovascular risk: a meta-analysis of individual patient data. Lancet 2014;384:591–598. 25131978.

5. Goff DC Jr, Lloyd-Jones DM, Bennett G, Coady S, D'Agostino RB, Gibbons R, Greenland P, Lackland DT, Levy D, O'Donnell CJ, Robinson JG, Schwartz JS, Shero ST, Smith SC Jr, Sorlie P, Stone NJ, Wilson PW, Jordan HS, Nevo L, Wnek J, Anderson JL, Halperin JL, Albert NM, Bozkurt B, Brindis RG, Curtis LH, DeMets D, Hochman JS, Kovacs RJ, Ohman EM, Pressler SJ, Sellke FW, Shen WK, Smith SC Jr, Tomaselli GF. American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;129:S49–S73. 24222018.

6. Stone NJ, Robinson JG, Lichtenstein AH, Bairey Merz CN, Blum CB, Eckel RH, Goldberg AC, Gordon D, Levy D, Lloyd-Jones DM, McBride P, Schwartz JS, Shero ST, Smith SC Jr, Watson K, Wilson PW, Eddleman KM, Jarrett NM, LaBresh K, Nevo L, Wnek J, Anderson JL, Halperin JL, Albert NM, Bozkurt B, Brindis RG, Curtis LH, DeMets D, Hochman JS, Kovacs RJ, Ohman EM, Pressler SJ, Sellke FW, Shen WK, Smith SC Jr, Tomaselli GF. American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;129:S1–S45. 24222016.

7. Grundy SM, Stone NJ, Bailey AL, Beam C, Birtcher KK, Blumenthal RS, Braun LT, de Ferranti S, Faiella-Tommasino J, Forman DE, Goldberg R, Heidenreich PA, Hlatky MA, Jones DW, Lloyd-Jones D, Lopez-Pajares N, Ndumele CE, Orringer CE, Peralta CA, Saseen JJ, Smith SC Jr, Sperling L, Virani SS, Yeboah J. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA guideline on the management of blood cholesterol: a report of the American College of Cardiology/ American Heart Association Task Force on Clinical Practice Guidelines. Circulation 2019;139:e1082–e1143. 30586774.

8. Whelton PK, Carey RM, Aronow WS, Casey DE Jr, Collins KJ, Dennison Himmelfarb C, DePalma SM, Gidding S, Jamerson KA, Jones DW, MacLaughlin EJ, Muntner P, Ovbiagele B, Smith SC Jr, Spencer CC, Stafford RS, Taler SJ, Thomas RJ, Williams KA Sr, Williamson JD, Wright JT Jr. 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol 2018;71:e127–e248. 29146535.

9. DeFilippis AP, Young R, McEvoy JW, Michos ED, Sandfort V, Kronmal RA, McClelland RL, Blaha MJ. Risk score overestimation: the impact of individual cardiovascular risk factors and preventive therapies on the performance of the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score in a modern multi-ethnic cohort. Eur Heart J 2017;38:598–608. 27436865.

10. Kavousi M, Leening MJ, Nanchen D, Greenland P, Graham IM, Steyerberg EW, Ikram MA, Stricker BH, Hofman A, Franco OH. Comparison of application of the ACC/AHA guidelines, Adult Treatment Panel III guidelines, and European Society of Cardiology guidelines for cardiovascular disease prevention in a European cohort. JAMA 2014;311:1416–1423. 24681960.

11. Rana JS, Tabada GH, Solomon MD, Lo JC, Jaffe MG, Sung SH, Ballantyne CM, Go AS. Accuracy of the atherosclerotic cardiovascular risk equation in a large contemporary, multiethnic population. J Am Coll Cardiol 2016;67:2118–2130. 27151343.

12. Volgman AS, Palaniappan LS, Aggarwal NT, Gupta M, Khandelwal A, Krishnan AV, Lichtman JH, Mehta LS, Patel HN, Shah KS, Shah SH, Watson KE. American Heart Association Council on Epidemiology and Prevention. ardiovascular Disease and Stroke in Women and Special Populations Committee of the Council on Clinical Cardiology. Council on Cardiovascular and Stroke Nursing. Council on Quality of Care and Outcomes Research. Stroke Council. Atherosclerotic cardiovascular disease in South Asians in the United States: epidemiology, risk factors, and treatments: a scientific statement from the American Heart Association. Circulation 2018;138:e1–e34. 29794080.

13. Hutchinson RN, Shin S. Systematic review of health disparities for cardiovascular diseases and associated factors among American Indian and Alaska Native populations. PLoS One 2014;9e80973. 24454685.

14. The ARIC investigators. The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol 1989;129:687–702. 2646917.

15. Fried LP, Borhani NO, Enright P, Furberg CD, Gardin JM, Kronmal RA, Kuller LH, Manolio TA, Mittelmark MB, Newman A, O'Leary DH, Psaty B, Rautaharju P, Tracy RP, Weiler PG. The Cardiovascular Health Study: design and rationale. Ann Epidemiol 1991;1:263–276. 1669507.

16. Friedman GD, Cutter GR, Donahue RP, Hughes GH, Hulley SB, Jacobs DR Jr, Liu K, Savage PJ. CARDIA: study design, recruitment, and some characteristics of the examined subjects. J Clin Epidemiol 1988;41:1105–1116. 3204420.

17. Lloyd-Jones DM, Leip EP, Larson MG, D'Agostino RB, Beiser A, Wilson PW, Wolf PA, Levy D. Prediction of lifetime risk for cardiovascular disease by risk factor burden at 50 years of age. Circulation 2006;113:791–798. 16461820.

18. Pencina MJ, D'Agostino RB Sr, Larson MG, Massaro JM, Vasan RS. Predicting the 30-year risk of cardiovascular disease: the Framingham Heart Study. Circulation 2009;119:3078–3084. 19506114.

19. Kim Y, Han BG. KoGES group. Cohort profile: the Korean Genome and Epidemiology Study (KoGES) Consortium. Int J Epidemiol 2017;46e20. 27085081.

20. Suh S, Baek J, Bae JC, Kim KN, Park MK, Kim DK, Cho NH, Lee MK. Sex factors in the metabolic syndrome as a predictor of cardiovascular disease. Endocrinol Metab (Seoul) 2014;29:522–529. 25559575.

21. D'Agostino RB Sr, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, Kannel WB. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117:743–753. 18212285.

22. Dawber TR, Meadors GF, Moore FE Jr. Epidemiological approaches to heart disease: the Framingham Study. Am J Public Health Nations Health 1951;41:279–281. 14819398.

23. Kannel WB, Feinleib M, McNamara PM, Garrison RJ, Castelli WP. An investigation of coronary heart disease in families. The Framingham offspring study. Am J Epidemiol 1979;110:281–290. 474565.

24. Jung KJ, Jang Y, Oh DJ, Oh BH, Lee SH, Park SW, Seung KB, Kim HK, Yun YD, Choi SH, Sung J, Lee TY, Kim SH, Koh SB, Kim MC, Chang Kim H, Kimm H, Nam C, Park S, Jee SH. The ACC/AHA 2013 pooled cohort equations compared to a Korean Risk Prediction Model for atherosclerotic cardiovascular disease. Atherosclerosis 2015;242:367–375. 26255683.

25. Jee SH, Batty GD, Jang Y, Oh DJ, Oh BH, Lee SH, Park SW, Seung KB, Kimm H, Kim SY, Mok Y, Kim HS, Lee DC, Choi SH, Kim MJ, Lee GJ, Sung J, Cho B, Kim ES, Yu BY, Lee TY, Kim JS, Lee YJ, Oh JK, Kim SH, Park JK, Koh SB, Park SB, Lee SY, Yoo CI, Kim MC, Kim HK, Park JS, Yun YD, Baek SJ, Samet JM, Woodward M. The Korean Heart Study: rationale, objectives, protocol, and preliminary results for a new prospective cohort study of 430,920 men and women. Eur J Prev Cardiol 2014;21:1484–1492. 23864362.

26. Appelman Y, van Rijn BB, Ten Haaf ME, Boersma E, Peters SA. Sex differences in cardiovascular risk factors and disease prevention. Atherosclerosis 2015;241:211–218. 25670232.

27. Mosca L, Barrett-Connor E, Wenger NK. Sex/gender differences in cardiovascular disease prevention: what a difference a decade makes. Circulation 2011;124:2145–2154. 22064958.

28. Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med 2015;34:1659–1680. 25684707.

29. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128–138. 20010215.

30. D'Agostino RB Sr, Grundy S, Sullivan LM, Wilson P. CHD Risk Prediction Group. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA 2001;286:180–187. 11448281.

31. DeFilippis AP, Young R, Carrubba CJ, McEvoy JW, Budoff MJ, Blumenthal RS, Kronmal RA, McClelland RL, Nasir K, Blaha MJ. An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort. Ann Intern Med 2015;162:266–275. 25686167.

32. WHO Expert Consultation. Appropriate body-mass index for Asian populations and its implications for policy and intervention strategies. Lancet 2004;363:157–163. 14726171.

33. Karthikeyan G, Teo KK, Islam S, McQueen MJ, Pais P, Wang X, Sato H, Lang CC, Sitthi-Amorn C, Pandey MR, Kazmi K, Sanderson JE, Yusuf S. Lipid profile, plasma apolipoproteins, and risk of a first myocardial infarction among Asians: an analysis from the INTERHEART Study. J Am Coll Cardiol 2009;53:244–253. 19147041.

34. Thom T, Haase N, Rosamond W, Howard VJ, Rumsfeld J, Manolio T, Zheng ZJ, Flegal K, O'Donnell C, Kittner S, Lloyd-Jones D, Goff DC Jr, Hong Y, Adams R, Friday G, Furie K, Gorelick P, Kissela B, Marler J, Meigs J, Roger V, Sidney S, Sorlie P, Steinberger J, Wasserthiel-Smoller S, Wilson M, Wolf P. American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics: 2006 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation 2006;113:e85–e151. 16407573.

35. Lee SW, Kim HC, Lee HS, Suh I. Thirty-year trends in mortality from cardiovascular diseases in Korea. Korean Circ J 2015;45:202–209. 26023308.

36. Kim HJ, Kim Y, Cho Y, Jun B, Oh KW. Trends in the prevalence of major cardiovascular disease risk factors among Korean adults: results from the Korea National Health and Nutrition Examination Survey, 1998-2012. Int J Cardiol 2014;174:64–72. 24742812.

37. Garcia M, Mulvagh SL, Merz CN, Buring JE, Manson JE. Cardiovascular disease in women: clinical perspectives. Circ Res 2016;118:1273–1293. 27081110.

38. EUGenMed Cardiovascular Clinical Study Group. Regitz-Zagrosek V, Oertelt-Prigione S, Prescott E, Franconi F, Gerdts E, Foryst-Ludwig A, Maas AH, Kautzky-Willer A, Knappe-Wegner D, Kintscher U, Ladwig KH, Schenck-Gustafsson K, Stangl V. Gender in cardiovascular diseases: impact on clinical manifestations, management, and outcomes. Eur Heart J 2016;37:24–34. 26530104.

39. Yusuf S, Hawken S, Ounpuu S, Dans T, Avezum A, Lanas F, McQueen M, Budaj A, Pais P, Varigos J, Lisheng L. INTERHEART Study Investigators. Effect of potentially modifiable risk factors associated with myocardial infarction in 52 countries (the INTERHEART study): case-control study. Lancet 2004;364:937–952. 15364185.

40. Camici PG, d'Amati G, Rimoldi O. Coronary microvascular dysfunction: mechanisms and functional assessment. Nat Rev Cardiol 2015;12:48–62. 25311229.

41. Reynolds HR, Srichai MB, Iqbal SN, Slater JN, Mancini GB, Feit F, Pena-Sing I, Axel L, Attubato MJ, Yatskar L, Kalhorn RT, Wood DA, Lobach IV, Hochman JS. Mechanisms of myocardial infarction in women without angiographically obstructive coronary artery disease. Circulation 2011;124:1414–1425. 21900087.

42. von Mering GO, Arant CB, Wessel TR, McGorray SP, Bairey Merz CN, Sharaf BL, Smith KM, Olson MB, Johnson BD, Sopko G, Handberg E, Pepine CJ, Kerensky RA. National Heart, Lung, and Blood Institute. Abnormal coronary vasomotion as a prognostic indicator of cardiovascular events in women: results from the National Heart, Lung, and Blood Institute-Sponsored Women's Ischemia Syndrome Evaluation (WISE). Circulation 2004;109:722–725. 14970106.

43. de Las Heras Gala T, Geisel MH, Peters A, Thorand B, Baumert J, Lehmann N, Jockel KH, Moebus S, Erbel R, Meisinger C, Mahabadi AA, Koenig W. KORA Study Group and the Heinz Nixdorf Recall Study Group. Recalibration of the ACC/AHA risk score in two population-based German cohorts. PLoS One 2016;11e0164688. 27732641.

44. Yadlowsky S, Hayward RA, Sussman JB, McClelland RL, Min YI, Basu S. Clinical implications of revised pooled cohort equations for estimating atherosclerotic cardiovascular disease risk. Ann Intern Med 2018;169:20–29. 29868850.

45. Lee CH, Woo YC, Lam JK, Fong CH, Cheung BM, Lam KS, Tan KC. Validation of the pooled cohort equations in a long-term cohort study of Hong Kong Chinese. J Clin Lipidol 2015;9:640–646. 26350809.

46. Majed B, Tafflet M, Kee F, Haas B, Ferrieres J, Montaye M, Ruidavets JB, Arveiler D, Yarnell J, Amouyel P, Ducimetiere P, Empana JP. PRIME study group. External validation of the 2008 Framingham cardiovascular risk equation for CHD and stroke events in a European population of middle-aged men. The PRIME study. Prev Med 2013;57:49–54. 23603213.

47. Chia YC, Gray SY, Ching SM, Lim HM, Chinna K. Validation of the Framingham general cardiovascular risk score in a multiethnic Asian population: a retrospective cohort study. BMJ Open 2015;5e007324.

SUPPLEMENTARY MATERIALS

Supplementary materials related to this article can be found online at https://doi.org/10.4093/dmj.2019.0061.

Supplementary Table 1

Baseline characteristics of study participants in the KOGES-Ansan and Ansung study and ASCVD risk prediction models

dmj-44-458-s001.pdf

Supplementary Table 2

Ten-year cumulative incidence of atherosclerotic cardiovascular disease

dmj-44-458-s002.pdf

Supplementary Table 3

Regression coefficients of the KOGES-Ansan and Ansung study and the pooled cohort equations

dmj-44-458-s003.pdf

Supplementary Fig. 1

Screening and selection of the study participants. KOGES, Korean Genome and Epidemiology Study; ASCVD, atherosclerotic cardiovascular disease.

dmj-44-458-s004.pdf

Article information Continued

Korean Diabetes Association

(open-access, http://creativecommons.org/licenses/by-nc/4.0/) :

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Characteristic	KOGES	PCE-WH^a	PCE-AA^a	FRS-CVD	KRPM	P value
Men	3,778	9,098	1,647	3,969	119,715
Age, yr	51.3±8.6	56.2±6.5	55.4±5.6	48.5±10.8	50.1±7.9	<0.001
Total cholesterol, mg/dL	192±36	211±38	208±43	213±39	197±35	<0.001
HDL-C, mg/dL	44±10	44±12	51±17	45±12	48±11	<0.001
Systolic BP, mm Hg	125±17	124±16	129±20	130±18	125±18	<0.001
BP treatment, %	8.5	16.9	31.2	10.1	NA	<0.001^b
Diabetes mellitus, %	7.8	8.8	15.9	6.5	10.5	<0.001
Current smoker, %	48.1	25.5	35.5	35.2	49.4	<0.001
Women	4,154	11,240	2,641	4,522	80,295
Age, yr	52.4±9.0	56.8±6.5	55.3±5.4	49.1±11.1	51.8±8.1	<0.001
Total cholesterol, mg/dL	191±35	221±41	214±44	215±44	201±37	<0.001
HDL-C, mg/dL	46±10	58±16	59±17	58±15	54±13	<0.001
Systolic BP, mm Hg	124±20	121±18	127±20	126±20	125±20	<0.001
BP treatment, %	13.1	18.5	40.8	11.8	NA	<0.001^b
Diabetes mellitus, %	6.0	6.3	17.4	3.8	7.4	<0.001
Current smoker, %	3.4	24.9	22.7	34.2	4.5	<0.001

Variable	Predicted risk, %	Observed risk, % (n)	Absolute difference	Discordance, %^a	C-statistic	Calibration
Variable	Predicted risk, %	Observed risk, % (n)	Absolute difference	Discordance, %^a	C-statistic	χ² statistic	P value
Men (n=3,778)
PCE-WH	8.51	8.05 (305)	0.46	6	0.731	10.59	0.226
PCE-AA	9.13	8.05 (305)	1.08	13	0.730	17.13	0.029
FRS-CVD	15.34	8.05 (305)	7.29	91	0.730	177.71	<0.001
KRPM	5.55	8.05 (305)	-2.50	-31	0.735	58.39	<0.001
Women (n=4,154)
PCE-WH	3.58	7.05 (293)	-3.47	-49	0.726	258.62	<0.001
PCE-AA	5.29	7.05 (293)	-1.76	-25	0.732	107.08	<0.001
FRS-CVD	7.23	7.05 (293)	0.18	3	0.726	24.70	0.002
KRPM	4.85	7.05 (293)	-2.20	-31	0.729	67.27	<0.001