Comparison of Operational Definition of Type 2 Diabetes Mellitus Based on Data from Korean National Health Insurance Service and Korea National Health and Nutrition Examination Survey

Article information

Diabetes Metab J. 2023;47(2):201-210

Publication date (electronic) : 2023 February 8

doi : https://doi.org/10.4093/dmj.2022.0375

Jong Ha Baek^1,2, Yong-Moon Park^3,4, Kyung Do Han⁵, Min Kyong Moon⁶, Jong Han Choi⁷, Seung-Hyun Ko^,8

¹Department of Internal Medicine, Gyeongsang National University Changwon Hospital, Gyeongsang National University College of Medicine, Changwon, Korea

²Institute of Health Science, Gyeongsang National University, Jinju, Korea

³Department of Epidemiology, Fay W. Boozman College of Public Health, University of Arkansas for Medical Sciences, Little Rock, AR, USA

⁴Winthrop P. Rockefeller Cancer Institute, University of Arkansas for Medical Sciences, Little Rock, AR, USA

⁵Department of Statistics and Actuarial Science, Soongsil University, Seoul, Korea

⁶Department of Internal Medicine, Seoul Metropolitan Government Seoul National University Boramae Medical Center, Seoul National University College of Medicine, Seoul, Korea

⁷Division of Endocrinology and Metabolism, Konkuk University Medical Center, Konkuk University School of Medicine, Seoul, Korea

⁸Division of Endocrinology and Metabolism, Department of Internal Medicine, St. Vincent’s Hospital, College of Medicine, The Catholic University of Korea, Suwon, Korea

Corresponding author: Seung-Hyun Ko https://orcid.org/0000-0003-3703-1479 Division of Endocrinology and Metabolism, Department of Internal Medicine, St. Vincent’s Hospital, College of Medicine, The Catholic University of Korea, 93 Jungbu-daero, Paldal-gu, Suwon 16247, Korea E-mail: kosh@catholic.ac.kr

Received 2022 October 26; Accepted 2022 December 5.

Abstract

Graphical abstract

INTRODUCTION

The prevalence of type 2 diabetes mellitus (T2DM) has increased worldwide, and diabetes itself is closely related to an increased risk of atherosclerotic cardiovascular diseases such as myocardial infarction and ischemic stroke, as well as mortality. As a result, population-based data have been widely used in epidemiologic studies [1-3] to identify individuals with diabetes and evaluate diabetes-related comorbidities and risk factors. The population-level classification of T2DM can also provide informative data to guide and prioritize populations at the greatest risk and those most likely to benefit from interventions and treatment. However, there is a limitation in the population-based claim database (DB) because accurate diagnoses cannot be made due to limited clinical and laboratory information, despite the advantage of the vast amount of data.

In Korea, two representative population-based DBs have been used, the Korea National Health and Nutrition Examination Survey (KNHANES) DB, with a cross-sectional design, and the National Health Insurance Service (NHIS) DB, with a national claims DB cohort design [4]. The Korean NHIS, a single-payer system for all residents, covers 97.1% of Koreans (approximately 50 million individuals), and this DB could be an efficient resource for diabetes research based on the entire population [5]. These big DBs have different advantages and disadvantages, depending on their characteristics.

Clinical measures, including glycosylated hemoglobin (HbA1c) and the oral glucose tolerance test (OGTT), are the gold standards for diagnosing diabetes [6]. However, it is difficult to routinely conduct an HbA1c test or OGTT in a study involving an entire population, especially for subjects with mild hyperglycemia. Instead, an operational definition was adopted to define diabetes using claims-based data and national health examination data in the NHIS DB. Generally, T2DM can be defined as the assignment of an International Classification of Disease, 10th Revision (ICD-10) code corresponding to T2DM (E11-14), with or without accompanying prescription codes for anti-diabetic drugs, or a high fasting glucose level (≥126 mg/dL) in the health check-up DB [7]. However, different operational definition criteria for diabetes were adopted for previous studies, depending on whether the diagnosis was based only on the corresponding ICD-10 codes [8,9], the use of concomitant drugs prescription were included [10-15], or fasting glucose results were included [16,17].

Whether the accuracy of defining diabetes based on claims data using diagnostic codes (ICD-10) with or without prescription codes (anti-diabetic drug use) is consistent with actual diabetes in the real-world is unknown. The quality of data must first be evaluated for fitness for use. Previous validation studies were performed based on comparisons with self-reports, telephone-based surveys, or medical chart reviews [18]. These methods may include biases, such as recall bias and selection bias, that affect accuracy and concordance. Our study aimed to evaluate the validity and reliability of the NHIS data-based definition of T2DM by comparing it with other population-based KNHANES data as a standard reference. The overall sensitivity, specificity, positive and negative predictive value, accuracy, and agreement were analyzed. We also compared the prevalence and concordance of T2DM when the two algorithms were applied, depending on whether the prescription codes and diagnostic codes were included in the criteria. To the best of our knowledge, this was the first study to validate the operational definition of T2DM using two big, linked Korean national DBs.

METHODS

The Institutional Review Board of The Catholic University of Korea (IRB No.: VC18FESI0240) approved this study. The study was conducted in compliance with the Declaration of Helsinki. Written informed consent by the subjects was waived due to a retrospective nature of our study and anonymous and de-identified information was used for analysis.

Data sources

The Korean NHIS program is a computerized DB containing all claims data, including patient demographics, drug prescriptions, diagnostic codes for the disease coding system (ICD), insurers’ payment coverage, patients’ deductions, and claimed treatment details [7]. Among the total datasets in the NHIS DB, qualifications, claims, health check-up DB, and death information were used. We investigated whether there were fasting glucose levels in the health check-up DB and whether there were ICD-10 codes corresponding to T2DM and claimed prescription data for anti-diabetic drugs in the Korean Health Insurance Review and Assessment. All Korean citizens are encouraged to receive regular biannual or pre-employment health evaluations provided by NHIS. This regular health examination included assessments of anthropometric measures, blood pressure, social history, physical activity levels, and laboratory tests after overnight fasting, including serum glucose, total cholesterol, creatinine, liver function, and urinalysis.

KNHANES is a population-based cross-sectional survey designed to assess Koreans’ health-related behavior, health conditions, and nutritional status [19]. A retrospective sample of non-institutionalized civilians was obtained from all geographic regions in the country. In the KNHANES data, we analyzed the laboratory test results (fasting glucose and HbA1c levels) and collected responses to a questionnaire on whether the people included took anti-diabetic drugs or were diagnosed with T2DM. Among the eight phases of the KNHANES, data from the IV to VII phases (2008 to 2017) were analyzed, and adults over 40 years old were included in the study. The subjects surveyed by the KNHANES each year were matched to the first claims data in the NHIS health check-up DB.

We identified a cohort of 39,701 subjects in the KNHANES from 2008 to 2017. Subjects who had no data on glucose levels in the medical check-up DB or did not undergo blood tests in a fasting state (for more than 8 hours) were excluded (n=1,598). Among them, 14,294 subjects in the NHIS health check-up DB matched those in the KNHANES. Finally, 13,006 subjects were included in the study, excluding those missing values for age, sex, body mass index, household income, alcohol or smoking status, regular exercise, or the presence of dyslipidemia, hypertension, or chronic kidney disease (CKD) in the KNHANES data (Fig. 1).

Fig. 1.

Study diagram. KNHANES, Korea National Health and Nutrition Examination Survey; NHIS, National Health Insurance System; HbA1c, glycosylated hemoglobin; BMI, body mass index; CKD, chronic kidney disease. ^aSystolic blood pressure ≥140 mm Hg and/or diastolic pressure ≥90 mm Hg or on medication, ^bTotal cholesterol ≥240 mm Hg and/or on medication, ^cEstimated glomerular filtration rate <60 mL/min/1.73 m².

RESULTS

The prevalence of T2DM according to operational definitions by the NHIS and KNHANES

The overall prevalence of T2DM satisfying KNHANES criteria was 14.2% (n=1,843). The prevalence of T2DM in the NHIS using algorithm 1 was 14.9% (n=1,942), and using algorithm 2, it was 20.8% (n=2,707) (Table 1). When classifying T2DM using the diagnostic criteria of the NHIS (algorithm 1) or KNHANES data, the prevalence of subjects who did not meet both the NHIS and KHNANES diagnostic criteria (true negative) was 82.1% (n=10,683); 381 subjects (2.9%) only met the KNHANES diagnostic criteria (false negative), 480 subjects met (3.7%) only the NHIS criteria (false positive), and 1,462 (11.2%) met both (true positive) (Table 2). When the condition of using an anti-diabetic drug was excluded from the NHIS criteria (algorithm 2), 10,025 (77.1%) subjects did not meet either set of criteria, 274 subjects (2.1%) met only the KHNANES diagnostic criteria, 1,138 (8.7%) met only the NHIS, and 1,569 (12.1%) met both criteria (Supplementary Table 1). According to algorithm 1, the subgroup that satisfied both criteria (NHIS+/KNHANES+) was older; had a higher proportion of male gender, hypertension, and CKD; higher HbA1c levels, and lower income and education levels than the subgroup that satisfied only one set of criteria (NHIS+/KNHANES-, NHIS-/KNHANES+) or were in the non-diabetic group (NHIS-/KNHANES-) (Table 2).

Table 1.

The prevalence of type 2 diabetes mellitus according to the NHIS and KNHANES diagnostic criteria stratified by the NHIS algorithm used

Table 2.

Characteristics of the study population classified as type 2 diabetes mellitus according to the KNHANES and NHIS criteria (algorithm 1)

Concordance measures

The overall sensitivity, specificity, PPV, NPV, accuracy, and Kappa coefficient of the NHIS diagnostic criteria (algorithm 1) compared to the KNHANES criteria was 79% (95% CI, 77 to 81), 96% (95% CI, 95 to 96), 75% (95% CI, 73 to 77), 97% (95% CI, 96 to 97), 93% (95% CI, 93 to 94), and 0.73 (95% CI, 0.72 to 0.75), respectively. When algorithm 2 was adopted in the NHIS criteria, sensitivity, specificity, PPV, NPV, accuracy, and the Kappa coefficient were 85% (95% CI, 84 to 87), 90% (95% CI, 89 to 90), 58% (95% CI, 56 to 60), 97% (95% CI, 97 to 98), 89% (95% CI, 89 to 90), and 0.63 (95% CI, 0.61 to 0.64) (Fig. 2). The mean sensitivity (ranging from 73% to 83%), specificity (ranging from 93% to 97%), PPV (ranging from 67% to 82%), NPV (ranging from 94% to 98%), accuracy (ranging from 91% to 95%), and agreement (Kappa index, ranging from 0.68 to 0.78) of the NHIS definition criteria (algorithm 1) were not different by age, sex, income level, education status, and accompanied hypertension or dyslipidemia (Table 3).

Fig. 2.

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and agreement according to the different algorithms of defining type 2 diabetes mellitus in Korean National Health Insurance System data. Algorithm 1: at least one of the following criteria was met: (1) fasting glucose ≥126 mg/dL or (2) International Classification of Disease, 10th Revision (ICD-10) codes corresponding to type 2 diabetes mellitus (E11-14) with accompanying prescription codes for any anti-diabetic drugs. Algorithm 2: at least one of the following criteria was met: (1) fasting glucose ≥126 mg/dL or (2) ICD-10 codes corresponding to type 2 diabetes mellitus (E11-14).

Table 3.

Sensitivity, specificity, predictive value, accuracy, and agreement of the operational definition of type 2 diabetes mellitus based on NHIS criteria (algorithm 1) compared to KNHANES criteria as a standard reference

DISCUSSION

Overall good validity and consistency of the diagnostic criteria using NHIS data were observed, which did not differ by age, sex, socioeconomic factors, or accompanied hypertension or dyslipidemia. When two diagnostic algorithms were applied to NHIS data according to whether the diagnostic codes were accompanied by prescription codes (algorithm 1) or not (algorithm 2), the prevalence of T2DM by algorithm 1 was lower than by algorithm 2, which was similar to the prevalence using the KNHANES data. In addition, although good reliability was observed for both algorithms, specificity and accuracy tended to increase in the algorithm that included both diagnostic and prescription codes (algorithm 1).

The prevalence of T2DM in the NHIS data using algorithm 1 (adopting both diagnostic and prescription codes) was lower, around 5.9% lower than when algorithm 2 (adopting only diagnostic codes) was applied. False positives (cases identified in NHIS claims data as having T2DM that were not diagnosed with T2DM by KNHANES criteria) increased when T2DM was defined only by diagnostic codes (8.7% in algorithm 2, 3.4% in algorithm 1). The overall prevalence of T2DM identified using algorithm 1 in this study was similar to the overall prevalence published in the 2021 Korea Diabetes Fact Sheet using KNHANES data (16.7%, approximately 6.05 million people) [15]. The mean HbA1c level in the NHIS+/KNHANES-group (false positives) was 5.9% using algorithm 1 and 5.7% using algorithm 2 in the study. There may be cases in which claims were issued for a T2DM diagnosis in subjects with prediabetes or early T2DM who did not require medications. Also, even though both algorithms (whether or not prescription claims data were included) provided good agreement based on the Kappa index, higher specificity, and accuracy for defining T2DM based on the NHIS were observed when claims for diagnostic codes were present along with prescription codes. When both diagnostic codes and prescription codes were included in the criteria for defining T2DM in the NHIS dataset, it helped to distinguish between patients who were in a prediabetic or early diabetic state and those who were in overt diabetes requiring treatment.

Concordance and the consistency of the diagnostic value based on NHIS criteria (algorithm 1) were not different according to age, sex, socioeconomic factors, and accompanied hypertension or dyslipidemia. The accuracy and specificity were over 90%, and the mean Kappa index showed good reliability (ranging from 0.68 to 0.78). These trends were consistent when algorithm 2 was applied (data not shown). A previous validation study compared accuracy and consistency using self-reports or telephone surveys as a reference standard [18]. Self-reports and telephone surveys are prone to recall bias, social desirability bias, poor understanding of the survey questions, incomplete knowledge, or their accurate diagnosis information. The literature review demonstrated that participants’ sociodemographic characteristics, such as age, gender, race, setting, and socioeconomic and health status, were associated with incomplete data linkage and the potential for systematic bias in reported outcomes [27]. Otherwise, our study used KNHANES data, a population-based surveillance system, as a reference standard. The KNHANES data has the advantage of minimizing selection bias compared to a diagnosis based on an electric medical chart review or interviews because the target population of the KNHANES comprises nationally representative non-institutionalized civilians in Korea. In addition, including clinical measures (HbA1c) as one of the diagnostic criteria in the reference standard for assessing validation can help overcome the potential limitation with systemic bias. Also, data linkage between the KNHANES and NHIS compensated for the shortcoming in the claims data, which was a lack of clinical information such as disease duration or glycemic control status, by adding information about self-reported surveys and urine or blood sample measurements in the KNHANES.

Validity of national claims administrative data was also evaluated in other countries such as Japan [28], Canada [29], and the USA [18]. Based on the Japanese national claim DB, the algorithm that contains both diagnosis-related codes for diabetes and medication codes had higher specificity (mean, 99.4% vs. 91.6%) and agreement (mean Kappa index, 0.80 vs. 0.49) than the algorithm that contains only diagnosis-related codes [28]. According to healthcare administrative data from Canada, compared with electronic medical records, the algorithm with the best specificity and PPV while maintaining sensitivity above 80% was either one hospitalization or physician claim and either one prescription for drug or diabetes-specific fee code at any time [29]. Validity of physician claims data-based on ICD-9 codes in the USA demonstrated that the sensitivity ranged from 26.9% to 97.0%, specificity ranged from 94.5% to 99.4%, and the Kappa index ranged from 0.8 to 0.9 [18]. Comparing the sensitivity, specificity, and Kappa agreement to other countries, the algorithm based on Korean NHIS data also demonstrated good validity and reliability.

Several limitations to this study should be considered. First, selection bias may have occurred because two-thirds of the subjects were excluded due to missing data on fasting glucose levels in the medical check-up DB or covariates in the KNHANES data, as well as cases where the person refused to provide personal information. In addition, only subjects aged 40 years or older were included in this study because national health check-up was conducted for 40 years or older. Second, among the diagnostic criteria in the KNHANES data used as a standard reference, questionnaires were also used to classify patients with T2DM through a self-reported survey. Other laboratory tests and data, such as the OGTT or hyperglycemia-accompanied symptoms, were not present in the data used to diagnose T2DM. As a result, the KNHANES data also did not fully reflect all patients with T2DM in real-world settings. Third, defining T2DM according to claims-based data can overlook patients with untreated diabetes or those who did not require treatment. Clinical factors such as disease duration, diabetes management status, or accompanied hypertension or dyslipidemia, were not assessed through the NHIS data. Despite these limitations, validating the operational diagnosis of T2DM by linking these two big national DBs, including clinical measures (HbA1c), represents a very important and timely investigation approach for future diabetes research in Korea.

In conclusion, population-based NHIS claims data can be useful in identifying subjects with T2DM by using diagnostic and prescription codes as diagnostic criteria in epidemiologic studies. The validity and accuracy of the population-based claims data for identifying T2DM were well documented and independent of sociodemographic and metabolic risk factors.

SUPPLEMENTARY MATERIALS

Article information Continued