2016 Classification Criteria for Macrophage Activation Syndrome Complicating Systemic Juvenile Idiopathic Arthritis: A European League Against Rheumatism/American College of Rheumatology/Paediatric Rheumatology International Trials Organisation Collaborative Initiative
This article is published simultaneously in the March 2016 issue of Annals of the Rheumatic Diseases.
Abstract
Objective
To develop criteria for the classification of macrophage activation syndrome (MAS) in patients with systemic juvenile idiopathic arthritis (JIA).
Methods
A multistep process, based on a combination of expert consensus and analysis of real patient data, was conducted. A panel of 28 experts was first asked to classify 428 patient profiles as having or not having MAS, based on clinical and laboratory features at the time of disease onset. The 428 profiles comprised 161 patients with systemic JIA–associated MAS and 267 patients with a condition that could potentially be confused with MAS (active systemic JIA without evidence of MAS, or systemic infection). Next, the ability of candidate criteria to classify individual patients as having MAS or not having MAS was assessed by evaluating the agreement between the classification yielded using the criteria and the consensus classification of the experts. The final criteria were selected in a consensus conference.
Results
Experts achieved consensus on the classification of 391 of the 428 patient profiles (91.4%). A total of 982 candidate criteria were tested statistically. The 37 best-performing criteria and 8 criteria obtained from the literature were evaluated at the consensus conference. During the conference, 82% consensus among experts was reached on the final MAS classification criteria. In validation analyses, these criteria had a sensitivity of 0.73 and a specificity of 0.99. Agreement between the classification (MAS or not MAS) obtained using the criteria and the original diagnosis made by the treating physician was high (κ = 0.76).
Conclusion
We have developed a set of classification criteria for MAS complicating systemic JIA and provided preliminary evidence of its validity. Use of these criteria will potentially improve understanding of MAS in systemic JIA and enhance efforts to discover effective therapies, by ensuring appropriate patient enrollment in studies.
This criteria set has been approved by the European League Against Rheumatism (EULAR) Executive Committee and the American College of Rheumatology (ACR) Board of Directors. This signifies that the criteria set has been quantitatively validated using patient data, and it has undergone validation based on an independent data set. All EULAR/ACR-approved criteria sets are expected to undergo intermittent updates.
The ACR is an independent, professional, medical and scientific society that does not guarantee, warrant, or endorse any commercial product or service.
This article is published simultaneously in the March 2016 issue of Annals of the Rheumatic Diseases.
Introduction
Macrophage activation syndrome (MAS) is the term used to describe a potentially life-threatening complication of systemic inflammatory disorders, which occurs most commonly in systemic juvenile idiopathic arthritis (JIA) and in its adult equivalent, adult-onset Still's disease 1-4, although its occurrence in patients with other autoimmune or autoinflammatory conditions, i.e., adult- and childhood-onset systemic lupus erythematosus 5, 6, Kawasaki disease 7, 8, and periodic fever syndromes 9, 10, is being reported with increased frequency. MAS is characterized by an overwhelming inflammatory reaction due to an uncontrolled and dysfunctional immune response involving the continual activation and expansion of T lymphocytes and macrophages, which results in massive hypersecretion of proinflammatory cytokines 11, 12.
Characteristic clinical features of MAS are high, nonremitting fever, hepatosplenomegaly, generalized lymphadenopathy, central nervous system dysfunction, and hemorrhagic manifestations. Typical laboratory abnormalities include pancytopenia, increased levels of ferritin, liver enzymes, lactate dehydrogenase, triglycerides, D-dimers, and soluble interleukin-2 (IL-2) receptor α (also known as soluble CD25 [sCD25]), and decreased fibrinogen levels. A typical histopathologic feature of MAS is the accumulation of well-differentiated macrophages exhibiting hemophagocytic activity in bone marrow biopsy specimens or aspirates 13. Although the prevalence of MAS among patients with systemic JIA has been estimated to be ∼10%, recent reports suggest that subclinical MAS may occur in as many as 30-40% of patients with systemic JIA 14, 15.
MAS can result in progressive multi-organ failure and eventually a fatal outcome if unrecognized. Recent studies indicate a mortality rate of 8% 16, 17, making timely diagnosis and prompt initiation of appropriate treatment imperative. However, early recognition of MAS is often challenging, given the lack of a single pathognomonic clinical or laboratory feature. Furthermore, histopathologic features of hemophagocytosis may not be present in the initial stages 18, 19 and lack specificity for hemophagocytic syndromes 20. In addition, features of MAS may be difficult to distinguish from other conditions that may present with overlapping manifestations, such as flares of systemic JIA or systemic infections. Recently, a wide disparity in the frequency and severity of the classic clinical and laboratory features across patients has been described 16, 17.
The difficulties in making the diagnosis of MAS and its clinical heterogeneity, together with the recent advances in its treatment and in understanding of its pathophysiology and underlying genetic defects 11, 21-23, emphasize the need for accurate criteria to aid physicians in appropriately classifying patients as having MAS to facilitate enrollment into clinical studies. The recognition that the syndrome is clinically similar to hemophagocytic lymphohistiocytosis (HLH) has led some to recommend the use of the HLH-2004 diagnostic guidelines 24. An alternative approach is based on application of the preliminary diagnostic guidelines for MAS complicating systemic JIA 25. However, although both sets of guidelines have been utilized for detecting MAS in patients with systemic JIA, each has several limitations 26. The primary purpose of the international collaborative project described herein, conducted under the auspices of the European League Against Rheumatism, the American College of Rheumatology, and the Paediatric Rheumatology International Trials Organisation (PRINTO), was to develop a set of classification criteria for MAS complicating systemic JIA, based on a combination of expert consensus, available evidence from the medical literature, and analysis of real patient data.
Methods
A multistep process was used in developing the classification criteria and included the following phases: 1) a Delphi survey of international pediatric rheumatologists, aimed at identifying MAS features potentially suitable for inclusion in classification criteria 27; 2) large-scale data collection on patients with systemic JIA–associated MAS and patients with 2 other conditions that potentially could be confused with MAS; 3) a web-based procedure for ascertaining consensus among experts; 4) selection of candidate criteria through statistical analyses; 5) selection of final classification criteria in a consensus conference; and 6) cross-sectional validation of final classification criteria. Health professionals and patient/parent representatives were not included in the study task force because the project did not involve any issues of specific interest to these stakeholders. In particular, none of the study assessments required the participation of health professionals, and no patient/parent-reported outcomes were incorporated.
Data collection on patients with MAS and patients with conditions that could be confused with MAS
The design, inclusion criteria, and data collection procedures of this portion of the project have been described in detail previously 16, 17, 26. Briefly, international pediatric rheumatologists and pediatric hematologists were invited to participate in a retrospective cohort study of patients with systemic JIA–associated MAS or with 1 of 2 conditions that could potentially be confused with MAS, i.e., active systemic JIA not complicated by MAS and systemic infection.
For patients with MAS, information on laboratory features at 3 time points (the last visit before onset of MAS, the time of MAS onset, and the period of full-blown MAS) was collected. Because the classification criteria were aimed at identification of MAS in its earlier stages, only laboratory data recorded at the time of onset were retained. Data at the time of presentation in patients with conditions that could be confused with MAS were also obtained. Except for blood cell counts and acute-phase reactant levels, values of laboratory parameters were tested using both the original values provided by each local laboratory and the values standardized according to the SI unit system based on their normal ranges, as previously reported 16.
A total of 1,111 patients (362 with systemic JIA–associated MAS, 404 with active systemic JIA without MAS, and 345 with systemic infection) were reported by 95 pediatric subspecialists practicing in 33 countries in 6 continents. Pediatric subspecialists who provided these data are listed in Appendix A. The features of the patients with MAS and the comparison patients have been described elsewhere 16, 17, 26.
Web-based procedures for ascertaining consensus among experts
At present, there is no single feature that is pathognomonic for MAS. Furthermore, no prior validated diagnostic or classification criteria are available. In order to classify patients as having or not having MAS, we therefore decided to use expert consensus as the “gold standard.” Based on publication records and experience in the care of children with MAS and related disorders, a panel of 28 experts (20 pediatric rheumatologists and 8 pediatric hematologists) was created.
The experts were asked to classify a total of 428 patient profiles as having or not having MAS, based on the clinical and laboratory features recorded at disease onset. The 428 profiles were selected randomly from among the 1,111 patients whose data were collected and comprised 161 patients with MAS, 140 patients with active systemic JIA without evidence of MAS, and 127 patients with systemic infection. Selection bias was unlikely, as the characteristics of patients who were selected and those who were not selected were comparable (data not shown). The experts were intentionally kept unaware of the original diagnosis and overall course of each patient.
Each patient profile included information about the presence or absence of key clinical manifestations, and the values of laboratory parameters and normal ranges at the respective institutions. Based on these data, all experts were asked to classify each patient as having or not having MAS. The minimum required level of agreement among experts was set at 80%. If an 80% consensus was not attained, the patient profile was discussed in a further round. Profiles for which consensus had not been achieved after the final round were declared uninterpretable and discarded from further analyses. Three rounds of voting were used, with prior vote data and recorded comments available to all of the participants before each vote took place, to augment the number of consensus decisions. All web-based consensus procedures were conducted by PRINTO.
Selection of best classification criteria through statistical analyses
All statistical analyses used for selecting the best classification criteria were conducted only on the sample of patients for whom the experts achieved consensus about the diagnosis of MAS or non-MAS. Cutoff values for laboratory tests were calculated with the receiver operator characteristic (ROC) curve method, by identifying the point on the ROC curve that best discriminated between patients classified by the experts as having MAS and those classified as not having MAS.
The aim of this exercise was to assess the ability of candidate criteria to classify individual patients as having or not having MAS, and to evaluate the agreement between the classification yielded by the criteria and the consensus classification of experts. Candidate classification criteria were partly derived from the literature and partly generated from the study data.
Literature criteria included the following: 1) the preliminary diagnostic guidelines for MAS complicating systemic JIA 25, 2) the same guidelines modified by addition of the item ferritin at various threshold levels (500, 1,000 or 1,500 ng/ml), and 3) the HLH-2004 diagnostic guidelines 24, adapted by eliminating 3 of the 8 items because information about presence of hemophagocytosis was not available for both comparison groups, and neither natural killer (NK) cell activity nor sCD25 levels were determined in all patients. Criteria obtained from the study data were generated in 2 ways: 1) through the evaluation, by the project steering committee, of all combinations of clinical and laboratory variables (see some examples in Supplementary Table 1, on the Arthritis & Rheumatology web site at http://onlinelibrary.wiley.com/doi/10.1002/art.39332/abstract) (combination of criteria approach), and 2) by assigning weights to clinical and laboratory variables, on the basis of their association with the diagnosis of MAS made by the experts, through multivariable logistic regression analysis. For each combination of variables that were significantly associated with the diagnosis of MAS in logistic regression models, the rule was to convert the odds ratio of each variable to its percentage value out of a total of 100%. Each set of criteria was then composed of a group of variables whose sum of weights made up a total score of 100 (MAS score). The cutoff value in the MAS score that was associated with the higher likelihood of the presence of MAS was obtained by calculating the point on the ROC curve that corresponded to the highest sensitivity and specificity.
A total of 982 candidate classification criteria were tested. For each set of criteria, we calculated the sensitivity (ability of the criteria to identify a patient as having MAS who had been classified as having MAS according to the expert panel), the specificity (ability of the criteria to identify a patient as not having MAS who had been classified as not having MAS by the experts), the positive and negative predictive value, the area under the ROC curve (AUC), and the kappa value for agreement between the classification yielded by the criteria and the classification made by the experts. Although there was one single model with the highest predictive value (criterion no. 929; Supplementary Table 1), we generated multiple combinations for comparison because we believed that less predictive models might have more face validity with the experts. Nevertheless, it was established that in order to qualify for inclusion in expert voting procedures at the consensus conference, a set of classification criteria should demonstrate a kappa value of ≥0.85, a sensitivity of ≥0.80, a specificity of ≥0.93, and an AUC of ≥0.90. An exception was made for the historical literature criteria, which were retained for further consideration even if they did not meet all statistical requirements.
Selection of the final classification criteria at consensus conference
The International Consensus Conference on MAS Classification Criteria was held in Genoa, Italy on March 21–22, 2014. The meeting was attended by all 28 experts who participated in web-based consensus evaluations and was facilitated by 2 moderators (HIB and NR) with expertise in nominal group technique. The overall goal of the meeting was to decide on a preliminary set of classification criteria using a combination of statistical and consensus formation techniques.
A plenary session was first held to present the scope, methodology, and flow of the project, the results of the Delphi survey, the characteristics of patients included in the data collection, the results of web-based consensus procedures and of statistical analyses of candidate classification criteria, and the methodology of the nominal group technique. Participants were then randomized into 2 equal-sized nominal groups and were asked to rank using nominal group technique, independently of each other and based on the evaluation of both ease of use and credibility (face/content validity) and statistical performance (particularly in terms of sensitivity, specificity, and kappa value), the 5 best classification criteria from 5 (highest) to 1 (lowest). All experts were connected by their laptops to a central computer and submitted all of their rankings electronically. A series of repeated independent voting sessions was held until the top 3 classification criteria were selected by each voting group. Then an 80% consensus was attained on the best (final) set of classification criteria, in a session with members of the 2 nominal groups combined.
Analysis of the association between the variables included in the final classification criteria and the web-based experts’ consensus evaluations
The association between the final classification criteria and the web-based evaluations made by the experts was assessed by multiple logistic regression analysis, which used as explanatory variables the individual items included in the final classification criteria and as the dependent variable the web-based expert consensus on classification of patients as having or not having MAS. The effect was expressed in terms of odds ratios, and 95% confidence intervals were calculated; statistical significance was tested by likelihood ratio test. The AUC of the model was used as an indicator of its predictive ability. The purpose of this post-consensus analysis was to evaluate which were the variables that most influenced the experts’ decision to classify the patients as having or not having MAS.
Validation of final classification criteria
Validation analysis was performed by assessing the performance of the criteria, in terms of sensitivity, specificity, negative predictive value, positive predictive value, AUC, and kappa value, in discriminating patients with MAS from patients with the 2 conditions with which MAS could be confused (combined in a single group), using the original diagnosis made by the caring physician (i.e., the investigator who entered the patient's data in the study web site) as the gold standard. This analysis was performed on the sample of patients not used for expert evaluations (n = 683). Only patients with available data on all items included in the final classification criteria were used for the analyses.
Results
Results of web-based process for ascertainment of expert consensus
After 3 rounds of web-based evaluations, the experts achieved consensus on the classification of 391 (91.4%) of the 428 patient profiles examined (Figure 1). A total of 95 patients were classified as having MAS by the experts, 88 of whom had also been diagnosed as having MAS by the treating physician; the original diagnosis had been systemic JIA without MAS in 3 patients and systemic infection in 4. A total of 296 patients were classified by the experts as not having MAS, 47 of whom had been diagnosed as having MAS by the treating physician. Thirty-seven patient profiles for which 80% consensus among experts was not reached were discarded. A comparison of clinical and laboratory features between patients diagnosed by the experts as having MAS and those diagnosed as not having MAS is shown in Table 1. Overall, patients whose condition was classified as MAS by the experts had more severe clinical and laboratory features than those classified as not having MAS.
Patient samples evaluated in the study and results of web-based expert evaluations. MAS = macrophage activation syndrome; sJIA = systemic juvenile idiopathic arthritis.
| Patients classified as having MAS | Patients classified as not having MAS | P | |||
|---|---|---|---|---|---|
| No. with available data | No. (%) or median (IQR)a | No. with available data | No. (%) or median (IQR)a | ||
| Clinical manifestations | |||||
| Fever | 94 | 93 (98.9) | 294 | 278 (94.6) | 0.08 |
| Hepatomegaly | 94 | 68 (72.3) | 295 | 75 (25.4) | <0.0001 |
| Splenomegaly | 92 | 53 (57.6) | 294 | 67 (22.8) | <0.0001 |
| Lymphadenopathy | 91 | 48 (52.8) | 292 | 73 (25.0) | <0.0001 |
| Central nervous system involvement | 93 | 40 (43.0) | 292 | 25 (8.6) | <0.0001 |
| Hemorrhagic manifestations | 92 | 25 (27.2) | 294 | 16 (5.4) | <0.0001 |
| Heart involvement | 94 | 27 (28.7) | 294 | 34 (11.6) | <0.0001 |
| Lung involvement | 95 | 27 (28.4) | 294 | 40 (13.6) | 0.0009 |
| Kidney involvement | 95 | 16 (16.8) | 295 | 16 (5.4) | 0.0004 |
| Laboratory results | |||||
| Hemoglobin, gm/dl | 95 | 9.9 (8.0–11.2) | 289 | 10.9 (9.4–12.2) | <0.0001 |
| White blood cell count, ×109/liter | 95 | 8.1 (3.2–12.8) | 289 | 15.3 (9.9–20.1) | <0.0001 |
| Neutrophil count, ×109/liter | 82 | 3.7 (1.5–8.0) | 236 | 9.4 (5.1–14.2) | <0.0001 |
| Platelet count, ×109/liter | 95 | 98 (57–141) | 290 | 385 (286–551) | <0.0001 |
| Erythrocyte sedimentation rate, mm/hour | 90 | 28 (17–65) | 245 | 70 (39–93) | <0.0001 |
| C-reactive protein, mg/dl | 85 | 8.7 (2.4–16.1) | 282 | 8.2 (2.4–15.6) | 0.63 |
| Aspartate aminotransferase, units/liter | 93 | 171 (98–436) | 284 | 30 (22–45) | <0.0001 |
| Alanine aminotransferase, units/liter | 91 | 115 (43–283) | 284 | 18 12-34 | <0.0001 |
| Lactate dehydrogenase, units/liter | 81 | 1,560 (801–2,400) | 248 | 482 (362–688) | <0.0001 |
| Triglycerides, mg/dl | 86 | 267 (192–358) | 186 | 123 (96–160) | <0.0001 |
| Albumin, gm/dl | 79 | 3.0 (2.6–3.5) | 252 | 3.7 (3.2–4.1) | <0.0001 |
| Serum sodium, mEq/liter | 80 | 136 (133–140) | 259 | 138 (136–141) | 0.003 |
| Fibrinogen, mg/dl | 88 | 220 (148–345) | 226 | 500 (356–650) | <0.0001 |
| Ferritin, ng/ml | 90 | 9,094 (2,000–19,767) | 244 | 268 (62–938) | <0.0001 |
| D-dimer, ng/ml | 48 | 3,579 (1,834–7,373) | 94 | 1,638 (528–3,325) | <0.0001 |
- a Clinical manifestations are reported as the number (%) and laboratory results as the median (interquartile range [IQR]).
Candidate clinical and laboratory variables determined by univariate analysis
In univariate analyses, the 10 variables found to have the greatest ability to distinguish patients with MAS from comparison patients were as follows: ferritin level, platelet count, levels of aspartate transaminase (AST), lactate dehydrogenase, triglycerides, alanine transaminase (ALT), and fibrinogen, central nervous system involvement, hepatomegaly, and hemorrhagic manifestations (Table 2). These variables qualified for inclusion in logistic regression analyses aimed at generating candidate classification criteria through the MAS score method described above. However, ALT levels were excluded owing to their close correlation with AST levels and a slightly lower statistical performance. They were replaced by neutrophil count or albumin level, depending on the model.
| No. of patients with available data | OR (95% CI)a | P | |
|---|---|---|---|
| Clinical features | |||
| Central nervous system involvement | 385 | 8.1 (4.5–14.4) | <0.0001 |
| Hepatomegaly | 389 | 7.7 (4.5–12.9) | <0.0001 |
| Hemorrhagic manifestations | 386 | 6.5 (3.3–12.8) | <0.0001 |
| Fever | 388 | 5.4 (0.7–40.9) | 0.106 |
| Splenomegaly | 386 | 4.6 (2.8–7.6) | <0.0001 |
| Lymphoadenopathy | 383 | 3.4 (2.1–5.5) | <0.0001 |
| Kidney involvement | 390 | 3.5 (1.7–7.4) | 0.0008 |
| Heart involvement | 388 | 3.1 (1.7–5.5) | 0.0001 |
| Lung involvement | 389 | 2.5 (1.4–4.4) | 0.0011 |
| Active arthritis | 390 | 1.6 (1.0–2.6) | 0.0587 |
| Laboratory features | |||
| Non-normalized values | |||
| Ferritin >684 ng/ml | 334 | 111.6 (26.7–465.8) | <0.0001 |
| Platelet count ≤181 × 109/liter | 385 | 84.3 (40–177.5) | <0.0001 |
| Aspartate aminotransferase >48 units/liter | 377 | 51.9 (21.7–124.4) | <0.0001 |
| Lactate dehydrogenase >853 units/liter | 329 | 20.0 (10.7–37.3) | <0.0001 |
| Triglycerides >156 mg/dl | 272 | 19.6 (9.4–40.8) | <0.0001 |
| Alanine aminotransferase >36 units/liter | 375 | 18.0 (9.4–34.3) | <0.0001 |
| Fibrinogen ≤360 mg/dl | 314 | 11.5 (6.3–21.0) | <0.0001 |
| Neutrophil count ≤3.7 × 109/liter | 318 | 5.9 (3.4–10.5) | <0.0001 |
| D-dimer >1,350 ng/ml | 142 | 5.7 (2.4–13.4) | <0.0001 |
| Erythrocyte sedimentation rate ≤30 mm/hour | 335 | 5.6 (3.3–9.5) | <0.0001 |
| Albumin ≤3.6 gm/dl | 331 | 5.5 (3.0–10.1) | <0.0001 |
| Hemoglobin ≤8.5 gm/dl | 384 | 4.9 (2.7–8.8) | <0.0001 |
| White blood cell count ≤10.2 × 109/liter | 384 | 4.6 (2.8–7.5) | <0.0001 |
| Serum sodium ≤133, mEq/liter | 339 | 2.9 (1.7–5.0) | 0.0002 |
| C-reactive protein >0.86 mg/dl | 367 | 2.4 (1.0–5.9) | 0.0501 |
| Normalized values | |||
| Ferritin >2,773.6 ng/ml | 334 | 23.9 (12.7–44.8) | <0.0001 |
| Aspartate aminotransferase >44.4 units/liter | 377 | 47.4 (21.6–103.9) | <0.0001 |
| Alanine aminotransferase >23 units/liter | 375 | 39.4 (14.1–110.5) | <0.0001 |
| Lactate dehydrogenase >238.8 units/liter | 329 | 39.2 (18.2–84.6) | <0.0001 |
| Triglycerides >193.9 mg/dl | 272 | 16.3 (8.7–30.8) | <0.0001 |
| Fibrinogen ≤318.9 mg/dl | 314 | 11.0 (6.1–20.0) | <0.0001 |
| Albumin ≤3.9 gm/dl | 331 | 6.4 (3.6–11.3) | <0.0001 |
| D-dimer >1,320.3 ng/ml | 142 | 5.2 (2.1–12.7) | <0.0001 |
| Serum sodium ≤135 mEq/liter | 339 | 3.3 (1.9–5.6) | <0.0001 |
- a OR = odds ratio; 95% CI = 95% confidence interval.
Candidate classification criteria sets
Of the 982 sets of criteria tested, 45 were retained for further evaluation at the consensus conference. Of these, 37 (20 generated through the combination of criteria approach and 17 obtained with the MAS score method) were represented by criteria that met the statistical requirements described in Methods, and 8 were criteria derived from the literature. The statistical performance of the 20 best candidate criteria is presented in Supplementary Table 1 (on the Arthritis & Rheumatology web site at http://onlinelibrary.wiley.com/doi/10.1002/art.39332/abstract). The definition of and statistics on all 982 criteria tested are available from the corresponding author upon request.
Final classification criteria selected at face-to-face conference
During the consensus conference, 7 voting sessions were held among the 28 experts until 3 top classification criteria sets remained (criteria nos. 466, 472, and 929; Supplementary Table 1). Notably, the same criteria were selected independently by the 2 nominal groups, confirming convergent validity of the selection process. After the last voting session, 82% consensus was reached on the final definition (criteria no. 472; Supplementary Table 1). A subsequent open face-to-face discussion among all experts led to the decision to include in the final definition the presence of fever as a mandatory criterion and the requirement that the patient should have known or suspected systemic JIA. The final classification criteria selected at the consensus conference are presented in Figure 2.
Criteria for the classification of macrophage activation syndrome in patients with systemic juvenile idiopathic arthritis. Laboratory abnormalities should not be otherwise explained by the patient's condition, such as concomitant immune-mediated thrombocytopenia, infectious hepatitis, visceral leishmaniasis, or familial hyperlipidemia.
Association between final classification criteria and experts’ web-based consensus evaluations
For this multivariable analysis, complete data were available on 227 patients. The logistic regression model to evaluate which were the variables included in the final classification criteria that most influenced the experts’ decision to classify a patient as having or not having MAS is presented in Table 3. All variables were independently correlated with the experts’ diagnoses. However, the association was much stronger for ferritin and platelet count. The AUC of the model was 0.99.
| Explanatory variable | OR (95% CI) | Pb |
|---|---|---|
| Ferritin >684 ng/ml | >999.999 (40.6−>999.999) | <0.0001 |
| Platelet count ≤181 × 109/liter | 237.0 (18.2−>999.999) | <0.0001 |
| Triglycerides >156 mg/dl | 18.3 (2.0–163.9) | 0.009 |
| Fibrinogen ≤360 mg/dl | 16.0 (2.0–126.4) | 0.009 |
| Aspartate aminotransferase >48 units/liter | 10.0 (1.3–78.4) | 0.029 |
- a The experts’ consensus on patients’ classification as having or not having macrophage activation syndrome (from the web-based process) was the dependent variable. Complete data were available on 227 patients. The area under the receiver operating characteristic curve of the model was 0.99. OR = odds ratio; 95% CI = 95% confidence interval.
- b By likelihood ratio test.
Preliminary validation of the final criteria set
The evaluation of the ability of the new classification criteria to discriminate MAS from comparator conditions in the patient sample not included in the expert evaluations (n = 415 of 683) showed a sensitivity of 0.73, a specificity of 0.99, a positive predictive value of 97.4%, a negative predictive value of 85.9%, an AUC of 0.86, and a kappa value for agreement between the diagnosis yielded by the criteria and the diagnosis made by the treating physician of 0.76.
Discussion
Using a consensus process as well as a statistical approach, we have developed a new set of classification criteria for MAS complicating systemic JIA (Figure 2). Because the criteria were established against comparison samples composed of patients with either a rheumatologic condition (active systemic JIA without evidence of MAS) or a nonrheumatologic condition (systemic infection), they may be of interest to a specialists in a range of disciplines.
The classification criteria include only laboratory variables and no clinical manifestations, with the exception of fever. This choice is in accordance with the common view that suspicion of MAS is most commonly raised by detection of subtle laboratory alterations, whereas clinical symptoms are often delayed and/or similar to those observed in other conditions 28. In a previous analysis of the MAS sample included in the present study, we found that 4 of the 5 laboratory tests that are part of the criteria (ferritin, platelet count, AST, and triglycerides) were among the parameters that showed a change of >50% between the last visit before the onset of MAS and the onset of MAS 16. In the same study, ferritin levels exhibited the largest change over time, which underscores the major importance of this feature in MAS detection and supports its use as a mandatory criterion. The key value of ferritin in the classification of MAS was corroborated by the observation that it was the parameter that had the greatest influence on the experts’ classification of patients as having or not having MAS (Table 3).
Although the presence of fever did not enable discrimination between MAS and comparator illnesses as it was found in all or nearly all patients in each sample, the expert panel considered fever a prerequisite for the classification of MAS. The cardinal diagnostic role of fever is substantiated by the observation that it was the mostly highly ranked clinical feature identified in the Delphi survey 27. Unfortunately, we lacked reliable information on the pattern of fever in the 3 patient groups. However, it is generally accepted that the onset of MAS is heralded by a shift from the high-spiking intermittent pattern typical of active systemic JIA to a continuous nonremitting pattern 3, 4, 29.
The detection of macrophage hemophagocytosis in bone marrow biopsy specimens or aspirates or reticuloendothelial organ biopsy specimens is another frequent and characteristic feature of MAS. However, because hemophagocytosis is often absent during the early stages of MAS 18, 19 and its demonstration requires an invasive procedure, the expert panel deemed it not necessary for the classification of systemic JIA–associated MAS. Notably, the demonstration of hemophagocytosis is not mandatory in either the HLH-2004 or the preliminary MAS diagnostic guidelines 24, 25.
Although possibly useful for diagnostic purposes, the classification criteria are primarily intended for use in clinical trials and research studies. The criteria exhibited high accuracy and face/content validity in consensus and statistical evaluations, but it should be taken into account that they were developed using expert consensus as the gold standard. It also should be noted that the experts were asked to differentiate MAS from non-MAS conditions by reviewing the clinical features and laboratory values recorded at a single point in time (i.e., at disease onset), and were unaware of the patient's clinical course, laboratory values over time, response to treatment, or outcome. This information was, however, available to the treating physician, who made the original diagnosis in the clinical setting. This disparity in the available information may partially explain the high proportion of patients who were diagnosed by the treating physician as having MAS but classified by the experts as not having MAS (47 of 161; 29.2%). It is conceivable that because the experts were provided with only the information relevant to the development of the criteria, they tended to confirm the diagnosis of MAS only in straightforward and unambiguous cases.
It is therefore important to emphasize that the classification criteria may not capture all instances of MAS seen in the routine clinical setting, particularly those with subtle onset or incomplete clinical expression. Notably, of the 47 patients for whom the treating physician's diagnosis of MAS was not confirmed by the experts, 30 (63.8%) also did not meet the final classification criteria, 11 (23.4%) could not be assessed due to lack of data on the laboratory variables needed to apply the criteria, and only 6 (12.8%) were classified as having MAS according to the final classification criteria. In addition, only 18 (46.6%) of the 37 patients for whom the experts could not agree on the diagnosis were classified as having MAS according to the final classification criteria. These findings underscore the consistency of the experts’ evaluations and support the validity of the final classification criteria.
The fact that the cutoff values for platelet count and fibrinogen level included in the criteria are within the normal range of routine laboratory assessments may be regarded as clinically implausible. The same may apply to the cutoffs for AST and triglycerides, which are only slightly above the upper limits of normal. However, it is widely recognized that children with active systemic JIA often have increased platelet counts (e.g., >600 − 800 × 109/liter) as well as elevated fibrinogen levels (e.g., >500–600 mg/dl) as part of the underlying inflammatory process 30, 31. Thus, a paradoxically normal platelet count or fibrinogen level in the setting of otherwise prominent systemic inflammation may raise the suspicion of MAS 16, 28. Because the levels of serum transaminases and triglycerides are generally normal in children with systemic JIA who do not have other coexistent pathologic conditions (e.g., infectious hepatitis or familial hyperlipidemia), their simple increase above the upper normal limits, combined with the other clinical and laboratory parameters included in the criteria, may be sufficient to herald the occurrence of MAS. This fits with the real-world patient data used in these studies to establish cutoff values that distinguish between children with JIA who have MAS and those who do not have MAS.
A secondary objective of the present project was analysis of the role of change in laboratory findings over time in the detection of MAS. However, this exercise was performed only for descriptive purposes, i.e., to identify and rank the laboratory parameters for which change over time was deemed by the experts as most important or useful for early detection of MAS. Because serial laboratory results were available for patients with MAS but not for the comparator groups, we could not establish the threshold level of change in each parameter that had the greatest sensitivity and specificity for the diagnosis of MAS. This precluded the ability to incorporate the change in laboratory values over time in the classification criteria. Due to space constraints, this analysis is reported in a separate manuscript 32.
Recently there have been several reports, from randomized controlled clinical trials and from postmarketing experience, of MAS occurring in patients with systemic JIA being treated with the cytokine blockers canakinumab and tocilizumab 33-35. Because these agents inhibit the biologic effects of IL-1 and IL-6, respectively, which are among the proinflammatory cytokines involved in the physiopathology of MAS 11, 36, it is conceivable that MAS episodes developing during treatment with these biologic agents may occur in the absence of fever or some of the typical laboratory abnormalities of the syndrome. Clinical symptoms in patients with systemic JIA–associated MAS receiving tocilizumab were found to be milder than those in patients not receiving this treatment 37. Preliminary analyses in patients who developed MAS while receiving tocilizumab or canakinumab have shown that a few cases did not meet the new criteria, due to the absence of fever or a peak ferritin level of <684 ng/ml 38, 39. More data from real-world clinical practice are needed to establish whether the criteria should be refined to increase their power to identify MAS occurring during treatment with IL-1 and IL-6 inhibitors.
Our study should be interpreted in light of some potential caveats. Patient data were collected through retrospective review of clinical charts, and retrospective analysis is subject to missing and possibly erroneous data. However, because all patient profiles were reviewed by the experts and the diagnosis of MAS or non-MAS was confirmed only when a high level of consensus was reached, the impact of this potential limitation was likely minimized. Some important diagnostic parameters of MAS, such as sCD25 and sCD163 levels and NK cell activity, could not be assessed due to their unavailability in some of the patient samples. However, these biomarkers are not routinely assessed, nor are they timely, in most pediatric rheumatology centers.
In summary, we have developed a set of classification criteria for MAS complicating systemic JIA and provided preliminary evidence of their validity. These criteria will help standardize the design and conduct of future clinical trials and research studies and contribute to enhancing knowledge and awareness of the syndrome.
ACKNOWLEDGMENTS
The authors thank the PRINTO employees, Simona Angioloni, Chiara Pallotti, Michele Pesce, Mariangela Rinaldi, and Luca Villa, for technical assistance during the study and for technical and secretarial assistance in the organization of the consensus conference.
AUTHOR CONTRIBUTIONS
All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. Ravelli had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study conception and design
Ravelli, Minoia, Davì, Horne, Martini, Ruperto, Cron.
Acquisition of data
Ravelli, Minoia, Davì, Horne, Aricò, Avcin, Behrens, De Benedetti, Filipovic, Grom, Henter, Ilowite, Jordan, Khubchandani, Kitoh, Lehmberg, Lovell, Miettunen, Nichols, Ozen, Pachlopnik Schmid, Ramanan, Russo, Schneider, Sterba, Uziel, Wallace, Wouters, Wulffraat, Demirkaya, Brunner, Martini, Ruperto, Cron.
Analysis and interpretation of data
Ravelli, Minoia, Horne, Bovis, Pistorio, Brunner, Ruperto, Cron.
APPENDIX A: PARTICIPATING PHYSICIANS
The following physicians contributed patient data for use in the study: Mario Abinun, MD (Newcastle, UK); Amita Aggarwal, MD (Lucknow, India); Jonathan Akikusa, MD (Melbourne, Victoria, Australia); Sulaiman M. Al-Mayouf, MD (Riyadh, Saudi Arabia); Maria Alessio, MD (Naples, Italy); Jordi Anton, MD (Barcelona, Spain); Maria Teresa Apaz, MD (Cordoba, Argentina); Itziar Astigarraga, MD (Bilbao, Spain); Nuray A. Ayaz, MD (Istanbul, Turkey); Patrizia Barone, MD (Catania, Italy); Blanca Bica, MD (Rio de Janeiro, Brazil); Isabel Bolt, MD (Berne, Switzerland); Luciana Breda, MD (Chieti, Italy); Vyacheslav Chasnyk, MD (Saint Petersburg, Russian Federation); Rolando Cimaz, MD (Florence, Italy); Fabrizia Corona, MD (Milan, Italy); Ruben Cuttica, MD (Buenos Aires, Argentina); Gianfranco D'Angelo, MD (Ancona, Italy); Zane Davidsone, MD (Riga, Latvia); Carmen De Cunto, MD (Buenos Aires, Argentina); Jaime De Inocencio, MD (Madrid, Spain); Eli Eisenstein, MD (Jerusalem, Israel); Sandra Enciso, MD (Mexico City, Mexico); Graciela Espada, MD (Buenos Aires, Argentina); Michel Fischbach, MD (Munster, Germany); Michael Frosch, MD (Munster, Germany); Romina Gallizzi, MD (Messina, Italy); Maria Luz Gamir, MD (Madrid, Spain); Yi-Jin Gao, MD (Shangai, China); Thomas Griffin, MD (Charlotte, NC); Soad Hashad, MD (Tripoli, Libya); Teresa Hennon, MD (Buffalo, NY); Gerd Horneff, MD (Sankt Augustin, Germany); Zeng Huasong, MD (Guangzhou, China); Adam Huber, MD (Halifax, Nova Scotia, Canada); Norman Ilowite, MD (New York, NY); Antonella Insalaco, MD (Rome, Italy); Maka Ioseliani, MD (Tbilisi, Georgia); Marijia Jelusic-Drazic, MD (Zagreb, Croatia); Michael Jeng, MD (Stanford, CA); Agneza Kapovic, MD (Zagreb, Croatia); Ozgur Kasapcopur, MD (Istanbul, Turkey); Toshiyuki Kitoh, MD (Nagakute, Japan); Isabelle Kone-Paut, MD (Paris, France); Sheila Knupp Feitosa de Oliveira, MD (Rio de Janeiro, Brazil); Bianca Lattanzi, MD (Ancona, Italy); Loredana Lepore, MD (Trieste, Italy); Caifeng Li, MD (Beijing, China); Jeffrey M. Lipton, MD (New York, NY); Silvia Magni-Manzoni, MD (Rome, Italy); Despoina Maritsi, MD (Athens, Greece); Deborah McCurdy, MD (Orange, CA); Rosa Merino, MD (Madrid, Spain); Velma Mulaosmanovic, MD (Sarajevo, Bosnia and Herzegovina); Susan Nielsen, MD (Copenhagen, Denmark); Priyankar Pal, MD (Kolkata, India); Sampath Prahalad, MD (Atlanta, GA); Donato Rigante, MD (Rome, Italy); Ingrida Rumba-Rozenfelde, MD (Riga, Latvia); Claudia Saad Magalhaes, MD (Botucatu, Brazil); Helga Sanner, MD (Oslo, Norway); Sujata Sawhney, MD (New Delhi, India); Wafaa M. Sewairi, MD (Riyadh, Saudi Arabia); Bita Shakoory, MD (Philadelphia, PA); Susan Shenoi, MD (Seattle, WA); Artur Silva Clovis, MD (Sao Paulo, Brazil); Valda Stanevicha, MD (Riga, Latvia); Kimo C. Stine, MD (Little Rock, AR); Gordana Susic, MD (Belgrade, Serbia); Flavio Sztajnbok, MD (Rio de Janeiro, Brazil); Syuji Takei, MD (Kagoshima City, Japan); Hasan Tezer, MD (Ankara, Turkey); Ralf Trauzeddel, MD (Berlin, Germany); Elena Tsitsami, MD (Athens, Greece); Erbil Unsal, MD (Izmir, Turkey); Olga Vougiouka, MD (Athens, Greece); Lehn K. Weaver, MD (Philadelphia, PA); Jennifer Weiss, MD (Hackensack, NJ); Sheila Weitzman, MD (Toronto, Ontario, Canada); Mabruka Zletni, MD (Tripoli, Libya).
