|Year : 2018 | Volume
| Issue : 4 | Page : 349-355
Using two scores for the prediction of mortality in pediatric intensive care units
Ashraf Abdelkader, Mohamed M Shaaban, Mahmoud Zahran
Department of Pediatrics, Faculty of Medicine, Al-Azhar University, Cairo, Egypt
|Date of Submission||03-Jun-2018|
|Date of Acceptance||27-Jan-2019|
|Date of Web Publication||23-Apr-2019|
Department of Pediatrics, Faculty of Medicine, Al-Azhar University, Cairo, 331
Source of Support: None, Conflict of Interest: None
Background Pediatric intensive care unit (PICU) has a specific location for the management of seriously ill children.
Aim of the work The purpose of the study was to compare two models [Pediatric Risk of Mortality III (PRISM III) and the Pediatric Index of Mortality 3 (PIM3) scores] for the prediction of mortality in PICU in KSA.
Patients and methods A prospective, cohort study was conducted and two mortality scores, PRISM III and PIM3, were applied on 68 children admitted to the PICU at As-Salama Hospital, Al Khobar, KSA over a period of 1 year from January till December 2016.
Results The mean age was 7.6±5.3 years with more men than women and the mean length of hospital stay was 9.8±7.0 days. The overall expected mortality using the PRISM III score was 6.7% whereas that by PIM3 was 7.4% and the observed mortality was 17.6%. Both tests underpredicted mortality at all probability levels. However, the degree of underprediction was less when the predicted mortality was more than 25%. Both tests showed excellent discrimination with a value of 0.94 (95% confidence interval, 0.86–1.0) with 94.1% sensitivity and 72.0% specificity; and of 0.93 (95% confidence interval, 0.87–0.99) with 82.4% sensitivity and 84.0% specificity, respectively. The Hosmer and Lemeshow goodness-of-fit test showed good calibration for PRISM III score (χ2=4.57, P=0.148) but poor calibration for PIM3 score (χ2=8.66, P=0.01).
Conclusion Both PRISM III and PIM3 scores underpredicted mortality at all probability levels. They offered good discrimination; however, the performance of the scoring system in the PICU patients was poor. PRISM III score showed good calibration while PIM3 score showed poor calibration.
Keywords: intensive care unit, Pediatric Index of Mortality 3 score, Pediatric Risk of Mortality III score
|How to cite this article:|
Abdelkader A, Shaaban MM, Zahran M. Using two scores for the prediction of mortality in pediatric intensive care units. Al-Azhar Assiut Med J 2018;16:349-55
|How to cite this URL:|
Abdelkader A, Shaaban MM, Zahran M. Using two scores for the prediction of mortality in pediatric intensive care units. Al-Azhar Assiut Med J [serial online] 2018 [cited 2019 Oct 16];16:349-55. Available from: http://www.azmj.eg.net/text.asp?2018/16/4/349/256759
| Introduction|| |
Critically ill patients are at an increased risk of death because of hemodynamic instability. They can be evaluated by measuring how much one or more physiologic variables deviates from the normal range . Pediatric intensive care unit (PICU) provides qualified care for critically ill children in order to decrease the mortality. This is achieved by intensive monitoring and treating of critically ill patients . Scoring systems for use in the PICU allow an assessment of the severity of disease and estimate mortality by providing quantitative statements regarding the severity of a disease, its prognosis, and its course .
The Pediatric Risk of Mortality III (PRISM III) score uses 17 physiologic variables and their ranges to facilitate an accurate estimation of mortality risk, control for severity of illness, and to assess quality of care .
Pediatric Index of Mortality 3 (PIM3) score was used for predicting the outcome of patients admitted to the PICU. Recent applications of PIM3 to other study populations have shown mixed results .
This study was carried out to compare the accuracy of the PRISM III and the PIM3 scores in children admitted to the PICU at As-Salama Hospital, Al Khobar, KSA within 24 h after admission and to compare their discriminative power and prediction of mortality.
| Aim of the work|| |
The aim of the study is to compare two models (PRISM III and the PIM3 scores) for prediction of mortality in PICU in KSA.
| Patients and methods|| |
Study design and setting
Data were collected prospectively as a cohort study over a period of 1 year from January till December 2016 on 68 children admitted to the PICU at As-Salama Hospital, Al Khobar, KSA, which includes 12 ICU beds equipped with a mechanical ventilator, infusion pump, and ECG/pulse oximeter monitors and receive medical and surgical patients from the age of 1 month till 18 years of age.
- Patients admitted to the PICU with medical or surgical problem with one or more system failure.
- Patients staying alive at least 12 h after admission. Readmissions to the PICU during the same hospitalization will be analyzed as separate patients because each admission presented a separate opportunity for an outcome.
- Age below 1 month or above 18 years.
- Patients who stay or die in less than 12 h in the PICU.
All cases were subjected to full history, complete physical examination, and basic investigations such as CBC, ESR, CRP, ABG, and chemistry panel. Special tests were done as needed.
PIM3 and PRISM III scores were calculated within 24 h of admission and the two scoring systems were compared.
PRISM III includes 17 variables that were separated into cardiovascular (systolic blood pressure, heart rate, temperature), neurologic (Glasgow coma score, pupillary response), respiratory (pH, PaO2, PCO2, HCO3), and chemical (serum creatinine, BUN, serum potassium, glucose, and hematologic, WBC count, platelet count, PT, PTT).
PRISM calculation is done through android software called the pediatric score (the Developer can be contacted at e-mail: email@example.com).
PIM3 includes 10 variables: systolic blood pressure (mmHg), pupillary reaction to light, PaO2 to FIO2 ratio, base excess (mmol/l), mechanical ventilation, elective admission to the ED, recovery after surgical procedure reason for PICU admission (cardiac bypass procedure performed or postoperative), and diagnosis (very high-risk diagnosis, high-risk diagnosis, and low-risk diagnosis present).
PIM3 calculation is done with an excel file from Australian & New Zealand Pediatric Intensive Care Registry (http://www.anzics.com.au/Downloads/PIM3%20Calculator.xlsx).
Ethical approval was obtained from the local ethics committees at As-Salama Hospital. An oral consent was obtained from the parents of the child prior to participation in the study with brief explanation on the objectives and benefits of the study with emphasis that personal data would be confidential and would be used for the scientific work only.
Statistical analysis was carried out using the SPSS computer package, version 21.0 (SPSS Inc., Chicago, Illinois, USA). For descriptive statistics: the mean±SD was used for quantitative variables, while the number and percentage were used for qualitative variables. χ2 test or Fisher’s exact test was used to assess the differences in frequency of qualitative variables. To assess the differences in the means of quantitative variables, independent samples t test was applied. The statistical methods were verified, assuming a significant level of P value less than 0.05.
To know how well PRISM III and PIM3 can predict mortality, the positive predictive value, negative predictive value, sensitivity, and specificity were used. The performance of the PRISM III and PIM3 scores was evaluated by assessing calibration and discrimination. The capacity of scores for discrimination between death and survival was calculated by the receiver operator characteristics (ROC) curve. Best cutoff values for the independent variables were determined by maximizing the Youden index (Se+Sp-1). Acceptable discrimination is represented by an area under the curve of 0.70–0.79, good discrimination by an area of 0.80–89, and excellent discrimination by an area of more than or equal to 0.90. For scores’ calibration, the Hosmer–Lemeshow goodness-of-fit test was used to test the agreement between observed and expected risks of mortality within standard risk categories [the standardized mortality ratio (SMR)] with 95% confidence intervals (CIs). Acceptable calibration is evidenced by a P value of more than or equal to 0.05. Values less than one imply good performance, and values greater than one imply poor performance ,.
| Results|| |
The study included 68 PICU patients meeting the inclusion criteria. Their mean age was 7.6±5.3 years ranged from 0.3 to 16.2 years, 57.4% were men and the mean length of hospital stay was 9.8±7.0 days that ranged from 1 to 22 days. The majority of patients was clinical (86.8%), about one-third (32.4%) had underlying chest disease, 42.6% required the use of mechanical ventilation, and 35.3% of them had hospital-acquired infection ([Table 1]).
Overall, 12 (17.6%) children died. The mean duration of hospital stay was significantly more among survivors (10.4±7.5 vs. 6.7±2.3; P=0.046). The variables that were found to be risk factors for mortality were mechanical ventilation (P=0.022), hospital‐acquired infection (P=0.019), and length of hospital stay (P=0.046) ([Table 2]).
|Table 2 Relation of different characteristics to actual outcome among the studied sample|
Click here to view
The total mean PRISM III and PIM3 scores were significantly lower in patients who survived (P <0.001). PRISM III score less than 3 was observed in 48 patients, all of them have survived. Five patients had a score of 3–5, of whom one (20.0%) had died. Scores of 5–10, 10–25, and more than 25 were observed in five, three, and seven patients, of whom two (40.0%), two (66.7%), and seven (100.0%) died, respectively. A PIM3 score of less than 3 was observed in 42 patients, of whom one (2.4%) had died. Eleven patients had a score of 3–5, of whom one (9.1%) had died. Scores of 5–15 and more than 15 were observed in five and 10 patients, of whom two (40.0%) and eight (80.0%) died, respectively ([Table 3]).
|Table 3 Relation of Pediatric Risk of Mortality III and Pediatric Index of Mortality 3 scores to actual outcome among the studied sample|
Click here to view
The calibration of the two models across various levels of probability of death showed that for the total studied sample, the SMRs at 95% CI using PRISM III and PIM3 scores were 2.5 (1.4–3.3) and 2.3 (1.1–3.5), respectively. The PRISM III and PIM3 scores underpredicted mortality at all probability levels. However, the degree of underprediction was less when the predicted mortality was more than 25% ([Table 4]).
|Table 4 Calibration of Pediatric Risk of Mortality III and Pediatric Index of Mortality 3 scores across various levels of probability of death|
Click here to view
The prognostic scoring performances are shown in [Table 5]. The area under the ROC curve (95% CI) for PRISM III and PIM3 scores showed excellent discrimination that yielded a value of 0.94 (95% CI, 0.86–1.0) with 94.1% sensitivity and 72.0% specificity; and of 0.93 (95% CI, 0.87–0.99) with 82.4% sensitivity and 84.0% specificity, respectively. The Hosmer and Lemeshow goodness-of-fit test showed a good calibration for PRISM III score (χ2=4.57, P=0.148) suggesting a significant prediction ([Figure 1]).
|Table 5 The prognostic scoring performances for Pediatric Risk of Mortality III and Pediatric Index of Mortality 3 scores|
Click here to view
|Figure 1 ROC curve for PRISM III score. PISM III, Pediatric Risk of Mortality III; ROC, receiver operator characteristics.|
Click here to view
| Discussion|| |
Patients admitted to the PICU often have multiorgan failure with high risk of mortality. The availability of appropriate test that predicts the mortality is imperative. Regular use of a scoring system provides an opportunity not only to predict the outcome, but also helps in improvement of the quality of life care within the limited resource .
Our results have shown a mortality of about 17.6% that was comparatively higher compared with many studies from developed countries . This can be explained by the severity of illness in PICU patients and the fact that our center is a referral center for other cities and registries of more serious disease. Contrary to our results, Choi et al.  reported a very low mortality of 2.6%. They gave explanation for this as sepsis was significantly underrepresented among their study population (2.3%).
The risk of mortality was higher with mechanical ventilation, hospital‐acquired infection, and short hospital stay. The mortality of ventilated neonates in our study was high (75%) which was comparable to the mortality reported by Hossain et al.  (70.6%) and Mathur et al.  (74%) but higher than figures of 46 and 43.3% reported by Sangeeta et al.  and Iqbal et al. , respectively and some other Western centers ,. This may be related to age differences, different methodologies, and quality of care.
In the Global Burden of Disease Study 2013, the majority of childhood deaths resulted from preventable causes and the first 24 h of hospitalization was the most vulnerable period with about one-third of mortalities occurring during this time .
In our study, age and sex showed no significant association with the outcomes (survivor/nonsurvivor). Similar results were reported by Das et al. .
Our findings declared that with increasing PRISM III score, there was an increase in mortality. PRISM III score offered a good discriminative power with the area under the ROC curve being 0.94 (95% CI, 0.86–1.0), which was close to that of Choi et al.  (0.95) and Pollack et al.  (0.90) but little higher than that obtained by Martha et al.  (0.87) and Varma et al.  (0.86) and much higher than Qureshi et al.  (0.78).
Also, there was significant correlation between PIM3 score and mortality (P<0.001) with excellent discrimination ability (area under the ROC curve 0.93, 95% CI, 0.87–0.99). In other validation studies, the area under the ROC curve for PIM3 was acceptable with values varied between 0.76 and 0.92 ,,,. This variation may be explained by the regional differences in study populations.
Overall, both scores exhibited good discriminative capacity between survivors and nonsurvivors and can be used as a tool with comparable performance for prognostic evaluation; however, they have been used to assess the risk of mortality when enrolling patients to clinical trials .
In our study, the observed mortality, in both scores, was higher than the expected mortality and prediction of mortality increased at higher scores. Late presentation to the hospital and delay in PICU admission might be the reason . In addition, different demographic characteristics of PICU population and a different pattern of disease may cause a falsely low PRISM score and underestimation of mortality .
The SMR was higher than 1, suggesting illness severity and/or poor performance of the ICU. However, SMR has several limitations as a quality measurement of ICUs as the time patterns of admission (i.e. early or late admission) can affect the physiological parameters recorded at the time of ICU admission and eventually the SMR. Variation in expected mortality according to the scoring system can also influence the SMR .
The poor performance of PRISM III and PIM3 scores in predicting mortality of pediatric patients was reported in several studies ,,.
The Hosmer–Lemeshow goodness-of-fit test showed good calibration for PRISM III score (χ2=4.57, P=0.148) but poor calibration for PIM3 score (χ2=8.66, P=0.01). Therefore, the PRISM III may be a reasonable choice for PICUs, despite its complexity. Similar good calibration for PRISM III score was reported by Gonçalves et al.  (χ2=3.82, P=0.282). In contrast to our results, Lee et al.  reported good calibration for PIM3 score (χ2=9.4, P=0.313). Leteurtre et al.  reported poor calibration for both scores (P<0.01).
Poor calibration has been attributed to various factors including variability of performance of the medical system, different case mix, disease pattern, differences in mortality rate, and failure of the scoring system equation to model the actual situation accurately ,.
Some researchers have suggested that, in the presence of good discrimination, poor calibration due to the source is correctable by using customized severity of illness models . However, Diamond  demonstrated that perfect calibration and perfect discrimination cannot coexist.
Our study has some limitations. First, the findings may not be generalizable to the entire pediatric population as it was a single-center study conducted at a tertiary hospital done in a small number of patients. Second, the scoring system was subjected to limitations as the PRISM score requires an observation period of 24 h, which represents a limitation of its use as an inclusion criterion in clinical trials.
In conclusion, both PRISM III and PIM3 scores underpredicted mortality at all probability levels. They offered good discrimination however; the performance of the scoring system in PICU patients was poor. PRISM III score showed good calibration while the PIM3 score showed poor calibration.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Teboul JL, Saugel B, Cecconi M, De Backer D, Hofer CK, Monnet X et al.
Less invasive hemodynamic monitoring in critically ill patients. Intensive Care Med 2016; 42:1350–1359.
Lemeshow S, Le Gall JR. Modelling the severity of illness of ICU patients: a systems update. J Am Med Assoc 2008; 272:1049–1055.
Filho RM, El Halal MG, Barbieri E, Trotta E, Carvalho PRA. Admission source and mortality in a pediatric intensive care unit. Indian J Crit Care Med 2012; 16:81–86.
] [Full text]
Pollack MM, Holubkov R, Funai T, Berger JT, Clark AE, Meert K et al.
Simultaneous prediction of new morbidity, mortality, and survival without new morbidity from pediatric intensive care: a new paradigm for outcomes assessment. Crit Care Med 2015; 43:1699–1709.
Lee OJ, Jung M, Kim M, Yang H, Cho J. Validation of the pediatric index of mortality 3 in a single pediatric intensive care unit in Korea. J Korean Med Sci 2017; 32:365–370.
Rapoport J, Teres D, Lemeshow S, Gehlbach S. A method for assessing the clinical performance and cost-effectiveness of intensive care units: a multicenter inception cohort study. Crit Care Med 1994; 22:1385–1391.
Nathanson BH, Higgins TL, Teres D, Copes WS, Kramer A, Stark M. A revised method to assess intensive care unit clinical performance and resource utilization. Crit Care Med 2007; 35:1853–1862.
Ghaffari J, Abbaskhanian A, Nazari Z. Mortality rate in pediatric intensive care unit (PICU): a local center experience. Int J Pediatr 2014; 13:81–88.
Wolfler A, Osello R, Gualino J, Calderini E, Vigna G, Santuz P et al.
The importance of mortality risk assessment: validation of the pediatric index of mortality 3 score. Pediatr Crit Care Med 2016; 17:251–256.
Choi KM, Ng DK, Wong SF, Kwok KL, Chow PY, Chan CH et al.
Assessment of the pediatric index of mortality (PIM) and the pediatric risk of mortality (PRISM) III score for prediction of mortality in a pediatric intensive care unit in Hong Kong. Hong Kong Med J 2005; 11:97–103.
Hossain MM, Mahfuza S, Abdullah MA, Hassan MN, Sahidullah M. Predictors of mortality in ventilated neonates in intensive care unit. Bangladesh J Child Health 2009; 33:77–82.
Mathur NB, Garg P, Mishra TK. Predictors of fatality in neonates requiring mechanical ventilation. Indian Pediatr 2005; 42:645–651.
Sangeeta ST, Rajesh KC, Anurakti S. Study of early predictors of fatality in mechanically ventilated neonates in NICU. Online J Health Allied Sci 2009; 8:3–9.
Iqbal Q, Younus M, Ahmed A, Ahmad I, Iqbal J, Charoo B et al.
Neonatal mechanical ventilation: indications and outcome. Indian J Crit Care Med 2015; 19:523–527.
] [Full text]
Slater A, Shann F, ANZICS Paediatric Study Group. The suitability of the pediatric index of mortality (PIM), PIM2, the pediatric risk of mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatr Crit Care Med 2004; 5:447–454.
van Kaam AH, Rimensberger PC, Borensztajn D, De Jaegere AP, Neovent Study Group. Ventilation practices in the neonatal intensive care unit: a cross-sectional study. J Pediatr 2010; 157:767–771.
Wang H, Naghavi M, Allen C, Barber R, Bhutta Z, GBD 2015 Mortality and Causes of Death Collaborators. Global, regional, and national life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death, 1980-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 2016; 388:1459–1544.
Das K, Kumar A, Paul P, Aggarwal K. Application of pediatric risk of mortality-III score to predict outcome in critically sick children admitted in a tertiary care pediatric unit in a developing country. Indian J Child Health 2015; 3:93–97.
Pollack M, Holubkov R, Funai T, Dean J, Berger J, Wessel D et al.
The pediatric risk of mortality score: update2015. Pediatr Crit Care Med 2016; 17:2–9.
Martha VF, Garcia PCR, Piva JP, Einloft PR, Bruno F, Rampon V. Comparison of two prognostic scores (PRISM and PIM) at a pediatric intensive care unit. J Pediatr (Rio J) 2005; 81:259–264.
Varma A, Damke S, Meshram R, Vagha J, Kher A, Vagha K. Prediction of mortality by pediatric risk of mortality (PRISM III) score in tertiary care rural hospital in India. Int J Contemp Pediatr 2017; 4:322–331.
Qureshi AU, Ali AS, Ahmad TM. Comparison of three prognostic scores (PRISM, PELOD and PIM 2) at pediatric intensive care unit under Pakistani circumstances. J Ayub Med Coll Abbottabad 2007; 19:49–53.
Straney L, Clements A, Parslow RC, Pearson G, Shann F, Alexander J et al.
Paediatric index of mortality 3: an updated model for predicting mortality in pediatric intensive care. Pediatr Crit Care Med 2013; 14:673–881.
Pearson G, Stickley J, Shann F. Calibration of the paediatric index of mortality in UK paediatric intensive care units. Arch Dis Child 2001; 84:125–128.
Vincent JL, Opal SM, Marshall JC. Ten reasons why we should NOT use severity scores as entry criteria for clinical trials or in our treatment. Crit Care Med 2010; 38:283–287.
Singhal D, Kumar N, Puliyel JM, Singh SK, Srinivas V. Prediction of mortality by application of PRISM score in intensive care unit. Indian Pediatr 2001; 38:714–719.
Hayes C, Lush RJ, Cornish JM, Foot AM, Henderson J, Jenkins I et al.
The outcome of children requiring admission to an intensive care unit following bone marrow transplantation. Br J Haematol 1998; 102:666–670.
Jacobe SJ, Hassan A, Veys P, Mok Q. Outcome of children requiring admission to an intensive care unit after bone marrow transplantation. Crit Care Med 2003; 31:1299–1305.
Tomaske M, Bosk A, Eyrich M, Bader P, Niethammer D. Risks of mortality in children admitted to the paediatric intensive care unit after haematopoietic stem cell transplantation. Br J Haematol 2003; 121:886–891.
Gonçalves JP, Severo M, Rocha C, Jardim J, Mota T, Ribeiro A. Performance of PRISM III and PELOD-2 scores in a pediatric intensive care unit. Eur J Pediatr 2015; 174:1305–1310.
Leteurtre S, Lectrec E, Wirth J, Noizet O, Magnenant E, Sadik A. Can generic mortality scores calculated 4 h after admission be used as inclusion criteria for clinical trials? Crit Care 2004; 8:185–193.
Markgraf R, Deutschinoff G, Pientka L, Scholten T. Comparison of acute physiology and chronic health evaluations II and III and simplified acute physiology score II: a prospective cohort study evaluating these methods to predict outcome in a German interdisciplinary intensive care unit. Crit Care Med 2000; 28:26–33.
Livingston BM, MacKirdy FN, Howie JC, Jones R, Norrie JD. Assessment of the performance of five intensive care scoring models within a large Scottish database. Crit Care Med 2000; 28:1820–1827.
Metnitz PG, Lang T, Vesely H, Valentin A, Le Gall JR. Ratios of observed to expected mortality are affected by differences in case mix and quality of care. Intensive Care Med 2000; 26:1466–1472.
Diamond GA. What price perfection? Calibration and discrimination of clinical prediction models. J Clin Epidemiol 1992; 45:85–89.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]