Development of Machine-Learning Models to Predict Ambulation Outcomes Following Spinal Metastasis Surgery

Article information

Asian Spine J. 2023;17(6):1013-1023
Publication date (electronic) : 2023 December 5
doi : https://doi.org/10.31616/asj.2023.0051
1Department of Orthopaedic Surgery, Faculty of Medicine Vajira Hospital, Navamindradhiraj University, Bangkok, Thailand
2Department of Orthopaedic Surgery, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
3Siriraj Informatics and Data Innovation Center, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
Corresponding author: Panya Luksanapruksa, Division of Spine Surgery, Department of Orthopaedic Surgery, Faculty of Medicine Siriraj Hospital, Mahidol University, 2 Wanglang Road, Bangkoknoi, Bangkok 10700, Thailand, Tel: +66-2-419-7969, Fax: +66-2-419-7961, E-mail: panya.luk@mahidol.ac.th
Received 2023 February 13; Revised 2023 June 30; Accepted 2023 July 10.

Abstract

Study Design

Retrospective cohort study.

Purpose

This study aimed to develop machine-learning algorithms to predict ambulation outcomes following surgery for spinal metastasis.

Overview of Literature

Postoperative ambulation status following spinal metastasis surgery is currently difficult to predict. The improved ability to predict this important postoperative outcome would facilitate management decision-making and help in determining realistic treatment goals.

Methods

This retrospective study included patients who underwent spinal metastasis at a university-based medical center in Thailand between January 2009 and November 2021. Collected data included preoperative parameters and ambulatory status 90 and 180 days following surgery. Thirteen machine-learning algorithms, namely, artificial neural network, logistic regression, CatBoost classifier, linear discriminant analysis, extreme gradient boosting, extra trees classifier, random forest classifier, gradient boosting classifier, light gradient boosting machine, naïve Bayes, K-neighbor classifier, Ada boost classifier, and decision tree classifier were developed to predict ambulatory status 90 and 180 days following surgery. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC) and F1-score.

Results

In total, 167 patients were enrolled. The number of patients classified as ambulatory 90 and 180 days following surgery was 140 (81.9%) and 137 (82.0%), respectively. The extreme gradient boosting algorithm was found to most accurately predict 180-day ambulatory outcome (AUC, 0.85; F1-score, 0.90), and the decision tree algorithm most accurately predicted 90-day ambulatory outcome (AUC, 0.94; F1-score, 0.88).

Conclusions

Machine-learning algorithms were effective in predicting ambulatory status following surgery for spinal metastasis. Based on our data, the extreme gradient boosting and decision tree best predicted postoperative ambulatory status 180 and 90 days after spinal metastasis surgery, respectively.

Introduction

The incidence of spinal metastasis is increasing as evidenced by recent studies that have reported that spinal metastasis occurs in 5%–10% of all patients with cancer [13]. To determine treatment options, several factors must be considered, such as disease factors, patient factors, and patient expectations. Considering all these factors, establishing clear and realistic treatment goals is important [4,5].

Treatments for spinal metastasis have rapidly improved to maximize survival and clinical outcomes [6]. However, despite advancements in treatment, some patients continue to have poor clinical outcomes and are unable to ambulate following spinal metastasis surgery [710]. A previous study proposed models for predicting ambulatory ability following spinal metastasis surgery, which were developed using conventional statistical methods; however, those models yielded only fair to moderate performance [11].

To yield improved benefits from vast amounts of exponentially generated data, artificial intelligence and machine learning (ML) were recently employed to develop new tools to improve spine treatment and research [12,13]. Several applications using ML in spine surgery were reported with promising results that outperformed conventional statistical methods [1417].

Postoperative ambulation status following spinal metastasis surgery is difficult to predict, and improved ability to predict this important postoperative outcome would facilitate management decision-making and help in determining clear and realistic treatment goals. Accordingly, this study aimed to develop ML algorithms to predict ambulation outcomes following surgery for spinal metastasis.

Materials and Methods

1. Guidelines

This study followed the “Transparent reporting of a multivariable prediction models for individual prognosis or diagnosis” guidelines and the “Guidelines for developing and reporting machine learning models in biomedical research.” All methods were performed in accordance with the relevant guidelines, regulations, and Declaration of Helsinki. The study protocol was approved by the Institutional Review Board of Siriraj Hospital (COA no., 978/2021, 937/2564 [IRB1]).

2. Patient selection

Consecutive patients who underwent surgery for spinal metastasis at a university-based medical center in Thailand between January 2009 and November 2021 were retrospectively enrolled. The inclusion criteria were as follows: (1) diagnosis of spinal metastasis, (2) age ≥18 years, and (3) history of surgery for cervical, thoracic, lumbar, and/or sacral metastasis/metastases. Patients who expired before 180 days following surgery or who had no records of their ambulatory status 180 days following surgery were excluded. Patients who could not ambulate because of causes other than myelopathy, such as intractable pain, general muscle weakness, or extraspinal problems, were also excluded from the study. Written informed consent was waived by the Siriraj Institutional Review Board because of the retrospective nature of this study.

3. Variables

Preoperative parameters were collected through a retrospective chart review. Factors that were previously reported to be significantly associated with ambulatory status following spinal metastasis surgery were collected, including age, sex, body mass index (BMI) (kg/m2), smoking status, American Society of Anesthesiologists classification, presence of myelopathy before surgery, duration of neurological deficit, Frankel grading, level of spinal compression, level of spinal metastasis, comorbidities, extraspinal bone metastasis, visceral metastasis, preoperative treatment (chemotherapy, radiotherapy, and targeted therapy), primary tumor origin, serum calcium level, albumin level, creatinine level, and preoperative ambulatory status [15,1822]. Primary tumor histology was also included to fully and clearly describe the primary tumor.

4. Outcomes

A study reported that functional recovery reached the plateau phase 6 months following spinal metastasis surgery [23]. Therefore, ambulatory status 180 days following surgery was selected as the primary outcome, and ambulatory status 90 days following surgery as the secondary outcome. Ambulatory status as “ambulator” was defined as patients who can walk (with or without a gait aid). Conversely, patients who could not walk were classified as “non-ambulators.” Patients were allocated to ambulatory and non-ambulatory groups according to their records when applicable. To blind the assessment of predictors from the results, predictors were separately reviewed from the outcomes by two orthopedic surgeons.

5. Preprocessing

Missing data were cleaned by eliminating patients who had no primary or secondary outcome data. In cases where preoperative data are unavailable, multiple imputations with chained equations were utilized.

To reduce the influence of different variable units and quantity levels, scale numerical variables were used to a standard deviation of 1 and a mean of 0, and dummy encoding was employed for categorical variables. Outliers whose laboratory values are three standard deviations from the average laboratory value at our hospital were removed.

6. Prediction models

The ML models included in this study were used in a previous study to evaluate survival among patients with metastatic disease [24]. To identify the best-performing model for both the primary and secondary outcomes, the performance of all the included ML models was compared.

Thirteen ML models were included in this study, namely, artificial neural network, logistic regression, CatBoost classifier, linear discriminant analysis, extreme gradient boosting, extra trees classifier, random forest classifier, gradient boosting classifier, light gradient boosting machine, naïve Bayes, K-neighbor classifier, Ada boost classifier, and decision tree classifier. All models were created with Python ver. 3.9 (https://www.python.org/) using Scikit-learn library ver. 1.0.1 (https://scikit-learn.org/stable/) under an open-source simplified BSD (Berkeley Software Distribution) license [25]. Grid search was used for hyperparameter tuning of each model with a random state equal to 1,337, and regularization techniques such as L2 regularization were used. For the neural network, Pytorch ver. 1.10 (https://pytorch.org/) was used in model development. After experimenting with various multilayer perceptrons, the optimal configuration was selected for comparison with other ML models. The size of the hidden layer was 10. The ReLU was selected as the activation function. The Adam optimizer with an initial learning rate of 0.001, beta 1 of 0.90, equal 2 of 0.999, and epsilon of 1e-8 was used.

The dataset was randomly divided into the training and testing sets at an 80:20 ratio. Model training was conducted using the training set with performance validation by fivefold cross-validation. A class weighting strategy was also used to ensure that the trained model would take each class into equal account despite class imbalance.

Model performance was evaluated using the testing dataset, and by evaluating and comparing the area under the receiver operating characteristic curve (AUC), F1-score, accuracy, kappa, and Matthews correlation coefficient among the 13 models. An AUC of 0.7–0.8 indicated fair performance, and an AUC of >0.8 indicated good performance. The F1-score, which is calculated using precision and recall parameters, has a maximum possible value of 1.0, which indicates perfect performance. In addition, accuracy, precision, and recall were provided, which are also performance evaluation criteria. However, these metrics include tradeoffs, such as the tradeoff between precision and recall; thus, the optimal model was selected for deployment using AUC.

Results

Although 405 patients with spinal metastasis met the inclusion criteria, only 245 were still alive 180 days following spinal metastasis surgery. Patients who met the exclusion criteria were excluded from the study. Finally, 167 participants were enrolled in this study, including 75 men (44.9%) and 92 women (55.1%). The mean age of all patients was 56.9±11.3 years. Moreover, 140 (81.9%) and 137 (82.0%) patients were classified as ambulatory 90 and 180 days following spinal metastasis surgery, respectively.

Missing data included BMI in seven patients (4.2%), serum calcium level in 21 (12.6%), serum creatinine level in 3 (1.8%), and level of surgery in 1 (0.6%). The baseline characteristics were compared between the ambulatory and non-ambulatory groups at 180 days (Table 1).

Baseline characteristics of patients who underwent surgery for spinal metastasis compared between the ambulatory and non-ambulatory at 180 days groups

Importance factors selected by the extreme gradient boosting classifier that significantly predicted the 180-day ambulatory outcome included serum albumin level, presence of symptomatic spinal compression at the thoracic level, preoperative neurological and ambulatory status, and BMI, as shown in Fig. 1. The importance factors selected by the decision tree algorithm that significantly predicted the 90-day ambulatory outcome included preoperative ambulatory status, age, serum albumin level, days of neurological deficit, and presence of symptomatic spinal compression at the thoracic level, as shown in Fig. 2.

Fig. 1

Feature importance values for the extreme gradient boosting model for predicting 180-day ambulatory status.

Fig. 2

Feature importance values for the decision tree model for predicting 90-day ambulatory status.

1. Model evaluation for the prediction of the 180-day ambulatory outcome

Among the 13 models that were evaluated, the extreme gradient boosting algorithm has the best performance for predicting the 180-day ambulatory outcome (AUC, 0.85; accuracy, 0.82; precision, 0.82; recall, 1; F1-score, 0.9) (Fig. 3). Data specific to the 180-day prediction performance of all evaluated models are presented in Table 2.

Fig. 3

Receiver operating characteristic (ROC) curve for predicting 180-day ambulatory status. The area under the ROC curve (AUC) for the extreme gradient boosting algorithm is 0.85.

Machine learning model performance in test set for predicting 180-day postsurgical ambulatory outcome

2. Model evaluation for prediction of 90-day ambulatory outcome

Of the 13 evaluated models, the decision tree algorithm demonstrated the best ability to predict 90-day postoperative ambulatory outcome (AUC, 0.94; accuracy, 0.82; precision, 1; recall, 0.79; and F1-score, 0.88) (Fig. 4). Details relating to the 90-day prediction performance of all models are shown in Table 3.

Fig. 4

Receiver operating characteristic (ROC) curve for predicting 90-day ambulatory status. The area under the ROC curve (AUC) for the Decision Tree model is 0.94.

Machine learning model performance in test set for predicting 90-day postsurgical ambulatory outcome

Discussion

Previous studies have reported the benefit of surgery in spinal metastasis relative to regaining ambulatory status, pain relief [8], quality-of-life score, and functional outcome score [23]. Despite promising results from surgery, 3.6%–15.3% of patients remained dependent, and postoperative complications were as high as 29%–34% [710]. Consistent with the rates reported from a previous study [10], 82% of patients with spinal metastasis in our study were ambulatory 180 days after their surgery.

Factors previously reported to be significantly associated with postoperative clinical outcome were baseline health-related quality of life, preoperative functional status, preoperative neurological function, interval from symptom onset to treatment, and chronology of motor deficit progression [10,19]. This study demonstrated similar factors for the 90-day outcome, which are related to the preoperative patient status, including preoperative neurological and ambulatory status. By contrast, disease-related factors were found to be most associated with the 180-day postoperative ambulatory outcome, such as level of compression and extent of metastasis. This finding may be explained by the scenario that after a recovery period, the effects of surgery may decrease, and the natural course of the disease may become more dominant.

A clear and realistic treatment goal requires accurate information. Previous studies have proposed models to predict ambulatory status following spinal metastasis surgery that were developed using conventional statistical methods. Numerous studies have demonstrated the efficacy of spinal metastasis surgery in improving ambulatory function using various clinical, radiographic, and treatment-related factors [2628]. Several studies have identified successful factors for postoperative ambulation. A meta-analysis of patients with spinal metastasis who underwent surgery found that pretreatment ambulatory status, the interval between symptom onset and treatment, and time to the development of motor deficits were associated with postoperative outcomes [10]. In a separate retrospective study, a motor grade of 4 or 5 and the occurrence of major complications were significant factors for the resumption of ambulation [29]. However, few studies have attempted to develop predictive tools. Ohashi et al. [11] retrospectively reviewed 82 cases and reported that ambulatory status recovery is correlated with a duration from the onset of neurological symptoms to gait disability of <5 days (AUC, 0.72) and a Tokuhashi score of <7.5 points (AUC, 0.71). In this study, we successfully developed 13 ML algorithms and identified the best predictive model for ambulatory status 180 (extreme gradient boosting) and 90 (decision tree) days following surgery with AUC values of 0.85 and 0.94, respectively.

In the 180- and 90-day groups, the extreme gradient boosting model and decision tree model yielded the best results, respectively. In contrast to the logistic regression, naïve Bayes, and neural network algorithms, the extreme gradient boosting and decision tree models were originally developed using a decision tree-based model, which is a practical strategy for evaluating relatively small imbalanced-class datasets, such as those used in this study. This may explain why the extreme gradient boosting and decision tree models outperformed the other ML models included in this study.

Since most of our patients were ambulators 180 days following surgery, this imbalance in data adversely affected ML algorithm development. To remedy this issue, we used a class weighting strategy to optimize the training process, and we included the F1-score for model evaluation. The F1-score provides valuable insights as a metric for examining imbalanced datasets. The extreme gradient boosting model, which was shown to best predict ambulatory status 180 days following surgery, had an improved F1-score of 0.90. Another common problem when developing an ML model is overfitting. To counter the potential of overfitting, a 5-fold cross-validation was implemented to continuously monitor model performance during training. Each model was then further evaluated using the testing dataset.

The previously published SORG ML algorithm was widely adopted for treatment decision-making and prediction of survival in patients with spinal metastatic diseases [17]. In addition to the survival rate, postoperative ambulatory status is also a very important factor. To our knowledge, this is the first study to report models that predict 180- and 90-day ambulatory outcomes following spinal metastasis surgery using ML algorithms. As previously mentioned, these models exhibit superior accuracy in predicting this critical factor compared with previously published tools, facilitating the establishment of realistic surgical goals, and aiding in treatment planning. Our combined ML model, which allows the user to predict either 180- or 90-day ambulatory status following surgery, has been deployed as an open-access web application, which can be found at https://share.streamlit.io/orthosiriraj/outcome_post_op_metas_spine/main/main.py.

For limitations in the data analysis, first, our model comparison did not provide the confidence interval of each model’s performance during the training phase. Second, only the best-performing model was used for the comparison among algorithms. Third, this study is also limited by its retrospective single-center design. Fourth, our center is a national tertiary referral hospital, which could limit the generalizability of our findings to other care settings. Fifth, a relatively small amount of included data could limit of the performance of ML. To remedy this limitation and continuously improve the performance of our developed algorithms, we will collect data to refine the performance of our ML models. More multicenter studies and external validation are needed to confirm the results of this study and establish the validity of these algorithms for use in real-world clinical practice.

Conclusions

ML algorithms are effective for predicting ambulatory status after surgery for spinal metastasis. The extreme gradient boosting and decision tree algorithms best predicted postoperative ambulatory status 180 and 90 days after spinal metastasis surgery, respectively. Once externally validated for use in routine clinical practice, these algorithms will improve case management decision-making and help in determining clear and realistic goals of treatment.

Acknowledgments

The authors gratefully acknowledge Miss Sirima Nilnok of the Research Unit of the Department of Orthopaedic Surgery, Faculty of Medicine Siriraj Hospital, Mahidol University for her assistance with statistical analysis, manuscript preparation, and coordination of the journal submission process.

Notes

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Author Contributions

PL, SW, PI, and PC designed the study. PC, BS, and PI collected, analyzed the data, and contributed substantially to interpretation of data. PL supervised the project. PC, BS, and PI drafted the article. All authors have read and approved the manuscript.

References

1. Klimo P Jr, Schmidt MH. Surgical management of spinal metastases. Oncologist 2004;9:188–96.
2. Luksanapruksa P, Santipas B, Ruangchainikom M, Korwutthikulrangsri E, Pichaisak W, Wilartratsami S. Epidemiologic study of operative treatment for spinal metastasis in Thailand : a review of national healthcare data from 2005 to 2014. J Korean Neurosurg Soc 2021;65:57–63.
3. Choi SH, Koo JW, Choe D, Kang CN. The incidence and management trends of metastatic spinal tumors in South Korea: a nationwide population-based study. Spine (Phila Pa 1976) 2020;45:E856–63.
4. Barton LB, Arant KR, Blucher JA, et al. Clinician experiences in treatment decision-making for patients with spinal metastases: a qualitative study. J Bone Joint Surg Am 2021;103:e1.
5. Lape EC, Katz JN, Blucher JA, et al. Patient experiences of decision-making in the treatment of spinal metastases: a qualitative study. Spine J 2020;20:905–14.
6. Heary RF, Bono CM. Metastatic spinal tumors. Neurosurg Focus 2001;11:e1.
7. Alamanda VK, Robinson MM, Kneisl JS, Patt JC. Functional and survival outcomes in patients undergoing surgical treatment for metastatic disease of the spine. J Spine Surg 2018;4:28–36.
8. Kim JM, Losina E, Bono CM, et al. Clinical outcome of metastatic spinal cord compression treated with surgical excision ± radiation versus radiation therapy alone: a systematic review of literature. Spine (Phila Pa 1976) 2012;37:78–84.
9. Park SJ, Lee CS, Chung SS. Surgical results of metastatic spinal cord compression (MSCC) from non-small cell lung cancer (NSCLC): analysis of functional outcome, survival time, and complication. Spine J 2016;16:322–8.
10. Liu YH, Hu YC, Yang XG, et al. Prognostic factors of ambulatory status for patients with metastatic spinal cord compression: a systematic review and meta-analysis. World Neurosurg 2018;116:e278–90.
11. Ohashi M, Hirano T, Watanabe K, et al. Preoperative prediction for regaining ambulatory ability in paretic non-ambulatory patients with metastatic spinal cord compression. Spinal Cord 2017;55:447–53.
12. Galbusera F, Casaroli G, Bassani T. Artificial intelligence and machine learning in spine research. JOR Spine 2019;2:e1044.
13. Rasouli JJ, Shao J, Neifert S, et al. Artificial intelligence and robotics in spine surgery. Global Spine J 2021;11:556–64.
14. Merali ZG, Witiw CD, Badhiwala JH, Wilson JR, Fehlings MG. Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy. PLoS One 2019;14:e0215133.
15. Paulino Pereira NR, Mclaughlin L, Janssen SJ, et al. The SORG nomogram accurately predicts 3- and 12-months survival for operable spine metastatic disease: external validation. J Surg Oncol 2017;115:1019–27.
16. Ahmed AK, Goodwin CR, Heravi A, et al. Predicting survival for metastatic spine disease: a comparison of nine scoring systems. Spine J 2018;18:1804–14.
17. Karhade AV, Ahmed AK, Pennington Z, et al. External validation of the SORG 90-day and 1-year machine learning algorithms for survival in spinal metastatic disease. Spine J 2020;20:14–21.
18. Moon KY, Chung CK, Jahng TA, Kim HJ, Kim CH. Postoperative survival and ambulatory outcome in metastatic spinal tumors : prognostic factor analysis. J Korean Neurosurg Soc 2011;50:216–23.
19. Feghali J, Pennington Z, Ehresman J, et al. Predicting postoperative quality-of-life outcomes in patients with metastatic spine disease: who benefits? J Neurosurg Spine 2020;34:383–9.
20. Luksanapruksa P, Buchowski JM, Hotchkiss W, Tongsai S, Wilartratsami S, Chotivichit A. Prognostic factors in patients with spinal metastasis: a systematic review and meta-analysis. Spine J 2017;17:689–708.
21. Cheung ZB, Vig KS, White SJW, et al. Impact of obesity on surgical outcomes following laminectomy for spinal metastases. Global Spine J 2019;9:254–9.
22. Truong VT, Shedid D, Al-Shakfa F, et al. Surgical intervention for patients with spinal metastasis from lung cancer: a retrospective study of 87 cases. Clin Spine Surg 2021;34:E133–40.
23. Paulino Pereira NR, Groot OQ, Verlaan JJ, et al. Quality of life changes after surgery for metastatic spinal disease: a systematic review and meta-analysis. Clin Spine Surg 2022;35:38–48.
24. Thio QC, Karhade AV, Bindels BJ, et al. Development and internal validation of machine learning algorithms for preoperative survival prediction of extremity metastatic disease. Clin Orthop Relat Res 2020;478:322–33.
25. Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: machine learning in python. J Mach Learn Res 2011;12:2825–30.
26. Kim YH, Ha KY, Park HY, et al. Simple and reliable magnetic resonance imaging parameter to predict postoperative ambulatory function in patients with metastatic epidural spinal cord compression. Global Spine J 2023;13:479–85.
27. Schoenfeld AJ, Losina E, Ferrone ML, et al. Ambulatory status after surgical and nonsurgical treatment for spinal metastasis. Cancer 2019;125:2631–7.
28. Hershkovich O, Sakhnini M, Gara S, Caspi I, Lotan R. Acute metastatic spinal cord compression: urgent surgery versus radiotherapy and treatment result prediction versus actual results. Curr Oncol 2022;29:7420–9.
29. Kim CH, Chung CK, Jahng TA, Kim HJ. Resumption of ambulatory status after surgery for nonambulatory patients with epidural spinal metastasis. Spine J 2011;11:1015–23.

Article information Continued

Fig. 1

Feature importance values for the extreme gradient boosting model for predicting 180-day ambulatory status.

Fig. 2

Feature importance values for the decision tree model for predicting 90-day ambulatory status.

Fig. 3

Receiver operating characteristic (ROC) curve for predicting 180-day ambulatory status. The area under the ROC curve (AUC) for the extreme gradient boosting algorithm is 0.85.

Fig. 4

Receiver operating characteristic (ROC) curve for predicting 90-day ambulatory status. The area under the ROC curve (AUC) for the Decision Tree model is 0.94.

Table 1

Baseline characteristics of patients who underwent surgery for spinal metastasis compared between the ambulatory and non-ambulatory at 180 days groups

Characteristic Missing data (%) Total no. Ambulatory at 180 days Non-ambulatory at 180 days
No. of patients 137 (82.0) 30 (18.0)
Postoperative follow-up (day) 181 (172–187) 182 (177–186)
Age (yr) 0 167 56.2±11.7 59.9±9.2
Gender 0 167
 Male 59 (43.1) 16 (53.3)
 Female 78 (56.9) 14 (46.7)
Body mass index (kg/m2) 4.2 160 22.48±3.84 22.92±3.29
Current smoker 0 167 14 (10.2) 5 (16.7)
ASA classification 0 167
 Class II 109 (79.6) 25 (83.3)
 Class III 28 (20.4) 5 (16.7)
Comorbidities 0 167
 Neurogenic bladder 25 (18.2) 15 (50.0)
 Myocardial infarction 1 (0.7) 1 (3.0)
 Pneumonia 2 (1.5) -
 Stroke 1 (0.7) -
 Delirium 2 (1.5) -
Presence of myelopathy 0 167 47 (34.3) 15 (50.0)
Duration of neurological deficit (day) 0 167 13.7±19.9 19.2±33.2
Frankel grading 0 167
 A 1 (0.7) 0
 B 9 (6.6) 11 (36.7)
 C 43 (31.4) 12 (40.0)
 D 64 (46.7) 4 (13.3)
 E 20 (14.6) 3 (10.0)
Symptomatic spinal compression 0 167
 Cervical 18 (13.1) 1 (3.3)
 Thoracic 37 (27.0) 23 (76.7)
 Lumbar 43 (31.4) 5 (16.7)
 Sacrum 1 (0.7) -
Level of metastasis 0 167
 Cervical 36 (26.3) 4 (13.3)
 Thoracic 81 (59.1) 27 (90.0)
 Lumbar 75 (54.7) 6 (20.0)
 Sacrum 16 (11.7) 1 (3.3)
Extraspinal bone metastasis 0 167 67 (48.9) 12 (40.0)
Visceral metastasis 0 167 33 (24.1) 6 (20.0)
Primary tumor source 0 167
 Breast 41 (29.9) 7 (23.3)
 Thyroid 10 (7.3) 1 (3.3)
 Kidney 5 (2.9) 1 (3.3)
 Lung 21 (15.3) 4 (13.3)
 Prostate 12 (8.8) 4 (13.3)
 Liver 4 (2.9) 2 (6.7)
 Hematologic 3 (2.2) 2 (6.7)
 Cholangiocarcinoma 4 (2.9) 2 (6.7)
 Nasopharyngeal 5 (3.6) 2 (6.7)
 Colorectal 5 (3.6) -
 Cervix 1 (0.7) -
 Unknown 18 (13.1) 4 (13.3)
 Others 9 (6.6) 1 (3.3)
Primary tumor histology 0 167
 Adenocarcinoma 85 (62.0) 16 (53.3)
 Squamous 9 (6.6) 3 (10.0)
 Follicular 6 (4.4) -
 Small cell 1 (0.7) -
 Clear cell 1 (0.7) 2 (6.7)
 Unknown 34 (24.8) 9 (30.0)
Preoperative calcium (mg/dL) 12.6 146 9.01±1.32 9.01±0.60
Preoperative albumin (g/dL) 0 167 4.00±0.53 3.75±0.43
Preoperative creatinine (mg/dL) 1.8 164 0.91±0.97 0.79±0.34
Preoperative treatment 0 167
 Chemotherapy 52 (38.0) 9 (30.0)
 Radiotherapy 40 (29.2) 4 (13.3)
 Molecular targeting therapy 6 (4.4) -
Preoperative ambulatory status 0 167
 Ambulator 83 (60.6) 5 (16.7)
 Non-ambulator 54 (39.4) 25 (83.3)

Values are presented as number (%), median (interquartile), or mean±standard deviation.

ASA, American Society of Anesthesiologists.

Table 2

Machine learning model performance in test set for predicting 180-day postsurgical ambulatory outcome

Model AUC Accuracy Recall Precision F1-score Kappa MCC
Extreme gradient boosting 0.8519 0.8182 1 0.8182 0.9 0 0
Light gradient boosting machine 0.8148 0.8182 0.8148 0.9565 0.88 0.5147 0.544
Extra trees classifier 0.7778 0.7879 0.8148 0.9167 0.8627 0.4031 0.417
Decision tree classifier 0.7778 0.8485 0.8889 0.9231 0.9057 0.5217 0.5241
Random forest classifier 0.7407 0.697 0.7037 0.9048 0.7917 0.2667 0.297
K neighbors classifier 0.7222 0.5758 0.5556 0.8824 0.6818 0.1348 0.1715
Ada boost classifier 0.716 0.6667 0.6667 0.9 0.766 0.2293 0.2631
Gradient boosting classifier 0.7037 0.6364 0.5926 0.9412 0.7273 0.2584 0.3287
Quadratic discriminant analysis 0.679 0.5758 0.5556 0.8824 0.6818 0.1348 0.1715
Linear discriminant analysis 0.6235 0.6667 0.7407 0.8333 0.7843 0.062 0.0642
Artificial neural network 0.6204 0.697 0.7407 0.8696 0.8 0.1912 0.202
Logistic regression 0.5556 0.6364 0.7037 0.8261 0.76 0.0294 0.0311
Naïve Bayes 0.466 0.5758 0.5926 0.8421 0.6957 0.061 0.0723
Support vector machine-linear kernel 0.4352 0.6061 0.7037 0.7917 0.7451 −0.1085 −0.1123

AUC, area under the receiver operating characteristic curve; MCC, Matthews correlation coefficient.

Table 3

Machine learning model performance in test set for predicting 90-day postsurgical ambulatory outcome

Model AUC Accuracy Recall Precision F1-score Kappa MCC
Decision tree classifier 0.9405 0.8235 0.7857 1 0.88 0.5641 0.6268
Gradient boosting classifier 0.9167 0.7941 0.7857 0.9565 0.8627 0.4664 0.5045
Random forest classifier 0.8988 0.7941 0.7857 0.9565 0.8627 0.4664 0.5045
Extreme gradient boosting 0.8988 0.8235 1 0.8235 0.9032 0 0
Light gradient boosting 0.8929 0.7647 0.75 0.9545 0.84 0.4188 0.4653
Ada boost classifier 0.8542 0.7353 0.6786 1 0.8085 0.427 0.521
K neighbors classifier 0.8423 0.6765 0.6429 0.9474 0.766 0.2996 0.3656
Extra trees classifier 0.8333 0.7941 0.7857 0.9565 0.8627 0.4664 0.5045
Linear discriminant analysis 0.631 0.6471 0.6429 0.9 0.75 0.2031 0.2398
Quadratic discriminant analysis 0.625 0.5882 0.6429 0.8182 0.72 −0.0171 −0.019
Logistic regression 0.619 0.6471 0.6429 0.9 0.75 0.2031 0.2398
Support vector machine-linear kernel 0.6012 0.5588 0.5357 0.8824 0.6667 0.1176 0.1543
Artificial neural network 0.5714 0.6176 0.6429 0.8571 0.7347 0.098 0.1121
Naïve Bayes 0.4107 0.6765 0.8214 0.7931 0.807 −0.1911 −0.1922

AUC, area under the receiver operating characteristic curve; MCC, Matthews correlation coefficient.