Step 1: Data Import & Split
First column must be ID matching data file's first column. Second column is group label (train/val/test).
Data Preview
Train
Validation
Test
Data File
Group Mapping
Step 2: Baseline Characteristics
Step 3: Univariate Analysis
Step 4: Multivariate Analysis
Step 5: Nomogram
Step 6: ROC Analysis
Step 7: Decision Curve Analysis
Step 8: Confusion Matrix
Step 9: Calibration Curve
Step 10: Generate Report
Help & Documentation

1. Data Structure Requirements

Your CSV file must contain:

  • A binary outcome variable (0/1, Yes/No, or factor with 2 levels)
  • At least one predictor variable (numeric or categorical)
  • Optional: an ID column (will be auto-removed)

2. Import Methods & Example Data

Three data input formats are supported. Prepare your files according to the examples below:

Method A — Separate Train/Validation/Test Files

Upload three independent CSV files. Each file must have the same column names.

train.csv (example, 5 rows):

patient_id age gender tumor_size grade outcome
P001 62 Male 2.5 High 1
P002 55 Female 1.8 Low 0
P003 70 Male 3.2 High 1
P004 48 Female 1.2 Low 0
P005 58 Male 2.0 Low 0

validation.csv (same column names, different rows):

patient_id age gender tumor_size grade outcome
P201 58 Female 2.1 Low 0
P202 65 Male 2.8 High 1

test.csv (same column names, different rows):

patient_id age gender tumor_size grade outcome
P301 60 Male 1.5 Low 0
P302 72 Female 3.5 High 1

Note: Column names must be identical across all three files. Factor levels are automatically unified. The outcome column should be the same in all files.

Method B — Single File + Group Mapping (B')

Upload one data file plus a two-column mapping file (ID, Group). The group labels are standardized automatically (train/val/test).

data.csv (example, 5 rows):

patient_id age gender tumor_size grade outcome
P001 62 Male 2.5 High 1
P002 55 Female 1.8 Low 0
P003 70 Male 3.2 High 1
P004 48 Female 1.2 Low 0
P005 58 Male 2.0 Low 0

group_map.csv (2 columns):

patient_id group
P001 train
P002 train
P003 validation
P004 test
P005 train

Accepted group labels: train / training / tr / learn / fit → train; val / validation / valid / dev / development → val; test / testing / te / eval / evaluation / holdout → test. Rows with IDs not found in the mapping are automatically assigned to train.

Method C — Single File with Auto-Split

Upload one CSV file. The app will randomly split it into train/validation/test by the ratios you specify (default 70:0:30).

data.csv (example, 5 rows):

patient_id age gender tumor_size grade outcome
P001 62 Male 2.5 High 1
P002 55 Female 1.8 Low 0
P003 70 Male 3.2 High 1
P004 48 Female 1.2 Low 0
P005 58 Male 2.0 Low 0

Parameters: Train ratio, Validation ratio, Test ratio (must sum to ≤1). Random seed is user-configurable (default 42) for reproducibility.

3. Workflow Steps

  1. Data Import: Load and split your data using one of the three methods above.
  2. Baseline Table: Descriptive statistics (continuous: mean±SD / median(IQR); categorical: n(%)) with P values for group differences.
  3. Univariate Analysis: Single-factor logistic regression for each predictor. Significant variables (P < threshold) are candidates for multivariate modeling.
  4. Multivariate Analysis: Multivariable logistic regression with backward elimination (AIC). You can force-include specific variables.
  5. Nomogram: Visualize the prediction model. Points are assigned to each variable level; total points are converted to predicted probability.
  6. ROC Analysis: Receiver Operating Characteristic curves with AUC (95% CI). DeLong test for pairwise AUC comparison across datasets.
  7. DCA: Decision Curve Analysis evaluates net benefit across threshold probabilities. Best decision range is identified automatically.
  8. Threshold Performance: Sensitivity, Specificity, PPV, NPV, accuracy, and F1 at every threshold (0–1). Optimal cutoff by Youden index.
  9. Confusion Matrix: Classification table (TP/FP/TN/FN) at the chosen cutoff, with performance metrics.
  10. Calibration: Bootstrap calibration plot (B resamples) and binned calibration with Hosmer-Lemeshow goodness-of-fit test.
  11. Report: Generate a comprehensive HTML report with all results and methodology.

4. Variable Selection in Multivariate

By default, variables with P < threshold in univariate analysis are used. You can:

  • Keep the default (auto-selected)
  • Remove unwanted variables by unchecking them
  • Add back non-significant variables by checking them
  • Use 'Force Include' to ensure specific variables are always in the model

5. Tips

  • Each step can be skipped by unchecking 'Execute this step'
  • Results are saved in memory until you refresh the page
  • Use 'Download All Results' to save everything as a ZIP file
  • If a step fails, check the error message and adjust parameters
  • Nomogram and Calibration use train set; ROC, DCA, Threshold, Confusion use test set
  • Categorical variables are auto-detected (character/factor or numeric with ≤10 unique values)

6. Factor Variable Handling

When using separate files (Method A) or Group Mapping (Method B), factor levels are automatically unified across train/validation/test sets. This ensures consistent model predictions across all datasets.

7. References & Bibliography

The following references describe the statistical methods used in this application:

  1. Hosmer DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression. 3rd ed. Wiley; 2013. doi:10.1002/9781118548387
  2. Harrell FE Jr. rms: Regression Modeling Strategies. R package version 6.7-1. 2024. https://CRAN.R-project.org/package=rms
  3. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361-387. doi:10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4
  4. Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. 2008;26(8):1364-1370. doi:10.1200/JCO.2007.12.9791
  5. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29-36. doi:10.1148/radiology.143.1.7063747
  6. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-845. doi:10.2307/2531595
  7. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565-574. doi:10.1177/0272989X06295361
  8. Vickers AJ, van Calster B, Steyerberg EW. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res. 2019;3:18. doi:10.1186/s41512-019-0064-7
  9. Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16(9):965-980. doi:10.1002/(SICI)1097-0258(19970515)16:9<965::AID-SIM509>3.0.CO;2-O
  10. Altman DG, Bland JM. Diagnostic tests 1: sensitivity and specificity. BMJ. 1994;308(6943):1552. doi:10.1136/bmj.308.6943.1552
  11. Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32-35. doi:10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3
  12. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. doi:10.1097/EDE.0b013e3181c30fb2

Version: 1.1.0 | Author: math81.com