Statistical model reference
XGBoost
Review when to use this method, its data requirements, implementation patterns, and interpretation guidance.
Description
A scalable, distributed gradient-boosted decision tree (GBDT) machine learning library that provides parallel tree boosting. Known for its speed, performance, and regularization techniques to prevent overfitting.
Use Cases
- structured/tabular data
- classification
- regression
- ranking
Requirements
- Sample Size: small, medium, large
- Missing Data: none, random, systematic
- Data Distribution: any
- Relationship Type: linear, nonlinear, interactions
Variable Types
Dependent Variables
- continuous
- binary
- categorical
Independent Variables
- continuous
- categorical
- binary
Implementation
from xgboost import XGBRegressor
model = XGBRegressor(objective='reg:squarederror', n_estimators=100)
model.fit(X_train, y_train)
predictions = model.predict(X_test)library(xgboost)
dtrain <- xgb.DMatrix(data = as.matrix(X_train), label = y_train)
model <- xgb.train(data = dtrain, nrounds = 100, objective = "reg:squarederror")* Requires Python integration or Modeler
BEGIN PROGRAM PYTHON.
from xgboost import XGBRegressor
model = XGBRegressor()
model.fit(spssDataset.X, spssDataset.y)
END PROGRAM.proc gradboost data=mydata;
target y / level=interval;
input x1-xn / level=all;
partition fraction(valid=0.3);
autotune;
run;* Requires Python integration
python:
from xgboost import XGBRegressor
model = XGBRegressor()
model.fit(X, y)
endSynthetic Data Example
Nonlinear dataset with interactions
R Code for Data Generation and Analysis
set.seed(123)
n <- 1000
x1 <- runif(n); x2 <- runif(n)
y <- 2*x1 + 3*x2^2 + 4*x1*x2 + rnorm(n)
library(xgboost)
dtrain <- xgb.DMatrix(data = cbind(x1,x2), label = y)
params <- list(objective = "reg:squarederror", max_depth = 3)
model <- xgb.train(params, dtrain, nrounds = 50)
xgb.plot.importance(xgb.importance(model = model))
Copy this code into your R environment to generate synthetic data and perform analysis with this model.
Expected Analysis Results
Console Output
Feature Importance:
Feature Gain Cover Frequency
1 x2 0.65 0.55 0.45
2 x1 0.35 0.45 0.55
These results are from running the R code on synthetic data. Your actual results may vary depending on your data.
Interpretation Guide
Need help interpreting the results of your XGBoost analysis? Our comprehensive interpretation guide explains:
- How to read and understand model outputs
- Interpreting coefficients and effect sizes correctly
- Understanding diagnostic plots and visualizations
- Common pitfalls and how to avoid them
- Making valid conclusions from your analysis