Model library Linear Regression
Statistical model reference

Linear Regression

Review when to use this method, its data requirements, implementation patterns, and interpretation guidance.

Description

A powerful statistical approach that models the linear relationship between a dependent variable and one or more independent variables. It finds the best-fitting straight line through the data by minimizing the sum of squared residuals, making it ideal for prediction, exploration of relationships, and testing causal hypotheses when assumptions are met.

Use Cases
  • prediction
  • exploration
  • inference
Requirements
  • Sample Size: small, medium, large
  • Missing Data: none, random, systematic
  • Data Distribution: normal, non_normal
  • Relationship Type: linear
Variable Types
Dependent Variables
  • continuous
Independent Variables
  • continuous
  • categorical
  • binary
Implementation
from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(X, y)
predictions = model.predict(X_test)
Documentation 
model <- lm(y ~ x1 + x2, data=df)
summary(model)
predictions <- predict(model, newdata=test_data)
Documentation 
REGRESSION
  /MISSING LISTWISE
  /STATISTICS COEFF OUTS R ANOVA
  /CRITERIA=PIN(.05) POUT(.10)
  /NOORIGIN
  /DEPENDENT y
  /METHOD=ENTER x1 x2
Documentation 
proc reg data=dataset;
  model y = x1 x2;
  run;
Documentation 
regress y x1 x2
Documentation
Synthetic Data Example

A dataset suitable for Linear Regression analysis

R Code for Data Generation and Analysis
# Generate synthetic data for linear regression
set.seed(123)
n <- 100  # sample size
x1 <- rnorm(n, mean = 10, sd = 2)  # continuous predictor
x2 <- rbinom(n, 1, 0.5)  # binary predictor
x3 <- factor(sample(1:3, n, replace = TRUE))  # categorical predictor with 3 levels
# Create outcome with a linear relationship plus some noise
y <- 2 + 0.5 * x1 + 1.5 * x2 + rnorm(n, mean = 0, sd = 1)
# Combine into a data frame
df <- data.frame(y = y, x1 = x1, x2 = x2, x3 = x3)

# Descriptive statistics
summary(df)
cor(df[, c("y", "x1", "x2")])
boxplot(y ~ x2, data = df, main = "Y by Binary Predictor", xlab = "X2", ylab = "Y")
plot(x1, y, main = "Scatterplot of Y vs X1", xlab = "X1", ylab = "Y")

# Model fitting
model <- lm(y ~ x1 + x2 + x3, data = df)
summary(model)

# Diagnostic plots
par(mfrow = c(2, 2))
plot(model)

# Predictions
new_data <- data.frame(x1 = c(8, 10, 12), x2 = c(0, 1, 0), x3 = factor(c(1, 2, 3), levels = 1:3))
predictions <- predict(model, newdata = new_data, interval = "confidence")
print(cbind(new_data, predictions))
Copy this code into your R environment to generate synthetic data and perform analysis with this model.
Expected Analysis Results
Console Output

> summary(df)
       y                x1             x2             x3    
 Min.   : 4.83   Min.   : 5.40   Min.   :0.00   1:28     
 1st Qu.: 7.68   1st Qu.: 8.68   1st Qu.:0.00   2:31     
 Median : 8.74   Median :10.17   Median :0.00   3:41     
 Mean   : 8.87   Mean   :10.07   Mean   :0.46          
 3rd Qu.:10.20   3rd Qu.:11.49   3rd Qu.:1.00          
 Max.   :12.61   Max.   :14.65   Max.   :1.00          

> cor(df[, c("y", "x1", "x2")])
          y        x1        x2
y  1.000000 0.8046874 0.4486060
x1 0.804687 1.0000000 0.0322727
x2 0.448606 0.0322727 1.0000000

> model <- lm(y ~ x1 + x2 + x3, data = df)
> summary(model)

Call:
lm(formula = y ~ x1 + x2 + x3, data = df)

Residuals:
     Min       1Q   Median       3Q      Max 
-2.05111 -0.62366  0.01062  0.70315  2.10890 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   2.1344     0.4940   4.321 3.92e-05 ***
x1            0.4921     0.0457  10.769  < 2e-16 ***
x2            1.4975     0.1866   8.025 3.41e-12 ***
x32           0.1977     0.2534   0.780   0.4373    
x33           0.1741     0.2309   0.754   0.4527    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.9766 on 95 degrees of freedom
Multiple R-squared:  0.7324,	Adjusted R-squared:  0.7208 
F-statistic: 64.93 on 4 and 95 DF,  p-value: < 2.2e-16

> new_data <- data.frame(x1 = c(8, 10, 12), x2 = c(0, 1, 0), x3 = factor(c(1, 2, 3), levels = 1:3))
> predictions <- predict(model, newdata = new_data, interval = "confidence")
> print(cbind(new_data, predictions))
   x1 x2 x3      fit      lwr      upr
1  8  0  1  6.071198  5.67851  6.46389
2 10  1  2  8.576220  8.17784  8.97460
3 12  0  3 10.146060  9.63766 10.65446

> plot(model)
These results are from running the R code on synthetic data. Your actual results may vary depending on your data.
Interpretation Guide

Need help interpreting the results of your Linear Regression analysis? Our comprehensive interpretation guide explains:

  • How to read and understand model outputs
  • Interpreting coefficients and effect sizes correctly
  • Understanding diagnostic plots and visualizations
  • Common pitfalls and how to avoid them
  • Making valid conclusions from your analysis
View Interpretation Guide Download Guide
Additional Resources
Tutorials and Guides
Quick-R Statistical Methods
Scikit-learn Tutorials
UCLA Statistical Methods
Academic Papers and References
Find Research Papers
Latest Statistical Methods
Back to Home View All Models

Statistical assistant