Linear Regression

• Regression analysis is one of the most important fields in statistics and machine learning.
• There are many regression methods available. Linear regression is one of them.
• Regression searches for relationships among variables.
• Linear regression is probably one of the most important and widely used regression techniques.
• It’s among the simplest regression methods.
• One of its main advantages is the ease of interpreting results.

When Do You Need Regression?

• you need regression to answer whether and how some phenomenon influences the other or how several variables are related.
• For example, you can use it to determine if and to what extent experience or gender impacts salaries.

Formulation

dependent variable 𝑦 
on the set of independent variables 𝐱 = (𝑥₁, …, 𝑥ᵣ),
where 𝑟 is the number of predictors, 
a linear relationship between 𝑦 and 𝐱: 𝑦 = 𝛽₀ + 𝛽₁𝑥₁ + ⋯ + 𝛽ᵣ𝑥ᵣ + 𝜀. 
this equation is the regression equation. 𝛽₀, 𝛽₁, …, 𝛽ᵣ are the regression coefficients, 
𝜀 is the random error.

Linear Regression types

• simple linear regression
• multiple linear regression
• polynomial regression

Underfitting and Overfitting

• Underfitting occurs when a model can’t accurately capture the dependencies among data, usually as a consequence of its own simplicity. It often yields a low 𝑅² with known data and bad generalization capabilities when applied with new data.

• Overfitting happens when a model learns both data dependencies and random fluctuations. In other words, a model learns the existing data too well. Complex models, which have many features or terms, are often prone to overfitting. When applied to known data, such models usually yield high 𝑅². However, they often don’t generalize well and have significantly lower 𝑅² when used with new data.

Simple Linear Regression With scikit-learn

There are five basic steps when you’re implementing linear regression:

• Import the packages and classes that you need.
• Provide data to work with, and eventually do appropriate transformations.
• Create a regression model and fit it with existing data.
• Check the results of model fitting to know whether the model is satisfactory.
• Apply the model for predictions.