Linear Regression from Scratch using Python

By Akhilesh Ketkar

The purpose of this project was to present the inner workings of Linear regression for better understanding using Python.

Coding linear regression algorithm from Scratch using NumPy in Python

 

First, We need to install the NumPy package using "Pip install numpy" in Command prompt. This line of code will import numpy.

import numpy

We need to set learning rate and iteration as arguments in the constructor. Second, most important thing is to define weight and bias variables for further use.

def __init__(self,learning_rate=0.01,n_iter=1000):
    #reference to the current instance of the class 
    self.learning_rate = learning_rate
    self.n_iter = n_iter
    
    #we will declare this value during fit
    self.weight = None
    self.bias = None

Then we'll define the fit function as follows,

def fit(self,X,y):
        #Fetching number of samples and features from the data
        n_samples, n_features = X.shape
        
        #Initiation weight and bias with zero
        self.weight = np.zeros(shape=(n_features))
        self.bias = 0
        
        
        
        #Gradien Descent 
        for _ in range(self.n_iter):
            #First predict the y using initial weight and bias
            y_predicted = np.dot(X,self.weight) + self.bias         
            #getting gradient             
            dw = (1/n_samples) * np.dot(X.T,(y_predicted - y))
            db = (1/n_samples) * np.sum(y_predicted - y)
            
            #updating values using gradient 
            self.weight = self.weight - (self.learning_rate)*dw
            self.bias = self.bias - (self.learning_rate)*db

here we've used NumPy.dot and NumPy.sum functions to calculate dot product and the sum of all elements respectively.

def predict(self,X):
    y_predicted = np.dot(X,self.weight) + self.bias
    return y_predicted

This predict function will return predicted values.