QDA – Quadratic Discriminant Analysis
- It applies to classification problems in supervised machine
- It calculates the likelihood that a data point belongs to each class using the Bayes theorem.
- Its goal is to simulate each class’s predictor distribution independently.
How it’s different from LDA?
Linear Discriminant Analysis (LDA) and QDA are similar. Both employed in classification with the goal of identifying the decision boundary or separation line, that separates several classes in the data. Their presumptions regarding the distribution of the data, however, vary:
LDA: This indicates that each class’s data points are dispersed similarly because it assumes that the covariance matrix, or spread measure, is shared by all classes.
QDA: Lets go of this presumption. It makes it possible for every class to have an individual covariance matrix, which helps it deal with scenarios in which data clusters have varying forms or orientations.
A straightforward illustration of how to apply QDA for categorization :
from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris
# Load the Iris dataset
iris = load_iris()
# Separate features (X) and target variable (y)
X = iris.data
y = iris.target
# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Create the QDA model
qda = QuadraticDiscriminantAnalysis()
# Train the model on the training data
qda.fit(X_train, y_train)
# Make predictions on the testing data
y_pred = qda.predict(X_test)
# Evaluate the model performance (accuracy in this case)
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
Advantages of QDA:
- Flexibility : In contrast to Linear Discriminant Analysis (LDA), QDA permits non-linear decision bounds. It can more precisely model complicated relationships in the data thanks to this flexibility.
- Interpretability: To categorize data points, it makes use of the well-known concept of Gaussian distributions and distances. In contrast to more intricate algorithms, this makes it simpler to describe the behavior of the model.
- Less Restrictive Assumptions: In contrast to several other classifiers, QDA makes less assumptions about the distribution of the underlying data. It is appropriate for a larger variety of datasets, as it does’t assume linear correlations between predictors and class labels.
Disadvantages of QDA:
- Prone to Overfitting : QDA becomes more prone to overfitting, especially when the number of observations is limited, due to the large number of parameters it needs to estimate. Regularization techniques may be necessary to mitigate overfitting.
- Assumption of Normality : If QDA encounters a situation where the predictors in each class do not adhere to a normal distribution, it might produce biased estimates and render predictions less reliable.
- Computational Cost: Calculating distinct covariance matrices for individual classes might incur computational costs, particularly when handling high-dimensional datasets with numerous features. Consequently, training QDA models could be slower in comparison to more straightforward algorithms.