Learn how to find and fix missing data in a Pandas DataFrame. Using simple methods like .isnull(), .isna(), fill missing values, or remove them to keep your data clean.
Creating a DataFrame by loading csv file
We can create DataFrame by loading csv file
The given fruits1.csv file has missing values.
Kindly observe the missing/NaN values in DataFrame.
import pandas as pd
df1 = pd.read_csv("fruits1.csv")
print(df1)
output:
isna() and isnull() method – Checking NaN values
isna() and isnull() are a predefined methods in DataFrame
We can access these methods by using DataFrame object.
By using these methods, we can check missing values exist in DataFrame or not.
If missing values are available then it return as True, otherwise False.
import pandas as pd
df1 = pd.read_csv("fruits1.csv")
df2 = df1.isna()
print(df1.head())
print()
print(df2.head())
output:
import pandas as pd
df1 = pd.read_csv("fruits1.csv")
df2 = df1.isnull()
print(df1.head())
print()
print(df2.head())
notnull() method – Checking NaN values
notnull() is a predefined method in DataFrame
We can access this method by using DataFrame object.
By using this method we can check missing values exist in DataFrame or not.
If missing values are available then it return as False, otherwise True
import pandas as pd
df1 = pd.read_csv("fruits1.csv")
df2 = df1.notnull()
print(df1.head())
print()
print(df2.head())
output:
dropna() method – Handling missing values
dropna() is a predefined method in DataFrame
We can access dropna() method by using DataFrame object.
This method drops the rows where at least one value is missing.
import pandas as pd
df1 = pd.read_csv("fruits1.csv")
df2 = df1.dropna()
print(df2)
output:
Counting NaN values in column wise
We can count number of missing values in DataFrame
By using isna() and sum() methods we can count the number of missing values in each column.
import pandas as pd
df1 = pd.read_csv('fruits1.csv')
s = df1.isna().sum()
print(s)
output:
fillna() method – Handling missing values
fillna() is a predefined method in DataFrame
We can access this method by using DataFrame object.
By using this method we can fill missing/NaN values with specific value.
1. fillna(0) -> This method fill NaN with Zero values
2. fillna(number) -> This method fill NaN with number|
import pandas as pd
df1 = pd.read_csv("fruits1.csv")
df2 = df1.fillna(0)
print(df1.head())
print()
print(df2.head())
output:
Reference links for Handling missing values:
Working with missing data — pandas 2.2.3 documentation
Missing values — pandas 2.2.3 documentation
Handling missing values is an important step in data cleaning. By identifying and addressing them properly, you ensure your Pandas DataFrame is accurate, reliable, and ready for analysis.