Python Pandas - Interpolation of Missing Values

Interpolation is a powerful technique in Pandas that used for handling the missing values in a dataset. This technique estimates the missing values based on other data points of the dataset. Pandas provides the interpolate() method for both DataFrame and Series objects to fill in missing values using various interpolation methods.

In this tutorial, we will learn about the interpolate() methods in Pandas for filling the missing values in a time series data, numeric data, and more using the different interpolation methods.

Basic Interpolation

The Pandas interpolate() method of the both DataFrame and Series objects is used to fills the missing values using different Interpolation strategies. By default, Pandas automatically uses linear interpolation as the default method.

Example

Here is a basic example of calling the interpolate() method for filling the missing values.

import numpy as np
import pandas as pd

df = pd.DataFrame({"A": [1.1, np.nan, 3.5, np.nan, np.nan, np.nan, 6.2, 7.9],
"B": [0.25, np.nan, np.nan, 4.7, 10, 14.7, 1.3, 9.2],
})

print("Original DataFrame:")
print(df)

# Using the  interpolate() method
result = df.interpolate()
print("\nResultant DataFrame after applying the interpolation:")
print(result)

Output

Following is the output of the above code −

Original DataFrame:
     A      B
0  1.1   0.25
1  NaN    NaN
2  3.5    NaN
3  NaN   4.70
4  NaN  10.00
5  NaN  14.70
6  6.2   1.30
7  7.9   9.20

Resultant DataFrame after applying the interpolation:
       A          B
0  1.100   0.250000
1  2.300   1.733333
2  3.500   3.216667
3  4.175   4.700000
4  4.850  10.000000
5  5.525  14.700000
6  6.200   1.300000
7  7.900   9.200000

Different Interpolating Methods

Pandas supports several interpolation methods, including linear, polynomial, pchip, akima, spline, and more. These methods provide flexibility for filling the missing values depending on the nature of your data.

Example

The following example demonstrates using the interpolate() method with the barycentric interpolation technique.

import numpy as np
import pandas as pd

df = pd.DataFrame({"A": [1.1, np.nan, 3.5, np.nan, np.nan, np.nan, 6.2, 7.9],
"B": [0.25, np.nan, np.nan, 4.7, 10, 14.7, 1.3, 9.2],
})

print("Original DataFrame:")
print(df)

# Applying the interpolate() with Barycentric method
result = df.interpolate(method='barycentric')

print("\nResultant DataFrame after applying the interpolation:")
print(result)

Output

Following is the output of the above code −

Original DataFrame:
     A      B
0  1.1   0.25
1  NaN    NaN
2  3.5    NaN
3  NaN   4.70
4  NaN  10.00
5  NaN  14.70
6  6.2   1.30
7  7.9   9.20

Resultant DataFrame after applying the interpolation:
          A          B
0  1.100000   0.250000
1  2.596429  57.242857
2  3.500000  24.940476
3  4.061429   4.700000
4  4.531429  10.000000
5  5.160714  14.700000
6  6.200000   1.300000
7  7.900000   9.200000

Handling Limits in Interpolation

By default, Pandas interpolation fills all the missing values, but you can limit how many consecutive NaN values are filled using the limit parameter of the interpolate() method.

Example

The following example demonstrates filling the missing values of a Pandas DataFrame by limiting the consecutive fills using the limit parameter of the interpolate() method.

import numpy as np
import pandas as pd

df = pd.DataFrame({"A": [1.1, np.nan, 3.5, np.nan, np.nan, np.nan, 6.2, 7.9],
"B": [0.25, np.nan, np.nan, 4.7, 10, 14.7, 1.3, 9.2],
})

print("Original DataFrame:")
print(df)

# Applying the interpolate() with limit
result = df.interpolate(method='spline', order=2, limit=1)

print("\nResultant DataFrame after applying the interpolation:")
print(result)

Output

Following is the output of the above code −

Original DataFrame:
     A      B
0  1.1   0.25
1  NaN    NaN
2  3.5    NaN
3  NaN   4.70
4  NaN  10.00
5  NaN  14.70
6  6.2   1.30
7  7.9   9.20

Resultant DataFrame after applying the interpolation:
          A          B
0  1.100000   0.250000
1  2.231383  -1.202052
2  3.500000        NaN
3  4.111529   4.700000
4       NaN  10.000000
5       NaN  14.700000
6  6.200000   1.300000
7  7.900000   9.200000

Interpolating Time Series Data

Interpolation can be applied to the Pandas time series data as well. It is useful when filling gaps in missing data points over time.

Example

Example statement −

import numpy as np
import pandas as pd

indx = pd.date_range("2024-01-01", periods=10, freq="D")
data = np.random.default_rng(2).integers(0, 10, 10).astype(np.float64)
s = pd.Series(data, index=indx)
s.iloc[[1, 2, 5, 6, 9]] = np.nan

print("Original Series:")
print(s)

result = s.interpolate(method="time")

print("\nResultant Time Series after applying the interpolation:")
print(result)

Output

Following is the output of the above code −

Original Series:
2024-01-01    8.0
2024-01-02    NaN
2024-01-03    NaN
2024-01-04    2.0
2024-01-05    4.0
2024-01-06    NaN
2024-01-07    NaN
2024-01-08    0.0
2024-01-09    3.0
2024-01-10    NaN
Freq: D, dtype: float64

Resultant Time Series after applying the interpolation:
2024-01-01    8.000000
2024-01-02    6.000000
2024-01-03    4.000000
2024-01-04    2.000000
2024-01-05    4.000000
2024-01-06    2.666667
2024-01-07    1.333333
2024-01-08    0.000000
2024-01-09    3.000000
2024-01-10    3.000000
Freq: D, dtype: float64