Pandas: Fill missing values in time series data

Last update on May 28 2022 13:39:50 (UTC/GMT +8 hours)

Pandas: DataFrame Exercise-74 with Solution

Write a Pandas program to fill missing values in time series data.

From Wikipedia , in the mathematical field of numerical analysis, interpolation is a type of estimation, a method of constructing new data points within the range of a discrete set of known data points.

Sample Solution :

Python Code :

import pandas as pd
import numpy as np
sdata = {"c1":[120, 130 ,140, 150, np.nan, 170], "c2":[7, np.nan, 10, np.nan, 5.5, 16.5]}
df = pd.DataFrame(sdata)
df.index = pd.util.testing.makeDateIndex()[0:6]
print("Original DataFrame:")
print(df)
print("\nDataFrame after interpolate:")
print(df.interpolate())

Sample Output:

Original DataFrame:
               c1    c2
2000-01-03  120.0   7.0
2000-01-04  130.0   NaN
2000-01-05  140.0  10.0
2000-01-06  150.0   NaN
2000-01-07    NaN   5.5
2000-01-10  170.0  16.5

DataFrame after interpolate:
               c1     c2
2000-01-03  120.0   7.00
2000-01-04  130.0   8.50
2000-01-05  140.0  10.00
2000-01-06  150.0   7.75
2000-01-07  160.0   5.50
2000-01-10  170.0  16.50

Python-Pandas Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Write a Pandas program to create DataFrames that contains random values, contains missing values, contains datetime values and contains mixed values.
Next: Write a Pandas program to use a local variable within a query.

What is the difficulty level of this exercise?

Test your Programming skills with w3resource's quiz.

Python: Tips of the Day

Find current directory and file's directory:

To get the full path to the directory a Python file is contained in, write this in that file:

import os 
dir_path = os.path.dirname(os.path.realpath(__file__))

(Note that the incantation above won't work if you've already used os.chdir() to change your current working directory, since the value of the __file__ constant is relative to the current working directory and is not changed by an os.chdir() call.)

To get the current working directory use

import os
cwd = os.getcwd()

Documentation references for the modules, constants and functions used above:

The os and os.path modules.
The __file__ constant
os.path.realpath(path) (returns "the canonical path of the specified filename, eliminating any symbolic links encountered in the path")
os.path.dirname(path) (returns "the directory name of pathname path")
os.getcwd() (returns "a string representing the current working directory")
os.chdir(path) ("change the current working directory to path")

Ref: https://bit.ly/3fy0R6m