Examples
Combine using a simple function that chooses the smaller column:
import numpy as np
import pandas as pd
df1 = pd.DataFrame({'X': [0, 0], 'Y': [5, 5]})
df2 = pd.DataFrame({'X': [2, 2], 'Y': [4, 4]})
take_smaller = lambda s1, s2: s1 if s1.sum() < s2.sum() else s2
df1.combine(df2, take_smaller)
Using a true element-wise combine function:
df1 = pd.DataFrame({'X': [5, 0], 'Y': [4, 6]})
df2 = pd.DataFrame({'X': [2, 2], 'Y': [3, 3]})
df1.combine(df2, np.minimum)
Using fill_value fills Nones prior to passing the column to the merge function:
df1 = pd.DataFrame({'X': [0, 0], 'Y': [None, 6]})
df2 = pd.DataFrame({'X': [2, 2], 'Y': [3, 3]})
df1.combine(df2, take_smaller, fill_value=-5)
If the same element in both dataframes is None, that None is preserved:
df1 = pd.DataFrame({'X': [0, 0], 'Y': [None, 6]})
df2 = pd.DataFrame({'X': [2, 2], 'Y': [None, 3]})
df1.combine(df2, take_smaller, fill_value=-5)
Use of overwrite and behavior when the axis differ between the dataframes:
df1 = pd.DataFrame({'X': [0, 0], 'Y': [6, 6]})
df2 = pd.DataFrame({'Y': [5, 5], 'Z': [-10, 2], }, index=[1, 2])
df1.combine(df2, take_smaller)
df1.combine(df2, take_smaller, overwrite=False)
Demonstrating the preference of the passed in dataframe:
df2 = pd.DataFrame({'Y': [4, 4], 'Z': [2, 2], }, index=[1, 2])
df2.combine(df1, take_smaller)
df2.combine(df1, take_smaller, overwrite=False)