Repository navigation
Query about interpolation in gridded_timeseries.py #140
Description
Have a query about the interpolation done in grid_variable function in gridded_timeseries.py
Had a situation where in some timesteps the profile had nan data, which when interpolated seems to produce more nans that I expected. Consider the toy problem
print("np.interp test")
depth_bins = np.array([10, 20, 30, 40, 50])
depths = np.array([12, 27, 35, 46])
values = np.array([1, 2, np.nan, 5])
print(depth_bins)
interp_values = np.interp(depth_bins, depths, values, left=np.nan, right=np.nan)
print(interp_values)
# interpolate non-nan
nmask = np.logical_or(np.isnan(depths), np.isnan(values))
interp_values = np.interp(depth_bins, depths[~nmask], values[~nmask], left=np.nan, right=np.nan)
print(interp_values)which produces
np.interp test
[10 20 30 40 50]
[ nan 1.53333333 nan nan nan]
[ nan 1.53333333 2.47368421 4.05263158 nan]
What is the reasoning to want depth_bins 30 and 40 to be set to nan?
Activity
mphemming commentedon Jun 16, 2022
We are also having issues with these stray nans. See attached plot of missing gridded data in late 2021 between 16-24 m depth. I have detailed my debugging steps below and added a quick fix which seems to solve the problem.
When running the for loop on line 190 in 'grid_variable' function in 'gridded_timeseries.py' for a date when there is missing gridded data, I get the following:
then, after sorting on line 198 using function 'sort_depths' I get the following:
The variable 'depth_mask' assigns the near surface data as False and the interpolated data is removed. Although I don't fully understand why this happens yet using the above VoI_values and depth as input. My guess is because of the nan and the incorrect sorting.
As a quick fix I added the following on lines 197-201 before the depth and VoI are sorted using function 'sort_depths':
The hourly data is now interpolated throughout the water column during this time, with the 16-24 m white space in the plot attached now interpolated over.
mhidas commentedon Jan 11, 2023
@sspagnol @mphemming Apologies for the long delay in responding to this query. I have finally had a brief look, and I agree that we could do better in the case illustrated by Simon's toy example. I'll try to spend some more time on this in the next month or two!