Cannot keep non-existent variable aerosol_optical_thickness_NIR

HARP
1

Hi,
I am trying to use harp to put the variable aerosol_optical_thickness_NIR into a regular gridded netCDF file. I am working with sentinel S5P offline Level 2 methane files (S5P_OFFL_L2__CH4____yyyy*.nc)

I get the error
cannot keep non-existent variable aerosol_optical_thickness_NIR
I noticed that when I print the variables in the netCDF file using harp.import_product(filename), print(data), aerosol_optical_thickness_NIR is not included in the listed

However, when I look at the variables in the netCDF header (using ncdump) the variable aerosol_optical_thickness_NIR is there.

Does harp.import_product not include all the variables that are in the netCDF file?

My code below.

Any advice on how to extract and regrid this variable would be great. Thanks

operations = “;”.join([
“keep(latitude_bounds,longitude_bounds,datetime_start,datetime_length,aerosol_optical_thickness_NIR)”,
“derive(datetime_stop {time} [days since 2000-01-01])”,
“derive(datetime_start [days since 2000-01-01])”,
“exclude(datetime_length)”,
“bin_spatial(1801,-90,0.1,3601,-180,0.1)”,
“derive(latitude {latitude})”,
“derive(longitude {longitude})”,
])

2

HARP indeed does not include all variables of the source product. It is an ‘importer’ and makes several translations to arrive at a HARP compliant representation of the data. For S5P CH4, the exact mapping of what variables are imported can be found at the bottom of the document page about this import. We currently only import the SWIR AOD, which is then available as aerosol_optical_depth.

3

Thank you so much for the quick reply. Could you recommend an alternative package that would do this?

4

You can always directly read the product via the netCDF4 library if you want to access data that is not supported by HARP.

But we could potentially also extend HARP to allow reading of this information. Is there a specific reason why you would specifically need the NIR AOD from, specifically, the CH4 product?

Also, if you want the AOD, have you considered using the pre-operational S5P AOD product?

5

I was hoping to create gridded netCDF files for all the variables that are used in the calculation of the qa_value. The purpose is to understand which variables are important in particular regions. I would like to recreate the data used in Figure 3. of this paper. Thank you for the link to the pre-operational AOD product. Would this data be the same that is used in the CH4 retrival algorthim?

6

I had a discussion internally, and we will add a HARP ingestion option to allow reading the NIR AOD data as wel. This should become available somewhere in March.

7

That is great. Thank you for adding this extra feature. It will be very useful.