# Cannot keep non-existent variable aerosol\_optical\_thickness\_NIR

**URL:** https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757
**Category:** HARP
**Created:** [February 27, 2024, 1:33pm UTC](https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757 "2024-02-27T13:33:17Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![sarahshannon](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/s/ad7895/32.png) [@sarahshannon](https://forum.atmospherictoolbox.org/u/sarahshannon)
#### Post date: [February 27, 2024, 1:33pm UTC](https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757/1 "2024-02-27T13:33:17Z")

</div>

Hi,  
I am trying to use harp to put the variable **aerosol\_optical\_thickness\_NIR** into a regular gridded netCDF file. I am working with sentinel S5P offline Level 2 methane files (S5P\_OFFL\_L2\_\_CH4\_\_\_\_yyyy\*.nc)

I get the error  
**cannot keep non-existent variable aerosol\_optical\_thickness\_NIR**  
I noticed that when I print the variables in the netCDF file using harp.import\_product(filename), print(data), aerosol\_optical\_thickness\_NIR is not included in the listed

However, when I look at the variables in the netCDF header (using ncdump) the variable aerosol\_optical\_thickness\_NIR is there.

Does harp.import\_product not include all the variables that are in the netCDF file?

My code below.

Any advice on how to extract and regrid this variable would be great. Thanks

operations = “;”.join([  
“keep(latitude\_bounds,longitude\_bounds,datetime\_start,datetime\_length,aerosol\_optical\_thickness\_NIR)”,  
“derive(datetime\_stop {time} [days since 2000-01-01])”,  
“derive(datetime\_start [days since 2000-01-01])”,  
“exclude(datetime\_length)”,  
“bin\_spatial(1801,-90,0.1,3601,-180,0.1)”,  
“derive(latitude {latitude})”,  
“derive(longitude {longitude})”,  
])

---

<div class="post-metadata">

### Author: ![sander.niemeijer](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/sander.niemeijer/32/5_2.png) [@sander.niemeijer](https://forum.atmospherictoolbox.org/u/sander.niemeijer)
#### Post date: [February 27, 2024, 1:47pm UTC](https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757/2 "2024-02-27T13:47:40Z")

</div>

HARP indeed does not include all variables of the source product. It is an ‘importer’ and makes several translations to arrive at a [HARP compliant representation](http://stcorp.github.io/harp/doc/html/conventions/index.html) of the data. For S5P CH4, the exact mapping of what variables are imported can be found at the bottom of the [document page about this import](http://stcorp.github.io/harp/doc/html/conventions/index.html). We currently only import the SWIR AOD, which is then available as `aerosol_optical_depth`.

---

<div class="post-metadata">

### Author: ![sarahshannon](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/s/ad7895/32.png) [@sarahshannon](https://forum.atmospherictoolbox.org/u/sarahshannon)
#### Post date: [February 27, 2024, 1:54pm UTC](https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757/3 "2024-02-27T13:54:39Z")

</div>

Thank you so much for the quick reply. Could you recommend an alternative package that would do this?

---

<div class="post-metadata">

### Author: ![sander.niemeijer](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/sander.niemeijer/32/5_2.png) [@sander.niemeijer](https://forum.atmospherictoolbox.org/u/sander.niemeijer)
#### Post date: [February 27, 2024, 2:15pm UTC](https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757/4 "2024-02-27T14:15:09Z")

</div>

You can always directly read the product via the netCDF4 library if you want to access data that is not supported by HARP.

But we could potentially also extend HARP to allow reading of this information. Is there a specific reason why you would specifically need the NIR AOD from, specifically, the CH4 product?

Also, if you want the AOD, have you considered using the [pre-operational S5P AOD product](https://data-portal.s5p-pal.com/products/aot.html)?

---

<div class="post-metadata">

### Author: ![sarahshannon](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/s/ad7895/32.png) [@sarahshannon](https://forum.atmospherictoolbox.org/u/sarahshannon)
#### Post date: [February 27, 2024, 2:47pm UTC](https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757/5 "2024-02-27T14:47:10Z")

</div>

I was hoping to create gridded netCDF files for all the variables that are used in the calculation of the qa\_value. The purpose is to understand which variables are important in particular regions. I would like to recreate the data used in Figure 3. of [this paper](https://www.nature.com/articles/s41598-023-41914-8). Thank you for the link to the pre-operational AOD product. Would this data be the same that is used in the CH4 retrival algorthim?

---

<div class="post-metadata">

### Author: ![sander.niemeijer](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/sander.niemeijer/32/5_2.png) [@sander.niemeijer](https://forum.atmospherictoolbox.org/u/sander.niemeijer)
#### Post date: [February 28, 2024, 8:50am UTC](https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757/6 "2024-02-28T08:50:50Z")

</div>

I had a discussion internally, and we will add a HARP ingestion option to allow reading the NIR AOD data as wel. This should become available somewhere in March.

---

<div class="post-metadata">

### Author: ![sarahshannon](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/s/ad7895/32.png) [@sarahshannon](https://forum.atmospherictoolbox.org/u/sarahshannon)
#### Post date: [February 28, 2024, 12:49pm UTC](https://forum.atmospherictoolbox.org/t/cannot-keep-non-existent-variable-aerosol-optical-thickness-nir/757/7 "2024-02-28T12:49:43Z")

</div>

That is great. Thank you for adding this extra feature. It will be very useful.
