Hi!
I’ve been having troubles importing OMI_MINDS_NO2d L3 datasets with HARP. Judging from the documentation it should be supported. Do you know what might be the issue?
Data is downloaded from NASA IT Security Warning Banner and the code simply looks like this (data path is altered here but should not matter to make the point):
import harp
prod = harp.import_product(‘./2004/OMI-Aura_L3-OMI_MINDS_NO2d_2004m1001_v01-01-2022m0218t093007.nc’)
I get an error message:
File ~/miniconda3/envs/clim/lib/python3.10/site-packages/harp/_harppy.py:1300, in import_product(filename, operations, options, reduce_operations, post_operations)
1297 # Import the product as a C product.
1298 if _lib.harp_import(_encode_path(filename), _encode_string(operations), _encode_string(options),
1299 c_product_ptr) != 0:
-> 1300 raise CLibraryError()
1302 try:
1303 # Raise an exception if the imported C product contains no variables, or variables without data.
1304 if _lib.harp_product_is_empty(c_product_ptr[0]) == 1:
CLibraryError: ./2004/OMI-Aura_L3-OMI_MINDS_NO2d_2004m1001_v01-01-2022m0218t093007.nc: unsupported product
Could it be a data version issue?
Thanks in advance for your support!
Somehow, confusingly, NASA is producing two different NO2d products. There is the OMNO2d 003 product and the OMI_MINDS_NO2d 1.1. The same issue also exists for the regular L2 products OMNO2 vs OMI_MINDS_NO2.
You can find the full overview of NASA OMI NO2 products here.
We currently don’t support the MINDS products, and I don’t understand why NASA is providing these next to the ‘original’ OMI NO2 products. (note also the difference in the end data: the OMNO2 products are produced faster than the MINDS ones).
It seems that the MINDS products are part of a multi-satellite product generation project. Since the core algorithm for this is the one used for OMI NO2, I do not expect much difference between the OMI MINDS NO2 and OMNO2 products. But adding support to HARP for the full suite of MINDS products (which includes SCIAMACHY, GOME-2, and Sentinel-5P) could be useful. We actually already created a ticket on this.
Yes, you are completely right. I actually tried the same thing with both of them before and still run into the same issue with the other one. Here I gave the example of MINDS since it actually seems like HARP supports it because of the table in documentation (Ingestion definitions — HARP 1.20 documentation):
We will fix the description. Thanks for pointing this out.
I am not sure what you mean with “still run into the same issue with the other one”. Are you saying that the OMNO2d 003 product also doesn’t work for you? Because I just tried to ingest OMI-Aura_L3-OMNO2d_2004m1001_v003-2019m1121t082956.he5 and that seemed to go fine.
Aha! I found the issue. Yes, initially also the OMNO2d 003 product could not be ingested, but since it worked for you, I checked what might be wrong in my installation. I was running HARP v1.17. Now updated to v1.20 and it works. Thanks a lot for your support!
