GOME2 SO2 SCD error in coda.fetch ingestion

CODA
1

Greetings Sander,

I am trying to ingest a GOME-2A file GOME_O3-SO2_L2_20190623230357_050_METOPA_65782_DLR_03.HDF5 using the following:

              product = harp.import_product(ffile, options = 'detailed_results=SO2;corrected=true;so2_column=15km', \
                         operations = 'solar_zenith_angle <= 75; scan_direction_type==0;SO2_column_number_density_validity!& 255;\
                         derive(SO2_column_number_density [DU]);derive(SO2_slant_column_number_density [DU]);\
                         derive(SO2_column_number_density_uncertainty [DU]); \
                         keep(latitude,longitude, solar_zenith_angle,\
                              SO2_column_number_density,SO2_column_number_density_uncertainty,\
                              SO2_slant_column_number_density, \
                              cloud_fraction,SO2_column_number_density_validity,  index)')
                                           
               with coda.open(ffile) as pf:
                    #  variables that are not in the ingestion definition need to be ingested by coda
                    # 0 is the first height, 15km
                    # 3 is the 4th species in the sequence, i.e. SO2
                    product.chisq = harp.Variable(coda.fetch(pf, "DETAILED_RESULTS/FittingChiSquare")[product.index.data,0], ["time"])
                    ESC = harp.Variable(coda.fetch(pf, "DETAILED_RESULTS/ESC")[product.index.data,3], ["time"])
                    ESC_Error = harp.Variable(coda.fetch(pf, "DETAILED_RESULTS/ESC_Error")[product.index.data,3], ["time"])
                    product.SO2_slant_column_number_density_uncertainty = harp.Variable(ESC_Error.data * 0.01 * ESC.data,["time"])

It is the second day of the Raikoke eruption, so there are plenty of valid data points. I tried with other files of the day as well, of course. I went through the ingestion definition, GOME2_L2_O3MOTO — HARP 1.25 documentation, and I understand that the 4th, i.e. python index 3, is the SO2 “window”. However, I get the following error in python.

IndexError Traceback (most recent call last)
Cell In[23], line 6
1 with coda.open(ffile) as pf:
2 # variables that are not in the ingestion definition need to be ingested by coda
3 # 0 is the first height, 15km
4 # 3 is the 4th species in the sequence, i.e. SO2
5 product.chisq = harp.Variable(coda.fetch(pf, “DETAILED_RESULTS/FittingChiSquare”)[product.index.data,0], [“time”])
----> 6 ESC = harp.Variable(coda.fetch(pf, “DETAILED_RESULTS/ESC”)[product.index.data,3], [“time”])
7 ESC_Error = harp.Variable(coda.fetch(pf, “DETAILED_RESULTS/ESC_Error”)[product.index.data,3], [“time”])
8 product.SO2_slant_column_number_density_uncertainty = harp.Variable(ESC_Error * 0.01 * ESC,[“time”])

IndexError: index 3 is out of bounds for axis 1 with size 2

I am using harp 1.23 on python 3.11.7. Any help very welcome as always!

Best wishes,
MariLiza

2

I think this is because you cannot consider the window index to be fixed.
See also the mapping description in HARP (bottom section). You need to use a window index using the index in MainSpecies[] that has the value ‘SO2’

1 Like
3

Well, this is a bit worrying. I always thought that the species within the GOME2 files are in the same order, always, as per Table in page 38 of the PUM, Microsoft Word - DLR_GOME-2_PUM_3B_Nov2019.

I.e. O3 is always 1st, SO2 is always 4th, etc.

Obviously, I was mistaken, print out of the META_DATA for the orbits of June 23rd 2019:

Indices of ‘SO2’: [3]
[‘O3’ ‘NO2’ ‘BrO’ ‘SO2’ ‘H2O’ ‘HCHO’]
Indices of ‘SO2’: [3]
[‘O3’ ‘NO2’ ‘BrO’ ‘SO2’ ‘H2O’ ‘HCHO’]
Indices of ‘SO2’: [3]
[‘O3’ ‘NO2’ ‘BrO’ ‘SO2’ ‘H2O’ ‘HCHO’]
Indices of ‘SO2’: [3]
[‘O3’ ‘NO2’ ‘BrO’ ‘SO2’ ‘H2O’ ‘HCHO’]
Indices of ‘SO2’: [3]
[‘O3’ ‘NO2’ ‘BrO’ ‘SO2’ ‘H2O’ ‘HCHO’]
Indices of ‘SO2’: [1]
[‘O3’ ‘SO2’]

I checked and this specific file’s filename only contains two species as well.

Will alter the python script, thanks!
MariLiza