Map Sentinel 5p CH4, SO2 and CO Data (Visan)

Your advice for making my monthly averages work in Panoply was great! I’ve finally completed my maps for methane and NO2. With CO and SO2 however, I’m still encountering difficulties. Since DU (Dobson Units) is the typial unit used for SO2 and ppbv is the typical unit used for CO, I looked into the difference between mixing ratios and column densities. I found that DU is a unit that measures column number density whereas ppbv, as you said, is a unit for mixing ratio. In Harp/Visan, both SO2 and CO (SO2_column_number_density and CO_column_number_density) come with the unit mol/m^2, a column density. I presume therefore that for SO2 it should be simple to derive DU given that they are the same type of unit but in your previous response you didn’t provide code for this.
Again I attempted to run this code but it didn’t work, if you could edit it so I can convert from the normal mol/m^2 unit to DU that would be great:
product = harp.import_product(r"/Users/jpoetzscher/Downloads/SO2 OCT 1/*.nc", operations=‘SO2_column_number_density_validity>50;keep(latitude_bounds,longitude_bounds,SO2_column_number_density);bin_spatial(1801,-90,0.1,3601,-180,0.1); derive(SO2_column_number_density [DU]) ’, post_operations=“bin();squash(time, (latitude_bounds,longitude_bounds));derive(latitude {latitude});derive(longitude {longitude});exclude(latitude_bounds,longitude_bounds,latitude_bounds_weight,longitude_bounds_weight,count,weight)”)

In terms of CO, I guess I need to translate from column number density to mixing ratio to go from mol/m^2 to ppbv. However, I integrated the line of code you provided in your previous response “derive(CO_column_volume_mixing_ratio {time} [ppbv])” and it didn’t work (Image below) I’d love to complete this today and I really appreciate your continuous help.

1 Like

The SO2 ingestion should work. What is the error that you are getting?

For CO, this doesn’t work since the keep() operation that is before the derivation throws out all the variables that are needed to derive the CO column vmr. To correct this we need to derive the xCO before the keep() operation, like:

product = harp.import_product(r"/Users/jpoetzscher/Downloads/CO OCT 1/*.nc",
    operations='CO_column_number_density_validity>50;derive(CO_column_volume_mixing_ratio {time} [ppbv]);keep(latitude_bounds,longitude_bounds,CO_column_volume_mixing_ratio);bin_spatial(1801,-90,0.1,3601,-180,0.1)',
    post_operations='bin();squash(time, (latitude_bounds,longitude_bounds));derive(latitude {latitude});derive(longitude {longitude});exclude(latitude_bounds,longitude_bounds,latitude_bounds_weight,longitude_bounds_weight,count,weight)')
1 Like

I think something just broke on my visan. When ever I try to input anything this happens

1 Like

Make sure to replace all ‘fancy’ quotes by regular quote characters.

yep that worked. So I must have messed something up with SO2 prior, because I ran it again and it worked. It looks a bit weird though with the underlying charcoal coloring. What is that? This is just for one day though so perhaps when I do the monthly average the grey will be hidden as there will be more colored pixels?

In terms of CO, the code ran error free but then when I go to plot it, it is in the form of mol/m^2

1 Like

I think I know what went wrong for the SO2 binning. You probably got the following error message

ERROR: variables don't have the same number of dimensions (SO2_column_number_density_weight)

This is a bug in HARP that I just fixed. You can work around it for the moment by removing the SO2_column_number_density_weight variable before binning (this will create slightly different values for the daily/monthly averages, but this should not be significant):

product = harp.import_product(r"/Users/jpoetzscher/Downloads/SO2 OCT 1/*.nc",
    operations='SO2_column_number_density_validity>50;keep(latitude_bounds,longitude_bounds,SO2_column_number_density);bin_spatial(1801,-90,0.1,3601,-180,0.1);derive(SO2_column_number_density [DU]);exclude(SO2_column_number_density_weight)',
    post_operations='bin();squash(time, (latitude_bounds,longitude_bounds));derive(latitude {latitude});derive(longitude {longitude});exclude(latitude_bounds,longitude_bounds,latitude_bounds_weight,longitude_bounds_weight,count,weight)')
1 Like

For the CO plot, you need to make sure to change CO_column_number_density to CO_column_volume_mixing_ratio in the keep() operation. Otherwise you just throw away the CO column vmr again :wink:

1 Like

Thanks! The CO worked!

The SO2 didn’t change it still has the grey background stuff but I’m confused, is that bad? I assumed thats just what the data looks like.

I am not sure what you mean with ‘grey background’. Do you mean the white spots? Those are locations where it wasn’t possible to calculate good SO2 values (due to our filtering on the quality flag). This could be because of clouds or other reasons. Having those gaps in a daily plot is quite normal.

No its difficult to explain so I just annotated the image and cicled the grey spots I’m referring to. It’s probably normal and I assume this is what the data should look like but I just want to make sure. If it is ok then we’ve basically worked through all the issues and so I want to thank you again.

Ah. Those black areas. This is also due to the color range clipping. If you would use a colorrange of e.g. [-3,3] you would also see values below 0.

These negative values are due to the high noise level. When you average over a longer time period this noise level should decrease. Another way to decrease the noise level is to use a coarser spatial grid.

I tried the NO2 code and found the single quotation marks-- ’ – should be changed to double quotation marks, then it can be run at VISAN.

Can this be done for NO2? I’m using following code:

harp.import_product(paths_2[i],
operations=“tropospheric_NO2_column_volume_mixing_ratio_validity>75;keep(latitude_bounds,longitude_bounds**,tropospheric_NO2_column_volume_mixing_ratio**);bin_spatial(7,35.7,0.1,7,14.1,0.1);derive(tropospheric_NO2_column_volume_mixing_ratio [ppbv])”,
post_operations= “bin();squash(time, (latitude_bounds,longitude_bounds));derive(latitude {latitude});derive(longitude {longitude});exclude(latitude_bounds,longitude_bounds,latitude_bounds_weight,longitude_bounds_weight,count,weight)”))

and getting the error: cannot filter on non-existent variable tropospheric_NO2_column_volume_mixing_ratio_validity

Thanks!

1 Like

Can this be done for NO2?

Why do you want the tropospheric NO2 in volume mixing ratios? This is not the natural quantity for NO2. It is also not what is in the product.
The way to handle NO2 is already shown in the first post of this topic.

Yes I was working fine with “tropospheric_NO2_column_number_density” but now I wish to correlate S5-P data with ground-base NO2 dataset in ppb. So this is not possible with the NO2 product?

If your ground based data is surface data, then this might be difficult. S5P provides a full tropospheric column, but is actually almost non sensitive at the surface (you can see this from the averaging kernel).
For validation you can check out what is done on the official validation server for S5P

1 Like

Hi I have been reading the conversation and has caused me a question.
If my purpose of evaluating air quality with satellite information and information from surface stations, wath variable I should use?.
And the variable nitrogendioxide_tropospheric_colum I find negative values, these ones refer to?.
Thanks

Hello @svniemeijer Sir,
Can I get to know the units for all the parametrs in which sentinel 5P is providing data and in which units we may use for better analysis

Thank You

The units in the products can be found in the user manuals of S5P.
The units that you get when ingesting with HARP are defined in the HARP ingestion documentation.
And what units to use for analysis is up to you. It depends on what you want to do.

Hello @svniemeijer Sir,
I have a query regarding the availability of Sentinel 5P Near Real Time data for 2019 now it was not available for whole world, is there any specific reason?