Degree of freedom filtering with HARP for CH4 S5P files

HARP
,
1

Hi everyone,

I am working on a project that is looking to use a time-series of S5P data to identify CH4 trends/ sources etc. I have never used S5P data before so please excuse any obvious mistakes I make. I initially tried to use arcGIS to convert from L2 to a L3 gridded product although this did not work. I have since found the HARP library and I have followed the RUS training webinar (pollution monitoring with Sentinel-5p) to understand how to process to L3 data.

The project requires two data quality filters to be applied to the CH4: a qa_value filter (to be set at 0.5 initially and then adjust to either increase or decrease if needed) and a degree of freedom filter of methane to only include >1 : (PRODUCT/SUPPORT_DATA/DETAILED_RESULTS/degrees_of_freedom_methane) to only include values >1.

I have checked the entire forum but nobody seems to mention the degree of freedom filter. I have also checked the ingestions page and this does not seem to mention a variable.
http://stcorp.github.io/harp/doc/html/ingestions/S5P_L2_CH4.html

Is there a way in which this can be done using HARP? If not i would appreciate any advice/ example on how I could do this?

Thanks
Ty

2

For the quality filtering you should use what is published by ESA, which is described in the PRF documents on the ESA Sentinel-5P site. The current filtering recommendation is only based on the QA value.

Be aware that in HARP you will have to use CH4_column_volume_mixing_ratio_dry_air_validity>50, since HARP maps the validity value directly to the stored integer value (and does not convert the value to a fractional floating point value).

1 Like
3

Hi Sander,

Many thanks for your reply. I will only filter using the QA value.

Thanks
ty

4

Hi Sander,

I am trying to keep the methane mixing ratio precision variable as I need to know the uncertainty associated with each Methane mixing ratio measurement. When i attempt to do with with harp i get the following issue:

CLibraryError: cannot keep non-existent variable CH4_column_volume_mixing_ratio_dry_air_uncertainty

I have checked the ingestion’s for CH4 (S5P_L2_CH4 — HARP 1.13 documentation) and this is the exact field name for the variable so it doesn’t make sense that it would say it is non existent.

I have attached a screenshot of the error. If I am doing something wrong please advise me.

thanks
Ty

image
image1023×646 14.8 KB

5

The bin_spatial() operation does not include any uncertainty propagation, so you will loose this variable when you regrid the data. As a result, you cannot keep that variable after regridding.

6

Thanks for the quick reply and for clearing this up. Is there another way in which the uncertainty values can be extracted (with the same spatial grid) .

thanks
Ty

7

I am not sure the word ‘extracted’ is appropriate here. If you are regridding the data you are processing the data and, as such, you would have to process (=propagate) the uncertainties equally.
If you want to extract data, in the sense of retrieving what is in the file, then you should stick to the original satellite grid.

Note that uncertainty propagation is something we want to introduce in the HARP toolset at some point. But this is not an easy topic.

8

Thanks for that explanation Sander - I can appreciate that it is difficult to develop tools for S5P. I chose the wrong word - I am looking to process the uncertainty to the same spatial grid as the processed methane values.

What approach would take to asses the uncertainty of the methane products processed using HARP, given that uncertainty propagation cannot currently be processed using HARP? Or is this something that you do not account for?

Best wishes
Ty

9

I am not sure I understand what you want to do.

If you just want to look at the uncertainties, then use the original L2 grid and don’t perform the bin_spatial operation to turn the data into a L3 grid.

If you want to turn the L2 data into a L3 grid and also have the uncertainties for that L3 grid, then you would currently have to perform those L2->L3 calculations yourself.

10

Hi,

Thanks again for the reply.
What I am looking to do is to generate a time averaged composite of CH4 (in a L3 grid) and provide an uncertainty value for each pixel within this (I have read when you are performing time averaging then the uncertainty should be doubled - “Uncertainties for the XCH4 is only based on the single sounding precision due to measurement noise. For applications requiring an overall uncertainty estimate, we propose to multiply the provided error by a factor 2, which reflects the scatter of single sounding errors in the TCCON validation”)

For example - for a major CH4 emitter (say a powerplant etc) I would like to be able to provide an uncertainty value for the pixel in the L3 grid, over a single observation and also over a time average (e.g. average the uncertainties).

Forgive my naivety, but if I am to generate the L3 uncertainty values myself (as HARP currently cannot) is this simply the case of gridding the L2 uncertainty values to the same spatial grid as the L3 CH4 (performed in HARP) or is there something more complex that needs to be done to correctly process this?

thanks again for your time,

Ty

11

You would have to propagate the uncertainties consistent with the way the data is regridded. HARP uses area weighted averaging, which means that you also need to use an area weighted average of the uncertainties (averaging in quadrature). Calculating these weights per pixel is not trivial.

I will see whether we can add uncertainty propagation for the bin_spatial operation in HARP (which will then only be for the random component of the uncertainty, not the total uncertainty). Since it is an isolated algorithm, it might not be the that difficult.

1 Like