Dear Sander,
As you know we are working with the S5P_PAL__L2__AER_OT data files. As the AOT is reported on five wavelengths, the validity parameter also has a second dimension, spectral, of 5.
Are you planning to introduce a way to filter the orbital files in such a way that each of the 5 AOTs is filtered with its equivalent validity parameter via harp? as you can imagine, if we just ask for *_validity>50 during ingestion, curious outputs are provided. 
Of course we are using a second step, keeping the validity flag and post-filtering, so it is not a curcial issue right now. It might however be a useful feature to have if say someone wishes to ingest and spatiotemporally bin all in one go using harp.
Best wishes,
MariLiza
To do this you will have to ingest the product 5 times individually for each wavelength, but you can actually already do this with the current HARP release:
harpdump -a 'index(spectral)==1;squash(spectral, (aerosol_optical_depth, aerosol_optical_depth_validity, single_scattering_albedo));aerosol_optical_depth_validity>50' S5P_PAL__L2__AER_OT_20231001T013131_20231001T031301_30910_03_010100_20231009T222210.nc
And with the just released HARP 1.20.1 version you can actually filter on wavelength:
harpdump -a 'wavelength==380;squash(spectral, (aerosol_optical_depth, aerosol_optical_depth_validity, single_scattering_albedo));aerosol_optical_depth_validity>50' S5P_PAL__L2__AER_OT_20231001T013131_20231001T031301_30910_03_010100_20231009T222210.nc
1 Like
Excellent! Our sys admin is about to install 1.20.1 so we just need a little loop for the five wavelengths and we are good to go.
Cheers.