Export daily level 3 products

This seems to be a simple task, but I haven’t yet found an easy and efficient way to do it.

I’m working on a python script where I can export daily (or even monthly) netcdf L3 files which I can later use for data analysis.

Let’s say I have 27 netcdf files for 2 days HCHO global coverage, how can I export 2 netcd files with the daily regridded data?

My current approach is to group the input files using wildcards, then I loop over the list with something like the following code. But this doesn’t seem to be the most efficient way.

product_path = '.../.../hcho_L2'  # level2 product path
export_path = '.../.../hcho_L3' # path for the created level3 files

input_files_OFFL = sorted(list(iglob(join(product_path, '**','*OFFL*HCHO*.nc'), recursive=True)))  # sorting level2 files into a list

operations = ";".join([
    "derive(tropospheric_HCHO_column_number_density [Pmolec/cm2])",
    "derive(latitude {latitude})",
    "derive(longitude {longitude})",

    "squash(time, (latitude, longitude, latitude_bounds, longitude_bounds))",

for i in input_files_OFFL:    
        harp_L2_L3 =harp.import_product(i, operations, reduce_operations=reduce_operations)
        export_folder = "{export_path}/{name}".format(export_path=export_path, name=i.split('/')[-1].replace('L2', 'L3'))  #renaming the new level3 files
        harp.export_product(harp_L2_L3, export_folder, file_format='netcdf')
    except harp.CLibraryError as e: # skipping corrupted files when generating the new list of level3 products

With the above code snippet I am able to convert every single L2 file to L3 file… but I wish I could export daily files rather then single retrieval L3 files. Can this be done with harpmerge?


You can do this with the above, but you have to split the filenames yourself into files per day.
If you have input_files_day1 and input_files_day2 you can just do harp_L2_L3 = harp.import_product(input_files_day1, operations, reduce_operations=reduce_operations). You should not loop over each individual product, but just provide all products from a single day to the harp.import_product function.