Unexplained harp error with v1.17

Dear Sander,

I am encountering a harp error, via python, on a code that I was using back in February. The code is attached to the end of this message. It reads, via coda, the NASA OMI L2G daily product and outputs in harp-compliant form gridded monthly data. Nothing fancy. I repeat, this was working in February, with the previous harp version.

The error is :

File ~/.conda/envs/myCondaEnvName/lib/python3.9/site-packages/harp/_harppy.py:1110 in _export_product

  • _export_variable(name, product[name], c_product)*

  • File ~/.conda/envs/myCondaEnvName/lib/python3.9/site-packages/harp/_harppy.py:1068 in _export_variable*

  • raise CLibraryError()*

CLibraryError: invalid unit ‘degree_north’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  • File ~/.conda/envs/myCondaEnvName/lib/python3.9/site-packages/spyder_kernels/py3compat.py:356 in compat_exec*

  • exec(code, globals, locals)*

  • File /mnt/lapsat/groups/lapsat/OMI_SO2/Programs/harp_convert_OMI_L3.py:99*

  • average = harp.execute_operations(productlist,*

  • File ~/.conda/envs/myCondaEnvName/lib/python3.9/site-packages/harp/_harppy.py:1517 in execute_operations*

  • _export_product(product, c_product_ptr[0])*

  • File ~/.conda/envs/myCondaEnvName/lib/python3.9/site-packages/harp/_harppy.py:1112 in _export_product*

  • raise Error(“variable ‘%r’ could not be exported (%s)” % (name, str(_error)))*

Error: variable ‘‘latitude’’ could not be exported (invalid unit ‘degree_north’)

My conda env only contains module via the conda forge channel, and is a dedicated harp env. Conda list replies, for harp & coda:
harp 1.17 py39r42h92210e6_0 conda-forge
coda 2.24.1 py39h92210e6_0 conda-forge

I am running the script on spyder with
*Python 3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 15:55:03) *
Type “copyright”, “credits” or “license” for more information.
IPython 8.8.0 – An enhanced Interactive Python.

Any ideas on how this may be going wrong?
Many thanks

#!/usr/bin/env python3

-- coding: utf-8 --

Created on Thu Jun 25 12:06:13 2020

@author: mariliza

import coda
import harp
import numpy
import os
import calendar
from datetime import datetime, date
import sys
import numpy as np
import fnmatch

#from os import listdir

twelve months

mon = [i+1 for i in range(12)]
matchex = [“%02d” %x for x in mon]

local variables

path = ‘/mnt/lapsat/groups/lapsat/OMI_SO2/EvenMoreData/’
outpath = ‘/mnt/lapsat/groups/lapsat/OMI_SO2/HARP_EvenMoreData/’

output directory

if not os.path.exists(outpath):

load all the filenames

filenames = []
for file in os.listdir(path):
if file.endswith(“.he5”):

loop in months

for minas in range(len(matchex)):

productlist = []

# locate only the files within that month

allfilenames = []
for ffile in filenames: #os.listdir(folder):
    # if fnmatch.fnmatch(ffile.decode(encoding='utf-8'), str('*2021m'+matchex[minas]+'*')):
    if fnmatch.fnmatch(ffile, str('OMI-Aura_L3-OMSO2e_2023m'+matchex[minas]+'*')):
if len(allfilenames) == 0:

# read in all the files in that month
for ffile in allfilenames:
    filename = os.fsdecode(ffile)
    infile = path+filename
    pf = coda.open(infile)
        latitude = coda.fetch(pf, 'HDFEOS/GRIDS/OMI_Total_Column_Amount_SO2/Data_Fields/Latitude')
        longitude = coda.fetch(pf, '/HDFEOS/GRIDS/OMI_Total_Column_Amount_SO2/Data_Fields/Longitude')
        so2 = coda.fetch(pf, '/HDFEOS/GRIDS/OMI_Total_Column_Amount_SO2/Data_Fields/ColumnAmountSO2')
        # so2_scd = coda.fetch(pf,'/HDFEOS/GRIDS/OMI_Total_Column_Amount_SO2/Data_Fields/SlantColumnAmountSO2')
        time = coda.fetch(pf,'/HDFEOS/GRIDS/OMI_Total_Column_Amount_SO2/Data_Fields/Time')
        so2_flag = coda.fetch(pf, '/HDFEOS/GRIDS/OMI_Total_Column_Amount_SO2/Data_Fields/QualityFlags_SO2')
        sza = coda.fetch(pf, '/HDFEOS/GRIDS/OMI_Total_Column_Amount_SO2/Data_Fields/SolarZenithAngle')
        # sys.exit()

    product = harp.Product()

    lat = latitude[:,0]   
    lon = longitude[0,:]

    product.latitude = harp.Variable(lat, ['latitude']) #, longitude'])
    product.latitude.unit = 'degree_north'
    product.longitude = harp.Variable(lon, ['longitude'])
    product.longitude.unit = 'degree_east'
    product.so2_column_number_density = harp.Variable(so2, ['latitude', 'longitude'])
    product.so2_column_number_density.unit = 'DU'
    # product.so2_slant_column_number_density = harp.Variable(so2_scd, ['latitude', 'longitude'])
    # product.so2_slant_column_number_density.unit = 'DU'
    product.time = harp.Variable(time, ['latitude', 'longitude'])
average = harp.execute_operations(productlist, 
                post_operations="bin();squash(time, (latitude,longitude))")

# sys.exit()
OutFile = outpath + '2023'+matchex[minas]+'.nc'
# filename[19:23] + filename[24:28] + '.nc'
harp.export_product(average, OutFile)

# sys.exit()

This might be because the udunits configuration files could not be found. Do you have a set of udunits2*.xml files in ~/.conda/envs/myCondaEnvName/share/harp?
Or do you maybe have a UDUNITS2_XML_PATH environment variable set that points to a wrong location?

1 Like

Yes, I do have a set of udinits2*.xml files in my env
[mariliza@aristotle5 ~]$ ls ~/.conda/envs/myCondaEnvName/share/harp
doc udunits2-accepted.xml udunits2-base.xml udunits2-common.xml udunits2-derived.xml udunits2-prefixes.xml udunits2.xml

But you are right! The UDUNITS2_XML_PATH environment variable set that is in my *bashrc file was pointing in the wrong direction. Our sysadmins were upgrading harp&coda for me to use with IDL and this was not updated.

I did not know that even though I am in my conda env, the udunits are picked from the *bashrc. A bit strange to me, but I am no expert.

I reloaded everything, works fine. Many thanks!