Hi,
I have been using HARP to preprocess S5P data on my computer. Recently, I wanted to migrate the processing pipeline to DataBricks. But installing HARP on DataBricks is challenging for me. Is there a place where whl can be found for HARP or if pip install is available?
Any help or suggestion is appreciated!
HARP is at its core a C library that again depends on other C libraries such as the HDF5 library. Packaging this all together into a single whl file is tricky and could create conflicts with other installed libraries.
The preferred approach is therefore to use conda environments that are able to correctly combine C libraries/tools and Python packages together in a single environment.
From what I can tell from the Databricks documentation they have support for conda. So I would suggest asking there on how to setup a conda environment with your selection of conda-forge packages (including HARP).
1 Like