# Problem with HARP/Python within a loop

**URL:** https://forum.atmospherictoolbox.org/t/problem-with-harp-python-within-a-loop/296
**Category:** HARP
**Tags:** python
**Created:** [March 22, 2021, 7:58pm UTC](https://forum.atmospherictoolbox.org/t/problem-with-harp-python-within-a-loop/296 "2021-03-22T19:58:17Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![corisanucci](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/corisanucci/32/120_2.png) [@corisanucci](https://forum.atmospherictoolbox.org/u/corisanucci)
#### Post date: [March 22, 2021, 7:58pm UTC](https://forum.atmospherictoolbox.org/t/problem-with-harp-python-within-a-loop/296/1 "2021-03-22T19:58:17Z")

</div>

Hi! i’m getting an error message in python when i want to extract the data from a harp imported product into an array, i dont know why this is happening. This is the line with the error:  
`Z = np.array(SCI_CH4_filt.CH4_column_number_density.data)`  
`AttributeError: Object 'Product' does not have attribute 'CH4_column_number_density'`

`SCI_CH4_filt` is the product I get after applying filters and `bin_spatial()` with harp. Inside it has the `CH4_column_number_density` field with the data that I am interested in extracting.

The reason I am totally lost is because the exact same line does not throw any errors and works perfectly within a test code for a single file. The error arises when I run the same code (exactly the same) but inside a for loop to extract data from multiple files, one at a time. The error arises for all files.

Has anyone encountered this problem maybe? Perhaps this is not the place for the query, if that is the case I will delete the comment of course, thank you.

---

<div class="post-metadata">

### Author: ![sander.niemeijer](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/sander.niemeijer/32/5_2.png) [@sander.niemeijer](https://forum.atmospherictoolbox.org/u/sander.niemeijer)
#### Post date: [March 23, 2021, 8:45am UTC](https://forum.atmospherictoolbox.org/t/problem-with-harp-python-within-a-loop/296/2 "2021-03-23T08:45:38Z")

</div>

Can you maybe show the full code that you are running in the for loop?

And normally Python just stops when an error occurs, so I am guessing you are only getting this error for the first product where this happens (not all of them). Unless you are using a try/except block to catch the error and continue.

---

<div class="post-metadata">

### Author: ![corisanucci](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/corisanucci/32/120_2.png) [@corisanucci](https://forum.atmospherictoolbox.org/u/corisanucci)
#### Post date: [March 23, 2021, 2:11pm UTC](https://forum.atmospherictoolbox.org/t/problem-with-harp-python-within-a-loop/296/3 "2021-03-23T14:11:02Z")

</div>

Yes, here I paste the code 🙂. I’m using try/except because every so often a file throws a NoDataError when calling the CH4 dataset to apply filters. But when I test this code it throws me the except message for all the files, that’s why I had to test without the try/except block to see what is happening (and there I found the AttributeError)

> ```
> for i in SCIAfiles:
> 
> ```

```
    # try: # puede haber archivos sin datos (NoDataError)

SCI_CH4 = harp.import_product(i, options='dataset=nad_ir1_ch4')  

product = coda.open(i) # abro el mismo archivo desde coda

SCI_CH4.err_CH4 = harp.Variable(coda.fetch(product, "nad_ir1_ch4",-1,"non_linear_fit_param_err",0), ["time"]) 
SCI_CH4.err_CO2 = harp.Variable(coda.fetch(product, "nad_ir1_ch4",-1,"non_linear_fit_param_err",1), ["time"]) 
SCI_CH4.err_H2O = harp.Variable(coda.fetch(product, "nad_ir1_ch4",-1,"non_linear_fit_param_err",2), ["time"])  
coda.close(product) 

SCI_CH4_filt = harp.execute_operations(SCI_CH4,
                                       operations='scan_direction_type == "forward";'
                                                  'latitude > -59 [degree_north];'
                                                  'latitude < -14 [degree_north];'
                                                  'longitude > -80 [degree_east];'
                                                  'longitude < -47 [degree_east];'
                                                  'CH4_column_number_density_validity=&49152;'
                                                  'err_CH4>=0;err_CO2>=0;err_H2O>=0;'
                                                  'err_CH4<0.005;err_CO2<0.01;'
                                                  'keep(longitude_bounds, latitude_bounds,
                                                  'CH4_column_number_density);'
                                                  'bin_spatial(86,-58,0.5,62,-79,0.5)')

# Genero una grilla/array
x = np.linspace(-48, -79, 62) # (lon_left, lon_right, ncols)
y = np.linspace(-15, -58, 86) # (lat_bottom, lat_top, nrows)
X, Y = np.meshgrid(x, y)

# Guardo los datos de CH4 en un array
Z = np.array(SCI_CH4_filt.CH4_column_number_density.data)

```

The code continues, what I do is bring the lat/lon grid and the CH4 data to a GeoTIFF. In order not to take up so much space here I copy this first part which is the one that throws me the error in the last line. The strange thing is that exactly the same code tested in a single file (that is, not inside the for loop) works fine. But inside the for loop that **same file** also throws an error, that’s why I don’t understand.

I am using **HARP 1.13** (the latest) in PyCharm. Yesterday I decided to try using harp.export\_product() to directly export the `SCI_CH4_filt` product as a NetCDF, it worked with the first test files but now when I want to do it it throws me another error 😂:

```
in export_product raise CLibraryError()
harp._harppy.CLibraryError: Permission denied (D:/Tesina/SCIAMACHY/SCIAMACHY_L3/L3_diarios_py/SCI_CH4_L3_20020802_011749.nc)

```

But I didn’t change any line! so, I don’t know why it stoped working haha.  
I appreciate any help or recommendation!

---

<div class="post-metadata">

### Author: ![corisanucci](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/corisanucci/32/120_2.png) [@corisanucci](https://forum.atmospherictoolbox.org/u/corisanucci)
#### Post date: [March 25, 2021, 3:12pm UTC](https://forum.atmospherictoolbox.org/t/problem-with-harp-python-within-a-loop/296/4 "2021-03-25T15:12:27Z")

</div>

Hello everyone! I was able to fix it, I removed the latitude longitude filter at the beginning (inside `harp.execute_operations()`) and it works. I don’t know exactly why, but the same problem appeared in R, apparently that filter leaves me the harp product empty, that’s why I couldn’t call the `CH4_column_number_density` data.

As for the other error (`harp._harppy.CLibraryError: Permission denied`), it was simply a working directory issue (I assigned the folder where I wanted to save the NetCDF as the working directory and voila).

I leave this here in case it helps someone else, as well as reading other topics on this forum has helped me a lot! 💪🏽
