# Harp error handling in IDL

**URL:** https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107
**Category:** HARP
**Created:** [April 7, 2020, 7:34pm UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107 "2020-04-07T19:34:33Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![mariliza](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/m/c77e96/32.png) [@mariliza](https://forum.atmospherictoolbox.org/u/mariliza)
#### Post date: [April 7, 2020, 7:34pm UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/1 "2020-04-07T19:34:34Z")

</div>

Hi Sander and colleagues,

Apologies in advance for a, I am sure, simpleton question!

I am ingesting S5P OFFL CO data using:

product = harp\_import(file,‘CO\_column\_number\_density\_validity\>50;derive(CO\_column\_volume\_mixing\_ratio {time} [ppbv]);area\_inside\_area((39.6335, 39.6335  
, 41.6335, 41.6335),(21.9572,23.9572,23.9572,21.9572));keep(latitude, longitude, solar\_zenith\_angle, CO\_column\_number\_density\_validity, orbit\_index, su  
rface\_pressure, CO\_column\_number\_density, sensor\_zenith\_angle, CO\_column\_number\_density\_uncertainty, scan\_subindex, H2O\_column\_number\_density, CO\_colum  
n\_volume\_mixing\_ratio,datetime\_start, datetime\_length, pressure\_bounds)’)

One of orbits does not pass the criteria and IDL replies with

% HARP-IDL ERROR -300: “invalid polygon (line segments overlap)”

The structure now contains the error

IDL\> help, product,/str  
\*\* Structure HARP\_ERROR, 2 tags, length=24, data length=18:  
ERRNO INT -300  
MESSAGE STRING ‘invalid polygon (line segments overlap)’

Can someone help me with catching the error and jumping through the rest of my analysis so that the code simply moves to the next file?

Thanks!  
MariLiza

---

<div class="post-metadata">

### Author: ![sander.niemeijer](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/sander.niemeijer/32/5_2.png) [@sander.niemeijer](https://forum.atmospherictoolbox.org/u/sander.niemeijer)
#### Post date: [April 7, 2020, 8:25pm UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/2 "2020-04-07T20:25:13Z")

</div>

Hi Mariliza,

Welcome to the forum!

The issue you raised is actually an interesting one. A similar problem was also found within the S5P-MPC as part of the automated validation (which is based on the HARP tools).

In that case the issue was found in S5P\_RPRO\_L2\_\_O3\_\_\_\_\_20180704T133322\_20180704T151649\_03748\_01\_010105\_20190212T074951.nc, but there are potentially other orbits that have a similar problem.

The problem is that for the S5P data at the end of the orbit there can be a bit of curl. You can see this if you plot the sensor zenith angle of the full product and zoom in on Alaska. You will then see this:

 ![l1_zenith_angle](https://forum.atmospherictoolbox.org/uploads/default/original/1X/19553df70ada94b8f4c16605f08b5b7fc9e37e43.png)  
This back curling in the southern corner causes at least one ground pixel to flip around and it ends up with a bow-tie shape (which is invalid in most GIS applications as it is in HARP).

The problem is actually not just in the L2, but also in L1B (S5P\_OFFL\_L1B\_RA\_BD1\_20180704T133421\_20180704T151550\_03748\_01\_010000\_20180704T170457.nc).  
It is actually an instrument sensing artefact. As we got clarified by the instrument team:

> Once a day the solar irradiance is measured over the North Pole.  
> To improve the accuracy of this measurement, it is executed at a reference azimuth angle to remove seasonal effects.  
> In order to achieve this reference azimuth angle, the spacecraft performs a yaw maneuver prior to the solar measurement.  
> When doing this, the instrument is still in radiance measurement mode.  
> The combination of the speed of the yaw angle change, together with the instrument wide swath, and the spacecraft along track speed causes the observed radiance scanlines to cross at certain times.
> 
> In addition, some measurements around nadir will not have rectangular ground pixel corner points, but rather the coverage resembles a bow-tie.  
> This may look strange, but is physically correct.

So there you have it.

Now, for your HARP operations. You are hitting this error because you are trying an `area_inside_area` matching filter. This filter will match _all_ satellite pixel areas against your area and therefore also try this weird pixel in Alaska. The `area_inside_area` operation is quite inefficient because of all these area matching operations. You should at least start of with a plain latitude/longitude filter (which is very efficient). And then, if this mid point filtering is not enough for you purposes, you could fine tune this by still adding the `area_inside_area` filter afterwards. And since you are then only filtering on satellite pixels that match the lat/lon range, you will not include this weird pixel at the end of the orbit anymore and thus your error should disappear.

So, use:

```
latitude>=39.6335;latitude<=41.6335;longitude>=21.9572;longitude<=23.9572

```

Or:

```
latitude>=39.6335;latitude<=41.6335;longitude>=21.9572;longitude<=23.9572;area_inside_area((39.6335, 39.6335, 41.6335, 41.6335),(21.9572,23.9572,23.9572,21.9572))

```

---

<div class="post-metadata">

### Author: ![ryan](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/r/a8b319/32.png) [@ryan](https://forum.atmospherictoolbox.org/u/ryan)
#### Post date: [April 8, 2020, 6:35am UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/3 "2020-04-08T06:35:51Z")

</div>

Very interesting!

Dear Sander,

May I ask what is the difference between **bin\_spatial()** and **area\_inside\_area()** to filter data. I know that bin\_spatial() also dictates the grid pixel size but I was also using it on S5P data to consider only an area of interest.

Thanks,  
Ryan

---

<div class="post-metadata">

### Author: ![sander.niemeijer](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/sander.niemeijer/32/5_2.png) [@sander.niemeijer](https://forum.atmospherictoolbox.org/u/sander.niemeijer)
#### Post date: [April 8, 2020, 6:53am UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/4 "2020-04-08T06:53:09Z")

</div>

The L2 data are considered a flat time series of pixels by HARP (so all data is only dependent on the time dimension). With `area_inside_area` you can filter this list to just those pixels that fall into a specific area (considering the area of each satellite pixel v.s. the area you define as parameter to the operation). But the result is still a time series of satellite pixels.

With `bin_spatial` you modify the data by turning it into grid with latitude and longitude dimensions (it is not a filter!). This will result in each target grid cell to have the weighted average of all satellite pixels that have an overlap.

---

<div class="post-metadata">

### Author: ![mariliza](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/m/c77e96/32.png) [@mariliza](https://forum.atmospherictoolbox.org/u/mariliza)
#### Post date: [April 8, 2020, 11:57am UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/5 "2020-04-08T11:57:26Z")

</div>

> [@svniemeijer](#):
>
> latitude\>=39.6335;latitude\<=41.6335;longitude\>=21.9572;longitude\<=23.9572

Dear Sander,  
Thank you so much for getting back to me so promptly!  
I would never have guessed, of course. Which brings to mind another question: what happens if the ingest command does not find any data that full fill the requirements? what does the command reply then? an empty product structure or another error? is there a way to catch that one?  
In case you are interested, the orbit that caused the original error is this:  
S5P\_OFFL\_L2\_\_CO\_\_\_\_\_20190706T103007\_20190706T121137\_08953\_01\_010302\_20190712T094815.nc  
Code works fine so far, after adding the min/max requirements.  
Many thanks,  
MariLiza

---

<div class="post-metadata">

### Author: ![mariliza](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/m/c77e96/32.png) [@mariliza](https://forum.atmospherictoolbox.org/u/mariliza)
#### Post date: [April 8, 2020, 11:59am UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/6 "2020-04-08T11:59:17Z")

</div>

Ah, while I was writing the reply, indeed I got an error:

CO/S5P\_OFFL\_L2\_\_CO\_\_\_\_\_20190731T091909\_20190731T110039\_09307\_01\_010302\_20190806T084611.nc  
% HARP-IDL ERROR -900: “no data left after operation”

🙂

MariLiza

---

<div class="post-metadata">

### Author: ![sander.niemeijer](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/sander.niemeijer/32/5_2.png) [@sander.niemeijer](https://forum.atmospherictoolbox.org/u/sander.niemeijer)
#### Post date: [April 8, 2020, 4:37pm UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/7 "2020-04-08T16:37:11Z")

</div>

The easiest way that I know of to check for an harp error in IDL is to look at the type of the return value.  
`typename(product)` will be `ANONYMOUS` when a HARP product is returned and will be `HARP_ERROR` if an error occurred.  
So just compare the type name against `HARP_ERROR` and if this succeeds then you can perform your error handling.

---

<div class="post-metadata">

### Author: ![mariliza](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/m/c77e96/32.png) [@mariliza](https://forum.atmospherictoolbox.org/u/mariliza)
#### Post date: [April 8, 2020, 5:51pm UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/8 "2020-04-08T17:51:23Z")

</div>

Thank you! Is there an HARP-related IDL “manual” somewhere perhaps and I couldn’t find it on GitHub? I recall from the era of BEAT a link with IDL examples, relevant commands, etc? or maybe I recall wrong. In any case, thank you again!  
MariLiza

---

<div class="post-metadata">

### Author: ![sander.niemeijer](https://forum.atmospherictoolbox.org/user_avatar/forum.atmospherictoolbox.org/sander.niemeijer/32/5_2.png) [@sander.niemeijer](https://forum.atmospherictoolbox.org/u/sander.niemeijer)
#### Post date: [April 8, 2020, 6:06pm UTC](https://forum.atmospherictoolbox.org/t/harp-error-handling-in-idl/107/9 "2020-04-08T18:06:28Z")

</div>

We have the [IDL interface](http://stcorp.github.io/harp/doc/html/idl.html) documentation, but that is indeed a bit minimal.

However, most users we know are using (or moving to) Python, so examples will mostly be provided in that langue.

Nevertheless, we now have a forum, which we didn’t have before. So if you need some custom examples, this forum will be the likely place for us to put it.
