# Problems with doas\_cl running tropomi inversion

**URL:** https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752
**Category:** QDOAS
**Tags:** sentinel-5p
**Created:** [January 27, 2024, 9:54am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752 "2024-01-27T09:54:54Z")
**Posts on this page:** 17
**Page:** 1

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [January 27, 2024, 9:54am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/1 "2024-01-27T09:54:54Z")

</div>

When using the doas\_cl command to invert one of tropomi’s radiation files, my results never come out, is this because the number of inversions is too high (3246\*450)? I have 500G of Linux memory, what should I do to get the inversion result properly.

 ![image](https://forum.atmospherictoolbox.org/uploads/default/original/1X/508468c816b2fc6cf65b22e8d5f55caf2b15caaa.png)

---

<div class="post-metadata">

### Author: ![thomasd](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/t/5f8ce5/32.png) [@thomasd](https://forum.atmospherictoolbox.org/u/thomasd)
#### Post date: [February 14, 2024, 8:52am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/2 "2024-02-14T08:52:22Z")

</div>

Hi,

(sorry for the delay, I was away from work for 2 weeks)

500G of memory should be more than enough. Did you enable output of analysis results in your configuration (see project properties → Output)?

Thomas

---

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [February 20, 2024, 9:40am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/4 "2024-02-20T09:40:12Z")

</div>

Thank you very much for your reply. I am able to invert a small number of Tropomi pixels and output the file. However, if you invert a Tropomi file (approximately 3246 \* 450 pixels) at once, it will result in slow runtime and the above error. Do you have any other methods?

---

<div class="post-metadata">

### Author: ![thomasd](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/t/5f8ce5/32.png) [@thomasd](https://forum.atmospherictoolbox.org/u/thomasd)
#### Post date: [February 20, 2024, 10:38am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/5 "2024-02-20T10:38:44Z")

</div>

Hi, the error message is not visible (anymore?), can you share it again?

---

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [February 20, 2024, 11:15am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/7 "2024-02-20T11:15:11Z")

</div>

![image](https://forum.atmospherictoolbox.org/uploads/default/original/1X/508468c816b2fc6cf65b22e8d5f55caf2b15caaa.png)

I think this is caused by too many pixels in Tropomi inversion, as a small amount of inversion can output files. What techniques did you use when inverting Tropomi?

---

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [February 20, 2024, 11:16am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/8 "2024-02-20T11:16:43Z")

</div>

![image](https://forum.atmospherictoolbox.org/uploads/default/original/1X/f2bcc1f30f594d060d9c4884fff44a46bd7a6716.png)  
This is my output settings

---

<div class="post-metadata">

### Author: ![thomasd](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/t/5f8ce5/32.png) [@thomasd](https://forum.atmospherictoolbox.org/u/thomasd)
#### Post date: [February 20, 2024, 11:55am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/9 "2024-02-20T11:55:50Z")

</div>

This does look like a memory issue. Could you show the result of running `free --giga -h -t` at the command line?

---

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [February 20, 2024, 12:49pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/11 "2024-02-20T12:49:02Z")

</div>

![image](https://forum.atmospherictoolbox.org/uploads/default/original/1X/cb11b1d53789dae0986349223bccedf03c9c2445.png)

I ran it again, this section needs to be displayed for a long time, and then an error occurred

 ![image](https://forum.atmospherictoolbox.org/uploads/default/original/1X/7e78ca879286f278bbde97c76cf3750e80f020d3.png)

---

<div class="post-metadata">

### Author: ![thomasd](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/t/5f8ce5/32.png) [@thomasd](https://forum.atmospherictoolbox.org/u/thomasd)
#### Post date: [February 20, 2024, 12:58pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/12 "2024-02-20T12:58:43Z")

</div>

I’m sorry to say, but it looks like you run into a qdoas bug. With such an amount of available memory, there should not be any problem. I will try to reproduce the problem here. Which version of qdoas are you using (output of doas\_cl --version?).

---

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [February 20, 2024, 1:13pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/13 "2024-02-20T13:13:02Z")

</div>

![image](https://forum.atmospherictoolbox.org/uploads/default/original/1X/7359f626fbcc867d59a525bd6fe37879ddce2afa.png)  
Tropomi satellite has over a million pixels in one file that need to be inverted. Is this the reason for this problem? Because if I only invert the part, I can produce the result file.

---

<div class="post-metadata">

### Author: ![thomasd](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/t/5f8ce5/32.png) [@thomasd](https://forum.atmospherictoolbox.org/u/thomasd)
#### Post date: [February 21, 2024, 1:50pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/14 "2024-02-21T13:50:41Z")

</div>

It seems like that has something to do with it, only we have not seen this behaviour before. Normally, about 10GB of memory is enough to process one Tropomi orbit.

If you could share your configuration file with me (via e-mail), that could be helfpful.

Thomas

---

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [February 21, 2024, 2:18pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/15 "2024-02-21T14:18:40Z")

</div>

Thanks for the reply， What is your e-mail address, please? I will send you the configuration file

---

<div class="post-metadata">

### Author: ![thomasd](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/t/5f8ce5/32.png) [@thomasd](https://forum.atmospherictoolbox.org/u/thomasd)
#### Post date: [February 21, 2024, 2:19pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/16 "2024-02-21T14:19:43Z")

</div>

Also, could you provide the output of `ulimit -a` on your system? This is to check if there is a limit on process memory usage.

Thomas

---

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [February 21, 2024, 2:28pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/17 "2024-02-21T14:28:19Z")

</div>

![image](https://forum.atmospherictoolbox.org/uploads/default/original/1X/654d786269d9891e3fc78fb71d8b4eb445b563f7.png)

---

<div class="post-metadata">

### Author: ![thomasd](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/t/5f8ce5/32.png) [@thomasd](https://forum.atmospherictoolbox.org/u/thomasd)
#### Post date: [February 22, 2024, 8:54am UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/18 "2024-02-22T08:54:29Z")

</div>

Hi, I’ve found the cause of the problem. There was a memory leak in our code, which has little impact in optical density fitting mode (and therefore went unnoticed), but has a huge impact in intensity fitting mode.

We’ll make a new qdoas release as soon as possible. In the mean time, I suggest you try the optical density fitting mode and see if you can get the results you need that way (it has the extra advantage of being much faster than intensity fitting 🙂 ).

Thank you for reporting the problem and for your help in identifying the cause.

Thomas

---

<div class="post-metadata">

### Author: ![thomasd](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/t/5f8ce5/32.png) [@thomasd](https://forum.atmospherictoolbox.org/u/thomasd)
#### Post date: [February 26, 2024, 3:06pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/19 "2024-02-26T15:06:36Z")

</div>

We’ve released Qdoas 3.6.3, which includes a fix for the memory issue. I recommend you upgrade to this version (the memory leak in version 3.6.2 also occurs when using optical density fitting, though it is not as bad as in intensity fitting mode, so you may not immediately notice it on a system with 500GB of memory).

---

<div class="post-metadata">

### Author: ![lijie](https://forum.atmospherictoolbox.org/letter_avatar_proxy/v4/letter/l/3d9bf3/32.png) [@lijie](https://forum.atmospherictoolbox.org/u/lijie)
#### Post date: [February 26, 2024, 3:18pm UTC](https://forum.atmospherictoolbox.org/t/problems-with-doas-cl-running-tropomi-inversion/752/20 "2024-02-26T15:18:12Z")

</div>

Thank you very much for your reply. I will update immediately.
