When using the doas_cl command to invert one of tropomi’s radiation files, my results never come out, is this because the number of inversions is too high (3246*450)? I have 500G of Linux memory, what should I do to get the inversion result properly.
Hi,
(sorry for the delay, I was away from work for 2 weeks)
500G of memory should be more than enough. Did you enable output of analysis results in your configuration (see project properties → Output)?
Thomas
Thank you very much for your reply. I am able to invert a small number of Tropomi pixels and output the file. However, if you invert a Tropomi file (approximately 3246 * 450 pixels) at once, it will result in slow runtime and the above error. Do you have any other methods?
Hi, the error message is not visible (anymore?), can you share it again?
I think this is caused by too many pixels in Tropomi inversion, as a small amount of inversion can output files. What techniques did you use when inverting Tropomi?
This does look like a memory issue. Could you show the result of running free --giga -h -t at the command line?
I’m sorry to say, but it looks like you run into a qdoas bug. With such an amount of available memory, there should not be any problem. I will try to reproduce the problem here. Which version of qdoas are you using (output of doas_cl --version?).
Tropomi satellite has over a million pixels in one file that need to be inverted. Is this the reason for this problem? Because if I only invert the part, I can produce the result file.
It seems like that has something to do with it, only we have not seen this behaviour before. Normally, about 10GB of memory is enough to process one Tropomi orbit.
If you could share your configuration file with me (via e-mail), that could be helfpful.
Thomas
Thanks for the reply， What is your e-mail address, please? I will send you the configuration file
Also, could you provide the output of ulimit -a on your system? This is to check if there is a limit on process memory usage.
Thomas
Hi, I’ve found the cause of the problem. There was a memory leak in our code, which has little impact in optical density fitting mode (and therefore went unnoticed), but has a huge impact in intensity fitting mode.
We’ll make a new qdoas release as soon as possible. In the mean time, I suggest you try the optical density fitting mode and see if you can get the results you need that way (it has the extra advantage of being much faster than intensity fitting 
).
Thank you for reporting the problem and for your help in identifying the cause.
Thomas
We’ve released Qdoas 3.6.3, which includes a fix for the memory issue. I recommend you upgrade to this version (the memory leak in version 3.6.2 also occurs when using optical density fitting, though it is not as bad as in intensity fitting mode, so you may not immediately notice it on a system with 500GB of memory).
Thank you very much for your reply. I will update immediately.