Use of QDOAS with hyperspectral data

QDOAS
1

Hi,

I’m interested in trying using PRISMA hyperspectral data with QDOAS to evaluate the possibility of trace gas retrieval. The data has the following characteristics:

  • Spectral resolution of about 10 nm
  • An amount of 240 bands in the spectral range from 400 nm to 2500 nm.

I’m currently facing issues and errors in trying to run the software. I tried to generate an input netcdf file with the same characteristics of the file accepted from the software in the case of TROPOMI, but with a different number of channels and spectral resolution.

I then tried to use the “APEX” format. However, I have not found information in the QDOAS manual on the structure of the APEX format, particularly regarding the format of the files containing the spectra to be analyzed and the reference spectrum. For example, if it is in netCDF format, how should it be structured? Specifically, what are the variable names containing the data for both the analysis files and the reference files?

Thank you in advance for taking into account my request and for your help.

2

Hi Davide,

indeed proper documentation for the APEX format is missing… I’ll briefly explain the structure of the files.

APEX files are NetCDF files with the following structure (CDL template):
example for the radiance:

netcdf apex_example {
dimensions:
	col_dim = 10 ;
	row_dim = 100 ;
	spectral_dim = 1280 ;
variables:
	double radiance(row_dim, col_dim, spectral_dim) ;
		radiance:comment = "Elastic earthshine radiance" ;
	double radiance_wavelength(spectral_dim) ;
		radiance_wavelength:Units = "nm" ;
	double solar_zenith_angle(row_dim, col_dim) ;
	double viewing_zenith_angle(row_dim, col_dim) ;
	double relative_azimuth_angle(row_dim, col_dim) ;
	double latitude(row_dim, col_dim) ;
	double longitude(row_dim, col_dim) ;
}

example CDL template for the reference:

netcdf apex_reference {
dimensions:
	col_dim = 10 ;
	spectral_dim = 1280 ;
variables:
	double reference_radiance(col_dim, spectral_dim) ;
		reference_radiance:comment = "Reference solar  irradiance" ;
	double reference_wavelength(spectral_dim) ;
		reference_wavelength:Units = "nm" ;
}

You can save these templates to files apex_example.cdl and apex_ref_example.cdl, and generate empty example netCDF files with the command ncgen <template_file.cdl> -o example_file.nc.

The dimension parameters given here are just examples, and the values should be adjusted to fit the PRISMA instrument. The meaning of the dimensions is:

spectral_dim: number of spectral bands in your data (from your question, I understand it would be 240 in your case?). This number must be the same between the radiance and reference files used in the analysis.

col_dim: this is meant for imager type instruments such as OMI or Tropomi, where the instrument consists of an array of detectors (if you are familiar with Tropomi L1B data, this equivalent to the pixel or ground_pixel dimensions of Tropomi). Qdoas will perform a separate wavelength / instrument response function calibration for every column. I’m not familiar with PRISMA, but if it is not an imager, col_dim should be 1. Again, col_dim of reference and radiance files should match.

row_dim: this is the number of observations per column (think of scanline in the Tropomi L1B).

The meaning of the variables:

radiance: the observed sepctra you want to analyse
radiance_wavelength: initial wavelength calibration for the radiance. Qdoas can optimize this using shift/stretch, but the initial values should be reasonable. A single wavelength calibration is expected for all rows and columns.
*_angle, latitude, longitude: observation geometry and location

reference_radiance: the radiance with respect to which you analyse the absorption in your DOAS analysis (can be solar irradiance or another radiance, depending on your method).
reference_wavelength: initial wavelength calibration for your reference.

I hope this helps you get started, please ask if anything is unclear.

Thomas

3

Interesting topic! @davide.desantis Have you figured out the correct way to input the hyperspectral data? I feel the NetCDF method sounds more straightforward. What kind of error did you get when you applied that?

@thomasd I have also tried to search the APEX keyword in the User Guide to understand how to use it, but I couldn’t find it. Any link of guide would be useful!

I have two main questions:

  • Should we use the same dimension name? such as row_dim, col_dim, spectral_dim.
  • How do we get the proper Reference solar irradiance? Is it already saved somewhere and we only need to do some interpolation?

Thanks for your help,
Xin

4

Should we use the same dimension name? such as row_dim, col_dim, spectral_dim.

Yes, the names of all variables and dimensions should be exactly as in the example provided. QDOAS uses these names to read the data from the file.

How do we get the proper Reference solar irradiance? Is it already saved somewhere and we only need to do some interpolation?

The reference spectrum should also be a measurement taken by your instrument. For satellites, this could be an irradiance measurement taken by the satellite, but also a radiance (or averaged radiance) taken over a “clean” background area, or area with known (or assumed, from a model?) concentration, …

1 Like
5

Hi, @thomasd ! I share the same interest as @davide.desantis in trying to analyze PRISMA hyperspectral data using the QDOAS software. However, the PRISMA option is not available from the satellite list in QDOAS. As a workaround, I need to choose a non-PRISMA option and convert the PRISMA data to that format. APEX might be a suitable option. Following the CDL template you provided, I generated a radiance.nc file using Python. I think the format of the nc file generated by this python code is the same as the ncgen method.

However, like Xin, I also have a question regarding solar irradiance. It appears that the PRISMA satellite data I downloaded does not include satellite-measured solar irradiance. So, as you suggested, I may need to model this reference spectrum. I’m unclear about the details of this modeling. For instance, should the solar irradiance be convolved directly to the satellite’s resolution, or is it necessary to assume an atmospheric profile (such as the US1976 profile) before modeling? Also, I didn’t fully understand what you meant by “clean.”

Additionally, I have a question about the satellite altitude. For example, PRISMA operates at an altitude of around 600 km, much higher than the altitude of airborne experiment like APEX. Could using the APEX option while actually working with PRISMA radiance data cause any issues?

Many thanks for your assistance!

6

Hi,

I’m not really familiar with PRISMA, and I don’t know if it has a solar irradiance measurement mode.

What I meant by “clean” is: you could also use as reference a regular observation (or an average of a number of observations) over an area where you know the concentration of the absorber you want to measure is very low, or has a homogeneous background value of which you have a reasonable estimate (e.g. from a model), so you can correct for the absorber optical depth in your reference spectrum.

I did not suggest to use a modeled spectrum as a reference in your DOAS analysis. One of the benefits of the DOAS method is that a lot of measurement artifacts and instrument calibration issues are avoided because you work with a ratio of two spectra measured by the same instrument. Comparing a simulated spectrum to a measured spectrum might give you less precision and/or could introduce bias. However, I don’t want to discourage you from trying :wink: Depending on the quality of your modeled spectrum and the specifics of your retrieval, it might also work…

Satellite altitude does not directly play a role in the slant column retrieval (which is all that QDOAS does…), so that should not be an issue.