QDOAS Latest changes
Version 3.2 - 12 September 2017
ASCII file format: QDOAS proposes a new option "column
extended", more flexible than the current "line" and "column"
ones. Spectra (preceeded or not by wavelengths) can be provided
as column vectors and records can be described by pre-defined
fields. See "Input File Format" annex for further
Super-Gaussian: this line shape is a generalization of the
Gaussian with the introduction of two independent parameters
determining the width and shape. This line shape was proposed
by Steffen Beirle (Max-Planck Institut für Chemie, Mainz), and
is discussed in a
paper in AMT. The current implementation in QDOAS is
operational but not optimized in terms of performance.
Sliding windows: during the calibration procedure, the
calibration interval is divided into a number of smaller
intervals. By default, the complete calibration interval is
divided into non-overlapping sub-windows of equal size. Now,
the user can specify the width of the sub-windows, independently
from their number.
Calculation of a pre-shift: if the wavelength calibration of
the reference spectrum is really too bad, the calibration
procedure couldn't converge and return a wrong value of the
shift that corresponds to a local minimum of the chi
square. QDOAS proposes to approximate the value of the shift by
calculating the correlation between the reference spectrum and
the solar spectrum convolved at the resolution of the instrument
and use this value as initial shift value of the usual NLLS
Export data/spectra: browses spectra and saves information and/or spectra in an ASCII file.
If only data on records are exported, the format of the generated file
is the same as QDOAS output; otherwise, the file including spectra
(and eventually the wavelengths) can be read by QDOAS with the new
"column extended" ASCII format.
Sequential analysis (use the
reference of the MAXDOAS scan) should work for most of ground-based
file formats including MFC formats generated by DOASIS and MS-DOAS.
The linear and non-linear fitting routines were tuned to improve
performance. This should lead to time savings of 10% up to 50%,
depending on the configuration used. The linear fitting routines now
use a QR decomposition from the GNU Scientific Library. The changes
in the non-linear fitting algorithm have a minimal impact on results,
leading to relative differences of 0.1% or less with respect to
results obtained with previous versions of QDOAS.
output files now contain fletcher32 checksums for all variables.
Instructions to compile QDOAS on macOS were updated, and, where
necessary, the code was adjusted to allow smooth compilation on macOS.
QDOAS can now read ascii input files with legacy Macintosh newline
encoding (single carriage return).
Improved handling of non-ascii characters in configuration file names.
QDOAS is now compatible with Qt5.
When processing GOME-2 files, geolocation data is
passed to the output also for pixels with invalid spectra.
The MFC binary format can now also be used on 64-bit linux systems.
Version 2.111 - 5 April 2016
- MFC original binary format generated with DOASIS or MS-DOAS is
- MAXDOAS measurements: extended options for the automatic selection of the zenith-sky spectrum of the scan, currently supported by a limited number of formats including MFC one. Please, contact authors if you want to apply this option to your format.
- BIRA-IASB airborne and mobile DOAS measurements are split.
MobileDOAS can be selected under ground-based measurements;
hundreds of seconds are accounted for in the fractional
- Extra output fields available for GOME-2: “MDR number”,
“Observation index” (=index within an MDR), “Observation Mode”
(“Nadir”/”North Pole Scanning”/…), solar zenith and azimuth
angles in satellite reference system, satellite latitude and
- Extra output fields for OMI: Spacecraft
altitude/longitude/latitude, instrument configuration id.
- QDOAS can now handle older OMI L1B files which do not contain
“XtrackQualityFlags”, when the configuration doesn't use these
- Fill Values are used in the output data when processing fails,
with proper fill value attributes set in netCDF and HDF-EOS5
- New output fields “number of wavelength bands used in the
fit”, “processing error flag”, “Analysis window center pixel
- “Spectrum” and “Irradiance” plots for satellite data now use
the wavelength grid from the L1B data if available (GOME-2,
OMI). The axis labels have been changed from “Counts” to the
physical units for the radiometrically calibrated
- New meta-data attributes in netCDF output files: “QDOASConfig”
contains the configuration file name, “QDOASConfigProject”
contains the name of project within that file that was used to
produce the results.
- Improved error messages for some incorrect configuration
- The calibration procedure can be preceded by the calculation of a pre-shift. This additional step approximates the value of the shift by calculating the correlation between the reference spectrum and the solar spectrum convolved at the resolution of the instrument using the initial value(s) of the slit function parameter(s). This step is useful if the wavelength calibration is not stable or when the initial wavelength calibration is really bad.
- New error message if a cross section file does not have the
expected number of columns. This implies that the two last
columns of previous pre-convolved ring cross sections generated
by the QDOAS ring tool should be removed. Such ring cross
sections are usually introduced to fit the wavelength
calibration. If the option “convolve ring” is used, the file
should have the 4 columns (the two last ones are the convolved
solar and Raman spectra). The QDOAS ring tool includes now a new option to save or not the convolved solar and Raman spectra and generates a four or two columns file.
- The new wavelength calibration procedure announced in version
2.109 was not actually used for GOME-2. This has now been
- A number of bugs related to netCDF output have been
- Fixed a bug in the output of calibration fields with more than
one dimension (e.g. calibration stretch).
- A minor bug appeared in version 2.108 (determination of the width of the calibration sub-windows) and has been fixed; it is then recommended to switch to this release.
- GOME-2 satellite altitude output fixed.
- When working with GOME-2 L1B files in previous versions of
QDOAS, the “Scan index” (1 = forward scan, 2 = backward scan, 0
= “other”) property was not stored properly.
- Milliseconds of OMI measurement timestamps are now properly
reported in the output.
- The configuration of the “2n-Lorentz” line shape is
changed. Until now, the user had to specify the polynomial
degree “2n” in the line shape properties when using a 2n-Lorentz
line shape. The interface made it possible to enter an odd
number, which lead to errors when running the analysis. The
interface now requests the number “n”, making such invalid
configurations impossible. Old configuration files will
automatically be adapted when saved with this new version.
- QDOAS can now provide output in the netCDF format, in addition
to the existing ASCII and HDF-EOS5 output options.
Version 2.109 - 9 April 2015
WAVELENGTH CALIBRATION FOR SATELLITE FORMATS
For GOME-2 and OMI, radiances are now interpolated onto the original level 1 irradiance wavelength grid before the start of the analysis, using the wavelength calibration provided in the level 1 files. For projects using a fitted shift/stretch for the radiance (as recommended), this change should have negligible impact on retrieved slant column densities. Fitted shift and stretch values will be different compared to previous versions, because part of the shift/stretch between radiance and irradiance is now accounted for by using the level 1 calibration. Though this change makes it possible to obtain results relying purely on the original level 1 wavelength calibration, we still recommend fitting an additional shift/stretch of the radiance in QDOAS for optimal accuracy.
- For OMI data, conversion of measurement timestamps from TAI to
UTC did not take into account UTC leap seconds, leading to an
error in the displayed datetime of up to 8 seconds (8 seconds for
data after 1 July 2012, when the last UTC leap second
occurred). Qdoas now displays the correct UTC time for each OMI
- Selecting 'output calibration' in a project that doesn't use
calibration no longer crashes the program.
- Calibration output for GOME/GOME-2 was broken in the previous
release; now fixed.
- Qdoas can now handle slit function files and ascii spectra with
a large number of columns (before, ASCII files with very long
lines could cause a problem).
- The output field 'Time (hh:mm:ss)' caused problems when writing
HDF-EOS5 output files due to restrictions on HDF-EOS5 field names.
It has been renamed to 'Time (hh-mm-ss)'.
- Reading ASCII files on systems with locale settings that expect
comma as a decimal separator (for example certain German language
locales) was broken.
- Trying to open an incorrect file in a GOME-2 project no longer
causes the program to crash. An error message is displayed
- When processing OMI or GOME-2 spectra using a reference spectrum
generated from averaged (earthshine) radiances, the “ref1/ref2
alignment” used to align this reference on to the irradiance grid
was not used anymore in the previous version 2.108.
- For OMI, the “ref1/ref2” alignment plot no longer causes the
calibration results plot to disappear.
- At the end of a processing session in the GUI, the session is
closed automatically; the user doesn't need to press the red 'stop'
button when all available records have been processed.
- Modified layout of the analysis window properties and project
properties menus, so they take up less space and fit on smaller
- There is now an option to use a linearized “offset” in optical
density fitting mode, where a polynomial divided by the reference
spectrum (instead of the analyzed spectrum) is added to the linear
- Minor change in the calculation of the “resol.” cross section in
order to better quantify the difference in resolution between the
spectrum and the reference.
- Ring tool: add convolution option “None” to disable convolution
by the slit function.
- Plots are now drawn with antialiasing, which gives a much
- Improved layout for plot images generated using the “export as
Version 2.108 - 29 April 2014
- HDF-EOS5 output did not work when no automatic calibration was
used for one of the analysis windows.
- When an analysis window is configured to re-use the fit result
for a given cross section from a previous analysis window, Qdoas
now displays an error message when it cannot find the exact same
cross section (i.e. same cross section file) in a previous
analysis window, instead of failing silently.
- When using a pre-convolved, 60 row high-resolution reference
spectrum for OMI, where each row is the result of the convolution
of a high-resolution solar spectrum with the slit function for the
corresponding row of the OMI detector, the spectrum for row 1 was
used in the wavelength calibration of the irradiance spectrum for
all rows, and the other rows were being ignored. Qdoas now uses
the correct spectrum to calibrate the wavelengths for each
- The calculation of undersampling was sometimes wrong due to a
bad selection of the slit function type.
- The name of the slit function was wrong in the headers.
- Resol cross section introduced in version 2.106 was centered
around 1. We subtract 1. now in order to have the cross section
around 0. The Fit Store button value was not correctly read from
the configuration file.
- The disabled status of analysis windows was not accounted for
- In the case of multiple analysis windows and different
criteria of selection for the reference spectrum, if the
Ref2/Ref1 alignment was not possible in an analysis window due
to a bad reference spectrum, the alignment of reference spectra
in the following analysis windows was not performed but the
spectra were processed anyway conducting to bad results.
- For multiple analysis windows using different reference
spectra (which leads to different wavelength calibrations for
those windows), calibration output only contained the results
for the first analysis window. Now, calibration data for all
analysis windows is included. Likewise, the display of
calibration results in the GUI now includes results for multiple
analysis windows, when different calibrations are used.
- When writing output files to directories yyyy/mm/dd, previous
versions of Qdoas used the date of the first successfully
processed spectrum to choose the output directory. For
satellite orbit files starting shortly before midnight, the
first successfully processed spectrum would often come from a
measurement taken after midnight, and therefore Qdoas put the
results in the directory for the next day. Now, Qdoas uses the
date of the first observation in a satellite orbit file to
determine the output directory, regardless of whether the
spectrum from this observation was processed.
- A time conversion bug was fixed. Due to this bug, results from
satellite measurements taken on the first day of a given year
would be written to a folder for the “zeroth” day of the
For MAXDOAS measurements, the automatic search of the reference
spectrum of the scan should work for CCD-EEV, ASCII, MFC STD and
MFC BIRA-IASB binary formats.
For MFC format, a converter exists to create daily binary files
(BIRA format) with spectra, dark currents and offsets. QDOAS
automatically corrects spectra using the dark currents and
Ring tool: new check button to normalize the Raman spectrum
before calculating the ring.
When the user has chosen to write output files, Qdoas checks if
it can write to the configured output directory before starting
Interface improvements: layout of calibration properties and
analysis window properties is now adapted to the font size to
improve readability; same for text fields in the slit function
properties tab, which were sometimes partly hidden, depending on
screen resolution and font size.
Configuration files: update of the xml format. The change is
backwards-compatible and only affects config files where an
instrument transmission function is specified (see the section
about the 'instrument' tab of the project properties in the
manual). Configuration files created with older versions of
Qdoas can still be used with this version, but for configuration
files saved with the new version, the transmission function will
not be loaded correctly in older versions of Qdoas.
In the output page of project properties, the color index field
is replaced by the bandwidth on which to calculate the flux (as
it is possible to save different central wavelengths separated
by semi-colons, color indexes can be calculated outside QDOAS)
Addition of a button to save successful records only (default
behavior) or to save all records including reference spectra and
records for which the analysis fails (log error on the spectrum
for example). Sometimes, it is useful to have all records (for
example to retrieve information on the reference spectra from
the output files or to always have the same number of records in
the output files whatever the analysis fails or not.
Qdoas now supports Qwt version 6.1.0, as well as 6.0.2.
Version 2.106 - 4 November 2013
- An error in the linear offset fitting routine, leading to
incorrect offset values, was fixed.
- Running analysis with incomplete calibration settings no
longer causes Qdoas to segfault. Qdoas now continues to work
after reporting the error.
- Output to a file in the current working directory without
leading "./" was broken, now fixed.
- The format of the residual output file (see manual, section
"Analysis windows properties") was slightly changed to be
compatible with the spike removal algorithm.
- Manual update/harmonization: all information regarding
configuration of the calibration procedure moved to a single
- Linux: installation script now automatically installs the
online Help files in the right location during "make install"
(default location: /home/username/doc/qdoas/Help).
- Resol cross section introduced in predefined parameters. It
is built from the reference spectrum with a gaussian FWHM of 0.05
- Add 2D slit function in the calibration procedure. A stretch
can be fitted as for 1D file.
- Possibility to save the record number of the automatically
selected reference spectrum
- Calculation of fluxes: the interval is defined now in
wavelengths instead of pixels
Version 2.105 - 29 May 2013
- HDF5 output of polynomial components works.
- Preconvolved solar reference spectra and cross sections are
no longer required to have the same number of points.
- The spike removal test now uses the RMS residual to decide if
a wavelength should be removed (previously, the average
magnitude was used).
Version 2.103 - 29 April 2013
- QDOAS can now process OMI level one files. Separate reference spectra/slit functions/calibration parameters are used for each of the 60 detector rows.
- OMI PixelQualityFlags and XtrackQualityFlags can be used to exclude pixels or spectra from the analysis.
- SCIAMACHY: possibility to correct the non linearity of the detector (new file to enter in the instrumental page of projects properties.
- GOME2: compatibility with BEAT 6.7.0/CODA 2.7.2 definitions.
- GOME2 and automatic reference selection: days with a lot of empty files make thes program crash.
- GOME-ERS: version 2.1 couldn't read anymore the geolocation coordinates of old versions of GOME.
Analysis windows properties
- Resol: in predefined parameters, this new parameter automatically generated from the reference spectrum allows to account for very small differences in the determined resolution.
- Asymmetric line shapes can be fitted with a asymmetric Gaussian line shapes (fit of an asymmetric factor in addition to the Gaussian FWHM Slit Function Parameter).
- Functions of type « File » can be fitted in the Calibration page of Projects properties; a different stretch factor is fitted for each side of the slit function; lookup tables of functions are accepted.
- Before, if the slit function was not fitted in the Calibration page of Projects properties, a pre-convolved solar spectrum was expected in this page. Now, the program uses the information of the Slit page to convolve the solar spectrum. If the slit function type is « None », the pre-convolved solar spectrum has to be entered in this Slit page.
- QDOAS S/W user manual has been updated with new features; algorithms description chapter has been completely reviewed.
- A « spike removal » function was added, which can automatically remove pixels with exceptionally high residuals.
- The display of analysis results in the spreadsheet was fixed so that long numbers are no longer truncated.
- Review Chi square/RMS formulas. The value previously reported as RMS was the square root of the reduced Chi square value. Now it is the RMS of the fitting residuals. Formulas are given in the user manual.
- A bug was fixed which caused the analysis to stop when using least squares weighting by instrumental errors.
- Pixels within gaps are now ignored in the plots.
- This version includes a number of small performance improvements.
- In intensity fitting only, possibility to authorize the concentrations of molecules within a given ranges of values (not recommended except for tests purposes).
- Doas_cl: new -xml switches to configure some options of the xml file during batch processing