USGS now delivers atmospherically corrected LANDSAT data

=>

USGS and NASA just released a new LANDSAT level 2A product (surface reflectances corrected for atmospheric effects with a cloud mask, a cloud shadow mask, and a water and snow mask). The thermal data are expressed as brightness temperatures at the top of atmosphere. This product is only available for Landsat 5 and Landsat 7 yet, but I know NASA is working on LANDSAT 8 atmospheric correction processor (Edit : LANDSAT 8 level 2A has been released in december 2014). The French Land Data Center Theia also distributes atmospherically corrected LANDSAT 8 data, but only over France.

 

Since May 2013, the atmospherically corrected LANDSAT 5 and LANDSAT 7 data are available on earthexplorer site. They are not so easy to find : click on the "data sets" tab and select your satellite in the "Landsat CDR" menu. You can order your data at once and the data will be available for download the next day (unless you ordered a few hundreds of them). It is only a pity that USGS does not own a large part of the historical LANDSAT 5 data acquired in ESA receiving stations. The data should be transferred one day, and a lot of users are awaiting for that.

 

Comparison of cloud masks obtained from MACCS (=MTCD) and LEDAPS on LANDSAT scenes in the USA. The agreement is very good in most cases, even if LEDAPS misses some clouds in some rare cases. One of the most distant points from the diagonal is explicited in next figure.

 

The LANDSAT surface reflectance images are produced with the LEDAPS processor. [Vermote et Masek 2006]. LEDAPS uses the DDV method to estimate the aerosol optical thickness, and a bunch of spectral  tests using only one date to detect clouds. Last year, a comparison was done with our method (MACCS), and the results were very close, event if the mono date method is not as accurate as the multi-temporal one (see the figures). LEDAPS is also less accurate for the cloud shadow detection which is even more tricky than the cloud detection. But nevertheless, it is evident that these new products will help many users in their applications.

 

The LANDSAT surface reflectance product is by now the only existing or foreseen level 2A product at the global level. The French Land Data Center THEIA also delivers Level 2A processed with the THEIA prototype for SPOT4 (Take5) (available here) and for the LANDSAT (5,7,8) data acquired since 2009 over France (available there), but they do not have a global observation capability.

 

Regarding Sentinel-2, ESA has decided not to produce level 2A products (which is a pity, even if we can understand that producing all Sentinel-2 data to level 1C is already a challenge). The French Land Data Center THEIA is already committed to process the whole Sentinel-2 data over France, and over other regions and countries to be hosen, covering up to 10 times France's area. but the application of MACCS to the global land surfaces is not decided yet, even if THEIA is working into it.

 

Comparison of MACCS (left) and LEDAPS (right), in a difficult case with thin clouds above bright surface. The thin clouds are much better detected by MACCS. In more standard cases, the cloud masks from both methods look alike.

 

 

Reference : Masek, J. G., Vermote, E. F., Saleous, N. E., Wolfe, R., Hall, F. G., Huemmrich, K. F., ... & Lim, T. K. (2006). A Landsat surface reflectance dataset for North America, 1990-2000. Geoscience and Remote Sensing Letters, IEEE, 3(1), 68-72.

How to estimate Aerosol Optical Thickness

=>

Caution ! This post contains formulas !


Aerosols play a great role in the atmospheric effects. Aerosols are particles suspended in the atmosphere, which can be of several types: sand or dust, soot from combustion, sulfates or sea salt, surrounded by water... Their size ranges between 0.1 micron and a few microns, depending on the type of aerosol or on the air moisture. Their quantity is also extremely variable : rain can suddenly reduce their abundance (known as "aerosol optical thickness"). The abundance variations result in great variations of observable reflectances from one day to the next, and it is therefore necessary to know the quantity and type of aerosols, in order to correct their effects.

 

Unfortunately, to correct the effects of aerosols, there is no global aerosol observation network, and the only available data are local observations from the few hundred points of Aeronet network. Therefore, this network can not be used operationally to correct the satellite images over large areas.

Weather forecast models just start predicting the amounts of aerosols, based on satellite observations and modeling of sources and sinks and of the transport of aerosols by the winds, but these data do not seem to have sufficient accuracy yet to be used for the atmospheric correction of images.

 

Our atmospheric correction method, named MACCS, is therefore based on an estimate of aerosol optical depth from the images themselves. To understand how this method works, one must already understand the effects of aerosols on radiation. We have seen in this post, that the effects of diffusion can be modelled as follows (assuming the corrected gas absorption):

ρTOA = ρatm +Td ρsurf

The reflectance at the top of the atmosphere ρTOA (Top of Atmosphere) is the sum of the atmospheric reflectance  ρatm and of the surface reflectance ρsurf transmitted by the atmosphere. We seek to know the surface reflectance, but for each measurement made at the top of the atmosphere, there are three unknowns to be determined. To separate the effects of the atmosphere and surface effects, we must use other information.

 

Dark pixel method

When the image includes a surface whose surface reflectance is nearly zero, the reflectance observed at the top of the atmosphere becomes ρTOA = ρatm. We can therefore deduce the atmospheric reflectance and using a radiative transfer model, the aerosols optical thickness (AOT). Finally, knowing the AOT, we can compute the diffuse transmission, and finally calculate ρsurf. An even simpler and more approximate version of this method consists in subtracting directly the reflectance of the dark pixel (or ρatm) to the entire image (neglecting the transmission) [Chavez, 1988].

 

However, this method assumes that there is a very dark area in the image (which is not always the case), and that the reflectance of the dark surface is known. The method also assumes that the amount of aerosols is constant over the image and it neglects the effect of terrain. The results obtained by this method can be quite inaccurate. In our method (MACCS), however, we use the method of black pixel determine the maximum value of the optical thickness in the area.

 

Multi Spectral Method, called "DDV"

If you know the type of aerosols in the atmosphere, it is possible to deduce the properties of aerosols in a spectral band from the optical properties in another spectral band.

 

If there are two spectral bands, there are two measures ρsurf and three unknowns (both surface reflectance in these bands, and the amount of aerosols). An additional equation can be obtained if we know the relationship between the surface reflectance of the two bands.

 

The method named "Dark Dense Vegetation" (DDV) is based on assumptions about relationships between surface reflectances of the dense vegetation exploiting the fact that the spectrum of dense green vegetation is quite constant. The most famous version of this method is that used by NASA for MODIS project [Remer 2005]. It connects the surface reflectance in the blue and red with those in the SWIR. This provides two equations for estimating the type of aerosol optical thickness. This method works well in temperate and boreal zones, but not in arid areas where it is difficult to find the dense vegetation. Early versions used the following equations:

 

ρBlue = 0.5 ∗ ρSWIR

ρRed = 0.25 ∗ ρSWIR

 

The following versions of the MODIS DDV algorithm are a bit more complicated but follow the same principle. Our work has shown that using the equation below allows a more accurate determination of the optical thickness, for less dense vegetation cover (NDVI to a 0.2) because bare soil brown also respect this relationship.

 

ρBlue = 0.5 * ρRed

(the exact value of the coefficient is adjusted according to the spectral bands of the instrument)


This version of the  method, however, does not allow to determine the aerosol model. In the case of SPOT4 (Take5), the absence of a blue band does not allow us to use this equation, resulting in a slight loss in accuracy.

This diagram shows that the correlation between surface reflectance above vegetation is much better for the (blue, red) couple of spectral bands than for couples including using (SWIR).

 

 

 

Multi Temporal Method

In most cases, the reflectance of the land surface changes slowly over time, while the aerosol optical properties vary rapidly from one day to another. We can therefore consider what changes from one image to another (apart from special cases often linked to human intervention) is associated with aerosols, and deduce the properties of aerosols and then correct for atmospheric effects. This method is too complex to be explained in detail here, interested readers can refer to [Hagolle 2008].

 

So that surface reflectance be nearly constant from one image to another, however, it is required that images be acquired at a constant angle. Indeed, the reflectance depend on the viewing angles : this is what we call directional effects. This method therefore applies only to satellite observations obtained with constant angle. It does not apply to standard SPOT data, but this condition is true for SPOT4 (Take5) data. It will also apply to Landsat Venμs and Sentinel-2.

 

Finally :

 

Validation of aerosol optical thickness (AOT) from time-series of FORMOSAT-2 images, depending on the method (multi-spectral, multi-temporal, combined), compared with the measurements provided by the Aeronet network of in-situ measurements. The multi-spectral method works best on sites covered with vegetation and is much less accurate on arid sites, while the multi-temporal method performs a little worse on green sites, but much better on dry sites. The combination of the two methods retains the best of the two basic methods.

The MACCS method, used for SPOT4 (Take5) experiment, and also for LANDSAT, VENμS and Sentinel-2 data, combines the three methods described above to obtain robust estimates of aerosol optical thickness. These methods work in many cases, but sometimes fail when the assumptions on which they are based prove to be incorrect. They generally tend to work better on vegetated areas rather than in arid areas. for now, they assume the model known aerosol and in the coming years, we will look for reliable ways to identify the type of aerosols.

 

References :
Chavez Jr, P. S. (1988). An improved dark-object subtraction technique for atmospheric scattering correction of multispectral data. Remote Sensing of Environment, 24(3), 459-479.

Remer, L. A., and Coauthors, 2005: The modis aerosol algorithm, products, and validation. J. Atmos. Sci., 62, 947–973.

Hagolle, O and co-authors, 2008. « Correction of aerosol effects on multi-temporal images acquired with constant viewing angles: Application to Formosat-2 images ». Remote sensing of environment 112 (4)

Hagolle, O.; Huc, M.; Villa Pascual, D.; Dedieu, G. A Multi-Temporal and Multi-Spectral Method to Estimate Aerosol Optical Thickness over Land, for the Atmospheric Correction of FormoSat-2, LandSat, VENμS and Sentinel-2 Images. Remote Sens. 2015, 7, 2668-2691.

Les aérosols jouent un rôle prépondérant dans les effets atmosphériques. Les aérosols sont des particules en suspension dans l'atmosphère, qui peuvent être de plusieurs types : grains de sable ou poussières, suies issues de combustion, sulfates ou sels marins entourés d'eau... Leur taille peut varier de 0.1 µm à quelques microns, en fonction du type d'aérosols ou de l'humidité de l'air. Quant à leur quantité, elle est extrêmement variable, une pluie pouvant réduire brutalement leur abondance (on parle d'"épaisseur optique d'aérosols"). Ils peuvent faire varier fortement d'un jour à l'autre les réflectances observables depuis le sommet de l'atmosphère et il est donc nécessaire de connaître leur quantité et leur type afin de pouvoir corriger leurs effets.

 

Malheureusement, pour corriger les effets des aérosols, on ne dispose pas de réseau global d'observation des aérosols, seulement d'observations locales, sur les quelques centaines de points du réseau Aeronet. Ce réseau ne peut donc pas être utilisé pour corriger opérationnellement les images de satellites sur de grandes étendues.
Des modèles météorologiques commencent à prédire les quantités d'aérosols, en se basant sur les observations de satellites et la modélisation des sources et du transport des aérosols par les vents, mais ces données ne semblent pas encore avoir une précision suffisante pour être utilisées pour la correction atmosphérique des images.

 

Notre méthode de correction atmosphérique (MACCS) repose donc sur une estimation de l'épaisseur optique des aérosols à partir des images elles-mêmes. Pour bien comprendre le fonctionnement de cette méthode, il faut déjà comprendre les effets des aérosols sur le rayonnement. On a vu, dans ce billet, que les effets de la diffusion peuvent être modélisés ainsi (on suppose l'absorption gazeuse corrigée) :

ρTOA = ρatm +Td ρsurf

La réflectance au sommet de l'atmosphère ρTOA (Top of Atmosphere) est la somme de la réflectance atmosphérique ρatm et de la réflectance de surface ρsurf transmise par l'atmosphère. On cherche à connaître la réflectance de surface, mais à chaque mesure réalisée au sommet de l'atmosphère, on a trois inconnues à déterminer. Pour séparer les effets de l'atmosphère et les effets de la surface, il faut donc utiliser d'autres informations.

 

Méthode du pixel noir

Lorsque l'image contient une surface dont la réflectance est quasi nulle, la réflectance observée au sommet de l'atmosphère devient ρTOA= ρatm. On peut donc en déduire la réflectance atmosphérique, et en utilisant un modèle de transfert radiatif, l'épaisseur optique des d'aérosols. On peut enfin en déduire la transmission diffuse, et finalement calculer ρsurf. Une version encore plus simple et plus approximative consiste à soustraire directement la réflectance du pixel sombre (soit ρatm) à toute l'image. [Chavez, 1988]

 

Cependant, cette méthode revient à supposer qu'il existe bien une surface très sombre dans l'image (ce qui n'est pas toujours le cas), et que la réflectance de cette surface sombre est connue. La méthode suppose aussi que la quantité d'aérosols est constante dans l'image et elle néglige les effets du relief. Les résultats obtenus par cette méthode peuvent donc être assez imprécis. Dans notre méthode (MACCS), nous utilisons cependant la méthode du pixel noir déterminer la valeur maximale de l'épaisseur optique dans la zone.

 

Méthode Multi Spectrale, dite "DDV"

Si on connaît le type d'aérosols présent dans l'atmosphère, il est possible de déduire les  propriétés des aérosols dans une bande spectrale, à partir des propriétés optiques dans une autre bande spectrale.

 

Si on dispose de deux bandes spectrales, on dispose de deux mesures ρsurf et de trois inconnues( les deux réflectances de surface dans ces bandes, et la quantité d'aérosols). Une équation supplémentaire peut être obtenue si on connaît la relation entre les réflectances de surface des deux bandes.

 

La méthode  méthode "Dark Dense Vegetation" (DDV ) est basée sur des hypothèses de relations entre réflectances de surface sur la végétation dense exploitant le fait que le spectre de la végétation dense et verte est un peu toujours le même. La version la plus connue de cette méthode est celle utilisée par la NASA pour le projet MODIS [Remer 2005]. Elle relie les réflectances de surface dans le bleu et dans le rouge avec celles dans le moyen infra-rouge. On dispose ainsi de deux équations qui permettent d’estimer le type d’aérosols et l’épaisseur optique. Cette méthode fonctionne bien en zones tempérées et boréales, mais pas en zones arides, où il est difficile de trouver de la végétation dense. Les premières versions utilisaient les équations suivante

ρBleu = 0.5 ∗ ρSWIR

ρRouge = 0.25 ∗ ρSWIR

Les versions suivantes ont un peu compliqué ces équations, sans en modifier le principe. Nos travaux ont montré que l’utilisation de l'équation ci dessous  (la valeur exacte du coefficient est à ajuster en fonction des bandes spectrales de l'instrument):

ρBleu = 0.5 ∗ ρRouge

permet une détermination plus précise de l’épaisseur optique, pour des couverts végétaux moins denses (jusqu’à un NDVI de 0.2), car les sols nus de couleur marron respectent aussi cette relation. La méthode ne permet pas, par contre, de déterminer le modèle d’aérosols. Dans le cas de SPOT4 (Take5) l'absence d'une bande bleue ne nous permet pas d'utiliser cette dernière équation, d’où une légère perte en précision.

Ce diagramme montre que la corrélation entre réflectances de surface au dessus de la végétation est bien meilleure pour le couple de bandes spectrales (bleu, rouge) que pour les couples incluant le moyen infra rouge. (SWIR)

 

Méthode Multi Temporelle

On observe dans la plupart des cas que les réflectances de la surface terrestre évoluent lentement avec le temps, alors que le propriétés optiques des aérosols varient très rapidement, d'un jour à l'autre. On peut donc considérer que ce qui change d'une image à l'autre (en dehors de cas particuliers souvent liées à des interventions humaines) est lié aux aérosols, et donc en déduire les propriétés des aérosols pour ensuite corriger les effets atmosphériques. Cette méthode est un peu trop complexe pour être expliquée en détails ici, les lecteurs intéressés pourront se reporter à [Hagolle 2008].

 

Pour que les réflectances de surface soient quasi constantes d'une image à l'autre, il faut cependant que les images soient acquises sous un angle de vue constant. Les changements d'angles d'observation font en effet varier les réflectances (ce phénomène sera prochainement expliqué dans un autre article). Cette méthode ne s'applique donc qu'aux seuls satellites permettant des observations à angle constant.  Elle ne s'applique donc pas aux données SPOT normales mais par contre convient parfaitement aux données SPOT4 (Take5). Elle s'appliquera aussi à Landsat, Venµs et Sentinel-2.

En résumé :

Performance de l'estimation de l'épaisseur optique des aérosols sur des séries temporelles d'images Formosat-2,, en fonction de la méthode (multi-spectrale, multi-temporelle, combinée), par comparaison avec les mesures fournies par le réseau de mesures in-situ Aeronet. La méthode multi spectrale fonctionne mieux sur des sites couverts de végétation et moins bien sur des sites arides, la méthode multi-temporelle marche un peu moins bien sur les sites verts, mais beaucoup mieux sur les sites arides. La combinaison des deux méthodes garde le meilleur des deux méthodes élémentaires.

 

Notre méthode MACCS, utilisée pour l'expérience SPOT4 (Take5), et pour les données LANDSAT, VENµS et Sentinel-2, combine les trois méthodes présentées ci-dessus pour obtenir des estimations robustes des épaisseurs optiques d'aérosols. Ces méthodes fonctionnent dans un grand nombre de cas, mais peuvent parfois échouer quand les hypothèses sur lesquelles elles reposent s'avèrent fausses. Elles ont en général tendance à mieux fonctionner sur des zones couvertes de végétation plutôt que dans des zones arides. pour le moment, elles supposent le modèle d'aérosol connu, et dans les prochaines années, nous chercherons des manières fiables d'identifier le type d'aérosols.

 

References :
Chavez Jr, P. S. (1988). An improved dark-object subtraction technique for atmospheric scattering correction of multispectral data. Remote Sensing of Environment, 24(3), 459-479.

Remer, L. A., and Coauthors, 2005: The modis aerosol algorithm, products, and validation. J. Atmos. Sci., 62, 947–973.

Hagolle, O and co-authors, 2008. « Correction of aerosol effects on multi-temporal images acquired with constant viewing angles: Application to Formosat-2 images ». Remote sensing of environment 112 (4)

The atmospheric effects : how they work.

=>

Earth surface observations by space-borne optical instruments are disrupted by the atmosphere. Two atmospheric effects combine to alter the images :

  • the light absorption by air molecules
  • the light scattering by molecules and aerosols

Here are two SPOT4 (Take5) images, acquired with a time gap of 5 days above Morocco. Because of atmospheric effects, the second image has less contrast and is"hazier" than the first one.

 

 

Light Absorption :
Atmospheric absorption : in blue, the surface reflectance of a vegetation pixel, as a function of wavelength. In red, the reflectance of the same pixel at the top of atmosphere.

The air molecules absorb the light within thin absorption bands. Within these absorption bands, the reflectance measured by the satellite is lessened, and in some cases, the light may be completely absorbed and the apparent reflectance at the top of atmosphere (TOA) is zero.  (for instance, at 1.4µm, in the figure on the right. Such a property is used to detect high clouds with Sentinel-2 or Landsat-8).

Thankfully, the satellite designers usually choose to locate the spectral bands away from strong absorption bands (but beware of satellite designers ;-) ). Within the satellite channels, the absorption is generally sufficiently low so that an approximate knowledge of the absorbent abundance is enough to obtain an accurate correction of absorption. Information on absorbing gases (ozone, water vapour) concentration may be found in weather analyses.

 

Light scattering

The air molecules scatter the light. A photon that passes close to a molecule will be deflected in another direction. As the air molecules are very small compared to visible light wavelengths, they will mainly scatter short wavelengths (in the blue range). The blue sky results from the scattering of sun light by air molecules, since the blue light in the sun spectrum is much scattered while the other wavelengths are mainly transmitted to the ground. A cloud also scatters the light, but its large particles (droplets, crystals) scatter all wavelengths, which explain its white colour.

 

Apart from clouds and air molecules, scattering may be due to aerosols. Aerosols are particles of diverse nature (sulphates, soot, dust...), suspended in the atmosphere. Their abundance, type and size are extremely variable, and their effect on light is also variable. Small aerosols will mostly scatter blue light, while larger aerosols will scatter all wavelengths. Some aerosols may also absorb light. All this variability makes the correction of their effect very tricky.

The above video, provided by NASA, gives an idea of the way aerosol properties may change from one day to the other, within a two years period. The colour gives an idea of aerosol types, while the colour intensity provides the aerosol optical thickness.

Simplified model :

In a very simplified way, atmospheric effects may be modelled as follows :

ρTOA= Tgatm +Td ρsurf)

where :

  • ρTOA is the Top of Atmosphere reflectance
  • ρsurf is the earth surface reflectance
  • ρatm is the atmospheric reflectance
  • Tg is the air molecules (gazeous) transmission (Tg<1)
  • Td is the transmission due to scattering (Td<1)

When aerosol quantity increases, ρatm increases while Td decreases. These two variables also depend on view and sun angles. The closer to vertical, the lower value of ρatm and the higher value of Td .

 

Adjacency effects :

The above model should only be applied to a uniform landscape. But above a standard landscape, a heavy loaded atmosphere will also blur the images. This is explained in another post.

Models, corrections.

Several models may be used to perform atmospheric corrections. For, approximate corrections, the SMAC model should be one of the simplest. SMAC be downloaded from the CESBIO site. The difficulty in using any atmospheric correction model lies in providing the necessary information on aerosol properties. We will talk about that in another post.

Other more accurate models may be used. In our case, in the MACCS processor, we pre-compute "Look-up Tables " using an accurate radiative transfer code (Successive Orders of Scattering), that simulates the light propagation through the atmosphere. But the use of a complex model is only justified if it is possible to obtain an accurate knowledge of the aerosol optical properties.

First Level 2A time series of SPOT4 (Take5) images

(aerosol images have been added at the end of the post)
=>

The verification of the various steps of our SPOT4(take5) processing scheme is going on. On Thursday, we received our first time series, I orthorectified them on Friday, and we were then able to start testing our level 2A processor with the first time series. The one displayed below was obtained on the CESBIO site in Tensift valley : Marrakech is near the center of the image, while the Atlas mountains are in the South East part of the image.

The images on the left column are ortho-rectified, and expressed in Top of Atmosphere reflectance (Level 1C product), while the right column displays the same images after atmospheric correction and cloud detection (Level 2A products), produced by Mireille Huc (CESBIO).

We quickly figured out that the cloud detection would be easy on these very clear images, even if on the February 10th, several diffuse plane contrails can be hardly seen but are partially detected, and some of their shadows as well (clouds are circled by red lines, while shadows are circled by a black line). No false cloud detection is visible. Water bodies and snow are also correctly detected for this first try (circled in blue and purple respectively)

The atmospheric correction, based on a multi-temporal method that detects the aerosols, enabled to detect that the image of February the 5th was hazier than the images of January 31st and February 10th.The February 5th image (left column) has a subtle blueish haze compared to the other dates. On the right column, the tint is roughly constant from one image to the other, which means that the aerosol detection and the atmospheric correction are working well. The aerosol images provided below are also very consistent, with the Atlas mountains playing their role of physical barrier blocking the aerosols on either side of the images. There is an aerosol measurement station on this site but it broke down at the end of January, just for the start of the experiment : Murphy's law...

So, we have reviewed and tested all the steps of the processing, but we still have to check that our methods are sufficiently robust to handle correctly the very diverse situations offered by the 42 sites. How do you say, in English "ce n'est pas une mince affaire" ?

Level 1C products expressed in reflectance at the top of atmosphere.
(c) CNES, processing : CESBIO
Level 2A products expressed in surface reflectance after atmospheric correction
(c) CNES, processing : CESBIO

Aerosol optical thickness images are displayed below. One can note that the image of the February 5th is consitent with a lot of aerosols in the North of the Atlas, and nearly no aerosols in the South. The mountains often act as barriers for the aerosols witch usually stay at a low altitude. The orange dots correspond to the snow mask whereas the red ones correspond to the cloud mask. The brighter spots on the last image may be artifacts.