MACCS qualified for Atmospheric Correction Euro and World Cup

Don't worry, I have not been converted to a football fan, it is still giving me allergies ! But after a post about sex ratio, we are still trying to increase the amount of connexions to this blog.

 

However, the title of this post is partly true, since, simultaneously, we are engaged to two competitions to compare cloud screening and atmospheric correction softwares applicable to Sentinel-2 or Landsat-8. The Atmospheric Correction Euro is organised by ESA, to select the method that will be used by ESA to systematically produce Sentinel-2 images to level 2A. The Euro world cup will last one year, with successive iterations between 4 producers, and a Jury of experts. Of course MACCS is participating, and we (DLR + CNES/CESBIO) are using this opportunity to merge MACCS and ATCOR methods, in order to use the methods from each of these two codes. In this framework, the resulting software we will get was renamed MAJA (say MAYA), for MACCS-ATCOR Joint Algorithm.

 

The Atmospheric Correction World cup is organized by NASA and ESA in the framework of the committee on earth observation satellites (CEOS). Here, about 12 codes will be compared, and the comparison will be done with LANDSAT 8 and Sentinel-2 data, it will seemingly focus on atmospheric correction rather than cloud screening, although it is not always easy to separate both aspects.

 

The World Cup will start this summer, while the Euro started last month. I have already shown a few validation results from this exercise :

- cloud masks comparison with Sen2cor

- validation of the consistency of processing among various tiles

- and thanks to Camille Desjardins (CNES), we just obtained validations of AOT estimates and Water Vapour estimates. The water vapour validation results are quite accurate and largely good enough to perform the atmospheric correction. The AOT validation results are decent, with rmse a little above what we observed for LANDSAT 8 over France. But the choice of sites is more difficult, with 2 arid sites, and the repetitivity of Sentinel-2 acquisitions is still far from what it should be (because of issues on board or on the ground segment). As already said in this blog, the accuracy of MACCS software increases with repetitivity.

Water vapour content validation wrt aeronet data for 4 sites (France, Morocco, Israel, SouthAfrica)

Aerosol optical thickness validation wrt aeronet data for 4 sites (France, Morocco, Israel, SouthAfrica)

 

We only are in the early stage of the EURO, and our wise coach did not introduce the best players in the first rounds. In next round, we will introduce MACCS 5.0, with its much enhanced shadows mask, and for the final stages, we will include the selected modules from ATCOR. But here is a glimpse of the MACCS 5.0 player.
On some images, the previous version of MACCS missed quite a lot of shadows, while with 5.0, they are detected (click on the images below to see the details). And by the way, the images were created within the Sen2Agri system, which is now taking shape and is a little ahead of MUSCATE in terms of versions. Many thanks to our colleagues from the Sen2Agri consortium (UCL, C-S France, C-S Romania, CESBIO).

Image generated at MUSCATE with MACCS 4.8, clouds and shadows are circled in green

Image generated at Sen2Agri with MACCS V5.0. The detected shadows are circled in yellow

And finally, I found an analogy between Football and Atmospheric Correction (I had to search a lot). Some say that football is a game which is played by two teams of eleven players and for which, at the end, Germany wins. For the Atmospheric Correction Euro, all the four teams competing include German institutes or companies. So here also, Germany wins.

Les données SPOT5 (Take5) de niveau 2A ont été retraitées

=>

Le projet SPOT5 (Take5) vient de publier une nouvelle version des données SPOT5 (Take5) de niveau 2 pour corriger un petit nombre de petits défauts détectés lors du premier traitement :

     

  • sur deux sites désertiques, parmi les plus brillants de la planète, en Libye et Tunisie, l'un des seuils de détection des nuages était réglé trop bas, et de grandes parts du désert étaient donc classées comme nuageuses. Le seuil a été augmenté spécifiquement pour ces deux sites.
  • Les produits de niveau 2A sur les sites nordiques n'étaient pas produits lorsqu'ils étaient complètement couverts de neige. Le problème a été résolu, mais il n'est pas possible d'y estimer l'épaisseur optique des aérosols lors des périodes de couverture neigeuse. Ils sont donc traités avec une épaisseur optique par défaut.
  • les ombres de nuages partiellement masquées par de grands nuages n'étaient souvent pas détectées. La méthode de détection des ombres de nuages a donc été entièrement revue, ce qui a permis une forte amélioration des masques d'ombres.
  • Les images d'épaisseur optique des aérosols étaient un peu trop bruitées, nous avons donc augmenté la taille de la fenêtre de lissage spatial.
  • un site (BrailaCasasuRomania) n'avait pas été traité à cause d'un problème sur le modèle numérique de terrain en entrée (et nous n'avions pas vu le message d'erreur sur notre système prototype). C'est maintenant corrigé, toutes nos excuses aux utilisateurs de ce site.

 

Quelques unes des images acquises sur le site de Braila, Roumanie

  • un site (DevonUK) avait de piètres performances de superposition (de l'ordre du pixel) en raison d'un problème avec l'image de référence. Cette image a été changée et les niveaux 1C et 2A reproduits.
  • L'ancienne version a été supprimée du site de distribution des données et remplacée par la nouvelle. The données reraitées sont disponibles ici :

    https://spot-take5.org

     

    Pour éviter toute confusion entre les anciennes et nouvelles versions sur vos diques, la nouvelle version peut être reconnue en vérifiant le champ ci dessous dans le header xml.

    <DATE_PROD>2016-xx-xx</DATE_PROD>


+

SPOT5 (Take5) reprocessed L2A data are online

=>

The SPOT5 (Take5) project just released a new version of all SPOT5 (Take5) L2A data to account for several small issues encountered during the first processing :

     

  • for two very bright desert sites (Libya, Tunisia), one cloud threshold was too low, and large parts of the desert were classified as clouds. The threshold has been changed specifically for these two sites.
  • the level 2A of Northern sites with a persistent snow cover were not issued in the first version. It has been solved, but as it is not possible to estimate aerosols above snow, they have been processed with a default Aerosol Optical Thickness value, as long as they are fully covered by snow.
  • the shadows below large clouds were often not detected. The shadow detection method has been completely reworked resulting in largely enhanced cloud shadow masks.
  • the Aerosol Optical Thickness images were a little too noisy, and the spatial smoothing of the values has been increased.
  • one site had been omitted because of a problem of DTM, which had been unnoticed (our processor is a prototype) :  BrailaCasasuRomania

Some extracts of the Level 2A products above the Romanian site of Braila.

  • one site (DevonUK) had a poor geometry because of a bad reference image. The reference image has been changed and the Level1C and level 2A have been reprocessed
  • The older version has been removed from the distribution site and replaced by the newer version. The reprocessed data are available here :

    https://spot-take5.org

     

    In order to avoid confusion between the older and newer version, you may check that the following HTML tag contains 2016 instead of 2015.

    <DATE_PROD>2016-xx-xx</DATE_PROD>


 

SMAC coefficients for quick Sentinel-2A atmospheric correction

=> 

The SMAC coefficients for Sentinel-2A are now available, and have been added to the CESBIO repository for SMAC coefficients. These coefficients were computed and checked by CNES colleagues in the Optical Measurement Physics service. If you happen to use these coefficients in your work, please thank CNES and CESBIO for providing them.

As already said in an earlier post :

The Simplified Model for Atmospheric Correction (SMAC) is the perfect model to perform easy, quick and not too dirty atmospheric corrections. It is based on very simple analytic formulas, based on the 5S model. The 49 coefficients of this model are fitted using a large number of radiative transfer simulations with the 6S model (the old historic version, not the recent vector version). This software is not very accurate (much less than MACCS), and it requires in-situ measurements for the aerosol optical thickness, and weather analyses for ozone and water vapour. If these data are available,  in most cases, its accuracy is within 2 and 3 percent, if we do not account for adjacency effects and slope effects, and it may be worse for large viewing and solar angles (above 70°) or within strong absorption bands.

 

We are also providing a python code to use these coefficients. This code is described and available here.

References

[1] Rahman, H., & Dedieu, G. (1994). SMAC: a simplified method for the atmospheric correction of satellite measurements in the solar spectrum. REMOTE SENSING15(1), 123-143.

 

Take 5 will be played one more week.

=> 

CNES just decided to extend the SPOT5 (Take5) experiment by one week until the 15th of September. This extension is a generous compensation for the loss of two periods of acquisition :

  • the first one was due to a failure of the receiving station, and resulted in the loss of a complete cycle of data, from May the 29th to June the 2nd.
  • the second and recent one was related to a bug in the programming software in a configuration that had not been tested yet. This one resulted in the lost of two days of acquisition, from August the 1st,, 21:26 :22 to August the 3rd, 22:25:44.

Thanks to this extension, we will still get 5 complete months of data for each site, and about 30 acquisitions per site, every 5 days.  After that, Sentinel-2 will soon take over from SPOT5 (Take5) acquisitions. Sentinel-2 acquisitions started to be almost systematic above Europe at the end of July, and the programmed zone will be extended progressively.

 

 

 

SPOT (Take5) distribution server is open

 

 

ESA and CNES are glad to announce they just opened the new distribution site for SPOT4 (Take5) and SPOT5 (Take5) with a brand new look and feel compared to the previous SPOT4 (Take5) site. All the SPOT4 (Take5) data are available and may be downloaded, but the SPOT5 (Take5) site is almost empty (except for two sites : "France : Pornic" and "SouthAfrica : Mooreesburg"). The SPOT5 (Take5) part will hopefully start to be filled next week.

 

Here is the address :

 

https://spot-take5.org

 

To download data, you will need to register only once ("sign up" button), and then to log on ( "sign in"  button). Then you may either download each product or add it to the cart and use the cart to download batches of products. I'll try to better describe how it works in the next days, but meanwhile I am quite sure you will be able to manage on your own. There are only two things you need to know :

  • on the left part of the page (see the image below), there is a very convenient button to add all Level1A or all level 1C to the cart for a given site.
  • you will have to add the downthemall plugin to your firefox browser in order to download the cart. There are similar plugins for the other browsers, such as Metalink downloader for Chrome (which I did not test).

 

 

SPOT5 est sur l'orbite Take5/ SPOT5 is on Take5 orbit

 

Tout s'est très bien passé ce matin, et SPOT5 a atteint sa nouvelle orbite, 2,5 km plus bas. Félicitations et un grand merci à toute l'équipe des opérations du CNES ! Les premières images seront acquises la semaine prochaine. Ne quittez pas !

 

On m'a de nouveau demandé comment un petit changement d'altitude de 2 km pouvait produire un changement aussi important de cycle orbital (de 26 à 5 jours). C'est expliqué ici.

 

Everything went well this morning, and SPOT5 is now on its new orbit, 2.5 km lower. Congratulations and many thanks to CNES operations teams ! The first images will be acquired next week. Stay tuned !

 

I had that question again : why does an altitude change by only 2 kms result in an orbit cycle change from 26 days to 5 days ? It is explained here.

 

Take5 goes to the movies

=>

How to go from 1 image every 5 days to 24 images per second ?

It was possible, thanks to CNES funding, thanks to an imaginative producer, Gérard Dedieu (who does not smoke cigars yet), thanks to a talented film director and scenarist, Thierry Gentet (the only film director who understands space mechanics), and thanks to his team, Mira Production, who are even able to shoot beautiful images in our  ... splendid CESBIO offices, and thanks to a series of promising actors and actresses Anne Jacquin, Valérie Demarez, Virginie Lafon, Valery Gond, Jean-Pierre Dedieu, and another one, the last one, who cannot say a full sentence before the 5th take.

 

We hope this little film will help you understand or explain the possibilites and opportunities offered by multi-temporal images at a high resolution, and that it will give you ideas to use the new SPOT5 (Take5) data.

 

Take5 fait du cinéma

=>

Comment passer d'une image tous les 5 jours à 24 images par secondes ?

C'est possible, grâce au soutien financier du CNES, grâce à un producteur plein d'idées, Gérard Dedieu (qui ne fume pas encore le cigare), grâce à un scénariste-réalisateur de talent (Thierry Gentet), le seul réalisateur de cinéma qui comprend la mécanique spatiale,  et grâce à son équipe (Mira Production) qui sait même mettre en valeur les ... magnifiques bureaux du CESBIO, et enfin grâce à une brochette d'actrices et d'acteurs de premier rôle, enfin dont c'était souvent le premier rôle : Anne Jacquin, Valérie Demarez, Virginie Lafon, Valery Gond, Jean-Pierre Dedieu et un autre, le dernier, qui n'arrive pas à faire une phrase complète avant la 5e prise.

 

Nous espérons que ce film vous aidera à comprendre ou à expliquer les intérêts et enjeux de l'imagerie multi-temporelle à haute résolution.