MAJA now corrects thin cirrus clouds on Landsat-8 and Sentinel-2 images

=>
Thanks to our collaboration with  DLR,, who developed the method [1], and with an ESA funding, we added within MAJA a correction for thin cirrus clouds that works for LANDSAT 8 and Sentinel-2. This correction uses the cirrus band  at 1.38 µm, which allows to estimate cirrus reflectance, which is then subtracted from the other bands with an empiric factor derived from the images.

 

Continue reading

Correction des cirrus sur les images de Sentinel-2 et Landsat 8

=>

Grâce à notre collaboration avec le DLR, qui a mis au point la méthode [1], nous avons mis en place dans MAJA une correction des cirrus fins pour LANDSAT 8 et Sentinel-2. Cette correction utilise la bande cirrus située à 1.38 µm, qui permet d'estimer la réflectance des cirrus qui est ensuite soustraite des autres bandes avec un facteur de proportionnalité calculé empiriquement.

 

Continue reading

First delivery of Venµs images

A first set of 35 images (L1C) is available for download on the following CNES web site:

https://theia.cnes.fr

 

Venµs is now systematically acquiring data on all selected sites since January 2018, although a few The Delta of Ebroacquisitions were missed from time to time for various technical reasons, which should become progressively less frequent.

 

The commissioning phase of the Venµs mission is not yet completed regarding image quality (radiometry and geometry). This delay is due to several issues, the most important of which are detailed hereafter. However, since these issues do not prevent the use of the data, CNES decided to process and distribute a first set of 35 Level-1 (L1) images for which the radiometric and geometric qualities are acceptable. Level 1 corresponds to orthorectified Top of the Atmosphere reflectances. This preliminary data set is intended to allow the users to familiarize with the data and their format.

 

These 35 images are only a subset of the data that are systematically acquired over the Venµs sites since January 2018. We plan to start the distribution of these L1 Venµs time series by April 2018. We expect to start the distribution of the Level-2 products (surface reflectances) by late May or beginning of June, 2018.

 

 

Main issues

 

  1. The absolute calibrations of the spectral bands B1 (415 nm) and B2 (440 nm), both located in the blue, are less accurate than the calibration of the other spectral bands. This issue prevents the use of B1 and B2 for retrieving the aerosol optical depth (AOD) or for water color applications.
  2. The accuracy of the satellite attitude restitution is for now lower than expected. Due to the fact that the different bands are not acquired simultaneously this has mainly impacts on the inter-band registration. For some sites with heavy cloud cover or uniform landscape, this issue also impacts the multi-temporal registration.

 

The characterization of radiometric and geometric performances is still ongoing. Efforts are also devoted to the improvement of the preprocessing algorithms.

As soon as significant progresses are made, the whole data set acquired since January 2018 will be reprocessed with the new parameters and algorithms.

The product format document is available here. A more complete version will soon be made available there.

 

The Venµs page on the Theia web site will keep you informed on the progress made.

 

Time series of Venµs images acquired over Oklahoma (USA) and processed at level 2A (with a first try of atmospheric effects correction applied)

 

 

 

 

 

[MUSCATE news] MAJA upgraded to version 2

Yesterday, the MUSCATE ground segment started delivering Sentinel-2A L2A products using the MAJA V2 processor, while MAJA V1 was used until now. MAJA V2 corrects for a few bugs (such as the pixels flagged as cloud or cloud shadow on the edge of the images) and adds three new features :

  • We implemented a correction for directional effects for a better estimate of Aerosol optical thickness. As you probably know, MAJA uses multi-temporal criteria to detect clouds and estimate aerosol content. These criteria suppose that surface reflectance does not change much from one date to the next one. Over a given tile, we can combine data coming from two adjacent orbits, with slightly different viewing angles, and the directional effect can induce some variations in the surface reflectance of successive acquisitions, which would be interpreted as atmospheric effects. We are now using a simple directional correction to reduce this noise source (D.Roy et al). As this directional correction is not perfectly accurate, it is not applied to the surface reflectance we deliver at the end of MAJA processing.

Elsa Bourgeois, from Cap Gemini, compared the performances of processing with or without directional correction. In both cases (S2A, top, S2B, bottom), there is only a very slight improvement of performances.

  • We implemented a cirrus correction, that comes from DLR works. This correction is not in production yet, as it needs more validation on a large data set. We simply parametrise MAJA with "cirrus_correction= false". A coming post will address this subject.
  • We have started using the new high cloud threshold that was refined in this post. It will result in more cloud cover, with a large difference over mountains.

 

All these small improvements are now implemented in version 1.6. They do not justify a reprocessing of the Sentinel-2 archive, although the accumulation of successive slight improvement could justify it. We intend to start a reprocessing when MAJA V3 is put in production.

Systematic validation of Sentinel-2 THEIA L2A products

=>

Very recently, Camille Desjardins (from CNES), who is handling the validation of the L2A products generated by THEIA, has set up a systematic validation of the products delivered by MAJA, with the help of an operational service from CNES (OT/PE) (Bruno Besson, and Nicolas Guilleminot from Thales Services, using tools developed by Aurélie Courtois, also from Thales)

 

Systematically, a comparison of AOT and water vapour is made for every Sentinel-2 L2A product from THEIA which observes one of the sites of the Aeronet network.

 

Both plots below show the results obtained during the month of February, for the Aerosol Optical Thickness (left), and for the water vapour content (right). Blue dots correspond to validations in ideal conditions (low cloud amount, no gap filling, and quality assured Aeronet data (Level 2.0). The red dots allow degraded conditions, and most of them correspond to the unavailability, yet, of version 2.0 Aeronet data. As data are processed in near real time, and level 2.0 data are made available a few months later, these plots rely mainly on Level 1.5 data, which are more prone to errors (such as a calibration drift... or the presence of a spider in the instrument tubes).

 

Aerosol optical thickness validation of Sentinel-2 L2A for all Aeronet match-ups gathered in February 2018 Water vapour validation of Sentinel-2 L2A for all Aeronet match-ups gathered in February 2018 (in g/cm2)

 

Continue reading

[MUSCATE News] Reprocessing of Sentinel-2B data aquired from July to October 2017

=>

MUSCATE is in a good shape these days thznkd to the continuous efforts of the development team (CNES and CAP GEMINI) who solved several issues. The counter of Sentinel-2 Level 2A products reached 70 000 products this night, just one month after reaching 60000. If we sum all the products delivered by MUSCATE, we are reaching 99 000 images. MUSCATE also distributes Sentinel-2 Snow masks over mountains, and Spot World Heritage data (old SPOT data reprocessed after ortho-rectification.and made available for free).

 

This good shape allows us to increase our production rhythm. We have started processing the Sentinel-2B data acquired between July and October 2017, as we had started processing S2B in November 2017 only. But since MAJA is a multi-temporal processor, we are in fact starting a complete reprocessing of the data, including S2A and S2B. The quality of S2A products should therefore also benefit from the improved repetitivity of observations.

 

This reprocessing will last several weeks. We are starting with data from France and will go on with our neighbouring European countries, then sites in Maghreb, the remaining sites of Africa, and finally, the rest of the sites in the world.

 

In case you have an urgent need for some tiles, please ask ! (of course, it is only applicable to the tiles already in our list)

 

 

 

Amélioration de la détection des nuages hauts dans MAJA pour les images de Sentinel-2

=>

La recherche pour l'amélioration des performances des méthodes de MAJA repose actuellement sur une toute petite équipe au CESBIO, composée de deux personnes, Bastien Rouquié, qui travaille sur l'amélioration des estimations d'aérosols, et moi, qui passe presque tout mon temps en réunions, ou a écrire des articles de blog (et parfois les deux en même temps). C'est pour cette raison que de nombreux points que nous aimerions améliorer sur MAJA ne s'améliorent que lentement.

 

Absorption atmosphérique. En bleu, la réflectance de surface pour un pixel couvert de végétation, en fonction de la longueur d'onde, en rouge la réflectance au sommet de l'atmosphère pour ce même pixel. A 1.38 µm, la vapeur d'eau absorbe totalement la lumière provenant de la surface au niveau de la mer.

J'ai pu récemment consacrer un peu de temps à l'amélioration de la détection des nuages, notamment au dessus de l'eau, et plus récemment, à améliorer la détection des nuages hauts, en utilisant la  bande "cirrus" située à 1.38 µm, qui est disponible sur Sentinel-2 et Landsat 8. Comme précisé dans ce post, la bande "cirrus" est située dans une forte bande d'absorption de la vapeur d'eau, telle qu'au niveau de la mer, les photons ont très peu de chances de faire le parcours du soleil au satellite en passant par la surface de la terre sans être absorbés. Comme la majorité de la vapeur d'eau est située dans les basses couches de l'atmosphère, les nuages situés plus haut dans l'atmosphère ont davantage de chances de renvoyer de la lumière jusqu'au satellite. C'est donc un moyen de détecter les nuages hauts.

 

La méthode de détection parait donc très simple, il suffit de faire un seuillage sur la réflectance dans la bande 1.38 µm.  Malheureusement, c'est plus compliqué que ça.

Continue reading

Theia produit au niveau 2A toute la bande côtière du Maghreb

Les tuiles en bleu viennent d'être ajoutées aux zones où Theia fournit des produits de niveau 2A.

=>

Depuis la découverte des récents bugs et la mise en place de moyens de les contourner, le rythme de  production de MUSCATE s'est accru, et nous avons donc pu étendre un peu les zones où nous fournissons des données Sentinel-2 (A&B) corrigées des effets atmosphériques, avec un bon masque de nuages: il s'agit de produits de niveau 2A fournis par la chaîne MAJA.

 

Nous venons de mettre en ligne toutes les données acquises sur la zone côtière du Maghreb, du Maroc à la frontière entre Algérie et Tunisie. Quelques tuiles manquantes ont également été rajoutées au sud du Maroc, et sur le Cap Bon, en Tunisie. Avec cet ajout de 50 tuiles, toutes les terres qui bordent la méditerranée occidentale sont donc suivies pat les produits de Theia (enfin, il manque les Baléares).

Continue reading

[MUSCATE] Theia releases Sentinel-2 L2A products on the whole Maghreb coastal region

The blue tiles were just added to the zones where Theia provides level 2A products with MAJA.

=>

Since we understood the bugs which were slowing MUSCATE, and we found ways to mitigate them, the production rhythm of MUSCATE improved and we were able to extend the zones where we provide Sentinel-2 (A&B) L2A products. L2A products provide surface reflectances after correction of atmospheric effects and with a high quality cloud mask. The products we deliver are provided by MAJA processor.

 

We just released all the data acquired on the Maghreb coastal zone, from Morocco to Tunisia. A few missing tiles have also been added in South Morocco, and on Cap Bon in Tunisia. With these new tiles, we now monitor all the lands that surround the occidental part of Mediterranean sea, adding 50 tiles to those already processed.

Continue reading

Improvements of MAJA cloud masks in production

We are continuously working on improving the Sentinel-2 L2A products delivered by Theia. Since the 2nd of February, a new parametrization was put in production in order to improve two points of the cloud masks. There is a page which summarizes the variations of Theia product versions. Because of some error within Muscate production centre, we are not able to change the product version when we change a parameter. You have to look at the production date to know which version was used. This inconvenience will be corrected soon.

Clouds above water

Although MAJA is optimized for cloud detection and atmospheric correction over land, it is important to detect clouds well over water. For instance to detect the shadows of these clouds that are cast over land. But until now, the clouds mask was missing quite a lot of clouds over water. This was not a surprise as we had passed only a few hours determining the thresholds to apply. We recently found some more time to obtain a better tuning of the detection thresholds above water.

Thresholds in the SWIR have been halved (from 0.08 to 0.04), in the absence of sunglint. When sunglint is likely, due to the geometry of acquisition, the threshold is still higher (0.016, but it was 0.25 before).This means that when the sunglint flag is raised, the accuracy of cloud detection is reduced.

Impact of new cloud detection threshold over water : left, old threshold, right, new threshold.

Cloud dilation

 

 

 

 

The plane contrail illustrates the difference in observation angles of the spectral bands of Sentinel-2. The cumulus clouds above has the same effect, although lower because the cloud is at a lower altitude.

As it may be seen on the image above, the cloud limits are fuzzy, and except for the big and nice cumulus clouds, the pixel next to the cloud mask is also affected by clouds. Even if the cloud has sharp limits, it also changes the illumination of neighbouring pixels, and, in the case of Sentinel-2, as all the bands do not observe exactly in the same direction, there is a "parallax effect" which results in different cloud positions depending on the spectral band. For all these reasons, we need to dilate the cloud mask (by the way, this is an identified drawback of Sen2cor, which does not dilates its clouds).

 

Our dilation buffer was originally 480m. Following complaints from some users working in very cloudy countries, we had reduced the buffer to 240m in may 2017. But we recently figured out that the reduced dilation was degrading our estimates of  aerosols, as undetected clouds are considered as aerosols.  Due to this, the new version which runs since the2nd of February has once again a dilation of 480m.

Left, aerosol validation results with a dilation of 240m, right with a dilation of 480m. The new result brings a significant improvement (Merci à Bastien Rouquié pour ce résultat)