New paper ! An active learning cloud detection tool to generate reference cloud masks for Sentinel-2. Application to the validation of MAJA, Sen2cor and FMask cloud masks

Example of reference cloud mask generated by ALCD, and comparison with the cloud masks generated by three operational processors (Sen2cor, FMask and MAJA). True positive invalid pixels appear in blue, true negative in green, false negative in red and false positive in purple..

It is not that frequent when the work of a trainee ends up as a peer reviewed publication, but Louis Baetens was a brilliant trainee. In a six months training period at CESBIO, funded by CNES, here is what Louis Baetens did:

  • developed an active learning method to generate reference cloud masks for Sentinel-2, using multi-temporal data as input
  • validated the quality of the produced masks (around 99% overall accuracy)
  • generated cloud and shadow masks covering 32 entire Sentinel-2 images
  • produced these same scenes with Sen2cor 2.5.5, FMask 4.0 and MAJA 3.3
  • evaluated the results using ALCD masks
  • wrote a report and a user manual for ALCD
  • released the masks and tools on open access platforms
  • And wrote (with Camille and myself) a scientific publication

 

The publication was just released by remote sensing :

Baetens, L.; Desjardins, C.; Hagolle, O. Validation of Copernicus Sentinel-2 Cloud Masks Obtained from MAJA, Sen2Cor, and FMask Processors Using Reference Cloud Masks Generated with a Supervised Active Learning Procedure. Remote Sens. 2019, 11, 433.

 

The remaining of the post provides a plain language summary (but it's better to read the paper !)

Continue reading

Apport des images radar et optiques pour la cartographie des surfaces irriguées

(English version below)

Dans le cadre du projet Simult’eau (partenaires : Arvalis, CACG, Chambres d’Agriculture du Tarn et des Hautes-Pyrénées, financement CASDAR) nous avons testé l’apport d’une utilisation combinée des images radar et optiques pour la cartographie des surfaces irriguées (maïs et soja) dans le Sud-Ouest de la France. Les résultats publiés dans Remote Sensing (https://www.mdpi.com/2072-4292/11/2/118) ont révélé que l’utilisation d’images radar Sentinel-1 combinées aux images optiques (Landsat-8) permettait de détecter les surfaces irriguées plus précocément qu’avec les images optiques seules. En effet ces dernières sont souvent perturbées par la présence de nuages qui rendent la détection impossible à certaines périodes de l'année. Ce résultat, qui doit être confirmé par des études complémentaires (autres lieux et autres dates), est très encourageant. Il ouvre de nouvelles perspectives pour une gestion "optimisée" des ressources en eau notamment pour des organismes tels que la CACG (Compagnie d'Aménagement des Coteaux de Gascogne) ou les Organismes de Gestion Collective de l’eau (OUGC). Les cartes produites sont en libre accès sur le site Theia: http://www.theia-land.fr/fr/ces-surfaces-irriguees.


Ces recherches se poursuivent actuellement dans le cadre de la thèse de
Yann Pageot financée par le CNES, l'Agence de l'Eau Adour-Garonne et la CACG.

 
Continue reading

S1-Tiling, on demand ortho-rectification of Sentinel-1 images on Sentinel-2 grid

=>

Sentinel-1 is currently the only system to provide SAR images regularly on all lands on the planet. Access to these time series of images opens an extraordinary range of applications.
In order to meet the needs of a large number of users, including our needs, we have created an automatic processing chain to generate "Analysis Ready" time series for a very large number of applications. Sentinel-1 data is ortho-rectified on the Sentinel-2 grid to promote joint use of both missions.

 

 

Continue reading

Sentinel-2 + OpenStreetMap = ♡

You may have heard about the tailings dam collapse at Feijão mine in Brumadinho, Brazil. The disaster occurred two days ago on Jan 25 and at least 58 people were found dead, while 300 are still missing. A Sentinel-2 acquisition was planned for today, therefore tonight I checked the EO Browser to see if the mud flow was visible.

Before/after images of Brumadinho mudflow from Sentinel2 imagery (false color composite using the near-infrared band)

Continue reading

Near-real time analysis of the 2018-2019 snow season in the Pyrenees and the Alps from satellite data

Here in southwest France ski lovers did not really enjoy the beginning of the snow season... But how does it compare to the previous years? Using Sentinel-2 and Landsat-8 data, we computed the snow cover duration since September 01 until January 20 for the past three snow seasons in the Alps and Pyrenees.

Snow cover duration (in days) from 01 September of year N-1 to 20 January of year N



Continue reading

Anak Krakatau before and after the December 2018 eruption

Anak Krakatau (Indonesia) erupted on 22 December 2018. During the eruption the collapse of the volcano summit triggered a tsunami in Sunda Strait causing a death toll of 437. The first post-event clear-sky image was finally acquired by Sentinel-2 today on 13 Jan 2019 (after 10 cloudy acquisitions). Here is an image comparison of the Krakatau Island before and after the eruption.

View in full screen

Improvement of water vapour retrieval in MAJA

Similarly to the aerosol retrieval, the retrieval of water vapour in MAJA atmospheric correction has also been improved, thanks to the work of Elsa Bourgeois (Cap Gemini) and Camille Desjardins (CNES). An accurate estimation of water vapour is not necessary to perform an accurate atmospheric correction, because water vapour absorption in most of Sentinel-2 bands is much lower than 5%. But the Sentinel-2 water vapour product could also prove useful, and when we plot validation results, showing a large bias for high water vapour contents is not nice.

 

 

 

Here is the kind of results we have been having with MAJA from the beginning, with a large bias when water vapour content is high :

Our very simple method uses the ratio between Sentinel-2 B9 and B8a bands to estimate the water vapour. B9 is located within a water vapour absorption band at 940 nm, while B8a serves as reference and is only moderately affected by water vapour. The ratio is converted thanks to the use of a Look-up table, which is obtained using radiative transfer calculations. Our method assumes that the water vapour is above the scattering layer, which is obviously not true. The errors due to this assumption increase with the amount of water vapour.

 

Elsa and Camille just empirically computed a new water vapour LUT to cancel this bias, and it works! As you can see, the RMS errors have been divided by a factor 2, from 0.2 g/cm2 to 0.1 g/cm2.

We will put this new parameter set in production in January within Theia, and make it available to the users of MAJA processor.

 

 

 

 

MAJA 3.1.2 with CAMS option finally validated

We had announced quite a long time ago the coming availability of MAJA 3.1 to correct for the atmospheric effects on Sentinel-2, Landsat 8 or Venµs satellites. This version brings a significant improvement in the estimation of Aerosol Optical Thickness, thanks to the use of Copernicus Atmosphere Monitoring Service (CAMS) data to constrain the aerosol type. The details of the methodscan be found here. Bastien Rouquié obtained them on our python prototype of MAJA.

 

We then implemented them in the operational and fast version of MAJA. If the validation tests of MAJA 3.1 were correct on the two test products we had defined, a large scale validation using 10 sites over two year time series showed that instead of improving, using the CAMS option was degrading the results. We had to search for the cause (a bad interpolation of CAMS data in space and time), and correct the errors and perform again a large validation.

 

This time, the validation results are improving a lot, as it may be seen on the figures below.

Without CAMS option With CAMS option

On the left column, we provide the results without activating CAMS option, while on the right, it is activated. The top row corresponds to the comparison between Aeronet AOT used as reference, and MAJA AOT, for eight sites in diverse landscapes. The bottom row provide an example on the well known validation site in Mongu, Zambia.The blue dots correspond to good quality aerosol measurements (no clouds, level 2.0 aeronet values), while red dots correspond to degraded conditions (with either clouds or not quality assured aeronet data (level 1.5 data)

 

Using CAMS to constrain the aerosol type improves the results by 25%, compared to the use of a continental aerosol model everywhere. Errors for the quality assured validation pixels decrease from 0.085 to 0.065 on the 8 sites, and from 0.143 to 0.094 on Mongu site in Zambia. This site has various types of aerosols depending on the season, including dust, biomass burning and continental aerosols. The results are still far from perfect, and we have work for the next 5 years, but it is still good to have them improved !

 

MAJA 3.1.2 is available starting from this link on github, as an executable program for linux. To be allowed to use it, you will have to sign the licence first, from this site.  If you want to use it for commercial applications, you should ask for a special licence (still for free), sending me an email. In January, I will provide the parameters to allow activate the CAMS options.

 

Regarding the production of Theia, our ground segment has been adapted to use MAJA version 3.1.2, and will soon be able to fetch the CAMS products from Copernicus Atmosphere. Then we will have an operational qualification phase, to check that we can download CAMS products in time for real time production. We should be able to start using in in February or March.  And after a few months, if the results are good, yoohoo, we will reprocess everything !

 

Many thanks to Bastien Rouquié, CESBIO, who did the scientific work, to Camille Desjardins w ho helpled with the validation, to Aurelien Bricier and Benjamin Esquis, at CS-SI for coding the operational version, and Peter Kettig (CNES) and Bruno Angeniol (Cap Gemini), and Bastien, for checking the consistency between prototype and operational versions.

 

 

 

Sentinel-2 Level3A time series (July to November 2018)

=>

As every month, Peter Kettig from CNES processed the Sentinel-2 L3A composites of France from the Month before. The full resolution data, and the corresponding data quality masks, can be downloaded from Theia's distribution server at CNES.

If you are not afraid to spend too much time while you have urgent things to do, you may have a look to the mosaic of Sentinel-2 monthly syntheses for each month since July over France. Each monthly synthesis is accessible using the following links :

Or you may also use the nice viewer below (merci Michel Lepage !) to compare with the previous months.

In November, in France, we had a... French November weather, and several zones stayed overcast for all Sentinel-2 overpasses during the synthesis period of 45 days. In that case, we try to provide a value, which is the minimum reflectance in the blue band. Of course this value is flagged as invalid. So the November synthesis is not as nice as the previous ones, due to the presence of remaining clouds. As in October (see below), we now also see artefacts at the edges of the swath.

Anyway, in many regions, the results are rather correct and they allow us to see the changes. Forests are now brown, soils are wetter and darker, winter crops have started, and the highest mountains are turning white.

In October, we had the first the opportunity to observe a neat swath edge effect in four months, near Cambrai, North of France. The Western part of the artefact is browner than the Eastern part. Because of the cloud cover, the average date used in the eastern part is several days before the average date of the western part. , due to the observation at very different dates on each side of the swath. So the only way to improve that with the current method would be to add a third or even a fourth Sentinel satellite.

See it full screen
Continue reading