A seamless and cloudless Sentinel-2 image of France in July 2018

=>
UPDATE: resolution improved to 20m !
 
I had to tinker a couple of days with Theia's Sentinel-2 monthly syntheses, in order to produce the mosaic below. For that, I used:

  • monthly syntheses of Sentinel-2 L3A products delivered by Theia in July 2018; this products are made from Level2A products, corrected from atmospheric effects, and provided with a good cloud mask, thanks to MAJA processor. Such L3A products will be delivered every month.
  • a script developed by Simon Gascoin
  • good advice from Simon and Michel Le Page
  • and Gdal, (Thanks Gdal !)

This mosaic best resolution is 20m. It is already requiring 8 GB. We could have provided it at 10m resolution, but it would have required 32 GB and several dozens of hours of computation.


See it full screen

Continue reading

La France sans nuages et sans coutures, vue par Sentinel-2 et Theia

=>
UPDATE: résolution améliorée à 20m !

Voilà, après quelques jours de bidouille, j'ai réussi à produire la mosaïque de la France à partir:

  • des synthèses mensuelles de produits Sentinel-2 de Niveau 3A fournis par Theia en juillet 2018; Ces produits sont réalisés à partir de données de niveau 2A, corrigé des effets atmosphériques, et avec une bonne détection des nuages, grâce à la chaîne MAJA. Ces données seront produites tous les mois
  • du script mis au point par Simon Gascoin
  • des conseils avisés de Simon et de Michel Le Page
  • et de Gdal, (Merci Gdal !)

La mosaïque vous est présentée à 20m de résolution, elle occupe déjà 8GO. Il aurait été possible de la produire à 10m, mais il aurait fallu 32 GO et quelques dizaines d'heures de plus.

Voir en plein écran

Continue reading

Belgium at 10 m resolution in July 2018

Using the new L3A product in Theia it is possible to make nice cloud-free mosaics from Sentinel-2 imagery. Here is an example for Belgium and the script to do it in your terminal.

 

Click here to view in full screen.
 


# download L3A images over Belgium
python theia_download.py -l 'Belgium' -d 2018-07-01 -f 2018-07-31 --level LEVEL3A -a config_theia.cfg
# unzip using GNU parallel
parallel unzip ::: SENTINEL2X_201807*zip
# make a mosaic of each band
# WARNING only works if all images have the same projection otherwise an extra step is required with gdalwarp
parallel gdalbuildvrt {}.vrt SENTINEL2X_201807*/*{}*.tif ::: B2 B3 B4
# stack the band mosaics
gdalbuildvrt -separate B432.vrt B4.vrt B3.vrt B2.vrt
# export as a RGB image at full resolution
gdal_translate -ot Byte -scale 0 2000 B432.vrt B432.tif
# Optionally clip the image using the polygon of the Belgium borders

gdalwarp -dstnodata 0 -q -cutline Belgium.kml -crop_to_cutline B432.tif B432_Belgium.tif
# make a tiled map to display in a browser
gdal2tiles.py -z 6-12 B432_Belgium.tif

 

NB) I used this command to generate the file Belgium.kml from the Eurostat Countries datasets:


ogr2ogr -f KML Belgium.kml -where "NAME_ENGL='Belgium'" CNTR_RG_01M_2016_4326.shp

[Nouvelles de MUSCATE] Un été très productif

=>

Cet été, alors que la plupart d'entre nous prenions des coups de soleil sur la plage, nous faisions dévorer par des moustiques, attrapions des courbatures en rando, ou faisions de longues siestes pour nous remettre de nuits fatigantes, les équipes de production et de distribution de MUSCATE ont eu la chance de bénéficier d'une climatisation efficace, de fauteuils confortables, d'ordinateur rapides et de l'accès à la meilleure cantine du monde au CNES. Il n'est donc pas surprenant que la production ait fait de gros progrès, mais nous pouvons néanmoins leur adresser un grand merci, car les résultats sont impressionnants.

 

Amélioration des performances de production

Theia's L2A counter reached 100 000 images on August 12th

 

Tout d'abord, l'anomalie qui avait perturbé la production à la fin du printemps a été résolue. C'était un problème de capacité de mémoire sur la machine qui héberge le catalogue interne de MUSCATE. Petit à petit, la besoin de mémoire dépassait le volume prévu, et les performances se dégradaient avant que le système ne plante, quasiment tous les jours. Il a suffi de quelques ajustement et d'un doublement de la mémoire pour que le problème disparaisse.

Ainsi, depuis début juillet, MUSCATE n'a plus connu que deux courtes interruptions dues à une phase de maintenance du centre informatique du CNES. Comme on peut le voir sur la figure qui suit,  la courbe orange, qui moyenne la production sur un mois n'a jamais été aussi haute, depuis le démarrage de la production.

 

Number of L2A produced each day by MUSCATE (after removing the products with more than 90% of clouds)

Continue reading

[MUSCATE News] A very productive summer

=>

This summer, while most of us were getting sunburnt near the coast, or getting sore legs after hiking in mountains, or having long naps after hectic nights, the lucky MUSCATE teams stayed at work, with a good and efficient air conditioning, convenient armchairs, fast computers and accessed to the best canteen in the world at CNES. So it's no surprise they progressed a lot in all aspects, but even if they benefited from perfect conditions, we should thank them a lot.

 

Improved production performances

Theia's L2A counter reached 100 000 images on August 12th

 

First of all, the colleagues solved the issue that delayed a lot MUSCATE production in late spring. It was a memory issue, with a heavy swap consumption that slowed then crashed the MUSCATE ground segment. Doubling the available memory on the catalogue server solved the issue.

As a result, since beginning of July, MUSCATE had only two interruptions of production, due to maintenance interventions on the CNES platform. As it may be seen below, the monthly average plot in orange shows that the production rate is at its best in two years.

 

Number of L2A produced each day by MUSCATE (after removing the products with more than 90% of clouds)

Continue reading

Three snow seasons in the Pyrenees through the eyes of Sentinel-2 and Landsat-8

On June 23 we will celebrate the third anniversary of Sentinel-2A in orbit. With three years of data we can start looking at the inter-annual variability of biophysical variables, like.. (random example), the snow cover.

 

This is what I attempted to do for the Theia workshop. I downloaded all available snow cover products from Theia over the Central Pyrenees (tile 31TCH) and I generated additional snow maps from the Theia Landsat-8 level-2A products using let-it-snow processor. Landsat-8 images enable to increase the frequency of observations when only Sentinel-2A was operational between 2015 to 2017.

 

I resampled the Landsat-8 snow maps to the same reference grid as Sentinel-2 at 20 m resolution using the nearest neighbor method. I cropped all snow maps to the intersection of the Sentinel-2 tile (green polygon) and Landsat-8 tile (red polygon).


When there was a snow map from Sentinel-2 (S2) and Landsat-8 (L8) on the same day, I merged them into a composite using a simple pixel-based rule:
Continue reading

Sen2agri system released

After 3 years of development, we are very happy to share the news of Sen2Agri system release. Sen2Agri system is a fully automatic production system to produce agriculture information from Sentinel-2 data, with a focus on food security applications. For this reason, the final user meeting was held in Rome at Food and Agriculture Organization and World Food Program. The Sen2Agri project was funded and managed by ESA, and developed by a consortium led by Université Catholique de Louvain, with CESBIO, CS France and CS-Romania.

A very attentive audience at the User Final Meeting, in the impressive World Food Programm conference room

The system manages the following operations :

  • Sentinel-2 and LANDSAT 8 data download,
  • L2A processing with MACCS/MAJA software (developed by CNES and CESBIO)
  • Monthly Synthesis product generation (with a method developed at CESBIO)
  • Generation of LAI products (based on a method developed at INRA, France, and updated, integrated to Orfeo Toolbox by CESBIO)
  • A Crop mask (issued several times per year), with two different methods :
    • without in situ data (method developed at UCL)
    • with in-situ data (method developed at CESBIO)
  • A crop type product (with a method developed at CESBIO, an early version of iota2 processor)

The scientific work behind the methods was described in a special issue of MDPI remote sensing.

Continue reading

Monthly cloud free syntheses merging Sentinel-2 and Landsat 8

To compute a cloud free synthesis of surface reflectances every month, a good repetitivity of observations is necessary. The weighted average method we developed at CESBIO, and which will be part of ESA's sen2agri system was coded by Cosmin Udroiu at CS Romania. It was meant to work with both Sentinel-2 sensors and an observation every fifth day. As we are still waiting for the launch of Sentinel-2B, the monthly syntheses obtained with Sentinel-2A alone really lack cloud free data.

 

On the left, the Sentinel-2A monthly synthesis, above Odessa (Ukraine) in May, and on the right its flag, with, in black, the areas flagged as cloud or cloud shadow. When a pixel is flagged as cloud or cloud shadow, the monthly synthesis provides the minimum blue reflectance, which tends to avoid clouds (if possible), but often selects cloud shadows.

 

Fortunately, the Sen2agri L3A processor is designed to work with LANDSAT 8 too, as both satellites have similar spectral bands, and as the MACCS atmospheric correction used to produce the L2A input products works for both sensors. Of course LANDSAT 8 geometric resolution is not that of Sentinel-2, so to avoid degrading Sentinel-2 imagery when LANDSAT8 data are available, we give Landsat 8 a very low weight in the weighted average. As a result, Landsat is really taken into account only when no cloud free Sentinel-2 was available during the synthesis period.

 

Same result as above, but including LANDSAT 8 data. A cloud free date at least is now found for every pixel. The water mask obtained from Level 2A product is a little wrong on the Landsat 8 image due to the presence of turbid waters and thin clouds. A solution for this problem will be implemented in next MACCS L2A version. Note that the monthly synthesis of both Sentinel-2 and LANDSAT-8 leaves nearly no visible artifacts on the lands.


For a better comparison of both versions, here is a little animation of composites with and without Landsat 8.

The Sen2Agri system is still in validation phase and should be released as open source next May, 6 months from now. The L3A synthesis processor will be also implemented within Theia and monthly L3A products will be distributed by Theia as it is already the case for L2A products.