[MUSCATE News] Unavailability of CNES HPC center on 4-5 December

Le centre de calcul (de haute performance) du CNES sera en maintenance les 4 et 5 décembre, dans le but d'accroitre sa robustesse et ses performances. En conséquence, MUSCATE ne sera pas en mesure de produire les données de Sentinel-2, Landsat et Venµs en temps réel. Pour une fois, nous ne serons donc pas en mesure de tenir nos engagements de production en moins de 2.5 jours, et nous espérons que cela ne vous dérangera pas trop dans vos travaux. Les données déjà produites resteront disponibles.

Pour la même raison, les traitements avec MAJA sur PEPS seront suspendus.

 

CNES High Performance Computing center will be on maintenance on the 4th and 5th of December, to improve its robustness and capacity. As a result, MUSCATE will not be able to produce Sentinel-2, Landsat and Venµs data during these days. Exceptionally, we will not meet our target to process these data in less than 2.5 days, and we hope it will not cause too much inconvenience. The already produced data will still be available.

For the same reason, MAJA processing on PEPS will be suspended.

Sentinel-2 Level3A time series (July to October 2018)

=>

As every month, Peter Kettig from CNES processed the Sentinel-2 L3A composites of France from the Month before. The data can be downloaded from Theia's distribution server at CNES.

If you are not afraid to spend too much time while you have urgent things to do, you may have a look to the mosaic of Sentinel-2 monthly syntheses for each month since July over France. Each monthly synthesis is accessible using the following links :

 
Or you may also use the nive viewer below (merci à Michel Lepage !) to compare with the previous months and see how France starts to become slightly greener, at least in the North, while the mountains are getting brown after the first cold days, and corn is drying in the South West.
 
It is also the opportunity to observe our first neat swath edge effect in four months, near Cambrai, North of France. The Western part of the artefact is browner than the Eastern part. Because of the cloud cover, the average date used in the eastern part is several days before the average date of the western part. The only way to reduce the effect is to increase the revisit time of Sentinels, with the future generations, to allows us to reduce the synthesis window length which is currently 45 days.
 

See it full screen
Continue reading

Theia just selected a new production zone for Sentinel-2 L2A in Sahel

=>

The Level 2A data production with MAJA processor, by MUSCATE processing center is now fast and efficient. Some margins have been found to add new zones to our production. (The zones already processed are displayed here). French public institutes of labs that would need new zones can tell us their needs, we should still have a few tiles left in our quota.

 

A few well informed colleagues acted fast, and Santiago Pena Luque (from CNES), who works for the SWOT satellite project made a large new proposal. The Senegal and Niger basins which have been chosen as integrative test sites by SWOT. So the main target is to monitor the height and flow of the rivers, but this implies to study rainfall, evapo-transpiration and run off. The Sentinel-2 data will provide the vegetation status. Of course, the same zone will also allow a large variety of applications related to climate, desertification, food production monitoring, starting from land cover. Several laboratories are associated to the proposal: namely, GET, LEGOS and CESBIO in Toulouse, and TETIS in Montpellier.

 

Here is the zone that was accepted by Theia today :

 


Sahel zone just accepted by Theia. In green, the tiles already produced by theia, in blue, the new tiles.

Continue reading

The Khumbu Icefall by Venµs

This is a time-lapse of all clear-sky images captured by Venµs over the Khumbu Icefall near Mount Everest since November 2017 (one year of data, 51 images at 5 m resolution).

Khumbu Icefall by Venµs

Here I used the Level-1C products (i.e. without atmospheric correction) because the Level-2A products are provided at a lower resolution (10 m). Anyway, the atmosphere is rather thin in this area..

To make this animation (without the date annotation to simplify):
1) download
python ./theia_download.py -l 'Nepal' -c VENUS -a config_theia.cfg -d 2017-11-01 -f 2018-12-01 --level 'LEVEL1C'
2) unzip
mkdir -p ../VENUS
parallel unzip -d ../VENUS ::: $(find . -name "VENUS*zip")

3) export as natural color pictures
cd ../VENUS
mkdir -p VIS
parallel gdal_translate -srcwin 4118 3132 1058 770 -of JPEG -b 7 -b 4 -b 3 -scale 0 800 0 255 -ot byte {} VIS/{/.}.jpg ::: $(find . -name VE*[0-9].DBL.TIF)

4) animate with imagemagick
convert -delay 10 VIS/*jpg anim.gif

Sentinel-2 Level3A time series (July, August, September 2018)

If you are not afraid to spend too much time while you have urgent things to do, you may have a look to the mosaic of Sentinel-2 monthly syntheses for September over France. You may access to each monthly synthesis using the following links :

 
Or you may also use the viewer below to compare with the previous months and see how France became brown in September :
 

See it full screen

The monthly syntheses are produced using the WASP processor, which is described here.

By comparing the various syntheses, you will see the evolution of the landscape, generally much brownler in September, but this representation will also help you spot the composite artefacts. These are not very numerous, but you will see them :

  • on some web browsers (firefox V58), geometrical differences appear even at a low resolution. Other browsers and versions do not have this defect. It is really not due to Sentinel-2 or Theia products
  • above water and snow (we must work on this defect)
  • where clouds have covered a place during the whole month of July or August. These pixels are flagged as invalid in the products (but not on the mosaic).
  • where clouds or shadows were not properly detected by MAJA
  • at the edges of Sentinel-2 swath. For the first time, in october, a swath edge is clearly visible near Cambrai. The area must have been quite cloudy, and we observe here a greener part on the right, observed later in October, that the browner part on the left. The only way to correct this kind of atefact while keeping a physical meaning to the reflectances, would be to improve Sentinel-2 revisit time
  • some tile edges in July, due to the fact that Level 3A products were not all generated for the 15th of July, but for dates between the 8th and the 26th. This has been corrected for the next months

 

Sentinel-HR, a metric resolution optical mission with global systematic and frequent observation, and free and open data

=>

Every four years, CNES brings together its scientific advising committees (and I have never been invited :( )  for its scientific prospective seminar. This seminar is where the scientific priorities for CNES's next space missions are defined. A few months before, a call for ideas for new missions is issued, and it was the opportunity to submit the idea of a mission called SENTINEL-HR (following Gérard Dedieu who had already proposed it two years ago).

 

Sentinel-HR mission

This mission would make it possible to observe all the lands at a metric resolution once a season. Taking into account the clouds, it would require a systematic revisit frequency of 20 to 30 days. As the mission name suggests, access to data would be free and open. The minimum configuration would have four bands (blue, green, red, near-infra-red), with optional bands in the short wave infra-red and the possibility of making stereoscopic observations.

2018 Image cover of France at a high resolution (6m), acquired by SPOT-6/7, for Theia/GeoSud project (1). Data were acquired between the 6 of april and the 11th of October .

 

As I already presented numerous tiles and for many years,  the benefit of Sentinel-2 data, compared to the SPOT missions of the past, is the ability to make frequent and systematic acquisitions, coupled with the free and open availability of data. This clearly explains the success of the SENTINEL-2 mission, which brought the number of users each year from a few thousands for SPOT to tens of thousands for SENTINEL-2.

Continue reading

Snow cover duration from 01 Sep 2017 to 31 Aug 2018 in the Alps and Pyrenees

=>

We are thinking to distribute a new product that would provide the snow cover duration in the Alps and the Pyrenees over a hydrological year (snow persistence map). Here is a preview of a "beta" version that is only 100 m resolution for now.
 

Snow cover duration from 01 Sep 2017 to 31 Aug 2018. Pixels with less than 60 days were masked out.


 
A few words on the method (see also here): this map is generated after a linear interpolation at the daily time step of every available Theia snow product. Here we have selected 26 Sentinel-2 tiles, which represents 3189 snow maps for the study period, that is 96 billion pixels to process.
 
With the accumulation of Sentinel-2 data it becomes possible to look at interannual variability. Here is for example a visual comparison of the snow cover duration calculated from January to May for three years in the Pyrenees (31TCH):

 Snow cover duration 31TCH

Snow Duration from Jan 01 to May 31 in the Pyrenees (tile 31TCH)

 
We can see longer snow durations for the year 2018, which was an exceptional year!
 

[MUSCATE News] A very productive summer

=>

Updated on October 4th

 

This summer, while most of us were getting sunburnt near the coast, or getting sore legs after hiking in mountains, or having long naps after hectic nights, the lucky MUSCATE teams stayed at work, with a good and efficient air conditioning, convenient armchairs, fast computers and accessed to the best canteen in the world at CNES. So it's no surprise they progressed a lot in all aspects, but even if they benefited from perfect conditions, we should thank them a lot.

 

Improved production performances

Theia's L2A counter reached 100 000 images on August 12th

 

First of all, the colleagues solved the issue that delayed a lot MUSCATE production in late spring. It was a memory issue, with a heavy swap consumption that slowed then crashed the MUSCATE ground segment. Doubling the available memory on the catalogue server solved the issue.

As a result, since beginning of July, MUSCATE had only two interruptions of production, due to maintenance interventions on the CNES platform. As it may be seen below, the monthly average plot in orange shows that the production rate is at its best in two years.

 

Number of L2A produced each day by MUSCATE (after removing the products with more than 90% of clouds)

Continue reading

On-demand SENTINEL2 L2A processing with MAJA on PEPS

Update : added a script to submit and download MAJA jobs on PEPS

 

Getting a Sentinel-2 L2A product (with atmospheric correction and cloud detection) anywhere has never been so easy.  Until now, you had two options to access to Sentinel-2 L2A products obtained from the MAJA processor :

 

 

Starting from today, there is another possibility: an on-demand MAJA processing from the French Sentinel collaborative ground segment PEPS.  The possibility is a proof of concept, and if a lot of people use it at once, we do not know how the processing time will evolve. But the only way to know it is to try it, hoping the processing and download capacity is sufficient. Anyway, it you intend to process a whole country, we hope you are from Belgium, and not from Australia.

 

However, if your site is above 60°N or under 60°S, you are still unlucky, as MAJA needs SRTM DEM to work. We will try to implement the use of Arctic DEM one day.

 

PEPS/MAJA processing page (don't worry, english language is also available)

Continue reading