On-demand SENTINEL2 L2A processing with MAJA on PEPS

Update : added a script to submit and download MAJA jobs on PEPS

 

Getting a Sentinel-2 L2A product (with atmospheric correction and cloud detection) anywhere has never been so easy.  Until now, you had two options to access to Sentinel-2 L2A products obtained from the MAJA processor :

 

 

Starting from today, there is another possibility: an on-demand MAJA processing from the French Sentinel collaborative ground segment PEPS.  The possibility is a proof of concept, and if a lot of people use it at once, we do not know how the processing time will evolve. But the only way to know it is to try it, hoping the processing and download capacity is sufficient. Anyway, it you intend to process a whole country, we hope you are from Belgium, and not from Australia.

 

However, if your site is above 60°N or under 60°S, you are still unlucky, as MAJA needs SRTM DEM to work. We will try to implement the use of Arctic DEM one day.

 

PEPS/MAJA processing page (don't worry, english language is also available)

So how does it work ?

  1. go to https://peps.cnes.fr, create an account, log in.
  2. Use the explore Menu, select your data. I like the semantic research possibility, for instance, you can ask for Sentinel-2 L1C Toulouse August, and you'll get all the products available in August.
  3. Select the ones you are interested and click on the gearing icon.
  4. Go to "your processing center" page (top left, with the gearing icon in black)
  5. Select the "MAJA atmospheric processor" and the products to process. It only works for Sentinel-2 L1C in single tile format, no need to try to process a Sentinel1 image,+ a S2 L2A image, or the old S2 format grouping several granules. Then press on "Process the products"
  6. Go to "My processing" page
  7. Wait (don't be fooled by the percentage, it goes fast at the beginning, but it takes between 1 and 2 hours)
  8. Download the results. If the image you chose contains more than 90 % of clouds, you get a filename which starts with "NOVALD" for "Not Valid".

 

OK, it's a lot of clicks, I am pretty sure you will not try to process the whole Australia like that. As this processing is made through the Wep Processing Service (WPS)  interface, there is a way to do that using only http requests, with no click. With the help of Christophe Taillan, I have implemented two sample scripts to help you do that (the first users will be beta testers). The first script submits Maja processes on PEPS, the second one checks thir completion and eventually download results.  The scripts are available on github.

 

The processing takes between  1 or 2 hours for one product.  This is due to the fact MAJA is a multi-temporal processor,it does not provide its best result when only processing one image, but when processing a whole tile series of images. When you ask for one product, PEPS actually processes, not 1, not 2, not 3... but 8 of them to get a good initialisation. But it only delivers the one you asked for. If the image you selected is not too recent, the 8 products will be posterior to that date. If the product was just acquired, the 8 products will be anterior.

 

This procedure is not optimal. if you want all the products in July 2018, for each product MAJA will use its backward mode, and process the seven products after that date. Some of these products will be therefore produced 8 times. As I said, it is a proof of concept, the PEPS team will soon propose more efficient ways of delivering the data.

 

What about the quality of results ?

Left THEIA product, right PEPS product. Both are generated by MAJA, but with a different history, as MAJA is a multi-temporal processor. The clouds are circled in green, the shadows in yellow, and water in blue

 

The above figure compares a MAJA product from Theia on the left, and a MAJA product on the right. Reflectances are nearly identical, cloud masks only slightly differ. The differences are due to the fact that both productions do not use exactly the same history or products. The Theia product should be a little better in general, but differences are low, provided the period is not too cloudy.

 

Just one warning, if the product you ask for is the only clear one in a very cloudy sequence of 8  images, do not expect a good result, due to the fact that we limit the history to 8 images. MAJA in THEIA context would produce a better result as it use longer periods of time.

 

The PEPS team will be happy to have feedback ! You might comment here or write to them.

By the way, who did what ?

Christophe Taillan did most of the development work, with some help from Erwann Poupart and myself, Vincent Garcia and the exploitation team make sure that everything works, Patrice Henry, the big boss, enabled the project, which was started by Mireille Paulin (to whom we think a lot). You may even see their photographs in the "who are we" page, here.

 

 

 

 

 

Posted under: Corrections (geometry, atmosphere, clouds), In English, PEPS

6 comments

  • Olivier Hagolle on 03/10/2018 at 11:45 said:

    Hi Mamy,
    that's indeed MAJA 3.1 which is running within PEPS. The reason it is not available yet for download is quite ridiculous, I am ashamed !

    At CNES, we have to deliver software through https://logiciels.cnes.fr .This site only accepts software with less than 500 MB. We are above this figure. And we have asked to raise the quota three months ago, with no answer...

    Sorry for that,
    Olivier

  • Charlotte Pelletier on 05/10/2018 at 05:02 said:

    Hi Olivier,

    Thanks a lot for this interesting post. I hope either the download of MAJA 3.1 or tips for using htpps requests will be available soon.

    Hopefully, I only want to process Victoria, not the whole Australia! So it should be fine :D

    Cheers,
    Charlotte.

    • Olivier Hagolle on 05/10/2018 at 08:40 said:

      Thanks Charlotte
      I have just added a link to submit and download MAJA L2A to PEPS.
      I'll be happy to have you as a beta user (see above)
      Cheers,
      Olivier

  • Simon on 05/10/2018 at 15:57 said:

    When a user queries a L2A product via peps_maja_process.py, or through the PEPS/MAJA webpage, does it check first if the product is available within the Theia L2A collection?

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>