Data upload on MUSCATE distribution server is slow

It seems that the upload of data to MUSCATE distribution server is quite slow these days. This results in a delay in the provision of data. As CNES is closed between Christmas and the new year, we will have to wait a little to have it repaired. We are sorry for that inconvenience. But you can use this opportunity to rest and spend time with your families without having to process our scenes :) . We wish you all a happy Christmas week.

Il semble que la mise à jour du serveur de distribution de MUSCATE très lente depuis quelques jours,c e qui cause d'importants retards de distribution. Comme le CNES est fermé entre Noel et le jour de l'an, nous devrons attendre pour réparer ce problème. Nous vous prions e nous en excuser. Mais vous pouvez en profiter pour passer des vacances en famille sans avoir à traiter nos données ;) . Passez tous de bonnes fêtes !

MAJA Sentinel-2 L2A processor downloaded more than 200 times

Since spring 2017, we have made the MAJA cloud screening and atmospheric correction processor available for commercial use. A bit later, end of June, the Sen2agri software package, which includes MAJA older version (named MACCS) , was also released to the public. We did not expect a large success, as these two packages are quite heavy ones, do not work on laptops, and require a specific linux system powerful computers (Red Hat or CentOS).

Anyway, the MAJA processor has had quite a large success, even if, I guess, it is far from the success of Sen2cor, which is much easier to install and use, even if the performances are not the same. The figures below correspond nearly to one download per day.

 

Number of downloads of MAJA (stand alone version) 93
Number of downloads of MACCS (Sen2Agri version)i 116

 

To celebrate this fact, we just published a brand new MAJA detailed description.

A brand new MAJA ATBD

I have always wanted to provide an Algorithm Theoretical Basis Document related to MAJA, but never had time, because I always had more urgent things to do. Some papers had been published, allowing MAJA users to get a good idea or the methods we use, but the published articles did not cover all the features of MAJA.

 

But this time, due to a contractual engagement with ESA, it was the urgent thing to do. So, at last, after a few weeks of hard work, here it is.

 

If you have already read the papers from our team, you will recognize some text published quite a long time ago, but we updated all the text and added some parts which had not been explained yet in journal publications, and of course the new parts recently added to MAJA. This ATBD is now in line with version 2.0 of MAJA.

 

The ATBD can be cited with the following reference :

Olivier Hagolle, Mireille Huc, Camille Descardins, Stefan Auer, Rudolf Richter, MAJA Algorithm Theoretical Basis Document, https://doi.org/10.5281/zenodo.1209633

[MUSCATE] A little error on Sentinel-2B L2A products processed before December 1st.

When we started making tests of the L2A production at MUSCATE with Sentinel-2B data, we considered using a small correction of Sentinel-2B to correct for a eventual bias between the reflectances of each satellite. Such differences were observed by colleagues form CNES, and were comprised between 0 and 2.5%.  Finally, we decided not to implement this in the operational line, because the figures we had were based on a small duration of acquisitions and were possibly not accurate enough.

 

Band name Coefficient
B1 1.002
B2 0.995
B3 1.000
B4 1.008
B5 1.009
B6 1.017
B7 1.014
B8 1.024
B8A 1.012
B9 1.000
B10 1.000
B11 1.004
B12 0.986

 

Due to a human error, it turns out that the parameter file that we used for these tests made its way to the operational line. MAJA parameters are kept under a GIT version manager, which should reduce the probablility of such errors, but that parameter file is a parameter of MUSCATE, not MAJA, which was not managed yet. And as you know, according to Mr Murphy, when something can turn wrong, it turns wrong. the test calibration adjustment made its way to the production line.

 

For the Sentinel-2B L2A products produced until the 30th of November, the L1C TOA reflectance  values were divided by the coefficients provided in the table above. On the first of December, this error was corrected. As the differences are quite small, we have not removed the Sentinel-2B products produced before December 1st, but we are going to reproduce them and replace them by the correct values during the coming weeks.

 

We are sorry for hat error, which will make us improve our verification procedure.

 

 

Building a global cropland mask, is not an easy task

Criticizing is easy, and doing is hard, especially when trying to create a global map of croplands. Some collegues from CESBIO have worked on that subject within the Sen2Agri project, and obtained good resuts, but only at the local or country scale. Finding a method that works everywhere must clearly be much harder.

These days, I have received a lot of emails, tweets and posts about a new cropland global product at 30 m resolution, edited by USGS. I have no doubt it was a serious work from a serious team, done with appropriate terrain data and methods, validation, and of course a tremendous data processing.

 

 

But there it is, I checked it over a lot of places that I know very well, and it seems to me that the cropland mask, at least in South West France, is clearly overestimated. Is it the same in tour region ? Here are some examples :

Continue reading

Le bulletin Theia est sorti !

De la part de Sophie Ayoubi, chargée de la communication de Theia

Le 8ème numéro du bulletin du pôle de données et de services surfaces continentales Theia vient de paraître.

A découvrir dans ce numéro :

  • Actualités
  • Nouveaux produits
  • Services et outils
  • Applications et thématiques
  • Développement instrumental
  • Accompagnement utilisateur

Téléchargez le bulletin de Theia en version impression (3.6 Mo) ou version allégée (932 Ko) ou feuilletez-le en ligne dans l'espace Calaméo de Theia.

Retrouvez l’ensemble des numéros dans la rubrique bulletins.

MUSCATE news : 50000 L2A products from Sentinel-2A and 2B

Since it became operationnal in December last year, MUSCATE has produced 50 000 level 2A products from Sentinel-2A. Let's recall what has been processed so far :
  • For 550 tiles, we have processed all Sentinel-2A data acquired since December 2015.
  • For 100 tiles, mainly in South America, and in Italy, we have processed all Sentinel-2A data from December 2016. We are currently catching up the backlog for Italy, and later on, for South American sites.
  • For all these 650 tiles, we are producing all Sentinel-2 data (Sentinel-2A and Sentinel-2B) in near real time. I think THEIA is the only place where you can download Sentinel-2B L2A data so far. ESA has not started that production yet (nah, nah, nah :) )
  • For all these 650 tiles, we have processed all Sentinel-2B data since beginning of October 2017. We will soon catch-up with the Sentinel-2B data acquired from July 2017.

 

 

See full screen

Map of the 650 tiles currently processed in near real time (in red). The blue tiles will be added beginning of next year.

 

All these products are available from https://theia.cnes.fr

 

Let's recall that MUSCATE uses the MAJA L2A processor, which uses multi-temporal criteria to perform a high quality cloud detection and atmospheric correction. Despite the recent installation of version 2.4, MUSCATE still regularly suffers from instability as soon as CNES High Performance Computer is overloaded. The problem does not lie in MAJA, but in the information exchanges between all the components of MUSCATE which need to respect an accurate timing (sorry, I am not able to explain better).

 

The exploitation team just installed a new version of MUSCATE (v 2.4.16.p2 (!)), which is expected to increase stability. But that's the theory, let's see if it works in the coming days and if we are able to increase our production rate.

 

 

La mission Sentinel-2 va vraiment bientôt fournir des données tous les 5 jours, partout !

=>

Mise à jour : L'ESA progresse rapidement, les acquisitions de toutes les terres ayant défilé sous S2B étaient disponibles sur PEPS le 12 Novembre.

 

La mission Sentinel-2, qui se compose de deux satellites jumeaux, S2A et S2B, a été déclarée opérationnelle par l'ESA au début du mois d'Octobre, même si chacun des satellites n'observait en fait que sur une orbite sur deux, sauf en Europe et en Afrique. Résultat, la répétitivité de 5 jours n'était disponible que sur ces deux continents, et passait à 10 jours ailleurs. Heureusement, dans le dernier Sentinel-2 mission status report, l'ESA a annoncé cette excellente nouvelle :

 

Sentinel-2A et -2B observent ensemble l'Europe, l'Afrique et le Groenland avec une répétitivité de 5 jours. Le reste du monde est observé tous les 10 jours. L'observation complète et systématique de toutes les terres tous les 5 jours sera obtenue quand la liaison EDRS deviendra opérationnelle. Ceci est attendu pour décembre 2017 à la fois pour S2A et S2B.

Continue reading

Sentinel-2 mission close to start observing all lands every 5th day, really !

=>

Update : ESA is progressing fast. All lands overflown by Sentinel-2B were available in PEPS on November 12th, thanks to EDRS relay satellites.

The Sentinel-2 mission, made of two twin satellites S2A and S2B, has been stated "operational" since beginning of October, even if each satellite was only observing on one orbit out of two everywhere except in Europe and Africa. As a result, the 5 days revisit was only available in these two continents.  But in the new Sentinel-2 mission status report, ESA announced this very good piece of news :

Sentinel-2A and -2B are together acquiring Europe, Africa and Greenland with 5 days revisit. The rest of the World is revisited every 10 days jointly by the two satellites. Full systematic 5-day revisit everywhere will be reached once EDRS downlink becomes available operationally, assumed during December 2017 for both S2A and S2B.

Continue reading

[MUSCATE News] Le traitement des données Sentinel-2B L2A a démarré

=>

L'ESA a déclaré Sentinel-2B opérationnel a la fin du mois d'Octobre, même si des données préliminaires étaient déjà disponibles depuis juillet. Depuis hier matin, MUSCATE distribue aussi les produits Sentinel-2B de niveau 2A en utilisant le processeur MAJA (les produits L2A sont exprimés en réflectance de surface après correction atmosphérique et sont munis d'un masque de nuage de bonne qualité). La production actuelle a démarré à partir des acquisitions du 1er octobre 2017 et se poursuivra en temps réel. De plus, nous ajouterons progressivement les produits Sentinel-2B acquis depuis juillet 2017.

 


Premiers produits S2B disponibles sur https://theia.cnes.fr

 

Comme toujours, les données peuvent être téléchargées gratuitement depuis https://theia.cnes.fr

Voici un petit exemple de série temporelle près de Baotou, en Chine, acquises en alternance par Sentinel-2A et Sentinel-2B. Les images se ressemblent beaucoup, excepté là ou la surface a clairement changé, dans la vallée Est-Ouest au centre de l'image.

Sentinel-2A Sentinel-2B Sentinel-2A

 

 

Comme le traitement MAJA est multi-temporel, sa précision bénéficiera de la répétitivité doublée des acquisitions, ce qui devrait avoir un effet sur nos résultats de validation à venir. A partir des productions de qualification de la nouvelle version, nous avons déjà vérifié que les réflectances fournies par les deux capteurs sont assez proches et concordent bien avec les mesures in situ obtenues avec la station de mesure de réflectance de surface du CNES à La Crau (Provence, France). Quelques exemples de résultats sont fournis ci-dessous.

Continue reading