Issue with Firefox 53 on Theia distribution server

We recently figured out that the Theia distribution server is not fully compatible with the most recent version of Firefox (V53), and that the authentication fails, preventing from downloading products with that version. The teams are working to solve this, but meanwhile, please wait before updating your firefox browser.

Nous nous sommes récemment aperçus que le serveur de distribution Theia n'est pas compatible avec la nouvelle version de Firefox. L'authentification plante, et il n'est donc pas possible de télécharger les données avec cette version. Les équipes techniques cherchent à résoudre ce problème, mais en attendant que le problème soit résolu, ne mettez pas à jour votre navigateur !

 

MACCS, renamed MAJA, now distributed as binary for non-commercial use

I have already mentioned that CNES teams working on MACCS cloud screening and atmospheric correction software and DLR teams working on ATCOR, decided to join their efforts to build a common software, named MAJA which stands for MACCS-ATCOR Joint Algorithm. And thankfully, a funding from ESA helps a lot this initiative.

 

The MAJA code relies on MACCS architecture which is designed to use multi-temporal criteria, but the successive MAJA versions will progressively include several features coming from ATCOR which did not exist in MACCS or were less elaborated and accurate.

 

We just released MAJA V1.0 which includes a few improvements (an enhanced mono-temporal mask coming from ATCOR has been included, as well as a modification of the estimation of water vapour). MAJA V1_0 is in fact just a little evolution compared to MACCS latest versions. MAJA V2.0 will include a directional correction (coming from CESBIO team), and a correction of cirrus clouds, that comes from DLR.  V1_0 version is now being integrated to Muscate, and it will soon run on our platform, while V2_0 should be delivered to CNES this week.

 

But the main news related to that is the fact that MAJA is now distributed as a binary code for linux Platforms RedHat and Cent OS versions 6 and 7 only. MAJA is distributed on CNES free software platform, and you will have to accept a licence and provide your email address. The licence is granted for non commercial use only. If you would like to get an extension of this licence for commercial use, please ask CNES.

MAJA tutorial page on github

MAJA was developed to process time series, if you only want to process a few images, please use SEN2COR. Given the size of Sentinel-2 images, MAJA is not meant to run on a laptop, it needs really serious computer resources, fast drives and processors, and several Gigabytes of Memory. I also have to stress that MAJA is not easy to use, and that solid knowledge in computers, linux, python might be necessary. MAJA also needs several stages of data preparation, including specific DTM generation.

Tutorial pages

A few beta-users are now testing the installation procedures. The first feedback is that the users often do not read the documentation carefully ;) . I strongly urge you to read the document provided in the package ! I have written a tutorial page on Github, and I hope you will manage to use it. In case of any problem, please do not comment here or send me an email, use the support address : maja-support@cnes.fr or add an issue to the github tutorial platform.

 

 

 

10000 L2A products available at Theia for Sentinel-2

The number of L2A products from Sentinel-2 available at Theia, just passed above 10 000 ! Data from France, Belgium, North of Spain, Morocco, Tunisia, Senegal, the Burkina-Mali transect, or La Reunion Island are now processed in real time : our requirement is 60 hours, after the data is available in Copernicus hub. Next site to be added is Madagascar.

 

Le compteur de produits de niveau 2A de Sentinel-2, disponibles à Theia, vient de passer la barre des 10 000 produits. Les données acquises par Sentinel-2 sur la France, la Belgique, le Nord de l'Espagne, la Tunisie, le Sénégal, le transect Burkina Mali, et l'île de la Réunion sont maintenant traitées en temps quasi réel : notre spécification est de les mettre à disposition en moins de 60 heures après leur apparition sur les serveurs du hub de Copernicus. Le prochain site couvrira une bonne partie de Madagascar.

THEIA/MUSCATE nears real time for Sentinel-2 L2A

THEIA MUSCATE production of Sentinel-2 L2A products nears real time, and L2A products acquired until March 2017 over France and Reunion Island have started to appear on our distribution server :

https://theia.cnes.fr

The production will progressively be extended over the whole France and over the other sites (Spain, Morocco, Belgium Tunisia, Senegal, Burkina, Mali...) and then follow the real time acquisitions with the shortest delay possible.

 

La production par MUSCATE des données de niveau 2A de Sentinel-2 s'approche du temps réel, et les données acquises jusqu'à mars 2017 sur la France et l’île de la Réunion ont commencé à apparaître sur le serveur de distribution.

https://theia.cnes.fr

 

La production va être progressivement étendue à toute la France et aux autres sites (Espagne, Maroc, Belgique Tunisie, Sénégal, Burkina Faso, Mali...), et suivra ensuite les acquisitions en temps réel avec le plus court délai possible


Quantitative comparison of cloud masks from MACCS/MAJA, Sen2Cor and GEOSYS (hand made)

=> 

As already explained in a previous post, we obtained some Sentinel-2 hand made cloud masks from GEOSYS company. We used those to validate the cloud masks from MACCS/MAJA. But we wanted to use them further to make a quantitative comparison with Sen2Cor cloud masks.
But this comparison required solving a little issue : GEOSYS cloud masks are generously dilated to avoid any risk to let a cloud pass through the operational processing. Those of MACCS:MAJA are also dilated while those of Sentinel-2 are not at all. In the following paragraphs, we'll explain how we solved that issue. Sen2cor (v2.3.0) has also three levels of cloud mask (High Medium and Low probability). We used here the Medium Probability mask. But let's start with the final result comparing the performances of Sen2Cor and MACCS:MAJA.

 

Overall accuracies for MACCS/MAJA, in red and Sen2cor, in blue for 11 images compared to GEOSYS cloud masks.

Continue reading

Comparaison quantative des masques de MAJA et Sen2Cor vis à vis des masques manuels de GEOSYS.

=>

Comme nous l'avons expliqué dans un article précédent, nous avons obtenu de la part de la société GEOSYS des masques de nuages de référence faits main pour Sentinel-2, qui nous ont permis de valider les masques de MACCS/MAJA. Nous avons voulu aller plus loin et comparer avec les masques de Sen2cor.

 

Mais cette comparaison nécessite de résoudre une petite difficulté : les masques de GEOSYS sont généreusement dilatés pour ne pas prendre le risque de laisser passer des nuages dans la chaîne de traitement. Ceux de MACCS/MAJA le sont aussi, alors que ceux de Sen2cor ne le sont pas du tout. Dans ce qui suit, j'ai utilisé les masques de SEN2COR Medium Probability, fournis par la version 2.3.0 de SEN2COR (qui fournit 3 niveaux (High, Medium et Low). Les précisions obtenues pour les deux chaînes sont celles exposées ci-dessous :

 

Comparaison des pourcentages de pixels bien classés par MACCS/MAJA (en rouge) et par Sen2cor, en bleu.

 

Le reste de l'article expose la méthodologie utilisée pour obtenir ce résultat et montre quelques exemples. Continue reading

THEIA diffuse un an de données Sentinel-2 L2A sur le Maroc

=>

Après la France, la Belgique, le Nord de l'Espagne, la Tunisie, un transect du Burkina Faso au Mali et quelques tuiles en Ethiopie, Theia vient de diffuser un an de données Sentinel-2 au niveau 2A au dessus d'une grande portion du Maroc, de Casablanca à Ouarzazate. Les produits de Niveau 2A fournissent des réfectances de surfaces corrigées des effets atmosphériques et accompagnées d'un masque de nuages de grande qualité.

 

Les données de niveau 2A sont, comme d'habitude, téléchargeables depuis https://theia.cnes.fr. Reportez vous à la page d'aide pour trouver la description du format. Pour télécharger les données sans cliquer, vous pouvez utiliser notre outil de téléchargement en python (après vous être inscrits sur le site de Theia).

Continue reading

Quantitative validation of MACCS cloud masks

=> 

At the annual review  of PEPS project (CNES Sentinel global mirror site), the GEOSYS company showed its operational activities centered on the use of Sentinel-2 data to provide advise on agriculture on many regions of the world. On this occasion, GEOSYS showed their cloud detection process for Sentinel-2 images. The Sen2cor solution was not considered reliable enough y GEOSYS, and the regions processed by MACCS within MUSCATE are far from covering all the regions of interest of the company. GEOSYS decided to rely on human operators to improve the cloud masking. For each processed Sentinel-2 image, a man made valid pixel mask is build ("valid" means without clouds and cloud shadows).

 

Continue reading