Plus de 1000 téléchargements en deux ans pour le code de correction atmosphérique MAJA

=>

Nous avons commencé à distribuer MAJA gratuitement en juillet 2017, il y a moins de deux ans, et le logiciel a déjà été téléchargé plus de 1000 fois (1068 précisément aujourd'hui). Celà représente en moyenne plus de deux téléchargements par jour de semaine, et ces dernières semaines, nous avons atteint environ 4 téléchargements par jour. Bien sûr, c'est probablement beaucoup moins que Sen2cor, qui dispose d'un financement confortable pour le rendre facile à installer et à utiliser sur un ordinateur personnel, sous Windows, IOS ou Linux. Mais les deux logiciels ne jouent pas dans la même catégorie: Sen2cor a été conçu pour fonctionner du côté client, tandis que MAJA a été conçu pour être robuste et efficace dans les environnements de production.

 

MAJA ne fonctionne que sur linux, et ses algorithmes multi-temporels le rendent moins facile à utiliser. MAJA fait des calculs beaucoup plus complexes que Sen2cor, en deux fois moins de temps que Sen2cor grâce à une bonne parallélisation utilisant la bibliothèque Orfeo Toolbox C++ Library. Malgré cette complexité, beaucoup d'utilisateurs semblent avoir réussi à le faire fonctionner.


Comparaison de séries temporelles d'images de MAJA (Gauche) and Sen2cor (droite) sur la région de Naples. Les masques de nuages sont soulignés en vert, et les ombres en jaune. Cliquez sur l'image pour l'agrandir.

La version de MAJA que nous recommandons actuellement est la version 3.3 qui apporte de nombreuses améliorations.

Deux outils permettent de faire tourner MAJA efficacement :
- Sen2Agri, pour lequel MAJA (et son ancienne version MACCS) a été téléchargé environs 700 fois
- Start-MAJA, qui est un petit orchestrateur que j'ai d'abord développé pour mon usage personnel, puis publié en open source sur github. Il a été téléchargé 450 fois et a reçu 18 étoiles sur la plateforme github du CNES , et 45 sur mon dépôt github personnel, où il était disponible avant de basculer vers celui du CNES.

Les téléchargements de STart-MAJA sur github pendant la dernière quinzaine

 

Cependant, il n'est pas facile de savoir combien d'utilisateurs ont réussi à l'installer correctement, et combien ont échoué. Nous savons que certains utilisateurs parviennent à l'installer facilement. Nous recevons souvent des commentaires et des questions lorsque les utilisateurs ne réussissent pas immédiatement, et nous ne connaissons pas la proportion de ceux qui abandonnent. Nous serions heureux de recevoir plus de retours sur les difficultés rencontrées et sur le taux de succès. N'hésitez pas à nous informer !

MAJA est un logiciel développé par la société CS-SI pour le compte du CNES (service DNO/OT/IS, Imagerie Spatiale). Les méthodes ont été définies par le CESBIO avec des apports du DLR. D'autres services du CNES (DSO/SI/MO, physique de la mesure optique) et compagnies (Magellium, Cap Gemini, Thales-IS) on apporté leur concours à son développement et sa validation.

 

Sen2Agri final version was just published

The Sen2Agri system was just released in version 2.0. As the project is now terminated and will not receive more funding from ESA, V2_0 is the final version, apart maybe for a few bug fixes.

 

Sen2Agri is a very complete ground segment that automates download and processing of Sentinel-2 data to obtain multi-temporal syntheses, crop/non crop, and crop type masks as well as biophysical variables and phenological indicators. It was designed to be able to process the Sentinel-2 data over whole countries.

 

The Sen2AGri system has been a huge success. I receive an email each time MACCS/MAJA is downloaded, and I just counted 651 downloads after sorting them out the double download,

The new version includes the most recent version of MAJA (3.3) for atmospheric correction and cloud detection (but without using the CAMS data, which are not always easy to access), an early version of WASP to make the composites, an early version of Iota2 to obtain crop types, a processor to obtain the crop/non Crop mask, and a processor to compute biophysical variables inspired by the works of F.Baret and M.Weiss at INRA.

 

The consortium, funded by ESA, was lead by Sophie Bontemps et Pierre Defourny at the Université Catholique de Louvain. The system was developed by CS-SI France and Romania, and most of the methods were designed in CESBIO.

 

 

 

Sen2agri system released

After 3 years of development, we are very happy to share the news of Sen2Agri system release. Sen2Agri system is a fully automatic production system to produce agriculture information from Sentinel-2 data, with a focus on food security applications. For this reason, the final user meeting was held in Rome at Food and Agriculture Organization and World Food Program. The Sen2Agri project was funded and managed by ESA, and developed by a consortium led by Université Catholique de Louvain, with CESBIO, CS France and CS-Romania.

A very attentive audience at the User Final Meeting, in the impressive World Food Programm conference room

The system manages the following operations :

  • Sentinel-2 and LANDSAT 8 data download,
  • L2A processing with MACCS/MAJA software (developed by CNES and CESBIO)
  • Monthly Synthesis product generation (with a method developed at CESBIO)
  • Generation of LAI products (based on a method developed at INRA, France, and updated, integrated to Orfeo Toolbox by CESBIO)
  • A Crop mask (issued several times per year), with two different methods :
    • without in situ data (method developed at UCL)
    • with in-situ data (method developed at CESBIO)
  • A crop type product (with a method developed at CESBIO, an early version of iota2 processor)

The scientific work behind the methods was described in a special issue of MDPI remote sensing.

Continue reading

Monthly cloud free syntheses merging Sentinel-2 and Landsat 8

To compute a cloud free synthesis of surface reflectances every month, a good repetitivity of observations is necessary. The weighted average method we developed at CESBIO, and which will be part of ESA's sen2agri system was coded by Cosmin Udroiu at CS Romania. It was meant to work with both Sentinel-2 sensors and an observation every fifth day. As we are still waiting for the launch of Sentinel-2B, the monthly syntheses obtained with Sentinel-2A alone really lack cloud free data.

 

On the left, the Sentinel-2A monthly synthesis, above Odessa (Ukraine) in May, and on the right its flag, with, in black, the areas flagged as cloud or cloud shadow. When a pixel is flagged as cloud or cloud shadow, the monthly synthesis provides the minimum blue reflectance, which tends to avoid clouds (if possible), but often selects cloud shadows.

 

Fortunately, the Sen2agri L3A processor is designed to work with LANDSAT 8 too, as both satellites have similar spectral bands, and as the MACCS atmospheric correction used to produce the L2A input products works for both sensors. Of course LANDSAT 8 geometric resolution is not that of Sentinel-2, so to avoid degrading Sentinel-2 imagery when LANDSAT8 data are available, we give Landsat 8 a very low weight in the weighted average. As a result, Landsat is really taken into account only when no cloud free Sentinel-2 was available during the synthesis period.

 

Same result as above, but including LANDSAT 8 data. A cloud free date at least is now found for every pixel. The water mask obtained from Level 2A product is a little wrong on the Landsat 8 image due to the presence of turbid waters and thin clouds. A solution for this problem will be implemented in next MACCS L2A version. Note that the monthly synthesis of both Sentinel-2 and LANDSAT-8 leaves nearly no visible artifacts on the lands.


For a better comparison of both versions, here is a little animation of composites with and without Landsat 8.

The Sen2Agri system is still in validation phase and should be released as open source next May, 6 months from now. The L3A synthesis processor will be also implemented within Theia and monthly L3A products will be distributed by Theia as it is already the case for L2A products.

Des cartes de cultures irriguées pour une meilleure gestion de l'eau en agriculture

=>

Dans un billet précédent, je vous avais brièvement présenté:

  • les problématiques liées à la consommation en eau inhérente à la culture du maïs irrigué en France;
  • les projets de recherche relatifs à cette thématique dans lesquels le Cesbio est impliqué.

Pour classer les surfaces agricoles irriguées, en cours de campagne et à l'échelle d'un territoire, nous nous sommes, dans un premier temps, focalisés sur l'utilisation d'images satellitaires optiques.
Dans les lignes qui vont suivre, je vais vous présenter le travail réalisé pour générer des cartes de cultures irriguées avec les séries temporelles Landsat-8 de niveau 2A mises à disposition par le pôle Theia.

Continue reading

First Sentinel-2 monthly cloud free syntheses from the Sen2Agri system

=> 

 

The Sen2Agri System, funded by ESA, is now on its pre-operational phase. It has just completed its first mass production and in this context, we just obtained our first Level 3A syntheses obtained with Sentinel-2A. Level 3A products are monthly syntheses of non cloudy pixels.

Syntheses

The monthly syntheses are produced by the Sen2Agri system. They use as input 2A level products processed by the MACCS processor, which provides surface reflectance along with cloud masks and shadow masks and snow and water masks. As their name suggests, the syntheses are produced once a month, but can be based on a bit longer time periods to increase chances to get cloud free observations.

Continue reading

Les premières synthèses mensuelles Sentinel-2 issues du système Sen2Agri

=>

 

Le système Sen2Agri, financé par l'ESA, est en phase de validation pré-opérationnelle. Il vient donc de réaliser ses premières productions de masse. Dans ce cadre, nous venons d'obtenir, avec Sentinel-2A, nos premières synthèses de Niveau 3A, c'est à dire des synthèses mensuelles des réflectances de surface des pixels non nuageux.

Synthèses

Les synthèses mensuelles sont produites par le système Sen2Agri. Elles utilisent en entrée des produits de niveau 2A traités par la chaîne MACCS, qui fournit des réflectances de surface accompagnées de masques de nuages et d'ombres et masques de neige et d'eau. Comme leur nom l'indique, les syntheses sont produites une fois par mois, mais peuvent se baser sur des périodes de temps un peu plus longues pour accroître les chances d'avoir des observations claires.

Continue reading

SPOT4 (Take 5) special issue is now complete

The last time I showed this post on the front page, in March, I thought the special issue about SPOT4 (Take5) was complete, but we were lacking a paper about snow cover, which came a little late, but still on time to join the special issue. The paper by Jean-Pierre Dedieu and colleagues was just released :

14 papers have finally been published in the SPOT4 (Take5) special issue in MDPI remote sensing. As this journal is an open access journal, all the papers may be accessed from the special issue webpage.

Some news about Sentinel-2 from Living Planet Symposium

 

If I haven't posted for more than a week, it is because I have been participating to the Living Planet Symposium 2016 in Prague. With the preparation, travel, and participation to this crazy symposium. I say crazy because it is packed with about 3000 people, of which I know only about 300. When you want to go from a room to another, it takes as much as 20 minutes as you meet at least 3 or 4 of your colleagues and have a chat. But I do not need to tell you, as very likely, you were there too !

One of the 6 to 8 rooms, packed with people at the same time

 

The good news is that I have had access to a lot of information. I will start by some news about Sentinel-2 of course. Some of these news are not good.

  • The launch of Sentinel-2B has been postponed by to 2017, probably between March and June, due to a delay with the Rockot launcher. This is very annoying as accounting with the time needed for the commissioning phase, it means we will only rely on a 10 days repetitivity during modt of 2017.
  • The availability of Sentinel-2 ortho-rectified data with ground control points has also been postponed to the end of 2016, while initially it was planned in June. ESA says it is due to the fact that the global reference images (GRI) are not ready yet worldwide. We thus will have to cope with registration errors of about 1 pixel within the same orbit and 2 pixels when comparing data from 2 orbits. ESA had announced last year that they would introduce the reference data per continents starting by Europe, but it seems they changed their minds. Still I was told that the GRI for Europe and Australia are available or will be very soon, so why not starting a prodution of ortho-rectified data on those continents ?
  • ESA is also going to change and shorten the very long names of their products and start to distribute data tile wise. Of course, this is good news as the choices made before were not convenient, and it is better correcting it now, as the data backlog is short, but it means everyone will have to change his software. This might delay several productions downstream ESA ground segment.
  • ESA has published a new version of Sen2cor, which is said to seriously enhance the scene classification which was really bad in the previous version, especially for its cloud and cloud shadows mask. I will test it of course, as soon as I find some time.

 

This image background image is a monthly synthesis of Sentinel-2 images of august 2016, covering the whole Czech Republic. It was processed by MACCS to level 2A  and then to level 3A by the synthesis method we developed at CESBIO and implemented within Sen2Agri package. The overlayed landcover map itself was generated by GISAT in the Czech Republic based mainly on Sentinel-1 data, as Sentinel-2 data last summer were still quite scarce. This poster was shown on the 10x10m advert on the congress centre, alternating with a nice bikini (sorry, I only have the Sentinel image !)

 

I have been very positively surprised by how our user community has started using the time series, instead of using images only. It is clear we have entered a new chapter of remote sensing history on the application side, with much more robust results. The case studies based on one image have completely disappeared on the presentations, even if they are still present on the posters. Sentinel-1 examples were impressive and joint uses of both Sentinels 1 & 2 are rising. As the recently launched Sentinel-3A seems to be working well, it is clear ESA has set up a great system Europe can be proud of. And on top of that, ESA really know how to organise a symposium !

 

2015

=>

Dear reader,  may 2015 bring you health and happyness !


2015 should be a great year for the time series of optical images at a high resolution, with the launch of Sentinel 2A in Mat, in less than four months ! We have been waiting for it for so long that we can hardly believe it will be launched so soon. And to help you wait until then. SPOT5 orbit will be changed, to observe a hundred of sites with a 5 day repetitivity similar to that of Sentinel-2, until the end of August.

 

This post is also an opportunity to recall what was achieved in 2014 :

  • In February, we started the Sen2Agri project for l'ESA, with our partners, UCL, CS-SI France and CS-SI Romania. This project which is keeping us busy busy aims at producing a system for automatic classification of crops at the scale of whole countries.
  • In March, THEIA released a new version of SPOT4 (Take5) data, in particular with an enhanced geometrical superposition. This data set has now been downloaded by more than 600 users.
  • In May, the production in near real time (with a delay of 15 to 20 days) of LANDSAT 8 L2A products over France started (this type of product is still not available at USGS, but not for long, I guess...), followed by the processing of de 3 years of LANDSAT 5 & 7 data. 5 years of data, from 2009 to 2014, with an interruption in 2012, after LANDSAT 5 end of life and before LANDSAT 8 launch.
  • In September, CNES finally decided to launch SPOT5 (Take5) experiment,  thanks to a large contribution from ESA. The call for sites proposal was launched by ESA in November, and was largely successful, with 62 proposals for a hundred of sites. The analysis of this proposal is now on-going, and the choice will be difficult as the proposed applications are rich and very diverse.
  • In September too, CESBIO hosted the meeting of THEIA Scientific Expertise Centres, which allowed to identify about 20 products that the centre could prepare in the coming years. These products are described in the recent THEIA bulletin (in French).
  • In November, CNES hosted the SPOT4 (Take5) users meeting, to summarize data use, with a hundred of participants and  23 excellent presentations of the applications allowed by the experiment.
  • In December, we finally launched the call for sites proposal for Venµs mission, which will be launched in 2016. The deadline is January the 28th, 2015.

 

We will have several topics to comment for this blog that starts its third year, with a large audience : it received 23000 visits, and 47000 pages were read, a 50 % increase compared to last year. These weeks, it is getting harder for me to maintain a rhythm of a post per week that I tried to keep in the first two years. Contributions of data users are welcome !

In white, the countries from which no visit was observed in 2014 (Ouganda, Afghanistan)

Liste of the 10 countries from which the most frequent visits are observed.