Quantitative comparison of cloud masks from MACCS/MAJA, Sen2Cor and GEOSYS (hand made)

=> 

As already explained in a previous post, we obtained some Sentinel-2 hand made cloud masks from GEOSYS company. We used those to validate the cloud masks from MACCS/MAJA. But we wanted to use them further to make a quantitative comparison with Sen2Cor cloud masks.
But this comparison required solving a little issue : GEOSYS cloud masks are generously dilated to avoid any risk to let a cloud pass through the operational processing. Those of MACCS:MAJA are also dilated while those of Sentinel-2 are not at all. In the following paragraphs, we'll explain how we solved that issue. Sen2cor (v2.3.0) has also three levels of cloud mask (High Medium and Low probability). We used here the Medium Probability mask. But let's start with the final result comparing the performances of Sen2Cor and MACCS:MAJA.

 

Overall accuracies for MACCS/MAJA, in red and Sen2cor, in blue for 11 images compared to GEOSYS cloud masks.

Continue reading

MACCS qualified for Atmospheric Correction Euro and World Cup

Don't worry, I have not been converted to a football fan, it is still giving me allergies ! But after a post about sex ratio, we are still trying to increase the amount of connexions to this blog.

 

However, the title of this post is partly true, since, simultaneously, we are engaged to two competitions to compare cloud screening and atmospheric correction softwares applicable to Sentinel-2 or Landsat-8. The Atmospheric Correction Euro is organised by ESA, to select the method that will be used by ESA to systematically produce Sentinel-2 images to level 2A. The Euro world cup will last one year, with successive iterations between 4 producers, and a Jury of experts. Of course MACCS is participating, and we (DLR + CNES/CESBIO) are using this opportunity to merge MACCS and ATCOR methods, in order to use the methods from each of these two codes. In this framework, the resulting software we will get was renamed MAJA (say MAYA), for MACCS-ATCOR Joint Algorithm.

 

The Atmospheric Correction World cup is organized by NASA and ESA in the framework of the committee on earth observation satellites (CEOS). Here, about 12 codes will be compared, and the comparison will be done with LANDSAT 8 and Sentinel-2 data, it will seemingly focus on atmospheric correction rather than cloud screening, although it is not always easy to separate both aspects.

 

The World Cup will start this summer, while the Euro started last month. I have already shown a few validation results from this exercise :

- cloud masks comparison with Sen2cor

- validation of the consistency of processing among various tiles

- and thanks to Camille Desjardins (CNES), we just obtained validations of AOT estimates and Water Vapour estimates. The water vapour validation results are quite accurate and largely good enough to perform the atmospheric correction. The AOT validation results are decent, with rmse a little above what we observed for LANDSAT 8 over France. But the choice of sites is more difficult, with 2 arid sites, and the repetitivity of Sentinel-2 acquisitions is still far from what it should be (because of issues on board or on the ground segment). As already said in this blog, the accuracy of MACCS software increases with repetitivity.

Water vapour content validation wrt aeronet data for 4 sites (France, Morocco, Israel, SouthAfrica)

Aerosol optical thickness validation wrt aeronet data for 4 sites (France, Morocco, Israel, SouthAfrica)

 

We only are in the early stage of the EURO, and our wise coach did not introduce the best players in the first rounds. In next round, we will introduce MACCS 5.0, with its much enhanced shadows mask, and for the final stages, we will include the selected modules from ATCOR. But here is a glimpse of the MACCS 5.0 player.
On some images, the previous version of MACCS missed quite a lot of shadows, while with 5.0, they are detected (click on the images below to see the details). And by the way, the images were created within the Sen2Agri system, which is now taking shape and is a little ahead of MUSCATE in terms of versions. Many thanks to our colleagues from the Sen2Agri consortium (UCL, C-S France, C-S Romania, CESBIO).

Image generated at MUSCATE with MACCS 4.8, clouds and shadows are circled in green

Image generated at Sen2Agri with MACCS V5.0. The detected shadows are circled in yellow

And finally, I found an analogy between Football and Atmospheric Correction (I had to search a lot). Some say that football is a game which is played by two teams of eleven players and for which, at the end, Germany wins. For the Atmospheric Correction Euro, all the four teams competing include German institutes or companies. So here also, Germany wins.