Quantitative comparison of cloud masks from MACCS/MAJA, Sen2Cor and GEOSYS (hand made)


As already explained in a previous post, we obtained some Sentinel-2 hand made cloud masks from GEOSYS company. We used those to validate the cloud masks from MACCS/MAJA. But we wanted to use them further to make a quantitative comparison with Sen2Cor cloud masks.
But this comparison required solving a little issue : GEOSYS cloud masks are generously dilated to avoid any risk to let a cloud pass through the operational processing. Those of MACCS:MAJA are also dilated while those of Sentinel-2 are not at all. In the following paragraphs, we'll explain how we solved that issue. Sen2cor (v2.3.0) has also three levels of cloud mask (High Medium and Low probability). We used here the Medium Probability mask. But let's start with the final result comparing the performances of Sen2Cor and MACCS:MAJA.


Overall accuracies for MACCS/MAJA, in red and Sen2cor, in blue for 11 images compared to GEOSYS cloud masks.

Continue reading

Comparaison quantative des masques de MAJA et Sen2Cor vis à vis des masques manuels de GEOSYS.


Comme nous l'avons expliqué dans un article précédent, nous avons obtenu de la part de la société GEOSYS des masques de nuages de référence faits main pour Sentinel-2, qui nous ont permis de valider les masques de MACCS/MAJA. Nous avons voulu aller plus loin et comparer avec les masques de Sen2cor.


Mais cette comparaison nécessite de résoudre une petite difficulté : les masques de GEOSYS sont généreusement dilatés pour ne pas prendre le risque de laisser passer des nuages dans la chaîne de traitement. Ceux de MACCS/MAJA le sont aussi, alors que ceux de Sen2cor ne le sont pas du tout. Dans ce qui suit, j'ai utilisé les masques de SEN2COR Medium Probability, fournis par la version 2.3.0 de SEN2COR (qui fournit 3 niveaux (High, Medium et Low). Les précisions obtenues pour les deux chaînes sont celles exposées ci-dessous :


Comparaison des pourcentages de pixels bien classés par MACCS/MAJA (en rouge) et par Sen2cor, en bleu.


Le reste de l'article expose la méthodologie utilisée pour obtenir ce résultat et montre quelques exemples. Continue reading

De l'importance d'un bon masque de nuage pour le traitement automatisé de séries temporelles


Série temporelle de réflectance au sommet de l'atmosphère pour les 4 bandes à 10m de résolution de Sentinel-2, issues des produits L1C


Le graphique ci-dessus montre une série temporelle de réflectance TOA rassemblée par Sentinel-2 sur un pixel choisi au hasard dans une tuile au centre de la France (tuile 31TDK, pixel 3000-7000), à partir de produits L1C. En regardant la série chronologique, il est assez difficile de dire quel type de surface a été observé, même si un cycle végétatif semble être présent. Comme nous le verrons ci-dessous, la plupart du bruit observé est dû à la présence de nuages ​​ou d'ombres de nuages.


La courbe ci-dessous montre qu'après avoir retiré tous les nuages ​​et leurs ombres, la réflectance au sommet de l'atmosphère est déjà plus lisse, et il est ainsi beaucoup plus facile de comprendre le type de surface observée. Continue reading

The importance of a good cloud mask for the operational use of Sentinel-2 data


Top of Atmosphere reflectance in the 4 high resolution channels of Sentinel-2

The above plot shows the TOA reflectance time series gathered by Sentinel-2 over a pixel chosen randomly in a tile in the centre of France (tile 31TDK, pixel 3000-7000), from L1C products.  Looking at the time series, it is rather difficult to tell what kind of surface was observed, even if a vegetation cycle seems to be present.  As we will see below, most of the observed noise is due to the presence of clouds or cloud shadows.

Continue reading

First test of Sen2cor 2.2.1 (and comparison with MACCS)

A couple of days before the living planet symposium, ESA issued a new version of Sen2cor (V2.2.1), the Sentinel-2 atmospheric correction toolbox. I had made a first comparison of MACCS and Sen2cor results using V2.0.6, regarding the cloud masks mainly, in which I concluded the scene classification was really bad, but probably due to bugs and lack of validation, and that we should wait for next version.


First of all, I found the installation of Sen2cor straightforward, at least on an Ubuntu 14.04 linux platform. You just have to read the doc and do as advised. The command line is also very simple and processing time correct, 30 minutes per tile (when processing without MNT), which is a little faster than MACCS (which does much more things). And just bravo ! to the team for being able to do that on all kinds of platforms.


I have made my first tests on a scene in Morocco (tile n°29RNQ), which I had already analysed with MACCS runs. If you look at the figures below, you will see that the claim of a large improvement of the scene classification is perfectly true. The obtained results are logical given the information used to detect the clouds, ie thresholds on reflectances and reflectances ratio, but no multi-temporal stuff. I have drown the contours of cloud masks in green, the contours of snow masks in pink and the contours of water in blue. For Sen2cor, I selected the clouds with medium or high probability, plus the high clouds. Here are the images, I will comment them below.



Sen2cor MACCS
2016-03-18 2016-03-18
2016-04-07 2016-04-07
2016-04-17 2016-04-17

What first strikes is the thickness of the contours in Sen2cor, which is due to the fact that scene classification is done at full resolution with Sen2cor, and at a lower resolution with MACCS and then interpolated. The edges of the clouds in sen2cor are made of a patchwork of cloud/no cloud pixels, which give a greater thickness to the contours. I know that's subjective but it seems that sen2cor finds too many clouds, for instance in the March image, at the North West of the scene.


On the second image, Sen2cor still detects too many clouds, and confuses snow and clouds even more than MACCS (which is far from perfect with partial snow cover).


Finally, on the third image, Sen2cor also misses a few clouds (small ones, but also some very bright low clouds, this is strange and I do not know how it happens. MACCS uses a 400m buffer around each cloud, in case it has fuzzy edges. This is very useful for all types of clouds except for the small cumulus clouds, which invade the north-eastern part of the image, where some clear pixels are lost. But to reduce the buffer only on cumulus clouds, it would be necessary to identify the type of cloud, which is not that easy.


One may also note that the tints of the images are very similar, which shows that the atmospheric corrections are equivalent. and the stability with time of these colours also show that they are good !


More data should be processed to confirm this first impression, but we may conclude that Sen2cor was indeed improved, and even if some classification errors can be found, the available information seems to be well used. As already said, MACCS seems more accurate in terms of cloud detection  thanks to the use of multi-temporal criteria, but because of that, it is less easy to use.


In a few weeks, you should have access to Sentinel-2 data processed with MACCS, either within THEIA ground segment, or through the Sen2Agri package.