This post is no longer up to date. Please use the following pages :
- a description of the distribution center and of how it works
- a program to automatically download the data without any click
This post is no longer up to date. Please use the following pages :
The second "Sentinel-2 for science" symposium , organised by ESA, took place in italy late may 2014. More than 400 future Sentinel-2 users participated, which is a record for a conference organised by ESA at ESA premises. Compared to the first Sentinel-2 users workshop, it turns out that most of the talks were based on time series of images, while this proportion was less than a third for the first users symposium (other talks were about spectral indexes, mono date model inversions, which is good science but is not specifically tailored for Sentinel-2). This shows that the Sentinel-2 users community state of preparation did a lot of progress during the two last years.
To this respect, it seems that the SPOT4(Take5) experiment has helped a lot, as at least 15 of the 55 talks (and a lot of posters) of the symposium were largely based on the data set. That was exactly the purpose of the experiment and I am quite please to see it succeeded. The data are still available there, and there are still a lot of things to do.
Here are the links to the 15 talks that use SPOT4 (Take5) data (I may have forgotten one of two, if so please tell me ! I have not found the links to the posters, if someone found them, please tell me !). You may also access the whole program here (some talks, although not based on SPOT4 (Take5), were also very stimulating )
Marc Leroy1, Olivier Hagolle2, Mireille Huc2, Mohammed Kadiri2, Gérard Dedieu2, Joëlle Donadieu1, Philippe Pacholczyk1, Céline L'Helguen1, Selma Cherchali1
1: CNES, France; 2: CESBIO
Olivier Hagolle1,3, Mireille Huc1,2, Mohamed Kadiri1,2, Dominique Clesse4, Sylvia Sylvander3, Marc Leroy3, Martin Claverie5, Gérard Dedieu1,3
1: CESBIO Umr 5126 CNRS-CNES-IRD-UPS, Toulouse, France; 2: CNRS,France; 3: CNES, France; 4: CAP GEMINI, France; 5: NASA/GSFC, USA
Eric Vermote1, Martin Claverie1,2, Jeffrey Masek3, Inbal Becker-Reshef2, Chris Justice2
1: NASA/GSFC Code 619; 2: University of Maryland, Dept of Geographical Sciences; 3: NASA/GSFC Code 618
Nataliia Kussul, Sergii Skakun, Ruslan Basarab
Space Research Institute NASU-SSAU, Ukraine
Sylvain Ferrant1,2, Simon Gascoin2,3, Amanda Veloso2, Martin Claverie4, Gérard Dedieu1,2, Valerie Demarez2,5, Eric Ceschia2,5, Patrick Durand6, Jean-luc Probst3,7, Vincent Bustillo2,5
1: CNES, France; 2: CESBIO, France; 3: CNRS, France; 4: University of Maryland; 5: University of Toulouse; 6: INRA, France; 7: ECOLAB, France
Based on Formosat-2 rather than SPOT4 (Take5), but these data are similar and produced with the same methods.
Elodie Vintrou1, Valentine Lebourgeois2, Agnès Bégué2, Dino Ienco3, Maguelonne Teisseire3, Pierre Todoroff1, Fidiniaina Ramahandry Andriandrahona4
1: CIRAD UR AIDA, Station Ligne Paradis, 7 chemin de l’Irat, 97410 Saint Pierre, La Réunion; 2: CIRAD UMR TETIS, Maison de la Télédétection, 500 rue J.F. Breton, Montpellier, France; 3: IRSTEA UMR TETIS, Maison de la Télédétection, 500 rue J.F. Breton, Montpellier, France; 4: FOFIFA, Station Régionale de Recherche FOFIFA Tsivatrinikamo ANTSIRABE 110, Madagascar
Defourny Pierre1, Bontemps Sophie1, Cara Cosmin4, Dedieu Gérard2, Guzzonato Eric3, Hagolle Olivier2, Inglada Jordi2, Rabaute Thierry3, Savinaud Mickael3, Sepulcre Guadalupe1, Valero Silvia2, Koetz Benjamin5
1: UCLouvain, Belgium; 2: CESBIO, France; 3: CS-Systèmes d’Information, France; 4: CS-Systèmes d’Information, Romania; 5: ESA, ESRIN, Italy
N. Knox1,2, L.T. Tsoeleng1, C. Adjorlolo1,2, T. Newby3
1: South African National Space Agency (SANSA), South Africa; 2: University of KwaZulu-Natal (UKZN), South Africa; 3: National Earth Observation and Space Secretariat (NEOSS), c/o SIIU - CSIR, South Africa.
Kamal Labbassi1, Nadia Akdim1, Silvia Maria Alfieri2,3, Massimo Menenti2
1: Chouaib Doukkaly University, Morocco; 2: Delft University of Technology, Netherlands; 3: Institute for Mediterranean Agricultural and Forest Systems, Italy
Christophe Sannier, Louis-Vincent Fichet
Astrid Verhegghen, Baudouin Desclée, Hugh Eva, Frédéric Achard
Joint Research Centre of the European Commission, Italy
Colette Meyer1, Hervé Yesou1, Stephen Clandillon1, Henri Giraud1, Jérôme Maxant1, Paul de Fraipont1, Arnaud Selle2
1: SERTIT, France; 2: CNES, France
Mapping estuarine turbidity using high and medium resolution time series imagery Virginie Lafon1, Arthur Robinet1, Tatiana Donnay2, David Doxaran2, Bertrand Lubac3, Eric Maneux1, Aldo Sottolichio3, Olivier Hagolle4, Alexandra Coynel3
1: GEO-Transfert, ADERA, Université de Bordeaux, France; 2: Laboratoire d'Océanographie de Villefranche, UMR 7093 - CNRS / UPMC, France; 3: UMR EPOC, Université de Bordeaux-CNRS, France; 4: CESBIO, CNRS,UPS, CNES, IRD, France
Carsten Brockmann1, Ruescas Ana1, Pinnock Simon2
1: Brockmann Consult GmbH, Germany; 2: ESA ESRIN, Italy
Kathrin Weise1, Marc Paganini2, Max Tobaschus1,3, Martin Faber1,3
1: Jena-Optronik GmbH, Germany; 2: European Space Agency, Italy; 3: Friedrich Schiller University Jena, Germnay
More than 10 years ago, on the Crau plain, in Provence, CNES set up an automatic calibration station to measure the atmospheric optical properties and the surface reflectances. This station, named ROSAS (RObotic Station for Atmosphere and Surface), is at the top of a 10 meter mast, and is equipped with a CIMEL instrument similar to the ones of the AERONET network that are used to characterize the atmospheric aerosols. But this one has been modified to observe also the ground. The initial objective of this station was to check the absolute calibration of optical remote sensing instruments with a high resolution (because the site uniformity is not sufficient for satellites with a kilometric resolution). But this station proves also useful to validate the surface reflectances from satellite level 2A products.
This work was done by some CNES colleagues, Vincent Lonjou, Sébastien Marcq et Aimé Meygret, using the level 2A products obtained from SPOT4 (Take5) experiment.
The ROSAS station needs 90 minutes to fully characterize the downward radiance and thus the atmosphere, and the upward radiance. The ratio of both measurements enable to compute the surface reflectance. However, the process is a little more complex than described here, as the surface around the mast is not perfectly uniform and the reflectances are affected by directional effects. A bidirectional model is therefore fitted to the measurements, and this model is then used to predict the reflectances measured by the satellite.
The ROSAS instrument, during the SPOT4 (Take5) experiment, was equipped with 10 spectral bands described in the table below. The instrument is now being modified in view of Sentinel-2 and Venµs launches, to accommodate new spectral bands, in the near infra-red mainly, where the sampling of the spectrum was not sufficient.
|Band||λ (nm), detector|
The agreement of ROSAS and SPOT4(Take5) surface reflectance measurement is excellent, in all band but near-infrared : better than 5% in the green (B1), red (B2) and SWIR (B4) channels, and 7-8% in the NIR (B3). The differences observed in the NIR are being investigated, but could be linked to the spectral interpolation, as SPOT4 B3 band is quite far from ROSAS spectral bands.
In the SWIR, the greater variations of surface reflectances with time may be noticed, with large reflectance drops after rains. The SWIR band is very sensitive to the soil moisture, at least when the vegetation cover is sparse, which is the case at La Crau. In the other bands, these variations are much less visible, and what should be noticed is the great stability of surface reflectances with time, thanks to the acquisitions with constant viewing angles and also to the quality of atmospheric correction...
A poster was shown by Aimé Meygret at the "Sentinel-2 for science" symposium in Frascati in may 2014.
It is our great pleasure to announce that the LANDSAT 8 level 2A data produced by THEIA are available at the following address.
The available data are all the data acquired by LANDSAT over France, for which a sufficient number of cloud free pixels were available. They were processed to Level 2A : they are expressed as surface reflectance after atmospheric correction, and are provided with a cloud mask. The way we produced them is explained here for LANDSAT 8 and here for LANDSAT 5 and 7.
The distribution server was developed by my CNES colleague Jérôme Gaspéri, helped by Rémi Mourembles from CAP Gemini ; it has a very simple but very modern interface, with only one simple field to formulate requests, which may be provided as sentences in day to day language. The tool indeed makes a semantic analysis of your requests. And it is meant to work as well on your computer, tablet or phone (but you should think before downloading a whole LANDSAT product on a smartphone).
Example of requests :
1) Date and locationLANDSAT7 images on Biarritz between january and june 2009LANDSAT8 images on Toulouse acquired in may 20132) Research on land cover characteristics :Herbaceous area on Jersey in 2013Images with forest in October 2013Images without forest in October 20133) Or any combination :Images with cultivated area and forest on Paris between March and August 2010Cultivated area on Bordeaux in August 20134) Telegraphic styleLANDSAT8 July 2013Arcachon LANDSAT5
To select the geographic extent, you could also zoom on the map to define the region of interest fom the corners of the displayed region.
Finally, to download the product, you need first to create an account, by clicking on the orange icon, and then you need to identify yourself. Every image can be downloaded by clicking on the download button or directly using its URL defined from the product name. I have to write an automatic download script, but you may already use the very convenient DownThemAll Firefox plugin. To use it, you will have first to login, then to ask Downthemall to download all the URLs thant end with "$download". (HowTo provided here)
The publication of these data is the result of years of work, at CESBIO and CNES, although their production takes less than 2 weeks. It is also the first cersion of this processing. Positive comments are welcome, as well as negative, they will be useful to enhance the service, before we start processing Sentinel-2, which should be launched next year.
Finally, we would like to thank our NASA and USGS colleagues who distribute the input data with no restriction, which allows THEIA to deliver fully open data. Please do not forget to tell us about what you did with the data, it is very important to elp us justify our funding requests.
That was fast ! The processing of all the LANDSAT 8 images taken above France in 2013 took less than 15 days. The first LANDSAT 8 images were taken in April 2013. The MUSCATE team processed the data for the THEIA land data center, using CNES computing center.
A few more days will be necessary to upload the data on the THEIA website and to check that the data are correct. Finally, the longest part in the processing is the downloading of the input Level 1T products from USGS earthexplorer website (equivalent to the Level 1C in THEIA's nomenclature).
The Level 2A data quality is quite good, as may be seen on the browse products on the right, as shown by images on the right, which come from the times series obtained on the tile of Paris. As usual on this blog, the clouds are circled in green, the shadows in black, the snow in pink and the water in blue. A few clouds are sometimes missed by our multi-temporal method, when the repetitivity of cloud free acquisition is too low, as in the image on the right which was acquired during a cloudy spring. The following images in the time series are not affected by this kind of defect.
This paper aims at describing the main steps of the processing.
We download the input data from the earthexplorer website, using an enhanced version of the script described here. These products are ortho-rectified by USGS, using a global data base of ground control points.
The location requirement for LANDSAT 8 is 50 m, which seems to be met by the L1T products. We found location errors around 1.5 pixels near Toulouse, but most regions seem to have better performances. USGS confirmed a 38m bias Southward near Toulouse and will try to correct them. Our processing does not correct for these small errors, and the next version of the USGS LANDSAT 8 processing only wil lcorrect for this bias.
Regarding LANDSAT 8 absolute radiometric calibration, we use the coefficient values recommended by LANDSAT 8 and provided with the L1T products.
Level 1T data are provided with the UTM projection. This projections uses three different zones over France, for which the registration of data is not direct. We decided to resample the data on a Lambert'93 projection, which is the official French projection.
We chose to tile the data in 110*110 km tiles spaced with a 100 km interval, as it will be done for Sentinel-2. The (1,1) tile is in the SouthWest corner of France. The tile of Toulouse is the 5th to the West, and the 2nd to the North. It is named D0005H0002 (D for "droite", H for "Haut")
For Corsica, a different tiling made of 2 tiles was defined.
For each tile, we provide the whole set of dates for which a LANDSAT 8 image intersects the tile. A few date may be missing, for several reasons, in general related to the cloud cover :
First of all, we would like to outline that our processor does not process the themal bands of LANDSAT 8.
For the visible, near and short wave infrared bands, we use the same method as for SPOT4(Take5). It involves also the MACCS processor, developed and maintained by Mireille Huc at CESBIO. It is based on multi-temporal methods for cloud screening, cloud shadow detection, water detection as well as for the estimation of the aerosol optical thickness.
However, thanks to LANDSAT 8 spectral bands, our processing was enriched compared to SPOT4 (Take5) : LANDSAT8's 1.38µm band enables an enhanced detection of high and thin clouds. And thanks to the blue band, we have an additional criterion to detect the aerosols, thanks to the quasi constant relationship between the surface reflectances in the blue and in the red above vegetation. The precision gain due to this criterion compensates for the precision loss due the lower repetitivity of LANDSAT8 images.
Level 2A images from Paris's tile, from 3 different LANDSAT 8 tracks (From left to right, tracks 200, 199, 198). The viewing angle differs as the image is from the west on the left image, at nadir in the center and from the east for the right image.
To enhance the cloud screening accuracy, we decided to use the data from adjacent satellite tracks within the same time series. These data are not acquired under exactly the same angle (+/- 7 degrees), which is the assumed by the multi-temporal method, but the difference is small enough to allow a large accuracy gain due to the enhanced repetitivity. However, because of this approximation, a few artefacts may be observed.
For a greater enhancement, we might also use LANDSAT 7 and LANDSAT 8 data in the same time series, but we will implement that later on...
For LANDSAT 8, we used the same format as for SPOT4 (Take5), excepted a few details, that I will describe soon...
As you may know, we have been selected for ESA's project "Sentinel-2 Agriculture". Among the tasks we must fulfill, we have to ask the users about their needs concerning the use of Sentinel-2 time series to monitor agriculture, and of course we need to write a synthesis.
ESA had already distributed a questionnaire at the S2 symposium in 2012, which was used as a basis to define the Sen2Agri project. My revered colleague (and boss) Gérard Dedieu, just cooked a new detailed survey form. If you are a potential user of remote sensed images for agriculture monitoring, you are very welcome to fill this survey.
Although the baseline of SenAgri products was already defined in the call for tender, your answers will be very useful to detail the product requirements, and to forward your needs to ESA and other space agencies, and to define the next versions of our products.
At the beginning of the week, the MUSCATE prototype processing center of THEIA started processing the LANDSAT 8 data available in France. The processing started with the 2013 data, which will be transformed into Level 2A products. As for SPOT4 (Take5), the level 2A products are expressed in surface reflectance after atmospheric correction, and are provided with a cloud mask, a cloud shadows mask, a water and snow mask.but in the case of LANDSAT 8, the products are split into tiles on a 100*100 km² grid, and each tile is 110*110 km² to allow an overlap of 10 km between tiles.
Landsat 8 data should progressively appear on THEIA's catalog in less than a month (but this is a risky assertion, as it is the first time we do this production and surprises may arise, although we spent a lot of time in validation). More details are available here.
As said in a previous post, we are testing various methods of level 3A production, using SPOT4 (Take 5). The Theia Land Data Center will the use these methods to process Sentinel 2 data. In case you did not click on the link above, let's recall that the level 3A products are monthly composite products of cloud free reflectances. For each pixel, our method computes the weighted average of the reflectances of the dates when the pixel is cloud free. For more details, you will need to follow this link.
The work of Mohamed Kadiri at CESBIO, which is funded by the CNES budget for Theia, adressed first the definition of quality indexes for composite products (for more details, may I suggest that you follow this link ?). This work showed that our product has nice performance, but we knew some one would ask us to compare them to the classical methods for level 3A products.
Therefore, we compared our product with the famous NDVI Maximum Value Composite (NDVI MVC), developped by our remote sensing ancestors, and used since the most remote antiquity to process AVHRR time series. This method consists in using for each pixel of the level 3A, the reflectances of the date which has the greatest NDVI. Why ? Mostly because the NDVI of a cloud is very low, often negative, and therefore this method will rather select cloud free pixels. The NDVI MVC comes from a time when the cloud masks were not very accurate.
|Example of a monthly synthesis obtained with the NDVI MVC methods||Example of a monthly synthesis obtained with the weighted average method|
This post uses the SPOT4-Take5 data to show a comparison of the performances obtained on the Versailles site, with the NDVI MVC method on the left, and the weighted average on the right. One can clearly see, on the left, the presence artefacts made of whiter and darker dots which are not seen on the image on the right. These artefacts appear when the selected date changes from one pixel to the other. These artefacts are much less visible on the vegetation covered plots, as, for this composite obtained in spring, the vegetation increases quickly, and all the pixels come from the last cloud free date of the synthesis.
If we have a look at our quality indicators, which were described in our previous post about composite products , it is obvious that the performances obtained by the weighted average method are much better than those of the NDVI MVC method, either as regards the similarity to the central date image of the Level 3A (in yellow, for the 70 % best pixels and in green for the 95% best pixels), and moreover as regards as the amplitude of artefacts (in blue). The abscissa of the plot is the half of the number of days used in the synthesis, and our recommended value is 21.
|NDVI Maximum Value Composite||Weighted Average Composite|
At CESBIO, we are developing land cover map production techniques, for high resolution image time series, similar to those which will soon be provided by Venµs and Sentinel-2. As soon as the SPOT4 (Take5) data were available over our study area (Sudmipy site in South West France), we decided to assess our processing chains on those data sets. The first results were quickly presented during Take5 user's meeting which was held last October.
In this post we describe the work carried out in order to produce these first land cover classifications with the SPOT4 (Take5) Sudmipy images (East and West areas) and we compare the results obtained over the common region to these two areas.
Prior to the work presented here, we organized a field data collection campaign which was synchronous to the satellite acquisitions. These data are needed to train the classifier training and validate the classification. The field work was conducted in 3 study areas (figure 1) which were visited 6 times between February and September 2013, and corresponded to a total of 2000 agricultural plots. This allowed to monitor the cultural cycle of Winter crops, Summer crops and their irrigation attribute, grasslands, forests and bulit-up areas. The final nomenclature consists in 16 land cover classes.
The goal was to assess the results of a classification using limited field data in terms of quantity but also in terms of spatial spread. We wanted also to check whether the East and West SPOT4 (Take5) tracks could be merged. To this end, we used the field data collected on the common area of the two tracks (in pink on the figure) and 5 level 2A images for each track acquired with a one day shift.
The first results of supervised SVM classification (using the ORFEO Toolbox) can be considered as very ipromising, since they allow to obtain more than 90% of correctly classified pixels for both the East and the West tracks and since the continuity between the two swaths is excellent. Some confusions can be observed between bare soils or mineral surfaces and Summer crops, but these errors should be reduced by using LANDSAT 8 images acquired during the Summer, when Summer crops will develop.
This zoom compares the results obtained on the common area of the two tracks (West to the left and East to the right). The two classifications were obtained independently, using the same method and the same training data, but with images acquired at different dates and with different viewing angles. The main errors are maize plots labeled as bare soil, which is not surprising, since this crop was just emerging when the last image was acquired. There are also confusions between wheat and barley, but even on the field, one has to be a specialist to tell them apart.
After performing these experiments, we were very satisfied with the operationnality of our tools. Given the data volume to be processed (about 10 GB of images) we could have expected very long computation times or a limitation in terms of memory limits of the software used (after all, we are just scientists in a lab!). You will not be surprised to know that our processing chains are based on Orfeo Toolbox. More precisely, the core of the chain uses the applications provided with OTB for supervised training and image classification. One just have to build a multi-channel image were each channel is a classification feature (reflectances, NDVI, etc.) and provide a vector data (a shapefile, for instance) containing the training (and validation) data. Then, a command line for the training (see the end of this post) and another one for the classification (idem) are enough.
Computation times are very interesting: several minutes for the training and several tens of minutes for the classification. One big advantage of OTB applications is that they automatically use all the available processors automatically (our server has 24 cores, but any off the shelf PC has between 4 and 12 cores nowadays!).
We are going to continue using these data, since we have other field data which are better spread over the area. This should allow us to obtain even better results. We will also use the Summer LANDSAT 8 images in order to avoid the above-mentioned errors on Summer crops.
We start by building a multi-channel image with the SPOT4 (Take5) data, not accounting for the cloud masks in this example :
otbcli_ConcatenateImages -il SPOT4_HRVIR_XS_20130217_N1_TUILE_CSudmipyE.TIF SPOT4_HRVIR_XS_20130222_N1_TUILE_CSudmipyE.TIF SPOT4_HRVIR_XS_20130304_N1_TUILE_CSudmipyE.TIF SPOT4_HRVIR_XS_20130413_N1_TUILE_CSudmipyE.TIF SPOT4_HRVIR_XS_20130607_N1_TUILE_CSudmipyE.TIF -out otbConcatImg_Spot4_Take5_5dat2013.tif
We compute the statistics of the images in order to normalize the features :
otbcli_ComputeImagesStatistics -il otbConcatImg_Spot4_Take5_5dat2013.tif -out EstimateImageStatistics_Take5_5dat2013.xml
We train a SVM with an RBF (Gaussian) kernel :
otbcli_TrainSVMImagesClassifier -io.il otbConcatImg_Spot4_Take5_5dat2013.tif -io.vd DT2013_Take5_CNES_1002_Erod_Perm_Dissolve16cl.shp -sample.vfn "Class" -io.imstat EstimateImageStatistics_Take5_5dat2013.xml -svm.opt 1 -svm.k rbf -io.out svmModel_Take5Est_5dat2013_train6.svm
And Voilà !, we perform the classification:
otbcli_ImageSVMClassifier -in otbConcatImg_Spot4_Take5_5dat2013.tif -mask EmpriseTake5_CnesAll.tif -imstat EstimateImageStatistics_Take5_5dat2013.xml -svm svmModel_Take5Est_5dat2013_train_6.svm -out ClasSVMTake5_5dat_16cl_6.tif
This post is an old one (last year), but I had not translated it.
The satellites observing the Earth at a high resolution may be divided in two categories according to their programming mode :
Users ask the provider to program an image above their site. The provider collects all demands and optimises the acquisition plan so that a maximum of user requests are satisfied. The provider often charges an extra cost if the user needs an image at a precise date, and in zones where satellite image demand is high, a user is never sure to get the image he requested, unless he pays for a higher priority.
SPOT, Pleiades, Ikonos, Quickbird, Formosat-2, Rapid Eye and most radar systems are of "SODA" type.
The image provider defines the zone to observe at the beginning or the satellite mission, and these zones are observed at each overpass of the satellite. In some cases (LANDSAT, Sentinel-2), the acquisition zones covers all lands, while on other cases (Venµs, SPOT4-(Take5)), the acquisition may only cover a few preselected sites.
Usually, SODA provide a better spatial resolution, while usually, the SAS provide a better temporal resolution. The SODA images must generally be purchased, since the resource is limited, while the SAS images are usually free of charge. There were periods when LANDSAT images were sold, but they encountered little commercial success, while their success is huge now that they are free of charge. Finally, the SODA are best suited to applications for which the acquisition date is not very important and for which a high resolution is essential, for instance urban studies or monitoring of ecological corridors, while the SAS are better suited to surfaces which quickly evolve, such as natural surfaces or farm lands, and they are best suited to automatically produce detailed land cover maps.
Oppositely to the US who, thanks to LANDSAT, have been working with SAS images, in Europe, users are much more trained to use SODA images such as the ones provided by SPOT. This situation should change radically, first with LANDSAT 8, which is much easier to access in Europe than LANDSAT 5, but above all with Sentinel-2, but the adaptation to this kind of data will require a lot of work and some time. New processing methods and new applications must be developed, which was one of the aim of SPOT4 (Take5) data set.