(updated on April the 24th, 2014, and on July the 24th, and on August the 19th, with a good simplification of interface) Le bout de code Python (2.7) dont le lien est fourni ci-dessous, permet de télécharger automatiquement des données LANDSAT, sur la version actuelle du site EarthExplorer. Il fonctionne pour LANDSAT 8 et pour LANDSAT5 et 7, et nécessite que les données soient en ligne. C’est le cas semble -t’il pour toutes les données LANDSAT 8, mais pour les données LANDSAT5 ou LANDSAT7, il peut être au préalable nécessaire de demander la production sur le site http://earthexplorer.usgs.gov. Pour que le programme fonctionne, il faut bien sûr un compte sur Earth explorer, et il vous faudra configurer vos comptes, mots de passe et répertoires dans le corps du programme. Ce programme peut être lancé de deux manières différentes :

  • En choisissant une scène LANDSAT définie par ses coordonnées WRS-2, par exemple, (198,030) pour Toulouse (option -s). Il faut aussi fournir une date de début de période (option -d), et eventuellement une date de fin (option -f), sinon, c’est la date du jour d’utilisation qui est prise par défaut comme date de fin.
    • exemple :
      • download_landsat_scene.py -o scene -d 20130128 -s 181025 -u usgs.txt -p proxy.txt --output /mnt/data/LANDSAT8/N0/
  • En fournissant en entrée un fichier avec la liste des produits à télécharger (un produit par ligne)
    • exemple : 
      python download_landsat_scene.py -o liste -l list2_landsat8.txt -u usgs.txt --output /mnt/data/LANDSAT8/N0/
    • Avec un fichier list2_landsat8.txt comme ci-dessous, il faut aller chercher les références des produits sur earthExplorer.
      • Tunisie LC81910352013160LGN00Tunisie LC81910362013160LGN00

Le fichier usgs.txt doit contenir votre compte et mot de passe sur la même ligne séparés par un espace.Si vous n’utilisez pas l’option –output, les fichiers seront téléchargés sur le répertoire /tmp/LANDSAT, à condition qu’il existe.Pour voir toutes les options :

download_landsat_scene.py -h

Code

J’ai déplacé le code sur GitHub, qui est beaucoup plus pratique et fait pour ça.https://github.com/olivierhagolle/LANDSAT-Download   The PYTHON 2.7  routine provided below enables to automatically download LANDSAT data, using the actual (April 2014) version of EartExplorer. It works for LANDSAT 8 and LANDSAT 5&7, but needs that the data be already online. It seems to be systematically the case for LANDSAT 8, but for the older LANDSAT, it may be necessary to first order for the production of L1T products., on the earthexplorer site. And of course, you will need to have an account and password on the Earthexplorer website, and change the username and password in the connect earthexplorer routines. This routine may be used in two ways :

  • by providing the WRS-2 coordinates of the LANDSAT scene, for instance, (198,030) for Toulouse (-s option). You will have to provide also a start date (-d option) and a final date (-f option). If the final date is not provided, the default final date is the current date. Example:
download_landsat_scene.py -o scene -d 20130128 -s 181025 -u usgs.txt -p proxy.txt --output /mnt/data/LANDSAT8/N0/
  • by providing a list of products to download, as in the example below:
python download_landsat_scene.py -o liste -l list2_landsat8.txt -u usgs.txt --output /mnt/data/LANDSAT8/N0/

with a file list2_landsat8.txt as provide below (the landsat references must exist in the earthexplorer catalog) :

Tunisie LC81910352013160LGN00Tunisie LC81910362013160LGN00

The usgs.txt must contain your username and password on the same line separated by a blank.If you do not use the –output option, the files will be downloaded to /tmp/Landsat (provided it exists)To see all the options :

download_landsat_scene.py -h

Code

I have moved the code to GitHub, which is much more convenient :https://github.com/olivierhagolle/LANDSAT-Download

30 thoughts on “Download_landsat_scene.py”

  1. I just corrected the value of date_asc, which was incorrect when doy was below 100. Landsat 8 first image was after doy 100 in 2013.date_asc= »%04d%03d »%(options.annee,doy)

    Répondre

  2. Olivier,Thanks for the script. It’s a little buggy (there is no -t option, for instance), but after some logical repairs, it seems to try out the downloads. However, I get HTTP Error: 500, for instance, when using:-o scene -a 2014 -p LC8 -d 60 -f 100 -s 198024LC81980242014060LGN01http://earthexplorer.usgs.gov/download/4923/LC81980242014060LGN01/STANDARD/EEHTTP Error: 500 http://earthexplorer.usgs.gov/download/4923/LC81980242014060LGN01/STANDARD/EETo understand this correctly, though, am I expected to first make this selection in EarthExplorer so that the selected imagery are queued as an order? In other words, is your script a replacement for the « Bulk Download App » provided by USGS?If your script can skip the pre-selection stage, it would be very useful indeed.With greetings,Guido Lemoine

    Répondre

  3. Sorry, I resolved the problem. -d 68 -f 100 works, and I had to change the version to 00.Wonderful. This is very useful!!

    Répondre

    1. Thanks for the comments Guido, you are right, I need to update the post, my own version has also evolved.\nThe idea for the code is to avoid the selection of data on Earth Explorer, but it only downloads images which have been processed to level 1C. On some LANDSAT tiles, you will need to first order the tiles which have not yet been processed.

      Répondre

  4. Hi to all,this algorithm could be very usefull but in my PC it doesn’t work.The error is in line 71 in g;10i’m not expert of python can you send me a reference where i can study it.tyRoberto

    Répondre

    1. Thanks Roberto,\nthere is indeed a problem with the tool that provides the syntax higlighting.\nIt seems to be replacing the «  » char by « > ».\nSo you could try to change these character sequences back to their true values in the code you copied and pasted.\n\nMeanwhile, I will try to find another syntax highlighting tool.\nHope the tool will work for you now.\nOlivier

      Répondre

      1. thank you oliver, I’m reading your reply only now, but i have studied python and i costumized the program for my python version and it work.Thank you, how can i cite you when i use this sript?

        Répondre

        1. Thanks Roberto. That’s great if it is useful,,but really do not need to cite it, it is just a script. \nBy the way, the script has been improved (with contributions), and moved to a github repository.\nhttp://olivierhagolle.github.io/LANDSAT-Download/

          Répondre

  5. hello,I’m trying to use landsat python code in order to download several images on new caledonia with a text file. ./landsat_download.py -o liste -l lst_landsat.txt Obviously i have changed login and pasword by i’ve a problem with connecting to the earthexplorer website. f = opener.open(‘https://earthexplorer.usgs.gov/login’, params)doesn’t work and last message error is:urllib2.URLError: . Could you please tell me if i have to replace user by Username on the website earthexplorer and passwd by password on earthexplorer user=’CNESNET\login’ passwd=’secret’ proxy_info = { ‘user’ : ‘Myself’, ‘pass’ : ‘secret’, ‘host’ : ‘proxy.esa.int’, ‘port’ : 8050Moreover what is ‘host’, do i have to change it. Thank you for your help.

    Répondre

    1. Bonjour Florian,\nIf your internet connection does not go through a proxy, you should uncomment \n#connect_earthexplorer_no_proxy()\nand comment the line :\nconnect_earthexplorer_proxy()\n\nA true programmer would have added an option in the command line, but I am not a true one… sorry.\nOlivier

      Répondre

      1. Hey, First of all, thank you for laying the groundwork for this. I have not been able to find a similar script so far. I haven’t been able to get it to work, but my USGS account seems to be on the fritz (like this script the official bulk download app, return authentication failed, even though I am 100% there is no typo in the login info). Anyway, you might want to fix the function def of the no proxy statement, it is missing an underscore. right now: def connect_earthexplorer_noproxy():should be: def connect_earthexplorer_no_proxy():Also, in the if loop, where you define what happens if option == ‘liste’, I believe one of the def connect_earthexplorer_proxy should be a …no_proxy. Again, thank you getting this started. Have you considered sharing this code on GitHub? People could more easily add to the progress of the script and add some functionality/fix bugs.

        Répondre

        1. Thanks for the suggestions, i’ll correct the typos. The wordpress plugin I use to provide the code is not easy to use and I have to make modifications in the code which do not allow then to test it. And I have moved the code to GitHub, which is really more convenient. Thanks for the suggestion. Olivier

          Répondre

  7. Hi,I’m trying to used landsat python. But I met errors.Could you tell me how can do it?Thanks C:\Users\Hoa\Desktop\Download_landsat_scence.py -o scene -a 2014 -d 228 -f 365 -s 123052/to/be/changed//SCENES/123_052/GZhttp://earthexplorer.usgs.gov/download/4923/LC81230522014228LGN00/STANDARD/EETraceback (most recent call last): File « C:\Users\Hoa\Desktop\Download_landsat_scence.py », line 168, in connect_earthexplorer_proxy() File « C:\Users\Hoa\Desktop\Download_landsat_scence.py », line 38, in connect_earthexplorer_proxy f = opener.open(‘https://earthexplorer.usgs.gov/login’, params) File « C:\Python27\ArcGIS10.1\lib\urllib2.py », line 394, in open response = self._open(req, data) File « C:\Python27\ArcGIS10.1\lib\urllib2.py », line 412, in _open ‘_open’, req) File « C:\Python27\ArcGIS10.1\lib\urllib2.py », line 372, in _call_chain result = func(*args) File « C:\Python27\ArcGIS10.1\lib\urllib2.py », line 1207, in https_open return self.do_open(httplib.HTTPSConnection, req) File « C:\Python27\ArcGIS10.1\lib\urllib2.py », line 1174, in do_open raise URLError(err)urllib2.URLError:

    Répondre

    1. Hi, the tool is not a professional one (you noticed). There are some lines that you must change within the code. I would not advise someone who does not know python.. For instance, the Landsat files are written to a directory /to/be/changed. Of course, this directory must be changed to be adapted to your platform. \n\nHowever I am trying to provide a more « profesional » code here : https://github.com/olivierhagolle/LANDSAT-Download\nOlivier

      Répondre

  8. Thank you for distributing the code. It is a great help! Note to potential users: python experience is necessary for debugging. This tool is appears to be very useful but will take some work to implement. I am running from a .bat file on windows. As indicated above the ‘-t’ in the example does not work. The example line (line 134) in the code itself worked after I included the full paths for python.exe, the program and usgs.txt.

    Répondre

  9. Hi Olivier, thanks for the download_landsat_scene.py script.I’ve added the capability to resume partially downloaded files, and made some other small improvements to the script.I would be happy to contribute all that. Just send me an email so I can send you the file.Regards,Mauro

    Répondre

    1. Great !\nThanks a lot for this useful improvement. I need to find some time to test it, but it is quite hard these days.\nOlivier

      Répondre

      1. Hi, I did some manual testing, just comparing (using diff) a file downloaded at once, with the same file downloaded with multiple interruptions and resumes.Also tested for some basic things like resuming a download vs. removing the file and downloading it from the beginning, resuming downloading a zero sized file, etc.

        Répondre

      2. Hi Olivier,I noticed that download resume only works with recent(Landsat 8) images.Added a range header check to the download resuming code. Now it falls back to re-downloading the whole file in case the server does not support download resuming(that is, for LT5/LE7 images).Regards.

        Répondre

  11. Hi,Thank you very much for automatic download algorithms, however I had a small question about the txt file for the user name. He did not recognize my login, it gives me the following error message: Authentification is probably wrong, I put the form in the txt file nom.adresse@mail.com mot de passe.Can you help me please.Thank

    Répondre

    1. Hi,\nan example of password file is provided here :\nhttps://github.com/olivierhagolle/LANDSAT-Download/blob/master/usgs.txt\n\nRather account name than email address.\nHope it works.\nOlivier

      Répondre

  12. Hi Olivier,Great work! It was working well yesterday and then I tried the script on my other desktop but run into issues and not quite sure how to resolve it.The query I ran was:C:\Python27_64bit\Lib\site-packages\LANDSAT-Download-master>download_landsat_scene.py -o scene -d 20161001 -s 170075 -u usgs.txt –output E:/Then the error I got was:None NoneSearching for images on (julian date): 2016278…http://earthexplorer.usgs.gov/download/4923/LC81700752016278LGN00/STANDARD/EEerror : file is in html and not an expected binary fileresult saved in ./error_output.htmlHowever when I paste « http://earthexplorer.usgs.gov/download/4923/LC81700752016278LGN00/STANDARD/EE » into my browser as a quick check, it finds the file and downloads.Please assist if you can, we are really keen to use this awesome script.ThanksJens

    Répondre

    1. I have updated the script a few days ago, having a similar problem as your’s.\nDo you use the very last version ?\n\nEarthexplorer server is down today. I’ll try that tomorrow. Please remind me if I forget.\nOlivier

      Répondre

  13. En que parte del código ingreso este comandopython LANDSAT8.py -o liste -l liste_landsat8_site.txt -u usgs.txt –output /mnt/data/LANDSAT8/N0/Necesito programar para bajar estas 2 imágenesLC80110622013147LGN00LC80020732014135LGN00Por favor alguien que me pueda apoyar..Gracias

    Répondre

  15. HiI am getting below error. Can anyone help me url= https://earthexplorer.usgs.gov/download/12864/LC80440342013154LGN01/STANDARD/EETraceback (most recent call last): File « Landsat_download.py », line 623, in main() File « Landsat_download.py », line 615, in main connect_earthexplorer_no_proxy(usgs) File « Landsat_download.py », line 67, in connect_earthexplorer_no_proxy data = urllib2.urlopen(« https://ers.cr.usgs.gov »).read() File « /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py », line 154, in urlopen return opener.open(url, data, timeout) File « /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py », line 431, in open response = self._open(req, data) File « /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py », line 449, in _open ‘_open’, req) File « /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py », line 409, in _call_chain result = func(*args) File « /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py », line 1240, in https_open context=self._context) File « /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py », line 1197, in do_open raise URLError(err)urllib2.URLError: Regardssbal

    Répondre

  16. Hi

    I am getting this errow after run the scritp
    >python download_landsat_scene.py -o scene -p LC8 -d 60 -f 100 -s 198024LC81980242014060LGN01 -u usgs.txt –output C:/Datasets/test
    None None
    error with usgs password file

    my usgs.txt just have a line with account password words
    I had try every suggestion here, but nothing works.

    Thanks in advance, JL

    Répondre

Répondre à Guido Lemoine Annuler la réponse

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *

Ce site utilise Akismet pour réduire les indésirables. En savoir plus sur comment les données de vos commentaires sont utilisées.