Kittens time series

=>

Optical remote sensing is great to map the snow cover extent in mountain regions as long as there is no cloud above the land surface. Radar remote sensing of the snow cover is not operational yet mainly because the backscatter from the snow surface is strongly dependent on the snowpack liquid water content. On the ground, however, thousands of people are observing the snow cover in the mountains, everyday. Some of them take photographs and kindly upload them to photo-sharing websites with a public license. Many of these photos are geotagged, either because the cameras have built-in GPS, or because the users added geographical coordinates when publishing their album.

 

A few years ago I decided to investigate a bit further the potential of the Flickr photo-sharing website to monitor the snow cover. Using the Flickr application programming interface, I queried all the public images metadata tagged at least with one of the following words: "snow", "neige", "nieve", "neu (snow in French, Spanish and Catalan languages). The search was limited to the geotagged photos located in the Pyrenees area. Using the date of shooting metadata it is possible to plot this time series of the aggregated number of "snow" pictures per month:

The number of public pictures available for a given time interval depends on several factors, which are not related to the snow cover, including the Flickr website popularity and the development of digital photography. To remove this "noise", I queried all the images tagged with "chat", "gat" or "gato" (cat in French, Spanish and Catalan languages). The "cat" tag was not considered in order to exclude the results from North America where Flickr got popular earlier than in Europe.

The number of "cat" images per month was used to fit a gauss model of the number of images uploaded in Flickr with time. This model was used to remove the trend in the numbers of snow-tagged photographs. The resulting time series is named a "snow picture index".

The comparison of the snow picture index with a time series of the snow cover area derived from MODIS shows that the kittens method works quite well. Spearman's correlation coefficient increases from 0.5 to 0.8. Below both time series were rescaled to 0-1 for visual comparison:

This study was restricted to 2003-2011. Since 2011 the advent of smartphones with built-in GPS and camera has strongly increased the amount of geotagged pictures. How to mine this big amount of data to extract further information on the snow cover? Here I relied on the tags that were added by the Flickr users. However, most photographs on the web are not tagged. To leverage the full potential of photo-sharing websites it would be necessary to find an automatic method to detect snow in digital photographs.

 

Just to get an idea about the difficulty of this task I had a look at the actual pictures. Again, using the Flickr API, I was able to download a sample (one tenth) of the photos that were returned by the "snow" query. Using Imagemagick's montage tool I made several mosaics of these photos. I could see there is not so much garbage in the photos. The "snow" tag seems to be a safe indicator that there is actually snow in the pictures. It's probably more reliable than any automatic classification method that could run on my computer...

Montage of a random sample of flickr images returned by the "snow" query

 

Another challenge is to project the snow pixels from the image to a real-world coordinates. Researchers at University of Bern, Switzerland are developing a semi-automatic procedure to generate snow cover maps based on images from many webcams. Maybe their tool could be used to produce useful "ground-truth" snow maps.

[edit 12-May-2016] Fanny Brun sent me this very recent paper by Fedorov et al. (2016) "Estimating snow cover from publicly available images". The authors present a very nice assessment of computer vision and image processing algorithms to extract georeferenced snow masks from flickr or webcam images. There's always more than one way to skin a cat...

 

Thank you

 

NB. This idea can be cited as Gascoin, S. (2016). Using kittens to unlock photo-sharing website datasets. Journal of Brief Ideas, doi:10.5281/zenodo.44809.

Posted under: Corrections (geometry, atmosphere, clouds), In English, snow-ice

Tagged as: , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>