I recently stumbled upon a fascinating dataset which contains digitised information from the log books of ships (mostly from Britain, France, Spain and The Netherlands) sailing between 1750 and 1850. The creation of this dataset was completed as part of the Climatological Database for the World’s Oceans 1750-1850 (CLIWOC) project. The routes are plotted from the lat/long positions derived from the ships’ logs. I have played around with the original data a little to clean it up (I removed routes where there was a gap of over 1000km between known points, and only mapped to the year 1800). As you can see the British (above) and Spanish and Dutch (below) had very different trading priorities over this period. What fascinates me most about these maps is the thousands (if not millions) of man hours required to create them. Today we churn out digital spatial information all the time without thinking, but for each set of coordinates contained in these maps a ship and her crew had to sail there and someone had to work out a location without GPS or reliable charts.

These maps were produced with the latest version of R‘s ggplot2, maptools, geosphere and png packages. Formatting the data took the most work (it was a very large MS Access database). I used ggplot’s annotation_raster() to add the compass rose and title.

Update: For some nice animations and a much better critical analysis of the data see Ben Schmidts blog.

The above map (and this one) was produced using R and ggplot2 and serve to demonstrate just how sophisticated R visualisations can be. We are used to seeing similar maps produced with conventional GIS platforms or software such as Processing but I hadn’t yet seen one from the R community (feel free to suggest some in the comments). The map contains three layers: buildings, water and the journey segments. The most challenging aspect was to change the standard line ends in geom_segment from “butt” to “round” in order that the lines appeared continuous and not with “cracks” in, see below.

I am grateful to Hadley and the rest of the ggplot2 Google Group for the solution. You can see it here. From this point I layered the plots using the geom_polygon() command for the buildings and water bodies and my new function geom_segment2() for the journey segments- these were simply the start and end latitudes and longitudes for each node in the road network and the number of times a cyclist passed between them. I have included the code below
 


#Code supplied by james cheshire Feb 2012
#load packages and enter development mode
library('devtools')
dev_mode()
library(ggplot2)
library(proto)

plon2


I have been using R (a free statistics and graphics software package) now for the past four years or so and I have seen it become an increasingly powerful method of both analysing and visualising spatial data. Crucially, more and more people are writing accessible tutorials (see here) for beginners and intermediate users and the development of packages such as ggplot2 have made it simpler than ever to produce fantastic graphics. You don’t get the interactivity you would with conventional GIS software such as ArcGIS when you produce the visualisation but you are much more flexible in terms of the combinations of plot types and the ease with which they can be combined. It is, for example, time consuming to produce multivariate symbols (such as those varying in size and colour) in ArcGIS but with R it is as simple* as one line of code. I have, for example, been able to add subtle transitions in the lines of the migration map above.  Unless you have massive files, plotting happens quickly and can be easily saved to vector formats for tweaking in a graphics package.

R’s utilisation has been tempered by its relatively sparse documentation and challenging usability. The R community is increasingly aware of this with packages such as DeducerSpatial providing a graphical user interface to some of R’s spatial data functionality. More and more tutorials are appearing and people have been inspired by some high profile maps made with R (see here) so I am confident that it will be increasingly seen as the engine for slightly glossier analysis and visualisation packages.

R can’t do everything- I find handling map projections a bit tricky and its not possible to pan and zoom the maps as they are being created. In some circumstances I can’t do without these functions so I opt for a traditional GIS. Also, for the programmers out there used to the likes of Python and Java, R can have quite a few quirks in its syntax so be patient. Despite it’s flaws, if you have a large data processing and visualisation task R is a great option. It offers a high degree of flexibility in terms of input data formats and with packages such as twitteR, RCurl, and XML it is easier than ever to import online data sources from social media sites and data feeds.  Aside from traditional export formats for the visualisations it has become incredibly simple to export interactive and animated graphics using the googleVIS package or igraph for network visualisations. Such flexibility is invaluable if you are seeking to create a variety of different graphics from a single datasource without having to format it for multiple software packages. The great thing with R is the sense that it still has masses of unrealised potential for  future spatial data visualizations. If you know of any good visualisations or tutorials please leave a comment!

I should also say that if you would like to learn how to do these sorts of visualisations (and more!) come and do our masters course!

*simple might be a slightly optimistic way of thinking about it if you haven’t used R before, but with a bit of practice you will ge there! 

Migrations of people have existed for millennia and occur at a range of scales and time-periods (from small-scale journeys to work through to intercontinental resettlement). As a geographer I have long been interested in these and thought it was about time I mapped them! Using data from the Global Migrant Origin Database (thanks Adam for the tip) and R, my favourite stats software, I have produced the maps you see here (click on them for higher resolution). Each line shows the origins and destinations of at least 4000 people in a given year (2000 in this case). The more red the line the more people it represents. I have used great circle distance to plot them onto the Earth.  The map below shows the same magnitude of flows but just for Europe. The Earth has been flattened for this one so the flows are represented by arbitrary arcs.

These visualisations aren’t perfect. Firstly they are based on a dataset where many of the movements are best guesses rather than measured data. You can read more about this here. It would also be great to have actual flows rather than inferred flows based on the number of migrants in each country. If I made these maps again I might draw lines between capital cities or population centres to avoid the impression that the majority of migrations to/ from Russia start/end in Siberia for example. There are of course endless ways of partitioning the data/ selecting the colours. Despite this I am really pleased with effect and the maps go some way to showing the dynamism in many 21st Century populations.

Technical Details

I think Paul Butler’s Facebook Map threw down the gauntlet to the R community in terms of the quality of visualisations that can be produced with the software so I was keen to see what I could do. To produce the maps I calculated the great circle distances using the geosphere package, I calculated my own arcs for the second map and used the maps package for my World outline. The visualisations (including projections) were done using ggplot2. Over the next few months I plan to stick together a more complete tutorial (PhD write-up permitting!).**UPDATE** the flowingdata blog has beaten me to it see here.

Buried in the London Datastore are the population estimates for each of the London Boroughs between 2001 – 2030. They predict a declining population for most boroughs with the exception of a few to the east. I was surprised by this general decline and also the numbers involved- I expected larger changes from one year to the next. I think this is because my perception of migration is of the volume of people moving rather than the net effects on the baseline population of these movements. I don’t envy the GLA for making predictions so far into the future, but can understand why they have to do it (think how long it took initiate Crossrail!). Last year I produced a simple animation showing past changes in London’s population density (data) and it provides a nice comparison to the above. In total I have squeezed 40 maps on this page!

Technical Stuff

These maps were all produced to demonstrate the mapping capabilities of R. The first uses ggplot2 (plus classInt + RColorBrewer) and is based on some code (see below) written by Mark Bulling. If you follow the code below you will end up with this map, not the one I have produced above. I will stick my code in a formal tutorial soon. The animation uses the standard plot functions (plus spatial packages) in R as per this example.

Data: all_date, Chart ID: MotionChart_2011-01-10-10-16-25

R version 2.12.1 (2010-12-16),

Google Terms of Use

Data: all_date, Chart ID: MotionChart_2011-01-10-10-10-46

R version 2.12.1 (2010-12-16),

Google Terms of Use

It was a bit fiddly to get the data formatted correctly and I couldn’t manage to get the complete dataset in one plot because my browser kept crashing (Chrome is best). Even with these teething problems it is a great way to get people creating better visualizations with their data. If you want to see Hans Rosling demonstrating these plots with his trademark enthusiasm I thoroughly recommend “The Joy of Stats” a program produced for the BBC. You can watch it here.

For those who want to create their own plots, I’m not proud of the code I used to format the data above so to get you started try this example (provided with the package).

library(googleVis)

data(Fruits)

M1 <- gvisMotionChart(Fruits, idvar=”Fruit”, timevar=”Year”)

plot(M1)

Thanks to the Recology blog for promoting this.

The visualisation above shows the average relative duration of Boris Bikers’ weekday journeys over a 4 month period at hourly intervals. For each time step the average journey time (in seconds) from each docking station has been calculated.This information is interesting because it shows the preference for short journeys around the City of London, whilst people on the outskirts of the the scheme (especially to the west) take longer journeys. I also like the the fact that journey times around Soho and the West End are longest around 23:00- perhaps correlating with the number of after-work drinks consumed. In one visualisation you get to see the changes in the cyclists behaviour- from the early morning commuters through to the late night cruisers

The data come from Transport for London’s recent release of 1.4 million Barclays Cycle Hire journeys to their developers area (thanks to this FOI request). The data are said include all the journeys between 30 July 2010 and 3 November 2010, except those starting between midnight and 6am. In this analysis journeys taking more than one hour are not included (there are relatively few and many were actually the bikes being removed for maintenance) and docking stations with fewer than 10 journeys within each hour across the time period have also been ignored.

The maps can be improved in many ways- stay tuned for more developments and I will also post something a bit more technical about the methods I used etc to create the map (I used a strange cocktail of R and ArcGIS 10) .

I also recommend Ollie O’Brien’s (@oobr) brilliant interactive visualisations these data.

Google Earth has become a popular way of disseminating spatial data. KML is the data format required to do this. It is possible to load almost any type of spatial data format into R and export it as a KML file. In my experience R seems much quicker at doing this than many well-known GIS platforms, such as ArcGIS. The worksheet below explains how.

Data and Package Requirements:

London Cycle Hire Locations. Download.

Install the following packages (if you haven’t already done so):

maptools, rgdal (Mac users may wish to see here first).

Click here to view the tutorial code.

“Your 2 November blog about rgdal on OSX is very misleading. The CRAN rgdal page: http://cran.r-project.org/web/packages/rgdal/index.html says all you need to know unless you need extra drivers, or already have PROJ.4 and GDAL installed. Just do:

setRepositories(ind=1:2)
install.packages(“rgdal”)

installs rgdal with all its external dependencies satisfied. It is kindly provided by Prof. Brian Ripley, and is presently running up-to-date GDAL.”******

This may offer a straightforward solution. I would be interested to hear how people get on.

After running a spatial data analysis with R session today, it became apparent that there are one or two teething problems installing the important rgdal package on Mac OS X operating systems. The usual install.packages(“rgdal”) won’t work. My colleague Jon Reades did some digging around to find this solution. I have tested it and it seems to work fine.

[Note that you'll need to be comfortable with the Terminal. If you're not, then find someone who is.]

1. Download the GDAL OS X install from kyngchaos
- http://www.kyngchaos.com/files/software/unixport/GDAL_Complete-1.7.dmg
(Looks like the basic page [for updates after 1.7 if you're reading this ages from now] is http://www.kyngchaos.com/software/frameworks)
- Install as per usual OS X install system
- Fire up the Terminal, then pico (or vi[m]) the .bash_login file
- Modify the PATH environment so that it reads:
export PATH=”/Library/Frameworks/GDAL.framework/Programs:$PATH”
[This is what enables the subsequent steps to find gdal-config]

2. Download and install proj4 from source
- http://trac.osgeo.org/proj/wiki/WikiStart#Download
- Download source code version proj-4.7.0.tar.gz
- Fire up the Terminal
> cd ~/Downloads/
> tar -xzvf proj-4.7.0.tar.gz
> cd proj-4.7.0
> ./configure
> make && make test
> sudo make install
[ should install to /usr/local/lib by default]

3. Download and install rgdal from source
- http://cran.r-project.org/src/contrib/rgdal_0.6-28.tar.gz
- Fire up the Terminal
> cd ~/Downloads/
> sudo R CMD INSTALL –configure-args=’–with-proj-include=/usr/local/lib’ rgdal_0.6-28.tar.gz

After all of this mucking about I was able to say:

> require(sp)
> require(rgdal)

And get a message indicating that GDAL was loaded successfully.

He also posted his solution on the Computational Mathematics Blog. If there is a better way I would be interested in hearing about it for future classes.

The ggplot2 package offers powerful tools to plot data in R. The plots are designed to comply with the “grammar of graphics” philosophy and can be produced to a publishable level relatively easily. For users wishing to create a good map without too much thought I would recommend this worksheet. For those without their own shapefiles who rely on the “maps” package they may wish to consult Hadley Wickham‘s ggplot2 book.

Data Requirements:

London Sport Participation Shapefile. Download (requires unzipping)

poly_coords function. Download

Install the following packages (if you haven’t already done so):

maptools, RColorBrewer, ggplot2

Click here to view the tutorial code.