Most government statistics are mapped according to official geographical units. Whilst such units are essential for data analysis and making decisions about, for example, government spending, they are hard for many people to relate to and they don’t particularly stand out on a map. This is why I tried a new method back in July 2012 to show life expectancy statistics in a fresh light by mapping them on to London Tube stations. The resulting ”Lives on the Line” map has been really popular with many people surprised at the extent of the variations in the data across London and also grateful for the way that it makes seemingly abstract statistics more easily accessible. To find out how I did it (and read some of the feedback) you can see here.

About the same time Mark Green, a PhD student from the University of Sheffield, had a similar idea to map levels of deprivation in London using an adaptation of Beck’s design. This map may be more familiar to people  than the geographically accurate version we used for Lives on the Line, and by using the size of the station circles you can see the extent to which levels of deprivation in London vary as you move along each line. Mark hasn’t managed to squeeze on any station labels so you may need to compare the map to the real thing to get a better idea of what’s going on. Mark’s map nicely illustrates some of the differences between inner and outer London (bigger circles are more deprived areas)  and also, to some extent, differences between east and west. To read more about the map see here (££).

Here is a small section of Mark’s map (I can’t show a better version as TfL will be unhappy):

In the past few months there have been a number of other visualisations of the variations in demographics along transport networks- mainly the London Tube. Danny Dorling’s latest book that walks readers along the 32 stops of the Central Line in London, exploring the lifestyles of those along it as portrayed by population data.

Although they aren’t quite maps, Rentonomy have produced graphs that show the average rents along each of the London Underground Lines. Whilst the information on these graphs is a fairly simplistic representation of some fairly complex patterns on the ground they are a quick and easy way of getting a rough idea of the sorts of places you can afford to rent in London.

Taking inspiration (I assume!) from what we have been doing for London, the New Yorker has done an interactive graphic showing “Inequality and New York’s Subway“. 

It would have been nice to see the values on the map, nonetheless the graphic has proved extremely popular and may prompt some more in North America. The power of subway maps lies in the ways that people can easily relate to them, and so I think they offer a great tool for showing the headlines of social (in)equality in our cities.

A lot of the above has been x-posted from mappinglondon. 

And finally…thanks to Moacir for alerting me to his version based on median income along the Chicago Subway:

Map design and cartography are now, more than ever, seen as just one set of tools in a growing data visualisation arsenal. This video features well-known data visualisers who talk through some of their guiding principles, offering many insights for those working with geographic data.

(via Visualising Data)

“Big Data” and “Smart Cities” are two concepts receiving a lot of attention at the moment. CASA‘s Director, Professor Mike Batty cuts through the hype to reveal some of the insights we can gain through the analysis and mining of large transportation datasets.

There are many useful introductory guides out there to R, but below is the kind of thing I now wish I’d been given when I first started using it – something with simple logically-progressive examples and minimal explanatory text. Copy the text below into a new script in R and run line-by-line to give a quick intro to many of R’s most basic principles and functionality. You can also download a text file with it here. It is by no means comprehensive, even at the most basic level, but still I hope someone finds it useful. You may want to look at RStudio as it is more user-friendly.

## A CRASH COURSE IN [R] PROGRAMMING
## Robin Edwards (geotheory.co.uk), March 2013
## In RStudio run through line-by-line using Ctrl + Enter

# basic R environmental functions
x=3.14159; y=’hello world’; z=TRUE # create some objects. In RStudio they’ll appear in ‘Workspace’
ls() # list the objects in the Workspace
print(y) # print information to R ‘Console’
rm(y) # remove an object
rm(list=ls()) # remove all
getwd() # find current working directory
setwd(“/Users/robinedwards/Documents”) # set working directory as preferred
print ( “R ignores the ‘white-space’ in command syntax” )

# use ‘?’ for help on any R function (if its library is loaded in the session)
?max
??csv # search for a text string in R documentation library
library(help=utils) # get help on a particular package (list its functions)

# ‘str’ is a powerful tool for investigating the underlying structure of any R object
str(max)

# CREATING AND MANIPULATING R OBJECTS

# assigning values to variables
n = 5 # is possible but
n 5 -> n
rm(n)

# R objects can be of various data types, but probably most common are ‘numeric’ and ‘character’
( num ( char <- ‘any text string’ )

# create a VECTOR (array) using the ‘c()’ concatenate function
( vec

# a vector series
( vec

# R vectors can be accessed in various ways using [ ] brackets
vec[3]
vec[3:6]
vec[ c(1,3,8) ]
vec[vec > 15]

# check a vector contains a value
5 %in% vec
12 %in% vec

# finding first index position of a matching value/sting
( x = c(‘one’, ‘five’, ‘two’, 3, ‘two’) )
match(‘two’, x)
match(c(‘two’,'five’), x)

# a MATRIX is a 2D vector (essentially a vector of vectors) of matching data type
( matrx = matrix(1:15, 3, 5) )
( matrx dim(matrx) print(matrx)
t(matrx) # a matrix can be easily transposed

# an ARRAY is a generic vector but with more flexibiity. A 1D array is the same as a normal vector,
# and a 2D array is like a matrix. But arrays can store data with ‘n’ dimensions:
( arry

# Using square brackets on arrays
arry[12] # a single criterion (argument) selects the array’s n’th record
arry[3,1,2] # or use multiple arguments that reflect the array’s dimensionality
arry[,,2]
arry[,1,]

# a DATA.FRAME is like a matrix, but accomodates fields (columns) with different data types
(df

# They can be viewed easily
View(df)

# examine their internal stucture
str(df)

# data interrogation with square brackets
df[1,]
df[2:3,]
df[,1]
df[2,1]

# data.frame and matrix objects can have field (column) and record (row) names
dimnames(df)
colnames(df)
names(df) # not for matrix objects
row.names(df)

# interrogate data.frames by field name using the ‘$’ operator. the result is a simple vector
df$name
df$name[2]

# names can be reassigned
names(df) row.names(df) print(df)

# check dimensions of vector/matrix/array/data.frame objects
length(vec)
dim(df)
dim(arry)
nrow(df)
ncol(df)

# R has various inbuilt data.frame datasets used to illustrate how functions operate e.g.
data()
InsectSprays # this guide makes use of these datasets
warpbreaks

# examine contents
head(InsectSprays) # list the top records of a vector / matrix / d.f.
tail(InsectSprays, n=3) # bottom the 3
summary(InsectSprays) # summarise a data vector

# aggregate() is a powerful function for summarising categorical data
aggregate(InsectSprays$count, by=list(InsectSprays$spray), FUN=mean)
sumInsects names(sumInsects) print(sumInsects)

# subset/apply filter to a data.frame
warpbreaks[warpbreaks$wool=='A',] # by 1 condition
warpbreaks[warpbreaks$tension %in% c('L','M'),] # multiple conditions

# adding entries is possible (if a bit tricky)
(newrow (warpbreaks

# but LISTS are better at this
lst = list()

# ways to assign/add items
lst[1] = “one”
lst[[2]] <- “two”
lst[length(lst)+1] <- “three”
print(lst)

# data retrieval
lst[[1]] # double brackets means the object returned is of the data class of the list item
lst[2:3] # selecting a more than 1 list item is possible with single brackets..
lst[c(1,3)] # but the object returned (from single bracket interrogation) is a list

# delete list items
lst[[3]] lst[1:2] lst

# entries can be any object type (like python), including other lists (double bracketting)
lst[[1]] lst[[2]] <- ‘item2′
lst
lst[[1]][[1]]

# Data in lists can also be stored and recalled by key word/number (like Python’s dictionary class)
dict dict['wed'] print(dict)
dict[['tues']]
dict[c('mon','wed')]

# reorder a vector with ‘sort’
vec sort(vec)

# or a dataframe with ‘order’
df[order(df$years),]

# LOGICAL objects (booleans) are binary true/false objects that facilitate conditional data processing
(bool (bool

# query an object’s data/structure type with ‘class()’
class(bool)
class(num) # numeric is the default data type for number objects
class(as.integer(num)) # integer class exists but is not default
class(char) # character class
class(’237′ ) # numbers aren’t always numeric type
as.numeric(’237′) # but can be converted
as.character(237) # and vice verse

# Child-objects are often of different class to parents
class(df)
class(df[,2])
class(df[,1])

# FACTOR objects are vectors of items that have been categorised by unique values
factr str(factr)
levels(factr)
table(factr)

# you may encounter problems converting a factor of numeric data to numeric type
as.numeric(factr)

# instead do this
as.numeric(as.character(factr))

# editing factors can be tricky
print(df)
df$person[1] <- ‘Matthew’

# instead convert to character or numeric etc
df$person df$person[1] <- ‘Matthew’
df$person levels(df$person)

# LOGICAL OPERATIONS
2 + 2 == 4 # ‘==’ denotes value equality
3 <= 2 # less than or equal to
3 >= 2 # greater than or equal to
‘string’ == “string”
‘b’ >= ‘a’ # strings can be ranked
3 != 3 # NOT operator
c(4,2,6) == c(4,2,8) # vector comparisons return locical vectors
TRUE == T # ‘T’ and ‘F’ default as boolean shortcuts (until overwritten)
TRUE & TRUE # AND operator
TRUE | FALSE # OR operator
F | F

# IF/ELSE statement (used in most logical procedures)
x if(x < 5){
print(‘x is less than 5′)
} else{
print(‘x is not less than 5′)
}

if(T|F) print(‘single liners can dispense with curly brackets’)
if(T&F) print(”) else print(“but then ‘else’ only works on the same line”)

# LOOPING FUNCTIONS – very useful for handling repetitive operations

# ‘FOR’ loop
for(i in 1:10){
print(paste(‘number ‘,i))
}

# WHILE loop (be careful to include safeguards to prevent infinite loops)
i = 30
while(i > 0){
print(paste(‘number ‘,i))
i = i – 3
}

# creating a function
multiply tot return(tot)
}

multiply(3,5)
# note ‘tot’ wasn’t remembered outside the function – functions are contained environments
# if required use ‘<<-’ for global assignment but be careful not to overwrite R’s internal objects
# its generally better to do this:
newVar

# handling ‘NA’ values
(x = 1:5)
x[8] = 8
x[3] = NA
print(x) # sometimes functions will fail because of NA values
na.omit(x) # iterates full list but ignores NAs
x[na.omit(x)]
is.na(x) # alternatively
x[!is.na(x)]

# useful basic math functions
seq(-2, 2, by=.2) # sequence of equal difference
seq(length=10, from=-5, by=.2) # with range defined by vector length
rnorm(20, mean = 0, sd = 1) # random normal distribution
runif(20, min=0, max=100) # array of random numbers
sample(0:100, 20, replace=TRUE) # array of random integers
table(warpbreaks[,2:3]) # array summary stats (powerful summary tool)
min(vec)
max(vec)
range(vec)
mean(vec)
median(vec)
sum(vec)
prod(vec)
abs(-5) # magnitude
sd(rnorm(10)) # standard deviation
4^2 # square
sqrt(16) # square root
5%%3 # modulo (remainder after subtraction of any multiple)
6%%2
for(i in 1:100) if(i%%20==0) print(i) # useful for running an operation every n’th cycle

# Importing and exporting data using comma-separated file
write.csv(df, ‘example.csv’) # save to csv file
rm(df)
(df

# PLOTTING IN R

# some basic functionality
plot(1:10)
plot(sort(rnorm(100)), pch=16, cex=0.5) # specifying point and size respectively
plot(x=1:25, y=25:1, pch=1:25) # x & y inputs, and showing the available point symbols
plot(sin, -pi, 2*pi)
hist(rnorm(1000), breaks=50)
barplot(sumInsects$sum, names.arg = sumInsects$group)
pie(sumInsects$sum, labels = sumInsects$group)

# plots with more visual components are built up incrementally
x plot(x, pch=17)
lines(x, col=’#00FF00′)
points(x+5, pch=16, col=’red’)

# stacking charts
warpbreaks
sumWB names(sumWB) sumWB
(data barplot(data, names.arg=c(‘Group A’,'Group B’),
legend.text=c(‘L’,'M’,'H’), args.legend = list(x = “right”))

barplot(data, names.arg=c(‘Group A’,'Group B’), beside=T,
legend.text=c(‘L’,'M’,'H’), args.legend = list(x = “topright”))

# ‘symbols()’ is a good way to represent a 3rd data dimension (use square root for area proportionality)
(cities lon=c(-0.1,-2.6,-2.2,-1.5), lat=c(51.5,51.4,53.5,53.8), pop=c(8,1,2.7,0.8)))
symbols(x=cities$lon, y=cities$lat, circles=sqrt(cities$pop), inches=0.3,
bg=’red’, fg=NULL, asp=T, xlab=’Longitude’, ylab=’Latitude’)
abline(h=(seq(51,53,1)), col=”lightgray”, lty=1)
abline(v=(seq(-4,1,1)), col=”lightgray”, lty=1)
text(x=cities$lon, y=cities$lat+0.2, labels=cities$city)

# But for much easier and more elegant data visualisation use GGPLOT2

# END OF SCRIPT

The publication of the International Benchmarking Review of UK Human Geography back in March was a major event for the discipline. The report concludes that the UK human geography community is setting the intellectual agenda for the subject and can therefore consider itself the best in the world. Whilst this is a great achievement, the “relative weakness in quantitative methods and GIS” (p16) was also highlighted. The authors flag the growing interest in data visualisation, and its value for information dissemination and impact, but suggest that human geographers may not be ahead of the curve in the same way as they are in other parts of the discipline.

Compelling data visualisation is the key to communicating results from research widely and, perhaps most importantly, a means to inspire current and future students to work with quantitative data. More maps (and other visualisations) than ever are being produced, but many are methodologically flawed or misleading because the people who created them are great with complex software and data manipulation, but less good at placing their visual outputs in context. I would argue that human geographers, with their knowledge of population processes, are uniquely placed to perform a “sanity check” on the many graphics that stem from analysis that has set off down the wrong path. Human geographers are also well-aware of the power of maps to change perceptions of the world (both for the better or worse) so they are perhaps a little more cautious in the way they portray data and make doubly sure that they have produced something that is both representative and defensible.

So, it is more important than ever for geographers to engage with the many relevant datasets that are now available and also with the communities (largely of computer scientists and designers) that are currently grappling with them. We don’t all have the skills to manipulate large datasets, or visualise them, but we can still contribute to the creative process and ensure the outcomes are representative of the data used to produce them. The benchmarking review lists four people/ groups that have made a start in this regard [Hans Rosling (of GapMinder), Simon Rogers (formerly, the Guardian), Danny Dorling (and colleagues) and CASA (where I work)]. Interestingly only two would probably badge themselves as geographers.  I thought I’d speculate a little about what makes each of these groups successful in this context and also add a few more groups that I think could offer the inspiration needed to get more people visualising human geography.

Hans Rosling and Gapminder

Gapminder helped to kick-start the interest in the power of data visualisation to explore the complexities of global development. The website, and energetic videos by Hans Rosling, have raised the profile of the many datasets compiled by global organisations, such as the UN, and offer a means for students to engage with data in an interesting way. For people starting to think about data, Gapminder should be one of the first websites they visit.

Simon Rogers & the Guardian Data Store

The Guardian Data Store and its associated blog have raised the profile of human geographers in the UK more than any other website over the past couple of years. Simon Rogers and his team have actively sought visualisations to feature (including many from this blog) and this has often resulted in interest from a range of other media outlets. Some of the graphics featured have been a bit dubious, but there has always been scope to comment and the team have taken criticisms on board. This has facilitated an important dialogue that is a great success story in terms of academic/ media engagement. The blog and the push for freely available data has been a great force for good and as Simon moves on to Twitter, I hope it will continue to promote the work of geographers.

Danny Dorling and Worldmapper

I think the work of Danny Dorling and his team at the University of Sheffield is notable for two reasons. The first is their promotion of cartograms. Cartograms distort the world according to a particular variable (in the map above it’s population) so that areas with the highest values become larger at the expense of areas with smaller values. The impact of this approach is the way that it draws attention to the places on the map that matter. This slightly trippy cartography is also a powerful way of changing perceptions of the world (so often based on the Mercator projection) and can form the basis for discussions about representations of geographic data.

Secondly, Danny has done a huge amount to promote spatial equity by making population data accessible to a wider audience. At the moment I am reading his latest book that walks readers along the 32 stops of the Central Line in London (a similar idea to the map I produced last year), exploring the lifestyles of those along it as portrayed by population data. Such data are used everyday by government and industry to alter people’s lives, but many are unaware about what it says about them and also how it can be used to promote equality. This kind of work is controversial at times but forms an important component of the informed social debates that human geographers can contribute to through good data visualisation.

The UCL Centre for Advanced Spatial Analysis (CASA)

At CASA we have produced a fair number of visualisations (many I have featured on this blog).  Most of what we do requires a collaboration between geographers, computer scientists and others to collect the data, format it and visualise it. We are lucky to have such a rich mix of disciplines in the department and I think it represents a successful template for geography departments more broadly as the need for quantitative methods increases with data volumes. We also run a great masters course…

Floatingsheep

Despite their slightly baaamy (see what I did there) name, the team behind floatingsheep are seriously getting to grips with the new forms of data produced online. With maps entitled “The Beer Belly of America” and “The Price of Weed“, students will be instantly engaged. There is also a long list of academic publications available that explore the more substantive research questions associated with the geography of the internet.

 New York Times Graphics Department

The New York Times Graphics Department excel at high quality data visualisation that is eye catching and informative. The graphic above speaks for itself.

Stamen

Stamen Design have produced a series of interactive visualisations from a range of datasets that have consistently raised the bar in terms of what is possible for geographic data. If you are creating anything interactive and online, their base maps are especially useful as they enhance the appearance of the data overlaid on top of them.  There is no longer any reason to use the default Google cartography!

Eric Fischer

Eric Fischer has led the way in producing stunning visualisations from social media data. Social media datasets, especially from Twitter, are creating a lot of hype at the moment in quantitive human geography and no-one maps them better than Eric.

Nathan Yau of Flowing Data

Finally, Nathan Yau’s flowingdata frequently features excellent examples data visualisation with geographic data. The blog has a huge range of examples and a number of tutorials that would be of benefit to human geographers, especially when teaching.

An updated version of the London Twitter Languages map I produced with Ed Manley recently appeared as a double page spread in the Guardian. The colour scheme is different to the original and we use a far larger dataset collected by John Barratt of Trendsmap. We want to keep this one print-only for the time being so we are keeping it offline.

The first is written by Richard Harris from the University of Bristol. You can download the worksheet here and the example data from here. I like this worksheet because it is very applied (it is using a real dataset on an important problem), it gets you working with spatial data quickly and it also offers an automated run-through of some of the more advanced stats. It provides a great general introduction to R that can be a basis for more advanced worksheets and tutorials.

The second worksheet is one that I have written as an introduction to spatial data visualisation with ggplot2 (a data visualisation package in R). Download instructions here and data here. The code takes you through some of the key functions I often use in ggplot2 to create maps. It also includes faceting which enables you to create lots of graphics as small multiples on a single page. I use this a lot for maps such as the one below of New York Twitter languages:

Here are a couple of additional worksheets from Chris Brunsdon (University of Liverpool) that may be of interest to those who want to do more advanced spatial analysis and I have a few tutorials here. I am sure there are many more good examples out there so feel free to post them in the comments.

Following the interest in our Twitter Tongues map for London, Ed Manley and I have teamed up with Trendsmap creator John Barratt to offer this snapshot of New York City’s Twitter languages. We have visualised the geography of about 8.5 million geo-located tweets collected between Jan 2010 and Feb 2013. Each tweet is marked by a slightly transparent dot coloured according to the language it was written in. Language was detected using Google’s translation tools. The above map (click for interactive version courtesy of Oliver O’Brien) has the top ten languages plotted together and the one below takes the top 24 in turn (excluding English) and orders them by popularity. English (in grey above) is by far the most popular with Spanish (in blue above) taking the top spot amongst the other language groups.  Portuguese and Japanese take third and fourth respectively. Midtown Manhattan and JFK International Airport have, perhaps unsurprisingly, the most linguistically diverse tweets whilst specific languages shine through in places such as Brighton Beach (Russian), the Bronx (Spanish) and towards Newark (Portuguese). You can also spot international clusters on Liberty Island and Ellis Island and if you look carefully the tracks of ferry boats between them. Ed has written up some more in depth analysis of the data here.

Making the Maps

For those interested, the maps above were produced using the R software platform with the ggplot2 package. Both coped surprisingly well with plotting 8.5 million points (it took about 15 minutes on my two year old iMac) and the results are really great. Here is the code I used to produce the black and white map above:

#two input data frames here. "lang_freqs" has the total frequency of each language and is ordered highest to lowest (this is used for the facet ordering) and "twit_lang" is a data frame with each tweet's location (lat, long) and its language (lang) (it therefore has 8.5 million rows).

#here I create a new column lang1 to twit_lang which is used to order the faceting.

labs<-as.factor(lang_freqs$lang)
twit_lang$lang1

p

p1<-c(geom_point(data=twit_lang,aes(x=long, y=lat),colour="white", alpha=0.1, size=1.2))

p+p1+ quiet + facet_wrap(~lang1, ncol=4) + opts(strip.text.x=theme_text(size=8))+opts(strip.background = theme_rect(colour="white", fill="white"))


Here is an animation that Ollie O’Brien created a couple of years ago for the “Sense and the City” exhibition at the London Transport Museum. The map shows the touch-ins (going into the network) and touch-outs (leaving the network) of Oyster cards at London’s tube and train stations, including a few beyond the Greater London boundary which still accept . As the animation moves forwards in 10-minute intervals during the typical weekday, the balance between touch-ins and touch-outs is shown by a colour scale. Red indicates the great majority of taps are touch-ins, and green indicates mainly touch-outs. White is the “neutral” colour, indicating that roughly as many people are entering the network as leaving it, at that period in time.