The London Data Table

The London Data Table was one of my personal favourites from the exhibition accompanying the CASA “Smart Cities” conference which took place at the University of London last Friday. The concept was thought up by Steven Gray and it consists of a wooden table, cut by programmable lathe into the outline of London. A special “short throw” projector with a fish-eye lens was purchased. It was mounted vertically on a converted basketball hoop stand, pointing downwards and outwards, allowing the content to be approached and examined without the projector getting in the way. Steven has blogged about the construction process.

I created a generic dark grey background map (from Ordnance Survey OpenData) with a blue River Thames as the main identifying feature. This was used by several authors, including myself, to create either Processing “sketches” in Java, or pre-recorded videos, which were displayed on the table during the exhibition. A simple Javascript script running on Node.JS was written to automatically cycle through the animations.

By ensuring that the background map and accompanying sketches/videos where “pixel perfect”, we were able to take advantage of having control of every individual pixel, producing the quite pleasing pixellated effect as seen in the below closeup of one of the sketches (a photo taken of a part of the table) – it is showing a bike share station animation that I created, based on the same data that powers the equivalent website.

The photo above shows the table running another Processing sketch, showing point information from CityDashboard and similar to the map view on the website, except that points are randomly and automatically selected to be displayed, as people stand beside and watch the table.

The most interesting sketch presented on the table (and shown on the right – photo by Helen) was built by Steven Gray and connected to a airplane sensor box, that picked up near-real-time broadcasts of location, speed and aircraft ID, of planes flying over London. The sketch stored recently received information, and so was able to project little images of plans, orientated correctly and with trails showing their recent path. Attached to each plane image was a a readout of height and speed, and most innovatively of all, a QR code was programmatically generated and rendered behind each plane, allowing smartphone users to scan it. QR codes are normally encoded URLs, and these ones were set to point to a flight information website, with the aircraft’s details preloaded, showing a photo, and the origin and destination at a glance.

The QR codes were able to be made very small – using a single projector pixel per QR code pixel and little error correction. Various smoothing and blurring digital effects having been switched off, and a digital connection between computer and projector used, to allow the sharpest possible representation. As a result, my iPhone was able to tell me more about the planes I was seeing fly, in near real time, around the table.

On Colour Ramps and City Dashboards

Here are the colour ramps I am using for numeric measures in the recently launched CityDashboard (which by the way now has a new URL – http://citydashboard.org/):

The colours have been designed to be clearly distinguishable from the white text that is on top of them.

Here is the PHP code that I’m using to choose the appropriate colour for each measure, and which I also used to produce the above ramps – the reverse colour and bad value handling is only implemented where I currently needed, ideally these would be implemented for all the ramps:

$na_rgb = 128;

function getGreyRedHex($val, $min, $max, $reverse=false, $processing=false)
{
	$val_0_255 = getNormalisedVal($val, $min, $max);
	$r = 128 + 0.5*intval($val_0_255);
	$g = 128 - 0.5*intval($val_0_255);
	$b = 128 - 0.5*intval($val_0_255); 
	return getHex($r, $g, $b, $processing);
}

function getGreyBlueHex($val, $min, $max, $reverse=false, $processing=false)
{
	$val_0_255 = getNormalisedVal($val, $min, $max);
	$r = 128 - 0.5*intval($val_0_255);
	$g = 128 - 0.5*intval($val_0_255);
	$b = 128 + 0.5*intval($val_0_255); 
	return getHex($r, $g, $b, $processing);
}

function getColdWarmHex($val, $min, $max, $reverse=false, $processing=false)
{
	$val_0_255 = getNormalisedVal($val, $min, $max);
	$r = intval($val_0_255);
	$g = 255 - 2*abs(127.5 - $r); 
	$b = 255 - $r;	
	if ($reverse)
	{
		$r_temp = $r;
		$r = $b;
		$b = $r_temp;
	}	 
	return getHex(0.8*$r, 0.8*$g, 0.8*$b, $processing);
}

function getGreenYellowRedHex($val, $min, $max, $reverse=false, $processing=false)
{
	global $na_rgb;
	if ($val === "n/a") { return getHex($na_rgb, $na_rgb, $na_rgb, $processing); }
	if ($val === "?") { return getHex($na_rgb, $na_rgb, $na_rgb, $processing); }
	$val_0_255 = getNormalisedVal($val, $min, $max);
	$r = intval($val_0_255);
	$g = 255 - intval($val_0_255);
	if ($g > 128) { $g = 128; }	
	$b = 0; 
	return getHex($r, $g, $b, $processing);
}

function getRedGreyGreenHex($val, $min, $max, $reverse=false, $processing=false)
{
	global $na_rgb;
	if ($val === "n/a") { return getHex($na_rgb, $na_rgb, $na_rgb, $processing); }
	$val_0_255 = getNormalisedVal($val, $min, $max);
	$r = 255 - intval($val_0_255);
	$g = intval($val_0_255);
	if ($g > 128) { $g = 128; } 
	$b = 128 - abs(127.5 - $val_0_255);
	return getHex($r, $g, $b, $processing);
}

function getNormalisedVal($val, $min, $max)
{
	if ($val < $min) { $val = $min; }
	if ($val > $max) { $val = $max;	}
	$range = $max - $min;
	return ($val - $min)*(255/$range); 
}

function getHex($r, $g, $b, $processing)
{
	$hex = str_pad(dechex($r), 2, "0", STR_PAD_LEFT) 
		. str_pad(dechex($g), 2, "0", STR_PAD_LEFT) 
		. str_pad(dechex($b), 2, "0", STR_PAD_LEFT);

	if ($processing) { return "FF" . $hex; }
	else { return "#" . $hex; }
}

I’ll be presenting CityDashboard at the forthcoming Wherecamp EU unconference which is taking place in Amsterdam this weekend.

CityDashboard

CityDashboard is the main project that I have been working on for the last few months. It aims to summarise quantitative data (both officially provided and crowd-sourced) for the major UK cities, in a single screen. Point data is also shown in an alternate map view.

It was launched at the CASA Smart Cities conference last Friday, for eight cities – London, Cardiff, Edinburgh, Glasgow, Manchester, Leeds, Birmingham and Newcastle. London has the most dashboard “modules” at present, with a number of London-specific modules from Transport for London, the Port of London Authority, and CASA’s own sensors. Other cities have several more generic modules (such as weather and Twitter trends) and more city-specific modules will be added to these in due course. I am also looking at improving the overall look and feel of the website, possibly by using the BBC Glow API that was suggested to me at the conference (but just now took me half an hour to find on the web!)

CityDashboard features specially curated Twitter lists. For each city, there is a general news list, featuring tweets from local newspapers, local correspondents for the BBC and other TV and radio channels, tourist organisations and the official accounts for the relevant local authorities. There is also a universities list, with the official Twitter accounts for the main universities in each city, as well as their student unions. It is hoped that this latter list with detail the latest university research outputs, coming out of that city. The account that manages the lists is CityDB and the lists take the form of, for example, http://twitter.com/citydb/london and http://twitter.com/citydb/london-uni. Anyone can subscribe to these lists, you don’t have to only view them through CityDashboard.

You can visit CityDashboard live, right now, at http://citydashboard.org/

The project is an output of NeISS, which is funded by JISC.

CASA Smart Cities

[Updated x2] Just a note to say that I will be presenting some of my work, at the CASA Smart Cities free one-day conference. Over 200 tickets have already gone, but there are, at the time of writing, a few left.

There will be an exhibition at the conference, some people in the department have been building some very cool things which will be unveiled there. Unfortunately I’m not allowed to talk about the very coolest one of all, but I have been allowed to post the above graphic which has got something to do with it…

(If you want to have a guess at what it is, leave a comment!)

[Update 3/4 - Tickets are sold out, however I think an extra batch will be available soon.]
[Update 13/4 - A few more tickets now available.]

London Bikeshare Expanding East and West

The Barclays Cycle Hire bikesharing system (map) in London is due for a major expansion on 8 March. Overnight on the 7th, operators will be working flat out to add 2300 1700 1900 new bikes into 4800 3000 3400 new stands, clustered in the 200 150 new docking stations that have been tested over the last few weeks, across the East End of London (Tower Hamlets, Shoreditch/Hackney Road and Canary Wharf). Also going live the same day will be a much smaller expansion west to the area round the Westfield shopping centre in Shepherd’s Bush in West London. Another small expansion around Camden Town has just been completed, adding several new stands to the northern tip of the system, including handily around Camden Town tube station and Camden Road train station, allowing commuters from the north and the Overground network (like me!) to avoid the expensive Zone 1 fares and Boris Bike the last few kilometres to work.

The expansion will move London up the league table of bike share cities from 7th to 5th – in a top 20 dominated by China. It will remain the second largest system outside of China, after Paris, although New York’s planned system will be even larger:

The Biggest Bike Sharing Cities (March 2012)

City Country Bikes
Mar 2012
Bikes
Nov 2011
(If Different)
1 Wuhan China 70000
2 Hangzhou China 60600
Bejing (planned) China 50000
3 Paris France 18000
New York (planned) United States 10000
4 Taizhou China 10000
5 London (8 March+) Great Britain 7200 5000
6 Yantai China 6000
7 Shanghai China 5700
Chicago (planned) United States 5000
8 Guangzhou China 5000
9 Barcelona Spain 4700 4400
10 Kaohsiung China 4500
11 Montreal Canada 4220*
12 Foshan China 4000
13 Lyon France 3400 3060
14 Zhangjiagang China 3200
15 Munich Germany 3000
16 Wuham/Qinshan China 3000
17 Toulouse France 2500
18 Brussels Belgium 2180 2060
19 Seville Spain 1950
20 Changshu China 1700 1440

* Currently closed for winter.

Data sources: Scheme operator websites (through my live map), http://publicbike.net/, http://citybik.es/, http://bike-sharing.blogspot.com/, press releases/newspaper articles on new schemes, Wikipedia articles. If you are aware of any mistakes, please let me know and I will correct. Photo: CC-By-NC IanVisits.

Reworking Booth: Geodemographics of Housing

[Update January 2013 - Scottish SIMD 2012 map added, more details.]

I’ve created a new visualisation, a dasymetric map of housing demographics which you can see here, which attempts to improve on the common thematic (a.k.a. choropleth) maps – a traditional example is shown below – where areas across the country are colour-coded according to some attribute. My visualisation clips the colour-coding to the building outlines in each area, leaving open ground, parks etc uncoloured.

The Traditional Approach

The shortcoming of choropleth maps is that each area is coloured uniformly. If the attribute being measured is a property of the houses in that area, such as much of the census data, then choropleth maps not only colour the houses in each area, but also the parks, rivers and mountains that might also be contained within the area, even though the data being displayed arguably only applies to the houses. This means that geodemographic classification results that predominate in rural areas tend to overwhelm a map at smaller scales – as can be seen in the map on the right – where the green represents a countryside geodemographic.

An alternative to choropleth maps is to use cartograms. These distort the area, elastically, to tessellating hexagonal groups or to circles (Dorling cartograms), to match typically population rather than geographic extent, so that the colours are represented more fairly, but cartograms are very difficult for most people to interpret and relate to familiar physical features. They can look very “alien”. One further alternative is dot distribution maps – these assign dots of colour, randomly within each area. This reduces the colour density correctly in sparsely populated areas, but distributes the dots evenly across empty parks and rows of houses, if both are in a single area, and imply single points of population.

Clipping the Choropleth Maps

My visualisation attempts be the best of both worlds, by retaining the familiar geographic shape of the UK and its towns and cities, but not swamping the map with colours in all areas, and indeed ensuring that unpopulated areas have no colour. This is possible because Ordnance Survey Open Data includes Vector Map District. The second release of this dataset improved the quality of building outlines considerably, allowing distinct rows of buildings on streets to be seen and even individual detached houses. Unfortunately building classifications are not included, so the process necessarily colours all buildings, rather than just the residential ones that formed part of the census data. This is why, for example, the Millennium Dome in Greenwich appears, even though no one (hopefully!) lives there.

The major shortcoming of doing this is that it falsely implies a higher level of precision within each Output Area, by often showing and colouring individual buildings, whereas the colour is representative as an average of the properties in the area concerned, rather than telling you something about that particular building itself. That is, the technique is showing no new or more detailed data than can be seen in the traditional choropleth maps, but tends to mislead the viewer otherwise. This is balanced by making the map seem more realistic, by not unformly covering everything in the area with a giant blob of a single colour.

The map can be considered to be a dasymetric map, albeit one where the spatial qualifier, population density, is one of two values – high (in a building) or zero (not in a building).

Booth’s Poverty Map

An inspiration for this kind of map is the Charles Booth Poverty Map of 1898-9, although my example is considerably less sophisticated. For this map, Booth (and his assistants) visited every house, to determine the demographic of the house, and then painstakingly coloured in the houses, along the streets. His map therefore did not suffer from the falsely implied accuracy – his map really was as accurate as it looks. The Museum of London, incidentally, has a “walk in” Booth poverty map, I featured it on Mapping London blog last year.

The photo above compares Booth’s map (from a photo of the map in the Museum exhibition, including a friend’s hand) with my map, for the Hackney area in London.

OAC, IMD and London

My main geodemographic map is showing the OAC (Output Area Classification), which was created by Dan Vickers in Sheffield in 2005, and is based on data from the 2001 census. The areas used are Output Areas, there are around 210,000 of them in the UK, each one with a population of roughly 250 people in 2001.

The OAC map is not particularly illuminating for London – the capital is considerably more ethnically diverse than most other parts of the country, but because the clustering process used to create OAC is run across the whole country uniformly, only one Supergroup appears to show such ethnically diverse areas – “7″ (Multicultural), rather than showing the variety within this group that extends across the capital. With this in mind I have created an alternative map, which colours the housing according to the IMD (Index of Multiple Deprivation) rankings. This covers England only, and the data is only available at larger spatial units, called LSOAs (Lower Super Output Areas) but is more up-to-date, being from 2010, and shows considerable more variety across London. Use the link at the bottom of the visualisation to switch between the two.

You can view the map here. It uses geolocation to attempt to zoom to your local area, if you allow it to – it will probably ask you to allow this when you visit the site.

Tube Colours

Transport for London (TfL) take their colours extremely seriously – the London Underground, in particularly, uses colour extensively to brand each line, and the maps and liveries are very well known.

The organisation has a colour guide to ensure that, when referencing the tube lines, the correct colour is used. Somewhat surprisingly, the guide includes hexadecimal (i.e. web) colours for only a “safe” palette – i.e. colours which would definitely work in very old web browsers. They don’t list the “true” hexadecimal for the colours, even though, confusingly, the colour shown is the true one. I couldn’t find anywhere on the web that did this either, all in one place, so here below is a summary. I’ve also included the safe colours so you can see the difference – but don’t use these unless you have to.

Line True Hexadecimal Web Safe Hexadecimal
Bakerloo #B36305 #996633
Central #E32017 #CC3333
Circle #FFD300 #FFCC00
District #00782A #006633
Hammersmith and City #F3A9BB #CC9999
Jubilee #A0A5A9 #868F98
Metropolitan #9B0056 #660066
Northern #000000 #000000
Piccadilly #003688 #000099
Victoria #0098D4 #0099CC
Waterloo and City #95CDBA #66CCCC
DLR #00A4A7 #009999
Overground #EE7C0E #FF6600
Tramlink #84B817 #66CC00

Don’t Zone-1 It When You Can Boris Bike It

[Updated with new connections.] Ever thought what the tube network would look like if you took out the expensive Zone 1? Me neither, until this morning, when I was wondering if it was possible to utilise my current “Boris Bike” bikeshare 24-hour membership to save a bit of money on commuting in to work.

Transport for London would really rather you didn’t take the tube into Zone 1. It’s often at capacity during the rush hour. The fares are priced accordingly – for example, to get a tube from Zone 3 to Zone 1, it costs £2.90 during the Peak Fare periods, compared with £1.40 if you only go from Zone 3 to Zone 2. Do that commute twice, and it’s a £3 saving missing out Zone 1, which more than makes up for the £1 Boris Bike 24-hour membership charge. So, I was wondering if it is viable to get off the tube a few stops early and Boris Bike the final mile or so.

Superimposing my London bike share map reveals directions from where such money-saving journeys may be possible. Plenty of opportunities from the north-west or the west, with St John’s Wood, Notting Hill and Earls Court having easy Boris Bike accessibility. Access from the south-west and south is also good, thanks to Vauxhall and Elephant & Castle. Things get a little trickier – as usual – in the south-east, where a 1km walk from Bermondsey, or a much longer walk from South Bermondsey are your only options – the region won’t even stand to benefit from the forthcoming scheme expansion. The east is also an option, with Whitechapel being very well stocked with Boris Bikes, and Wapping not being too far either. The east is set to benefit too from the imminent expansion of the Barclays Cycle Hire scheme, to use its full name, to cover all of Tower Hamlets. The north-east is OK, with Hoxton an option, although it’s a shame the docks don’t extend up to Highbury & Islington station, a major interchange. The north is also good, thanks to the legendary Mornington Crescent station.

Four of the Best Zone 2 & Bike Opportunities

  • Mornington Crescent (Northern Line) – right beside a big docking station. Don’t go to Euston (Zone 1). It’s also easy to get to Mornington Crescent from the Overground, thanks to an unmarked but official Out of Station Interchange.
  • Notting Hill Gate (Central Line) – the tube takes you to the top of the hill, then go east by Boris Bike.
  • Earl’s Court (District/Piccadilly) – it’s a long way to Charing Cross, but if you don’t need to go that far, the bike is a good way to travel.
  • Whitechapel (District/Hammersmith & City) – two big stands very close to the station, and an easy cycle along CS2 into the City.

So, all in all, not too bad. Whether it’s worth the extra time walking to a convenient docking station, and the worry of finding it all out of bikes, to save a pound or two, is another thing…

+ =

See my Mapping London article for more detail about the No Zone 1 map. You can click on the map above for a larger version. An SVG version (i.e. editable) without the bike docking stations, is downloadable here although there is lots of missing detail beyond Zone 2.

The map was based on a Wikimedia/Wikipedia file, which I then augmented (to show the Overground and some selected regular railway lines) with OpenStreetMap data, by producing a map in GEMMA of railway=rail features. I also added some unmarked Out of Station Interchanges, thanks to this FOI request. Photos by Gnatallica, Clotheyes and Wwarby on Flickr.

[Update: An earlier version of the article and map made reference to Shoreditch High Street station which I incorrectly thought was in Zone 2 for "old East London Line" journeys - it appears this is not actually the case - the anomaly that I was misremembering is that for short journeys from the station, the Peak Fare increases do not apply. I've also updated the map a few times since posting this article, to add in a few missing stations and also the locations of the big terminus stations in London. I've also added some "Out of Station Interchanges" on the Overground - many of these aren't marked on official TfL maps, but are valid interchanges, i.e. you don't get charged for two journeys.]

Magical Maps: An Evening at the Art of Mapping Exhibition

I was at a talk last night organised by the Londonist – Magical Maps. It took place in the TAG Fine Arts‘ Air Gallery in Mayfair. TAG Fine Arts currently have an exhibition there – The Art of Mapping, and the audience was surrounded by the various artworks – some made from maps, some as interpretations of maps and some simply with a geographical theme.

The speakers were Stephen Walter, an artist who drew the “Island” London map that was a hit at the British Library’s recent Magnificent Maps exhibition; John Kennedy, a London cabbie, blogger and chronologer of bollards and other London objects; and UCL CASA’s very own James Cheshire, recent PhD, now lecturer.

James was up first and showed some of the recent visualisation work by CASA. You can see most of what he showed in an article here.

Then John explained The Knowledge, taking the audience on a verbal taxi journey, from the gallery to the Elephant and Castle – “the true centre of London” – and back, taking into account various banned turns and other taxi restrictions! His journey was peppered with anecdotes of the various places encountered.

Finally, Stephen outlined his artistic career, showing how he switched from traditional photography and painting, to producing the monochromatic repeated symbols on landscapes, that envolved into maps and finally his famous Island map. He also showed some works in progress, such as a map of London’s underground structures (tube lines, water pipes, the Mail Rail etc).

Matt from the Londonist then chaired a Q&A session with the panel, and finally there was another chance to look around and think about the artworks on the walls. I was most intrigued by the below image, which lists and shows all the bridges across the Thames in London, the top line being Teddington Lock and the bottom being the Thames Barrier – but I don’t know why the lines cross over and wiggle. Perhaps it is just purely artistic, but the data visualiser in me hopes there is a deeper meaning and an embedded infographic…

An excellent evening with three very different but equally engaging speakers and some very interesting things to look at.

The exhibition is open for another week, it’s quite small and free, and there is a complementary exhibition book, so take a visit to Mayfair and have a look! There is a talk tomorrow by some of the artists.

The Tech City Map

Last week the government launched the Tech City map. It is not the first – there have been several previous maps, e.g. by the Guardian, of technology startups in the Old Street/Shoreditch area of London, but this is the highest profile one produced yet. I’m not sure whether a map itself is useful to the community but it serves perhaps to increase the identity and branding of the area – and awareness of it for the general public.

Incidentally, the area covered is also known as the Silicon Roundabout area, it often incorrectly referred to the Westminster set as “East London” (I would more term it as North-East inner-city London). I would consider more for the area around the Olympic Park to be true East London, i.e. east of Charing Cross, north of the river and east of the City – areas such as the East End, Stratford, Barking and West Ham.

As for the cartography, there has been some criticism by some people of the mass of straight blue lines, representing retweets of one company’s message by others. I prefer to see it as a piece of “art” showing the interconnectiveness of the organisations. No one’s trying to do serious quantitative impact work with this, it’s just for visual impact and identification of a new organically-created geographical area.

It’s nice also that the custom style functionality of Google Maps v3 API is being used – giving the streets a nice blue hue rather than the very familiar (and quite distracting) regular colours on Google Maps. Of course Google has recently introduced quite low limits (2500 map loads) for free usage of its custom style functionality, I wonder if this website eventually breaching such a limit if it gets hit with another blaze of publicity?

Perhaps the most exciting news that came out of last week’s announcements was that UCL, Imperial and Cisco are collaborating to build a new Future Cities research facility in the Tech City area, which would be focusing on urban modelling and technology. This is music to my ears – I’m not sure whether UCL CASA, the lab where I work, has plans ever have a presence there but it would be an amazing place to work. Maybe there will be a CASA East in the not too distant future?