Category Archives: Mashups

Rise of the Colourful Circles – Election 2010 Visualisation

Posted on by
Reply

election

I’ve fixed and tidied up a visualisation I created back in 2010, which showed the results of that year’s General Election in the UK. Newer versions of OpenLayers had broken it (specifically the use of addUniqueValueRule with a custom context resulted in no circles appearing) and also the UI looked rather rudimentary. Now it has rounded corners, transparency, more spacing and a prettier font!

Although it was my first “coloured circles” visualisation (the Bike Share Map followed on from it a few months later when London’s system launched) it was my most sophisticated, with the circles having different coloured areas and borders, and changing in size – plus a view where the colour itself is calculated from the numeric values – select the “Constituency Colour” from the first pop-down.

The key benefit of the circles other the traditional “colour in the constituency” election map is that sparsely populated rural areas did not dominate the map. It also means that, when viewing the results from individual parties, that each pixel of each colour represents exactly the same number of votes – whether in central London or the Highlands of Scotland.

The background map is not great at all – a mess of greys and names. At the time, I was strictly keeping colour out of it, so that the only colour was the data being visualised. The early Bike Share Map also had the uninspiring background, with a dark grey river flowing past lighter grey lands. These days I’ve relented – a small amount of colour is OK, as long as the shades are pastel and appropriate, and the key data’s colours are vibrant.

You can see the map here.

Update to CityDashboard CSV API & iPad Wall!

Posted on by
Reply

I’ve made some minor alterations to the CSV API for CityDashboard. The main changes are in the metadata rows (the top two) rather than the subsequent rows. Specifically, the top metadata row has now split out the description, source and source URL – which were previously rather messily combined into a bit of HTML – into three text fields; and the second metadata row now uses properly formatted names for value titles, i.e. including spaces, and units, for example “broken_pc” now becomes “% docks/bikes broken”.

The reason for these changes is to accommodate a new and exciting use of the API here at CASA – our lab hardware specialist has recently been hard at work building an “iPad wall” and one of the visualisations in it is of CityDashboard data. Here’s what the uncompleted – but operational – iPad wall looks like (source):

It’s a physical CityDashboard!

I also took the opportunity to fix a few bugs and typos – mainly just cosmetic, but including a pretty silly one for the Mappiness-sourced data that was over-reporting the true value by a large and variable amount. Entirely my fault. That will serve me right for doing a coding change during a colleague’s Ph.D viva drinks reception! I also handle temporarily unavailable source feeds a little better – they’ll now appear unavailable for one complete update cycle but it means the source server doesn’t get repeatedly hammered until it comes back up again.

Boundary Change Map

Posted on by
Reply

I pulled together this interactive map of Proposed Constituency Boundary Changes in England, after the information was released by the Boundary Commission for England last week. My colleague James Cheshire highlighted that this kind of map could be illuminating, particularly as the official maps are simple greyscale PDFs of each new constituency boundary, without the old boundaries or adjoining constituencies for context, and with one document per constituency!

Click the image above to go to the interactive map, then use the slider to fade between the current and proposed boundaries. The new boundaries have been put together to have roughly the same populations in each one (72000-80000 people), and also the total number of constituencies has been dropped by around 5-10%. They are just proposed ones, and are themselves revised from an earlier version.

There are some interesting patterns – many urban areas, such as London, have undergone very significant redrawings, while many rural areas – historically with higher constituency populations – remain untouched. For example, Tottenham loses its identity as a single constituency, the southern half being assimulated into Stamford Hill and the northern half into Edmonton. Slough has a big bite taken out of its SW corner, the people here potentially being represented by a Windsor MP in the future. Much of north Yorkshire is unchanged however.

We didn’t use vector-based boundaries here, even though this would have made it more interactive, because of the size of the boundary files – simplifying them to reduce the size would have been tricky (as it would have made unmoved boundaries move slightly) and the necessary simplification might have distorted the boundaries too much.

As with all my more recent web visualisations, social media (Twitter and Facebook) buttons are included, and geolocation is used to default the view to the user’s location, if they are in England.

On a technical note, this is my first pure HTML5 map. It also takes advantage of simpler ways of setting up maps in the latest release of OpenLayers, 2.12. It means it is out-of-the-box compatible with mobile browsers, and the HTML, JavaScript (including a JQueryUI slider) and CSS adds up to less than 200 lines of code – the only other code used being a couple of Mapnik XML stylesheets for rendering the two maps themselves.

Thanks to James Cheshire for the idea and getting hold of the data.

Run Every Street in Edinburgh – in Strict Alphabetical Order

Posted on by
Reply

…it sounds like one heck of a lot of running. But Murray Strain, one of Scotland’s top terrain runners, is counting on it for his basic training. He’s logging the whole venture, which is based on his trusty Edinburgh A-Z. If two adjacent streets with very similar names are nonetheless separated in the A-Z index by one on the far side of the city, it means a couple of legs right across the city.

Since he started the exercise last year Murray’s got through all the As, and is currently midway through the Bs. I’ve produce a couple of GEMMA maps, one showing the A-Bs (above, As are red and Bs are orange) and one showing the A-Gs (below, in rainbow order). That’s a lot of streets. N.B. The maps in fact show all linear features in the area in OpenStreetMap, so the odd named cycleway and waterway has crept in there too. But the ~95% of the coloured lines will be the streets that Murray will be run.

In order to produce the map, I’ve added a new feature to GEMMA – it now allows you specify only one desired geometry type, i.e. points, lines OR polygons, when adding an OpenStreetMap layer to your map. Previously, you got all three types, although you could reduce each to a dot if desired. This example also highlights the need for legends on the PDF maps that GEMMA produces – a larger coding change, so one that would make it into a future version 2 of GEMMA.

The Tech City Map

Posted on by
1

Last week the government launched the Tech City map. It is not the first – there have been several previous maps, e.g. by the Guardian, of technology startups in the Old Street/Shoreditch area of London, but this is the highest profile one produced yet. I’m not sure whether a map itself is useful to the community but it serves perhaps to increase the identity and branding of the area – and awareness of it for the general public.

Incidentally, the area covered is also known as the Silicon Roundabout area, it often incorrectly referred to the Westminster set as “East London” (I would more term it as North-East inner-city London). I would consider more for the area around the Olympic Park to be true East London, i.e. east of Charing Cross, north of the river and east of the City – areas such as the East End, Stratford, Barking and West Ham.

As for the cartography, there has been some criticism by some people of the mass of straight blue lines, representing retweets of one company’s message by others. I prefer to see it as a piece of “art” showing the interconnectiveness of the organisations. No one’s trying to do serious quantitative impact work with this, it’s just for visual impact and identification of a new organically-created geographical area.

It’s nice also that the custom style functionality of Google Maps v3 API is being used – giving the streets a nice blue hue rather than the very familiar (and quite distracting) regular colours on Google Maps. Of course Google has recently introduced quite low limits (2500 map loads) for free usage of its custom style functionality, I wonder if this website eventually breaching such a limit if it gets hit with another blaze of publicity?

Perhaps the most exciting news that came out of last week’s announcements was that UCL, Imperial and Cisco are collaborating to build a new Future Cities research facility in the Tech City area, which would be focusing on urban modelling and technology. This is music to my ears – I’m not sure whether UCL CASA, the lab where I work, has plans ever have a presence there but it would be an amazing place to work. Maybe there will be a CASA East in the not too distant future?

WMS and SVG Input with Mapnik and Cairo

Posted on by
Reply

A rather techy post from me, just before the Final Project Post for GEMMA (my current project at CASA) is submitted, summing the project and applications up. Why? I wanted to share with you two rather specialist bits of GEMMA that required quite a bit of head-scratching and trial-and-error, in the hope that, for some developer out there, they will be of use in their own work. I’m cross-posting this from the GEMMA project blog.

One of the core features of GEMMA that I have always desired is the ability to output the created map as a PDF. Not just any PDF, but a vector-based one – the idea that it will be razor-sharp when you print it out, rather than just looking like a screen grab. I had written a basic PDF creator, using Mapnik and Cairo, for OpenOrienteeringMap (OOM), an earlier side-project, and because the GEMMA project is about embracing and extending our existing technologies and knowledge at CASA, rather than reinventing the wheel, I was keen to utilise this code in GEMMA.

Most of the layers were quite straightforward – the two OpenStreetMap layers (background and feature) are very similar indeed to OOM, while the Markers and Data Collector layers were also reasonably easy to do – once I had imported (for the former) and hand-crafted (for the latter) suitable SVG images, so that they would stay looking nice when on the PDF. The trickiest layer was the MapTube layer. For the terms of this project, the MapTube imagery is not a vector layer, i.e. we are not using WFS. However I was still keen to include this layer in the PDF, so I turned to Richard Milton (the creator of MapTube) and discovered there is an WMS service that will stitch together the tiled images and serve them across the net. I could combine this requesting the WMS images on the GEMMA server (not the client!), converting them to temporary files, and then using a RasterSymbolizer in Mapnik 2, and an associated GDAL filetype.

The trickiest part was setting georeferencing information for the WMS image. Georeferencing is used to request the image, but it is also needed to position the image above or below the other Mapnik layers. Initially it looked like I would have to manually create a “worldfile”, but eventually I found a possibly undocumented Mapnik feature which allows manual specification of the bounding box.

I’ve not seen this done anywhere else before, although I presume people have just done it and not written it down on the web, so here’s my take, in Python.

First we get our WMS image. MAP_W and MAP_H are the size of the map area on the “sheet” in metres. We request it with a resolution of 5000 pixels per metre, which should produce a crisp looking image without stressing the server too much.

mb = map.envelope()
url = maptube_wms_path + "/?request=GetMap&service=WMS&version=1.3.0"
url = url + &format=image/png&crs=EPSG:900913"
url = url + "&width=" + str(int(MAP_W*5000)) 
url = url + "&height=" + str(int(MAP_H*5000))
url = url + "&bbox=" + str(mb.minx) + "," + str(mb.miny) + "," 
url = url + str(mb.maxx) + "," + str(mb.maxy)
url = url + "&layers=MAPID" + str(maptubeid)

furl = urllib2.urlopen(url, timeout=30)

Mapnik doesn’t work directly with images, but files, so we create a temporary file:

ftmp = tempfile.NamedTemporaryFile(suffix = '.png')
filename = ftmp.name
ftmp.write(furl.read())

Next we set up the layer and style. It’s nice that we can pass the opacity, set on the GEMMA website, straight into the layer in Mapnik. 

			
style = mapnik.Style()
rule = mapnik.Rule()
rs = mapnik.RasterSymbolizer()
rs.opacity = opacity
rule.symbols.append(rs)
style.rules.append(rule)
map.append_style(filename,style)
lyr = mapnik.Layer(filename)

Here’s the key step, where we manually provide georeferencing information. epsg900913 is the usual Proj4 string for this coordinate reference system.

lyr.datasource = mapnik.Gdal(base='',file=filename, bbox=(mb.minx, mb.miny, mb.maxx, mb.maxy)) #Override GDAL
lyr.srs = epsg900913

Finally:

lyr.styles.append(filename)
map.layers.append(lyr)

I’m excited about one other piece of code in the PDF generation process, as again it involves jumping through some hoops, that are only lightly documented – adding an SVG “logo” – the SVG in this case being the GEMMA gerbil logo, that Steve (co-developer) created from Illustrator. Cairo does not allow native SVG import (only export) but you can use the RSVG Python package to pull this in. I’m being a bit lazy in hard-coding widths and scales here, because the logo never changes. There are more sophisticated calls, e.g. svg.props.width, that could be useful.

	
svg = rsvg.Handle(gemma_path + "/images/logo.svg")
ctx = cairo.Context(surface)
ctx.translate((MAP_WM+MAP_W)*S2P-ADORN_LOGO_SIZE, CONTENT_NM*S2P)
ctx.scale(0.062, 0.062)
svg.render_cairo(ctx)

Note that we are calling render_cairo, a function in RSVG, rather than a native Cairo function that we do for all the other layers in the PDF.

The screenshot above contains data from the OpenStreetMap project.

Heatmaps are Simple with HTML 5 and Canvas

Posted on by
1

Cross-posted from my orienteering blog.

As a Saturday-lunchtime project, I have created a heatmap of where the 2700-odd geolocated orienteering races have been held in Great Britain in the last two years.

The heatmap quickly shows clusters around the main urban areas, where the population sizes supply participation for many local events being put on. Another major bright-spot is the Lake District extract on the right – which contains a large amount of high quality terrain for events. Other areas, such as the Cotswolds NW of Oxford, seem to be somewhat underused.

If you have a browser than can handle the HTML5 Canvas tag (i.e. not Internet Explorer!) you can view the heatmap here. Zoom into your local town or city to see if events have been held there – when zooming in, you’ll need to adjust the two sliders most of the way to the right, so that individual events show up. With the individual settings, a single, isolated event will have very little impact on the heatmap.

The heatmap was possible thanks to the excellent Heatmap library produced for OpenLayers by Bjoern Hoehrmann. The map is powered by OpenLayers, with an OpenStreetMap basemap. I’ve used a custom colour ramp, based on one supplied by Colorbrewer. The custom map adornments are supplied by MapBox.

Creating a heatmap like this is very easily, with just a few lines of Javascript needed to add objects in. The Heatmap library does the rest. See Bjoern’s example for the documentation.

The First Million London Bike Share Journeys

Posted on by
13

Thanks to a FOI request from Adrian Short, Transport for London have recently released to their developers area details of 1.4 million bike share journeys. The data is believed to include all the journeys between 30 July 2010 and 3 November 2010, except those starting between midnight and 6am.

I’ve created a map which visualises these journeys – select a docking station and a time, and it will show the journeys that start/end at that dock, depending on the options chosen.

You can see the map here. On launching the site, an initial docking station – one outside Waterloo station – is selected, and an “interesting” timeframe is chosen – the morning of 4 October, which was a day impacted by a tube strike.

Heavy usage along the Broad Walk through Kensington Gardens, particularly at weekends:

The predominant flows from a docking station near King’s Cross station, in weekday mornings, are outwards (red lines), particularly south towards the river. Only a few inbound journeys happen (blue lines):

The reverse is true in weekday evenings, as commuters head back to the stations:

The map bears a resemblance to my live Barclays Cycle Hire scheme status map, as I’m reusing a lot of the same code and graphics.

More Circles on a Map – Orienteering Fixtures

Posted on by
Reply

Five years ago, I created a mashup of forthcoming orienteering fixtures in Great Britain, as listed by the sport’s national governing body, British Orienteering, on its website. It was based on the Google Maps v2 API, and a regular scraping of the HTML on their website, and was a set of pins on a map, coloured by the number of weeks to the event. On clicking a pin, you got a popup balloon with details of the event, and a link to the organising club’s website. A postcode locator, based on data from the NPEMap project, was added, so you could focus on events in your local area. You could also filter out far away events.

A couple of years later, British Orienteering’s web developers added their own map to their website – Google Maps v2 API based, with pins coloured by the number of weeks to the event, and a popup balloon, a postcode search and distance filter etc etc… The Unique Selling Point of my fixtures map was lost.

So, when a rewrite of British Orienteering’s website just before Christmas broke my map, I took the opportunity to rewrite it, as a vacation project, using the technologies I’ve been using a lot in 2010 – namely OpenLayers, OpenStreetMap, OS OpenData and coloured vector circles. The map is bigger, brighter, and hopefully more useable than the official map and my previous version.

You can see the new map here – with a mass of dots representing forthcoming fixtures, and circles surrounding the “home” postcode, backed by OpenStreetMap, with the postcode locator based on CodePoint Open from Ordnance Survey OpenData. Only the locator uses a database, the rest of the webpage is constructed on-the-fly from a webpage regularly copied from the British Orienteering website.

Not Scarborough…

The map remains subject to the quality of the data entered on the corresponding list – there is some limited tidying up of the data, but it’s difficult to correct grid references that result in events being in the sea – there’s currently one in the Irish Sea, as the event registrant entered “GR” as the grid reference letters, and this just so happens to be the location of the GR myriad. There is still work to be done on my new map, such as spotting obvious errors like this, guessing locations where a grid reference isn’t supplied, and perhaps including Northern Ireland’s events.

Incidentally, my original orienteering web map, which inspired my fixtures map, was one showing orienteering maps, it was written way back in August 2004, using a Flash mapping package by Map Bureau, with dots superimposed on top of a map pinched from Wikipedia. We’ve come a long way.

Mapping our Tweeting MPs with MapTube

Posted on by
2

One of the geometries available in MapTube shows the 2010 parliamentary constituencies. I’ve used to create a map, based on the list of MPs’ Twitter pages listed at Tweetminster and updated to reflect the boundary changes earlier this year. The map is clickable – choose a blue constituency, i.e. one which has a tweeting MP, and click on it to get a direct link to their Twitter page.

There’s more MPs on Twitter than you might think – over 200 currently, which is roughly one-third of all the MPs currently in parliament.

The map is OK on its own, but it’s real power comes when you can combine it with another political one, using MapTube’s map search functionality. You have to be careful when combining maps together, as maps with lots of colours will quickly become tricky to interpret when the colours, with varying translucencies, are laid on top of each other. However, with simple maps, it works quite well. There’s currently one showing which MPs have agreed to back an Early Day Motion banning Wild Animals in Circuses.

If you overlay both these maps together, you can interpolate the colours to quickly identify which MPs are on Twitter but (for example) have not agreed to back the motion, and then tweet them! In this example, the “haven’t signed” colour is red, and the “has Twitter” colour is blue, so the “target” list is purple.

The map doesn’t include Northern Ireland as we don’t have the constituency boundary geometries for those.