I can’t believe it’s #IronViz Feeder season already. It seems like just yesterday I was watching Curtis Harris crush it in Austin with NYC taxi cycle plots!
For those that don’t know, #IronViz is a contest at Tableau Conference where three contestants viz it out on stage building amazing visualizations in just 20 minutes. There are three feeder competitions to decide who gets to compete on stage.
Usually, there is a topic for each feeder contest, but the last two feeders have been feature specific. I am not a fan of this trend and hope the contest returns to topics again. I think making it feature specific limits who can or will participate.
This feeder is centered around the geospatial connector in Tableau 10.2 which allows for connections to shp and kml files. I love maps so I immediately started downloading shp files of all kinds of topics.
I looked at tornadoes, wind storms, bike data, fault lines, Chernobyl radiation exclusion zone, jack the ripper victims, ebola outbreaks, Jon Snow cholera data, Hurricane Katrina, Scotch distilleries, timezones, restricted fly zones, the Roman empire, shark tracking etc. You get the picture…I was all over the place looking for the right dataset.
Unfortunately, I got this error message a lot.
I had actually dm’d Kent Marten at Tableau (before the topic was announced) and he said lines were not supported yet, but they are on the list of things to add. He did mention I could try and buffer the lines in QGIS. I had never done this before, but it would prove to be helpful advice in the end.
I buffered the lines in a tornado shapefile and joined it with mobile home percentages by county (both files from NOAA). I thought this looked pretty cool so I ran with this topic for awhile…
I showed it to my favorite zen master, Pooja, and to recent IronViz champ, Curtis. There was one ‘yea’ and one ‘nay’ from my crack peer review team. Without naming anyone, Pooja, I’ll say one person said it’s not very IronVizzy. I took the feedback to heart and moved on.
I ended up finding a bunch of bathymetry layers from Natural Earth Data. The term “bathymetry” originally referred to the ocean’s depth relative to sea level, although it has come to mean “submarine topography,” or the depths and shapes of underwater terrain. ~ NOAA
I merged all these layers in QGIS and added a new field called Layer (learned some new tricks) so I could use this dimension later on in Tableau. I built this map…
Now, I thought this was awesome, but it wasn’t a story. I was hoping I could find some shark tracking data somewhere and join it to this, but my google skills failed me miserably. However, my search terms of ‘bathymetry shapefiles and shark’ led me to John Nelson’s blog, Adventures in Mapping.
You bet your ass I clicked on “Sharknado Map”…
Did someone say Sharknado? Surely, it isn’t based off the movie…
John Nelson created this beauty of a visualization that includes bathymetry layers in the ocean, coastal areas for great white and hammerhead sharks in 150, 200 and 250 mi buffers, NOAA hurricane data, hexagons to highlight concentration areas, custom shapes, and some pretty funny annotations.
This is really amazing work. I was enamored with this for a few reasons: 1) I was looking for shark data and already using the bathymetry layers; 2) because it it IronViz time and let’s face it, this is just badass; 3) I love maps, but am a relative newbie to QGIS; and 4) because at a glance it looked impossible to do in Tableau. John created this using ArcGIS and his blog post has links to his entire process and most of the files he used.
I wanted to challenge myself to build this in Tableau, using QGIS for anything GIS related, and to make it interactive. I thought I could learn a lot while trying to recreate this with different tools.
I pretty quickly figured out I needed to merge/modify the shapefiles so I could get them all into Tableau. Tableau maps only allow for a dual axis (I can’t wait for the additional layers we saw at TC16) so I determined I needed to merge all my polygon shapefiles together and all my point shapefiles together. In total, there are 20 layers in this map. There are 17 polygon layers (11 Bathymetry, 5 Coastal Areas and the Hexagons) and 3 point layers (category 3, 4 and 5 hurricanes). After merging the files in QGIS, I added a new field called Layer so I would have a single dimension I could use for sorting and color in Tableau.
The hexagons were a bit of a challenge. I even contacted John to ask permission to recreate the viz and for a little direction on the hexes. He told me to try the MMQGIS plugin in QGIS to create the hexagons and encouraged me to use these techniques on future visualizations (I can’t wait).
I created the 3,000+ hexagons successfully, but they covered the entire layer…
I manually went through the attribute table and clipped the hexagons that coincided with the coastal areas and hurricane layers. This is a photo I sent Pooja while complaining a little. You can see I got the Pacific Coast cut out…only 2,000+ more to go!
This grid layer in QGIS is a polygon layer. They appear transparent because I set the fill to transparent in QGIS. This would not work for me in Tableau. I was going to put the hurricane points on top of the polygon layers. This means the solid hexagons in tableau would obscure the hurricane data.
I was able to load just this layer into Tableau and make it appear transparent, by dialing opacity to 0% and selecting a border, but…
I can’t dial the opacity to 0% on my polygon shapefile because all the other layers would be invisible.
Notice Curtis is DMing me back while I am complaining again.
So back to QGIS. After I clipped the hexagon layer, I added a row id field so they would have a unique id for use in tableau. Then, I converted the polygons to lines, followed by buffering the lines back into polygons, but with the centers cut out. This would allow for users to hover on the points underneath the hexagon.
That was a fun problem to solve and I have more new tricks. Hat tips to John for the plugin and Kent for the line buffer.
Since this is a geospatial contest, I added another element to the original visualization. I loaded a populated places dataset from Natural Earth Data into QGIS along with the hurricane points. I used the NNJOIN plugin to determine the nearest neighbor (closest City, State, and Country) to each point of the hurricane. This data was added to the hurricane tooltips.
I did run into a hiccup when trying to extract the data for Tableau Public. I had originally done a cartesian join that resulted in about 34 Million records and Tableau Public will only allow 15 Million to be uploaded.
I ended up pasting in two rows of data and doing a join on a calculation. This cut the data from 34 Million records to 12 Thousand!!!!!!
The rest was a bit of formatting and dashboarding tricks.
- I colored both axes based on the new Layer field.
- I used 40% opacity on the polygons to allow all the layers to bleed together – this is also what makes the orange coastal border where great whites and hammerheads live.
- I used 20% opacity on the hurricane points to allow the hexagons that are underneath them to show through.
- I created the custom legends using icons from John’s blog and from one of the hexagons. I used some background image tricks to make the legends appear transparent.
- I created custom multi-line tooltips for all the layers using a few calculation tricks.
- I used hover actions on all the legends to highlight the map.
- The lines connecting the map to the text boxes are blank annotations with line formatting. I couldn’t just drop some PNGs on there because the points underneath would not be interactive.
- I created a sources sheet with some information and links in the tooltips.
Wrap It Up Already
In the end, I think I was able to create a strikingly-similar visualization to the original with some enhancements. The map is now fully interactive, I added a nearest neighbor algorithm, centered the map, added a hex legend, dropped an annotation from the original, lightened some colors, etc. Again, my goal was not to copy the original, but instead to use different tools and push the limits of what is possible in Tableau.
I want to thank John Nelson again for allowing me to recreate his visualization. I learned a lot from this exercise.
Thank you to Pooja Gandhi and Curtis Harris for peer reviews. You two rock!
Thanks to Rody Zakovich for being a sounding board when I screwed up my join!
Unfortunately, the workbook did not work on Tableau Public, I got this error before it renders the dashboard. This was also tested on Tableau Server 10.2 with the same result.
I opened a support case and it looks like whatever the issue is will be fixed in 10.2.1, but Tableau was able to hook me up with a twb hack to get it on public!
Maybe this is a good reason not to center a contest around a new feature?
Regardless, thanks Kent and the hardworking devs for the work-around!
Below is a GIF of the highlight actions. There is quite a bit more to the workbook though.