D3coder

Importing JSON Data into D3

EvanZ — Mon, 15 Oct 2012 23:38:02 +0000

I just wanted to write a quick post showing a relatively simple example of importing and visualizing data stored in JSON format. I’m using shot location data from a basketball game between the Golden State Warriors and Denver Nuggets that took place recently. I wrote a Ruby scraper which pulls the shot data from ESPN (using the Nokogiri gem), and simply pipes it to a JSON data file (using the json gem).

The JSON data looks like…

[{"id":"4004005102","player_name":"Andre  Iguodala","desc":"Made 17ft jumper 11:37 in 1st Qtr.","player_id":"2386","quarter":"1","x":"42","y":"86","made":"true","home":"h"},{"id":"4004005103","player_name":"Stephen Curry","desc":"Miss 22ft jumper 11:19 in 1st Qtr.","player_id":"3975","quarter":"1","x":"37","y":"25","made":"false","home":"a"},...

Result

The "h" and "a" stand for home and away, respectively. Made shots are colored blue and missed shots are red. If you hover over a shot, a tooltip will give you some more detail about the shooter and time of the game.

Method

First define the svg drawing area:

var screenHeight = 400,
	screenWidth = 800,
	vis = d3.select("#chart").append("svg")
    .attr("width", screenWidth)
    .attr("height", screenHeight),
    padding = 20;

Next, call the function to import the data and set up a couple of scales to transform the raw (x,y) coordinates (given in ft) to screen coordinates:


d3.json("shots.json", function(shots) {
	var scaleScreenX = d3.scale.linear()
				.domain([0,100])
				.range([padding,screenWidth-padding]),
		scaleScreenY = d3.scale.linear()
				.domain([0,50])
				.range([padding,screenHeight-padding]);

Draw a filled circle for each shot by binding the data to svg elements:


	vis.selectAll("circle")
		.data(shots)
		.enter()
		.append("circle")
		.attr("cx", function(d) {
        	return scaleScreenY(d.y)+5*(Math.random()-0.5);
   			})
   		.attr("cy", function(d) {
        	return scaleScreenX(d.x)+5*(Math.random()-0.5);
   			})
   		.attr("r", function() {return 2*Math.random()+6;})
   		.attr("fill", function(d) { return d.desc.match(/Made/) ? "steelblue" : "red" })
   		.attr("opacity",0.60)
   		.attr("stroke-width", 2)
   		.attr("stroke","black")
   		.append("svg:title")
   		.text(function(d) { return d.player_name+" "+d.desc; });

And do the same thing for adding the text labels to each shot:


   	vis.selectAll("text")
   		.data(shots)
  	 	.enter()
   		.append("text")
   		.text(function(d) {
       		 return d.home;
  			 })
  		.attr("x", function(d) {
			return scaleScreenY(d.y)-2;
		   })
	    .attr("y", function(d) {
			return scaleScreenX(d.x) + 3;
	   		})
	   	.attr("font-family", "serif")
   		.attr("font-size", "12px")
   		.attr("fill", "black");
});

That’s it, really. In the future, I need to overlay the shots onto the background of an actual basketball court. Also, I’d like to add some filters for selecting games, players, teams, seasons, etc, and maybe add some whiz-bang D3 transitions in there as well.

Drawing Chemical Structures with Force Layout

EvanZ — Sun, 01 Jul 2012 21:28:24 +0000

I’m trying to learn how to use the different layouts in D3, so I can start making more sophisticated visualizations. The force layout lets you specify an array of nodes and links between those nodes, and the nature of the physical interaction between the nodes. I realized a few hours into playing around with it, that this could be a great toolbox for visualizing chemical structures. I’ll show you how in this post.

Let’s draw Alanine! Alanine is the simplest amino acid, consisting of 3 carbons, 1 nitrogen, 2 oxygens, and 7 hydrogen atoms. Try clicking on an atom and dragging it!

Result

Method

To start, we need to define an array of nodes:

			var nodes = [
				{"atom":"C", "size":12},
				{"atom":"C", "size":12},
				{"atom":"C", "size":12},
				{"atom":"N", "size":7},
				{"atom":"H", "size":1},
				{"atom":"O", "size":16},
				{"atom":"O", "size":16},
				{"atom":"H", "size":1},
				{"atom":"H", "size":1},
				{"atom":"H", "size":1},
				{"atom":"H", "size":1},
				{"atom":"H", "size":1},
				{"atom":"H", "size":1} 
			],

Each object in the array represents one of the atoms. We’ll use the “atom” key later to draw the atomic symbol, and we’ll use the “size” key to vary the size of the node, which will be represented by a circle. An array of links is needed to connect the atoms:



			links = [
				{"source":0, "target":1},
				{"source":1, "target":2},
				{"source":1, "target":3},
				{"source":2, "target":5},
				{"source":2, "target":6},
				{"source":1, "target":4},
				{"source":3, "target":10},
				{"source":3, "target":11},
				{"source":0, "target":7},
				{"source":0, "target":8},
				{"source":0, "target":9},
				{"source":5, "target":12}
			],

Next we start up the force layout, bring in the nodes and links, and set a few of the “physical” parameters:


			force = d3.layout.force()
				.nodes(nodes)
				.links(links)
				.charge(-1500)
				.friction(0.8)
				.gravity(0.5)
				.size([w,h])
				.start(),

Initialize the visualization elements for the links (“bonds”):


			link = vis.selectAll("line")
				.data(links)
				.enter()
					.append("line")
					.attr("class","link"),

And do the same for the nodes (“atoms”):


			node = vis.selectAll(".node")
				.data(nodes)
				.enter()
					.append("g")
					.attr("class","node")
					.call(force.drag); //allows you to drag atoms
			
				node.append("circle")
					.attr("r", function(d) {
						return Math.pow(40*d.size,1/3);
						})
					.attr("fill",function(d) {
						return fill(d.size);
						})
					.attr("stroke","black")
					.attr("stroke-width",2);
					
				node.append("text")
					.attr("dx",function(d) {
						return Math.pow(40*d.size,1/3)+1;
						})
					.attr("dy",".35em")
					.text(function(d) {return d.atom;});

At each “tick” of the simulation, the following function updates the viz:


			force.on("tick", function() {
				link.attr("x1", function(d) { return d.source.x; })
					.attr("y1", function(d) { return d.source.y; })
					.attr("x2", function(d) { return d.target.x; })
					.attr("y2", function(d) { return d.target.y; });
					
				node.attr("transform", function(d) { 
					return "translate(" + d.x + "," + d.y + ")"; 
					});		
				});

This is a pretty good start for a chemical visualization library. I need to figure out how to implement double and triple bonds, and also figure out how to simulate different bond strengths.

Full Code

      
		Chemical structure visualized using D3.
	
            view raw
            gistfile1.html
            hosted with ❤ by GitHub

My First Chart

EvanZ — Sun, 17 Jun 2012 23:24:18 +0000

Let’s make a chart! I’m going to try to replicate a chart from Golden State of Mind where we were voting on players for a NBA draft board (fear not, you won’t have to know what the heck that means to follow along here). Here’s a snapshot of it:

Snapshot of poll results that we’re trying to replicate.

The Result

Here’s my version followed by the code to implement it:

The Method

First, we define our dataset, which is an array of objects:

var players = [
		{"First": "Harrison", "Last": "Barnes", "Votes": 61},
		{"First":"Andre", "Last":"Drummond", "Votes":27},
		{"First":"Bradley", "Last":"Beal", "Votes":"58"},
		{"First":"Meyers", "Last":"Leonard", "Votes":8},
		{"First":"Jared", "Last":"Sullinger", "Votes":8},
		{"First":"Damian", "Last":"Lillard", "Votes":5},
		{"First":"Terrence", "Last":"Jones", "Votes":2},
		{"First":"Austin", "Last":"Rivers", "Votes":2},
		{"First":"Perry", "Last":"Jones", "Votes":2},
		{"First":"Others","Last":"","Votes":6}
		],

There are surely many different ways to read in the data, but that one made sense to me. Now we’ll define some other variables that will help us make the chart later:

    maxVotes = Math.max.apply(Math,players.map(function(o){return o.Votes;})),
    minVotes = Math.min.apply(Math,players.map(function(o){return o.Votes;})),
    numVotes = players.reduce(function(a,b) {return a+parseInt(b.Votes);},0),
    xOffset = 50,
    numPlayers = players.length,
    svgHeight = numPlayers*55,
    svgWidth = maxVotes + 200,
    yOffset = svgHeight/(numPlayers+1);

We’ll sort the array by votes in descending order (actually this is an improvement over the above chart):

    function sorter(a,b) {
	 return b.Votes - a.Votes;
	 }
    players.sort(sorter);

Next we insert the svg and draw a border around it:

    d3.select("body").append("svg")
        .attr("width",svgWidth)
        .attr("height",svgHeight);
    d3.select("svg").append("rect")
        .attr("x",0)
        .attr("y",0)
        .attr("width",svgWidth)
        .attr("height",svgHeight)
        .attr("style","fill-opacity:0; stroke:black;stroke-width:6px");

Then we draw the bars:

    d3.select("svg")
        .append("g").attr("class","bars")
        .selectAll("rect")
        .data(players)
        .enter()
            .append("rect")
                .attr("x",xOffset)
                .attr("style","fill:gold; stroke:black")
                .attr("y", function(d,i) {
                    return yOffset*i+50;
                    })
                .attr("width", function(d,i) {
                    return  d.Votes*2;
                    })
                .attr("height", 20);

Followed by the labels above each bar containing player names:

    d3.select("svg")
        .append("g").attr("class","names")
        .selectAll("text")
        .data(players)
        .enter()
            .append("text")
                .attr("x", xOffset)
                .attr("y", function(d,i) {
                    return yOffset*i+45;
                    })
                .attr("style","font-family: sans-serif; font-size: 12pt")
                .text(function(d) {return d.First+" "+d.Last;});

The number of votes each player received is placed to the right of the bars (I think an improvement in readability over the original):

    d3.select("svg")
        .append("g").attr("class","votes")
        .selectAll("text")
        .data(players)
        .enter()
            .append("text")
                .attr("x", function(d,i) {
                    return xOffset+d.Votes*2+5;
                    })
                .attr("y", function(d,i) {
                    return yOffset*i+45+20;
                    })
                .attr("style","font-family: sans-serif; font-size: 10pt")
                .text(function(d) {return parseInt(d.Votes) + " votes";});

And the % of total votes each player received is placed to the left of the bars:

    d3.select("svg")
        .append("g").attr("class","percent")
        .selectAll("text")
        .data(players)
        .enter()
            .append("text")
                .attr("x", 10)
                .attr("y", function(d,i) {
                    return yOffset*i+45+20;
                    })
                .attr("style","font-family: sans-serif; font-size: 10pt")
                .text(function(d) {
                    return d3.format(".2p")(d.Votes/numVotes);
                    });

Finally, I put the total number of votes below the chart:

    d3.select("body").append("div")
        .text(numVotes+" votes");

Code

      
            view raw
            index.html
            hosted with ❤ by GitHub

Getting Started with Data Joins

EvanZ — Sat, 16 Jun 2012 07:27:23 +0000

To begin to use d3 effectively, I realize I need to have a very good grasp of how data joins work. The idea of the “data join” is essentially what it sounds like: the act of joining (or binding) data to DOM elements. Here I’ll demonstrate a few things I’ve picked up so far. You can follow along here by creating an empty HTML document, loading it with Chrome, and using the JavaScript console to type the commands and see the results.

First, let’s create an array containing some data. In this case, a set of names:

> var names = ["Sam", "Diane", "Norm", "Cliff"]

Now we create an empty table row that we can populate with these names:

> d3.select("body").append("table").append("tr")

This should produce an empty table containing one (empty) row in the html:
...

...

Before joining data to the table the “correct” way, I think it’s instructive to first see what happens when we try to do it incorrectly. Here’s one way that fails:

> d3.select("tr").append("td").data(names).text(function(d) {return d;})

This will result in only one column, having joined just the first element of the array, and the output in your browser will look like:

Sam

Interestingly, if you run the same command a second time, you will simply get:

Sam Sam

We can start over from scratch to get it right, but even from here we can fix it:

> d3.select("tr").selectAll("td").data(names).text(function(d) {return d;})

This gives:

Sam Diane

Since we had already created two td elements, the last command joined the first two elements of the names array to those elements. This is an update according to the terminology of d3. However, we still want to get those other two names in the table. To do this, we must use the enter() function and append two columns:

> d3.select("tr").selectAll("td").data(names).enter().append("td").text(function(d) {return d;})

Sam Diane Norm Cliff

What this command did was figure out that there were 4 elements in the data array, but only two td elements to join, so it put the “extra” two elements (Norm and Cliff) into the enter array. It then appended two new td elements and bound Norm and Cliff to those.

You might think it also re-wrote the first two names, but that is not the case. It’s an important point, so let’s extend the example further. From here, let’s shift the names in the array and add one more:

> names.shift(1); > names.push("Sam"); > names.push("Carla"); > names.push("Woody");

Now names = ["Diane", "Norm", "Cliff", "Sam", "Carla", "Woody"].

Let’s see what happens when we re-run the command from before:

> d3.select("tr").selectAll("td").data(names).enter().append("td").text(function(d) {return d;})

Sam Diane Norm Cliff Carla Woody

Ah, notice that while the last two (“entering”) elements have been added to the table, the first four elements did not change. To change those elements, we must run an update:

> d3.select("tr").selectAll("td").data(names).text(function(d) {return d;})

Diane Norm Cliff Sam Carla Woody

Note that an update is simply the previous command minus the enter() and append() functions.

So how do we delete elements? Let’s get rid of Diane (don’t worry, in time you’ll get over it).

> names = names.slice(1)

["Norm", "Cliff", "Sam", "Carla", "Woody"]

We can run the update first and then remove the extra element or vice-versa. Let’s run the update first:

> d3.select("tr").selectAll("td").data(names).text(function(d) {return d;})

Norm Cliff Sam Carla Woody Woody

To get rid of the extra Woody, we use the exit() function:

> d3.selectAll("td").data(names).exit().remove()

Norm Cliff Sam Carla Woody

Voila! Just to show one more piece of functionality, be aware that you can use the data() function to simply return the current data join:

> d3.selectAll("td").data()

> ["Norm", "Cliff", "Sam", "Carla", "Woody"]

This seems like a good place to stop for now. There is a lot more to the data function that I haven’t quite wrapped my head around yet, namely, the second argument (not used here) which creates a key function to replace the default one which simply uses the array index. In the meantime, I hope this post has been useful to you in some way.

A Simple Beginning

EvanZ — Sat, 19 May 2012 20:36:19 +0000

These are my first baby steps in D3, a new(ish) JavaScript library that was developed by Mike Bostock (i.e. a data science guru) for creating data visualizations. In fact, D3 stands for “Data Driven Documents”, and as the name suggests, it’s really a general purpose framework for doing pretty much anything one could want to do with data these days. I’m not only new to D3, but also to JS, which I am forcing myself to learn so I can take advantage of D3 (having seen the coolness of it, I couldn’t let something as small as not knowing JS hold me back).

There doesn’t appear to be a D3 book yet, so one has to go about finding tutorials in various places. There are plenty of examples going around, though. I found this tutorial from Scott Murray (@alignedleft) particularly valuable.

As I learn D3, I intend to post from time to time during each stage of my development. Hopefully, progress will be evident in both my understanding of D3 and JavaScript. Here, I just wanted to get going by doing a very simple animation of a “gun” “shooting” a “bullet”. To make it just a little bit more interesting, I added a random component which changes how far the bullet goes each time. Click on the “Shoot!” button to refresh the page which will re-load (i.e. restart) the animation.

To simulate recoil in the gun, I used the .ease("bounce") function, and to simulate the bullet slowing down, I used .ease("exp-out"). I tried to make it somewhat realistic by multiplying the recoil distance and bullet travel by the same random factor (factor = Math.random()). Everything else should be fairly self-explanatory. One thing I like about D3 is that it almost seems to document itself (how meta!).

Simulation of gun firing bullet

HTML/JS Code

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53


  
    
  
  
   id="chart">
   id="button">
  	 type="button" id="Button1" value="Shoot!" onclick="history.go(0)"/>

view raw gistfile1.html hosted with ❤ by GitHub