D3.js - D3.js Tutorial

D3.js (or just D3 for Data-Driven Documents) is a JavaScript library for producing dynamic, interactive data visualizations in web browsers. It makes use of the widely implemented SVG, HTML5, and CSS standards. It is the successor to the earlier Protovis framework. In contrast to many other libraries, D3.js allows great control over the final visual result. Its development was noted in 2011, as version 2.0.0 was released in August 2011.

Data visualization company Datameer officially uses D3.js as its core technology, while The New York Times sometimes uses it for rich graphs. It is used by the iD editor for editing OpenStreetMap. D3.js has been extensively used for GIS map making, managing both GeoJSON and Topojson files as input.

Context

The first web browsers appeared in the early 1990s. They were initially capable of displaying static web pages only: the only way for a user to interact with the web was through clicking links and scrolling pages. There were many efforts to overcome such limitations. One of the most significant was the integration of JavaScript as the scripting language for web browsers. JavaScript gradually became the de facto standard language for creating web pages with rich user interactivity. This played a crucial role in the decision to use JavaScript as the language of D3.js.

At the same time, researchers, engineers, and practitioners from various branches of engineering and science looked for tools that would enable web browsers to visually present data within web pages. There were multiple projects with that goal, each of which had its successes and failures, and inspired the subsequent ones. The most notable examples were the Prefuse, Flare, and Protovis toolkits, which can all be considered as direct predecessors of D3.js.

Prefuse was a visualization toolkit created in 2005 that required usage of Java, and visualizations were rendered within browsers with a Java plug-in. Flare was a similar toolkit from 2007 that used ActionScript, and required a Flash plug-in for rendering.

In 2009, based on the experience of developing and utilizing Prefuse and Flare, Professor Jeff Heer, Ph.D. student Mike Bostock, and M.S. student Vadim Ogievetsky of Stanford University's Stanford Visualization Group created Protovis, a JavaScript library to generate SVG graphics from data. The library received notable acceptance both by data visualization practitioners and academics.

In 2011, the development of Protovis was stopped to focus on a new project, D3.js. Informed by experiences with Protovis, Bostock, along with Heer and Ogievetsky, developed D3.js to provide a more expressive framework that, at the same time, focuses on web standards and provides improved performance.

Technical principles

Embedded within an HTML webpage, the JavaScript D3.js library uses pre-built JavaScript functions to select elements, create SVG objects, style them, or add transitions, dynamic effects or tooltips to them. These objects can also be widely styled using CSS. Large datasets can be easily bound to SVG objects using simple D3.js functions to generate rich text/graphic charts and diagrams. The data can be in various formats, most commonly JSON, comma-separated values (CSV) or geoJSON, but, if required, JavaScript functions can be written to read other data formats.

Selections

The central principle of D3.js design is to enable the programmer to first use a CSS-style selector to select a given set of Document Object Model (DOM) nodes, then use operators to manipulate them in a similar manner to jQuery. For example, by using D3.js, one may select all HTML <p>...</p> elements, and then change their text color, e.g. to lavender:

The selection can be based on tag (as in the above example), class, identifier, attribute, or place in the hierarchy. Once elements are selected, one can apply operations to them. This includes getting and setting attributes, display texts, and styles (as in the above example). Elements may also be added and removed. This process of modifying, creating and removing HTML elements can be made dependent on data, which is the basic concept of D3.js.

Transitions

By declaring a transition, values for attributes and styles can be smoothly interpolated over a certain time. The following code will make all HTML <p>...</p> elements on a page gradually change their text color to pink:

Data-binding

For more advanced uses, loaded data drives the creation of elements. D3.js loads a given dataset, then, for each of its elements, creates an SVG object with associated properties (shape, colors, values) and behaviors (transitions, events).

Generated SVG graphics are designed according to the provided data.

Appending nodes using data

Once a dataset is bound to a document, use of D3.js typically follows a pattern wherein an explicit .enter() function, an implicit "update," and an explicit .exit() function is invoked for each item in the bound dataset. Any methods chained after the .enter() command will be called for each item in the dataset not already represented by a DOM node in the selection (the previous selectAll()). Likewise, the implicit update function is called on all existing selected nodes for which there is a corresponding item in the dataset, and .exit() is called on all existing selected nodes that do not have an item in the dataset to bind to them. The D3.js documentation provides several examples of how this works.

API structure

D3.js API contains several hundred functions, and they can be grouped into following logical units:

• Selections
• Transitions
• Arrays
• Math
• Color
• Scales
• SVG
• Time
• Layouts
• Geography
• Geometry
• Behaviors

Maths

• Generation of pseudorandom numbers with normal, log-normal, Bates, and Irwin-Hall distributions.
• Transformations in 2D: translation, rotation, skew, and scaling.

Arrays

D3.js array operations are built to complement existing array support in JavaScript (mutator methods: sort, reverse, splice, shift and unshift; accessor methods: concat, join, slice, indexOf and lastIndexOf; iteration methods: filter, every, forEach, map, some, reduce and reduceRight). D3.js extends this functionality with:

• Functions for finding minimum, maximum, extent, sum, mean, median, and quantile of an array.
• Functions for ordering, shuffling, permuting, merging, and bisecting arrays.
• Functions for nesting arrays.
• Functions for manipulating associative arrays.
• Support for map and set collections.

Geometry

• Computing convex hull of a set of points.
• Computing Voronoi tesselation of a set of points.
• Support for point quadtree data structure.
• Support for basic operations on polygon.

Color

• Support for RGB, HSL, HCL, and L*a*b* color representation.
• Brightening, darkening, and interpolation of colors.

Similar tools and libraries

• AnyChart
• Datacopia
• gnuplot
• Matlab
• Matplotlib
• Plotly
• Processing
• Qlik
• R
• Raphaël
• SAS
• Tableau Software
• Zoomdata

References

Further reading

Background on data visualization and information graphics:

• Kirk, Andy (December 2012), Data Visualization: a successful design process (1st ed.), Birmingham: Packt Publishing, ISBN 978-1-84969-346-2 
• Cairo, Alberto (August 2012), The Functional Art (1st ed.), Berkeley: Peachpit, ISBN 978-0321834737 

Background on D3.js itself:

• Bostock, Michael; Ogievetsky, Vadim; Heer, Jeffrey (October 2011), "D3: Data-Driven Documents", IEEE Transactions on Visualization and Computer Graphics, IEEE Press, 17 (12): 2301â€"2309, doi:10.1109/TVCG.2011.185, PMID 22034350 

Using D3.js - beginner level:

• Murray, Scott (March 2013), Interactive Data Visualization for the Web, An Introduction to Designing with D3 (1st ed.), Sebastopol, California: O’Reilly Media, ISBN 978-1-4493-3973-9 
• Timms, Simon (September 2013), Social Data Visualization with HTML5 and JavaScript (1st ed.), Birmingham: Packt Publishing, ISBN 978-1-7821-6654-2 

Using D3.js - intermediate level:

• Dewar, Mike (June 2012), Steele, Julie; Blanchette, Meghan, eds., Getting Started with D3, Creating Data-Driven Documents (1st ed.), Sebastopol, California: O’Reilly Media, ISBN 978-1-4493-2879-5 
• Qi Zhu, Nick (October 2013), Data Visualization with D3.js Cookbook (1st ed.), Birmingham: Packt Publishing, ISBN 978-1-7821-6216-2 

Others

• Newton, Thomas; Villareal, Oscar (2014). Learning D3.js Mapping. Packt Publishing. p. 126. ISBN 9781783985609. 
• Navarro Castillo, Pablo (2014), Mastering D3.js, Birmingham: Packt Publishing, p. 352, ISBN 9781783286270 
• Teller, Swizec (2013), Data Visualization with d3.js, Birmingham: Packt Publishing, p. 194, ISBN 9781782160007 
• Newton, Thomas; Villarreal, Oscar (2014), Learning D3.js Mapping, Birmingham: Packt Publishing, p. 126, ISBN 9781783985609 
• Viau, Christophe (2013), Developing a D3.js Edge: Constructing Reusable D3 Components and Charts, Bleeding Edge Press, p. 268, ISBN 9781939902023 
• Meeks, Elijah (2014), D3.js in Action, Manning Publications, p. 325, ISBN 9781617292118 
• Maclean, Malcolm (2014), D3 Tips and Tricks, Leanpub, p. 580 
• King, Ritchie (2014), Visual Storytelling with D3: An Introduction to Data Visualization in JavaScript, Addison-Wesley Data & Analytics Series, p. 288 

Videos

• Gopal, Nikhil (October 2014), D3 and CoffeeScript: A Python Programmer's Introduction to Web Visualizations, Sebastopol, California: O’Reilly Media 
• King, Ritchie (December 2014), D3 Visualization LiveLessons: An Introduction to Data Visualization in JavaScript, Addison-Wesley Professional 

External links

• Official website
• D3.js Gallery
• Blocksplorer, search for blocks by methods used