25 Transforms

How is a measurement distributed across your rows? A transform derives a new mark from one that already exists. It modifies how data reaches the mark, preprocessing it statistically before any pixel is drawn.

The family has other names. ggplot2 says stat, and the chapter Wilkinson devotes to it says statistics (Wilkinson, 2005); SQL says aggregate, dplyr says summarize, a spreadsheet says pivot. Each of those covers about half of what gog groups here. mean, count and density really are statistics, and they replace rows with a summary. But bin cuts an axis into cells, bounds reshapes two columns you already have and computes nothing at all, and stack, dodge, jitter and repel are about collisions between marks rather than about the values behind them. Calling all of that a statistic would misname most of it, so the word here is the most general one that is still accurate: a transform is anything that changes what reaches the mark.

Transforms are applied with the * operator:

mark * transform

The result is a derived mark type, not a new atom to memorize.

Transform	What it does
`bin`	cuts a continuous axis into equal-width cells, and counts the rows in each
`smooth`	fits a LOESS curve through the `(x, y)` pairs
`count`	counts the rows at each distinct `x`
`density`	estimates the distribution of `x` with a Gaussian kernel
`proportion`	divides the measurement by its total, so the shares sum to 1
`sum`	totals `y` within each `x`
`mean`	averages `y` within each `x`
`median`	the middle `y` within each `x`
`max`	the largest `y` within each `x`
`min`	the smallest `y` within each `x`
`quantile`	the `y` at one probability within each `x`, `quantile(0.9)`
`range`	a band of `y` per group, as a pair; the whole group, or two quantiles you name
`confidence`	the mean’s confidence interval per group: low, high and center
`deviation`	the spread of `y` per group: the mean plus and minus a standard deviation

The middle six (sum, mean, median, max, min, quantile) form the aggregation family. All share the same pattern: group by x, reduce y to one value. range, confidence and deviation group the same way but write a low and a high, the extents an interval spans.

25.1 One variable, or two?

Fourteen of the transforms fall into two families by how many columns they read, and that number is exactly what decides whether you write y(). The rest of the kernel’s twenty-three come later in this chapter: bounds, which reshapes rather than computes; partition, flow, layout and cluster, which each compute a whole picture; and the four collision modifiers.

Inventing transforms (bin, count, density) read a single variable and make the measured axis from it. proportion joins them whenever it stands alone, because the only measurement it then has to rescale is the tally it makes itself. You name only the variable whose shape you are asking about, and the transform invents the y column and labels it (“Count”, “Proportion”, “Density”). bar * bin + x(life) and line * density + x(life) are the same grammatical shape: one name in, a new axis out. The engine is not failing to notice a missing y(). There is exactly one sensible y here, the count or the density, so it fills that in.

Reading transforms (smooth, sum, mean, median, max, min, quantile, range, confidence, deviation) read two columns and reshape the second against the first. They fit or aggregate an existing y, so a y() must be in scope: bar * mean + x(country) + y(gold) has no way to guess which column to average. Leave y() off and the engine invents nothing: only you know which column was meant, so it stops and names what is missing (add y(<column>)). range, confidence and deviation are the three that read a y like the others but write more than one: range a low and a high, the other two a low, high and center, the extents an interval spans.

So y()’s presence is not a quirk to memorize per transform; the family a transform belongs to decides it. This is one instance of Explicit Over Implicit: the short form is allowed exactly when it cannot be misread, and refused when it could.

A statistic runs within each group when color (or group) is bound. bar * bin + color(species) is one histogram per species, see overlaid histograms, not one combined bar that has discarded the split; and the same rule gives a density curve, a count, or a mean per group. Each output keeps its group, so the mark colors it and the legend agrees.

25.2 `bin`: histogram

The first question to ask of one continuous column is how its values are spread. Where are they dense, where are they sparse, and is there one peak or two?

bin is the first inventing transform. It cuts x into equal-width bins and counts the rows in each. Combine it with bar to draw a histogram:

`gapminder_2007`: first 5 of 142 rows
country	continent	year	life	population	gdp
Afghanistan	Asia	2007	43.828	31889923	974.5803
Albania	Europe	2007	76.423	3600523	5937.0295
Algeria	Africa	2007	72.301	33333216	6223.3675
Angola	Africa	2007	42.731	12420476	4797.2313
Argentina	Americas	2007	75.320	40301927	12779.3796

data(gapminder_2007) + bar * bin + x(life) +
  x_label("Life expectancy (years)") +
  title("Histogram of Life Expectancy, 2007")

(data(gapminder_2007) + bar * bin + x(col.life) +
  x_label("Life expectancy (years)") +
  title("Histogram of Life Expectancy, 2007"))

data(gapminder_2007) + bar * bin + x(:life) +
  x_label("Life expectancy (years)") +
  title("Histogram of Life Expectancy, 2007")

plot(data(gapminder_2007), layer(bar, bin), x(col.life),
  x_label("Life expectancy (years)"),
  title("Histogram of Life Expectancy, 2007"))

“Given gapminder 2007: bars derived by bin, x is life.”

Bin count defaults to Sturges’ rule (Sturges, 1926): k = ⌈log₂(n)⌉ + 1. Notice there is no y() in the expression: bin invents the count column internally, and the y-axis is labeled “Count” automatically.

25.2.1 Choosing the bin count

A histogram’s shape depends on how it is cut, and the default cut may hide a gap or split a peak. You will often want a finer cut, or bins whose width means something in the data’s units.

Sturges’ rule is a sensible default, not a mandate. Pass a number to set the bin count, or name a width to fix the bin width in the data’s own units:

data(gapminder_2007) + bar * bin(30) + x(life) +
  x_label("Life expectancy (years)") + title("Thirty bins: a finer histogram")

(data(gapminder_2007) + bar * bin(30) + x(col.life) +
  x_label("Life expectancy (years)") + title("Thirty bins: a finer histogram"))

data(gapminder_2007) + bar * bin(30) + x(:life) +
  x_label("Life expectancy (years)") +
  title("Thirty bins: a finer histogram")

plot(data(gapminder_2007), layer(bar, bin(30)), x(col.life),
  x_label("Life expectancy (years)"),
  title("Thirty bins: a finer histogram"))

“Given gapminder 2007: bars derived by bin into 30, x is life.”

data(gapminder_2007) + bar * bin(width = 5) + x(life) +
  x_label("Life expectancy (years)") + title("Bins five years wide")

(data(gapminder_2007) + bar * bin(width = 5) + x(col.life) +
  x_label("Life expectancy (years)") + title("Bins five years wide"))

data(gapminder_2007) + bar * bin(width = 5) + x(:life) +
  x_label("Life expectancy (years)") + title("Bins five years wide")

plot(data(gapminder_2007), layer(bar, bin({ width: 5 })), x(col.life),
  x_label("Life expectancy (years)"), title("Bins five years wide"))

“Given gapminder 2007: bars derived by bin at width 5, x is life.”

The positional number is the count: bin(30) is thirty bins. A width is the less common request, so it is said out loud, bin(width = 5); the two are mutually exclusive. Writing bin(30, width = 5) refuses with direction rather than guessing which you meant: the same reason bin(30) alone cannot mean a width, since on this data five bins and five-wide bins are opposite plots.

25.2.2 Why `bar * bin`, not a `histogram()` shortcut?

Because bin does the same computation regardless of the mark, and the mark chooses how the bins are drawn. bar * bin fills bars. line * bin connects the bin centers into a frequency polygon, and step * bin traces the histogram as a staircase outline. Add stack and a fourth reading appears, the dot plot further down this chapter: one dot per observation rather than one mark per bin. Adding a histogram() shortcut would violate Law 2 (No Exceptions) by special-casing a particular mark+transform pair.

25.2.3 The same `bin`, cutting two axes

All twelve gapminder years on one scatter make 1704 points, and where they overlap you cannot see how dense the cloud is. The answer is to count rows in cells of the plane, not only along one axis.

The mark decides something else too: how many axes get cut. Each mark above leaves one axis free to measure the count along, so bin cuts the other one. A zone measures nothing by length, and it needs an extent on both axes to be a rectangle. So the same word cuts both, and the count goes to color, the only channel left holding a number:

`gm_all`: first 5 of 1704 rows
country	continent	year	life	population	gdp
Afghanistan	Asia	1952	28.801	8425333	779.4453
Afghanistan	Asia	1957	30.332	9240934	820.8530
Afghanistan	Asia	1962	31.997	10267083	853.1007
Afghanistan	Asia	1967	34.020	11537966	836.1971
Afghanistan	Asia	1972	36.088	13079460	739.9811

data(gm_all) + zone * bin(24) + x(gdp, scale = "log") + y(life) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("One bin, two axes: the heatmap")

(data(gm_all) + zone * bin(24) + x(col.gdp, scale = "log") + y(col.life) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("One bin, two axes: the heatmap"))

data(gm_all) + zone * bin(24) + x(:gdp, scale = "log") + y(:life) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("One bin, two axes: the heatmap")

plot(data(gm_all), layer(zone, bin(24)), x(col.gdp, { scale: "log" }),
  y(col.life), x_label("GDP per capita"), y_label("Life expectancy"),
  title("One bin, two axes: the heatmap"))

“Given all the gapminder years: zones derived by bin into 24, x is gdp on a log scale, y is life.”

That is a heatmap, with no atom added to the kernel.

25.2.4 One of each: the mixed mesh

Five continents make five histograms, and comparing five panels is harder than comparing five rows of one picture.

“Cuts both” needs one qualification. What bin cuts is every axis that has a width to cut, and a categorical axis has none, because it arrived already cut: a category is a cell. Give a zone one of each and the transform does the only coherent thing, which is to cut the continuous axis and leave the other alone:

data(gm_all) + zone * bin(20) + x(life) + y(continent) +
  palette("viridis") + title("One axis cut, one already cut")

(data(gm_all) + zone * bin(20) + x(col.life) + y(col.continent) +
  palette("viridis") + title("One axis cut, one already cut"))

data(gm_all) + zone * bin(20) + x(:life) + y(:continent) +
  palette("viridis") + title("One axis cut, one already cut")

plot(data(gm_all), layer(zone, bin(20)), x(col.life), y(col.continent),
  palette("viridis"), title("One axis cut, one already cut"))

“Given all the gapminder years: zones derived by bin into 20, x is life, y is continent, with the viridis palette.”

A row of cells per continent, every row cut on the same edges, so a column compares straight down. It is a distribution per category shown as shade rather than as height, which is why five of them fit where one histogram would go. The Zone chapter reads all three cases as one rule.

25.2.5 Which mesh: `tiling`

bin also decides the shape of the cells it cuts, because a different mesh puts different rows in different cells and so changes the counts. Rectangles are the default; hexagons stagger alternate rows, which stops the eye reading the mesh’s own alignment as structure in the data (Carr et al., 1987):

data(gm_all) + zone * bin(20, tiling = "hex") + x(gdp, scale = "log") + y(life) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("Carr's staggered mesh: tiling = hex")

(data(gm_all) + zone * bin(20, tiling = "hex") + x(col.gdp, scale = "log") + y(col.life) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("Carr's staggered mesh: tiling = hex"))

data(gm_all) + zone * bin(20, tiling = "hex") + x(:gdp, scale = "log") +
  y(:life) + x_label("GDP per capita") + y_label("Life expectancy") +
  title("Carr's staggered mesh: tiling = hex")

plot(data(gm_all), layer(zone, bin(20, { tiling: "hex" })),
  x(col.gdp, { scale: "log" }), y(col.life), x_label("GDP per capita"),
  y_label("Life expectancy"),
  title("Carr's staggered mesh: tiling = hex"))

“Given all the gapminder years: zones derived by bin into 20 on hexagons, x is gdp on a log scale, y is life.”

A tiling means nothing to a bin that cuts one axis, since there the cells are intervals and an interval has no shape, so bar * bin(tiling = ) is refused pointing at zone. The Zone chapter works through both meshes, including why the empty cells are left as panel.

25.3 `count`: frequency aggregation

The first question to ask of a categorical column is how many rows fall in each category. A table with one row per country does not hold that number anywhere.

count counts the number of rows for each unique value of x, and it works on both categorical and continuous columns. It is the categorical counterpart to bin: bin groups a continuous column; count groups a categorical one.

# How many countries are in each continent?
data(gapminder_2007) + bar * count + x(continent) +
  title("Countries per Continent, 2007")

(data(gapminder_2007) + bar * count + x(col.continent) +
  title("Countries per Continent, 2007"))

data(gapminder_2007) + bar * count + x(:continent) +
  title("Countries per Continent, 2007")

plot(data(gapminder_2007), layer(bar, count), x(col.continent),
  title("Countries per Continent, 2007"))

“Given gapminder 2007: bars derived by count, x is continent.”

The bars show it. Histogram bars touch, because the slices adjoin on one continuum. Bar chart bars stand apart, because the categories are separate.

count, bin, and density are the three inventing transforms, and proportion behaves as one when it stands alone, so y() is optional for all four and the y-axis labels itself automatically (“Count”, “Proportion”, “Density”). Bind y(my_name) only when you want a custom axis label.

25.3.1 The same `count`, tallying two axes

With two categorical columns the question becomes how often each pair occurs, such as how many winds came from the west in winter.

bin cuts one axis on a bar and two on a zone, and count divides the same way for the same reason. A bar has a length to show the tally along, so it groups by one axis; a zone measures nothing by length, so it tallies rows into the cell where two categories cross and the count goes to color:

`winds`: first 5 of 264 rows
direction	bearing	speed	season
N	19.612260	10.1	Winter
N	357.049499	11.0	Winter
N	349.709737	13.8	Winter
N	2.602499	7.4	Winter
N	338.210715	3.7	Summer

data(winds) + zone * count + x(direction) + y(season) +
  title("One count, two axes: the tile plot")

(data(winds) + zone * count + x(col.direction) + y(col.season) +
  title("One count, two axes: the tile plot"))

data(winds) + zone * count + x(:direction) + y(:season) +
  title("One count, two axes: the tile plot")

plot(data(winds), layer(zone, count), x(col.direction), y(col.season),
  title("One count, two axes: the tile plot"))

That is the tile plot (a confusion matrix is the same sentence with different columns), and like the heatmap, no atom was added to the kernel.

The pair of them states the whole rule: bin cuts, count tallies. A continuous axis has to be cut into cells before there is anything to count in; a categorical axis arrives already cut, because a category is a cell. So each is refused where the other belongs, and each refusal names the other.

Counting the axes one at a time gives three plots, not two. Cut both axes and it is the heatmap; cut neither and it is the tile plot; cut one of two and it is the mixed mesh above. A bin is refused only where there is nothing left for it to cut at all.

25.4 `proportion`: relative frequency

Fifty-two of the 142 countries are in Africa, and that fact reads differently as a count than as a share of the whole.

proportion divides a measurement by its total, producing shares in [0, 1] that sum to 1. Used on its own it has nothing to divide yet, so it tallies the rows first exactly as count does:

# What share of countries belongs to each continent?
data(gapminder_2007) + bar * proportion + x(continent) +
  title("Continent share, 2007")

(data(gapminder_2007) + bar * proportion + x(col.continent) +
  title("Continent share, 2007"))

data(gapminder_2007) + bar * proportion + x(:continent) +
  title("Continent share, 2007")

plot(data(gapminder_2007), layer(bar, proportion), x(col.continent),
  title("Continent share, 2007"))

“Given gapminder 2007: bars derived by proportion, x is continent.”

Use proportion when you care about relative composition rather than raw counts, for example “what fraction of respondents chose each option?” Because count and proportion differ only by a constant factor (total n), order works identically for both: sorting by the y column produces the same order.

It follows count into two dimensions as well, so zone * proportion reads a tile plot as shares (see Zone).

25.4.1 The total is the whole plot, always

Once a second column splits the bars, each segment could be a share of its own bar or of the whole plot.

The denominator is every row the plot draws, in every context. That is the one thing the word is allowed to mean. Split the bars by a second column and they still sum to 1 between them, so each bar reads as this pair’s share of everything:

data(winds) + bar * proportion + x(direction) + color(season) +
  title("Every bar is a share of all 264 observations")

(data(winds) + bar * proportion + x(col.direction) + color(col.season) +
  title("Every bar is a share of all 264 observations"))

data(winds) + bar * proportion + x(:direction) + color(:season) +
  title("Every bar is a share of all 264 observations")

plot(data(winds), layer(bar, proportion), x(col.direction),
  color(col.season),
  title("Every bar is a share of all 264 observations"))

“Given the winds: bars derived by proportion, x is direction, color by season.”

Add up the sixteen segments there and you get 1, not 2. What you cannot get that way is the conditional reading: “of the summer winds, what share came from the west”. That reading is the facet. It cuts the data into panels before any of this runs, so each panel normalizes inside itself:

data(winds) + bar * proportion + x(direction) | facet(season)

data(winds) + bar * proportion + x(col.direction) | facet(col.season)

data(winds) + bar * proportion + x(:direction) | facet(:season)

plot(data(winds), layer(bar, proportion), x(col.direction),
  across(col.season))

“Given the winds: bars derived by proportion, x is direction, split into panel columns by season.”

Read the two plots against each other and the difference is the whole distinction: in the first, summer’s bars and winter’s bars divide one total between them; in the second each season gets its own.

25.4.2 Normalizing something other than a tally

Two histograms from tables of different sizes cannot be compared by their counts, and a continent’s population means more as a share of the world’s.

Because proportion divides whatever measurement it finds, it composes with any transform that leaves one number per cell. Put it after a bin and the histogram’s own counts come back as fractions, the relative-frequency histogram, on exactly the mesh a plain bar * bin cuts:

data(gm_all) + bar * bin(20) * proportion + x(life) +
  title("Relative frequency, not counts")

(data(gm_all) + bar * bin(20) * proportion + x(col.life) +
  title("Relative frequency, not counts"))

data(gm_all) + bar * bin(20) * proportion + x(:life) +
  title("Relative frequency, not counts")

plot(data(gm_all), layer(bar, bin(20), proportion), x(col.life),
  title("Relative frequency, not counts"))

“Given all the gapminder years: bars derived by bin into 20 and proportion, x is life.”

Put it after a summary and the column you named is the thing rescaled, so each slot reads as its share of the grand total:

data(gapminder_2007) + bar * sum * proportion + x(continent) + y(population) +
  title("Each continent's share of world population, 2007")

(data(gapminder_2007) + bar * sum * proportion + x(col.continent) + y(col.population) +
  title("Each continent's share of world population, 2007"))

data(gapminder_2007) + bar * sum * proportion + x(:continent) +
  y(:population) +
  title("Each continent's share of world population, 2007")

plot(data(gapminder_2007), layer(bar, sum, proportion), x(col.continent),
  y(col.population),
  title("Each continent's share of world population, 2007"))

“Given gapminder 2007: bars derived by sum and proportion, x is continent, y is population.”

What it refuses is what leaves it nothing to divide by. A density already integrates to 1, and its cells are sample points rather than parts of a whole. A smooth gives a value at each x, so adding those values up answers nothing. A transform that writes a low and a high leaves two numbers per cell, and a span is not part of any total.

data(gm_all) + line * density * proportion + x(life)

data(gm_all) + line * density * proportion + x(col.life)

data(gm_all) + line * density * proportion + x(:life)

plot(data(gm_all), layer(line, density, proportion), x(col.life))

Error:
! gog: `line * density * proportion` has nothing to normalize — a density already integrates to 1, and its cells are points where the estimate was sampled rather than parts of a whole, so the sum of its heights is not a quantity anything is a share of. For the estimated shape, `line * density + x(<number>)`; for shares of the rows themselves, cut them into cells first: `bar * bin * proportion + x(<number>)` is the same curve's histogram, read as fractions.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

25.5 `density`: kernel density estimate

A histogram’s outline changes with every choice of bin, and its steps belong to the bins rather than to the data. A smooth curve describes the same distribution without the steps.

density is the third inventing transform: it estimates the distribution bin counts, as a smooth curve. The curve is the probability density function of x, estimated with a Gaussian kernel and evaluated at 256 evenly spaced points. By default the bandwidth (how wide each kernel is, and so how smooth the curve) is chosen automatically by Silverman’s rule-of-thumb (Silverman, 1986), IQR-adjusted for robustness.

Combine it with line to draw a smooth density curve:

data(gapminder_2007) + line * density + x(life) +
  x_label("Life expectancy (years)") +
  title("Kernel Density: Life Expectancy 2007")

(data(gapminder_2007) + line * density + x(col.life) +
  x_label("Life expectancy (years)") +
  title("Kernel Density: Life Expectancy 2007"))

data(gapminder_2007) + line * density + x(:life) +
  x_label("Life expectancy (years)") +
  title("Kernel Density: Life Expectancy 2007")

plot(data(gapminder_2007), layer(line, density), x(col.life),
  x_label("Life expectancy (years)"),
  title("Kernel Density: Life Expectancy 2007"))

“Given gapminder 2007: a line derived by density, x is life.”

As with the other inventing transforms, there is no y() to write: the density column and its axis label are both made for you.

25.5.1 Choosing the bandwidth

The automatic curve can be smoother than the data supports, hiding a second peak, or rougher, showing bumps that are only noise.

The automatic bandwidth is a sensible default, not a mandate, and it is density’s one parameter, just as the bin count is bin’s. It is set the same two ways. A positional number is an adjust multiplier on the automatic bandwidth: density(2) is twice as smooth, density(0.5) half as smooth and so more jagged.

data(gapminder_2007) + line * density(2) + x(life) +
  x_label("Life expectancy (years)") + title("density(2): twice as smooth")

(data(gapminder_2007) + line * density(2) + x(col.life) +
  x_label("Life expectancy (years)") + title("density(2): twice as smooth"))

data(gapminder_2007) + line * density(2) + x(:life) +
  x_label("Life expectancy (years)") +
  title("density(2): twice as smooth")

plot(data(gapminder_2007), layer(line, density(2)), x(col.life),
  x_label("Life expectancy (years)"),
  title("density(2): twice as smooth"))

“Given gapminder 2007: a line derived by density adjusted by 2, x is life.”

To fix the bandwidth in the data’s own units instead, name it. Here bandwidth = 1 sets each kernel one year wide, narrower than the default, so the curve keeps more of the distribution’s fine structure:

data(gapminder_2007) + line * density(bandwidth = 1) + x(life) +
  x_label("Life expectancy (years)") + title("bandwidth = 1 year: an absolute width")

(data(gapminder_2007) + line * density(bandwidth = 1) + x(col.life) +
  x_label("Life expectancy (years)") + title("bandwidth = 1 year: an absolute width"))

data(gapminder_2007) + line * density(bandwidth = 1) + x(:life) +
  x_label("Life expectancy (years)") +
  title("bandwidth = 1 year: an absolute width")

plot(data(gapminder_2007), layer(line, density({ bandwidth: 1 })),
  x(col.life), x_label("Life expectancy (years)"),
  title("bandwidth = 1 year: an absolute width"))

“Given gapminder 2007: a line derived by density at bandwidth 1, x is life.”

This works exactly as bin does. The positional number is the adjust multiplier: the common request, since you rarely know the bandwidth you want in advance and only need the automatic one a little smoother or rougher. An absolute width is the less common request, so it is said out loud, density(bandwidth = 1); the two are mutually exclusive, and density(2, bandwidth = 1) refuses with direction rather than guessing which you meant.

25.5.2 One transform, four readings

Once you have a curve for one column, you will want the same estimate over two columns, or one curve per category, side by side. Each of those is still density.

density estimates a distribution; what it is drawn as depends on how much room the bindings leave it, and the answer is different in each case rather than being four separate transforms sharing one name.

Bind one number and it is a curve along the free axis, which is everything above. Bind two numbers to a mark with no measure axis and it is a field over the plane: path * density traces its contours and zone * density paints it cell by cell, with surface * density raising it into the cube. Bind a category and a number and there is no free axis left, so the estimate is drawn across the category’s slot, one distribution per group, which is the violin:

data(gapminder_2007) + ribbon * density + x(continent) + y(life) +
  y_label("Life expectancy") + title("ribbon * density: the slot reading")

(data(gapminder_2007) + ribbon * density + x(col.continent) + y(col.life) +
  y_label("Life expectancy") + title("ribbon * density: the slot reading"))

data(gapminder_2007) + ribbon * density + x(:continent) + y(:life) +
  y_label("Life expectancy") + title("ribbon * density: the slot reading")

plot(data(gapminder_2007), layer(ribbon, density), x(col.continent),
  y(col.life), y_label("Life expectancy"),
  title("ribbon * density: the slot reading"))

“Given gapminder 2007: ribbons derived by density, x is continent, y is life.”

Each of density’s parameters belongs to one of those readings, and asking for one in a reading it cannot mean is refused rather than ignored. bandwidth is a length in one column’s units, so it means nothing to a field, whose two axes measure different quantities. levels counts lines of equal density, and a curve has none to count. The slot reading has two parameters of its own. compare says what a violin’s width means from one slot to the next. reach says how far the shape extends, measured in slots, and past half a slot the shapes reach into their neighbors: the ridgeline. Neither means anything to a plot with no slots:

data(gapminder_2007) + line * density(compare = "count") + x(life)

data(gapminder_2007) + line * density(compare = "count") + x(col.life)

data(gapminder_2007) + line * density(compare = "count") + x(:life)

plot(data(gapminder_2007), layer(line, density({ compare: "count" })),
  x(col.life))

Error:
! gog: `density(compare = )` says what a violin's width means from one slot to the next, and `line * density` here has no slots — it estimates one curve along one axis. Bind a category to give it slots to compare: `ribbon * density(compare = "count") + x(<category>) + y(<number>)`. Otherwise drop `compare`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

25.6 What `bin`, `density` and `smooth` refuse

A histogram of continent is an easy thing to ask for. The refusal is easier to act on once you know what these three transforms share.

The two families above sort the transforms by how many columns they read. A second property divides them a different way: these three all describe how values are spread along an axis, so the axis they read has to carry a number. A category is one slot rather than a stretch of line, so there is nothing to cut into intervals, nothing for a curve to spread over, and nothing to fit through.

The violin above looks like an exception, and it is worth being precise about why it is not one. There the category is not the axis being estimated along: y(life) is, and the category only says which rows each estimate is made from. Take the number away and there is nothing left to spread over, so the refusal returns, which is why every chunk below binds a category and nothing else.

Each refusal names the atom that asks the same question of categories. Binning one is really counting it:

data(gapminder_2007) + bar * bin + x(continent)

data(gapminder_2007) + bar * bin + x(col.continent)

data(gapminder_2007) + bar * bin + x(:continent)

plot(data(gapminder_2007), layer(bar, bin), x(col.continent))

Error:
! gog: `bin` cuts a continuous axis into intervals, and `x(continent)` is categorical — a category is one slot, with no width to cut. To tally rows per category, `count` is the transform that does it: `bar * count`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

A density over categories is really a share of the total:

data(gapminder_2007) + bar * density + x(continent)

data(gapminder_2007) + bar * density + x(col.continent)

data(gapminder_2007) + bar * density + x(:continent)

plot(data(gapminder_2007), layer(bar, density), x(col.continent))

Error:
! gog: `density` estimates a continuous distribution, and `x(continent)` is categorical — there is no number line for the curve to spread along. For the share of rows in each category, that is `bar * proportion`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

And a curve fitted across categories is really a summary within each one:

data(gapminder_2007) + point * smooth + x(continent) + y(life)

data(gapminder_2007) + point * smooth + x(col.continent) + y(col.life)

data(gapminder_2007) + point * smooth + x(:continent) + y(:life)

plot(data(gapminder_2007), layer(point, smooth), x(col.continent),
  y(col.life))

Error:
! gog: `smooth` fits a curve of `y` against `x`, and `x(continent)` is categorical — a fit needs a number line to run along. For a typical value per category, `bar * mean` (or `point * mean`) says it directly.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

Which axis must be continuous is read off the bindings rather than fixed to x. The horizontal histogram is bar * bin + y(life): it cuts y, because y is the axis it was handed and x is the one it invents. smooth is the strict one of the three, since it fits y against x and so needs a number on both.

25.6.1 A fit needs rows to fit through

It is easy to ask for a trend through very few rows: a table filtered to its latest years, or a group with two members.

Having numbers is not the same as having enough of them. smooth fits a curve, and two points are a straight line while one is a point, so it asks for at least three:

two_readings <- subset(gm_all, country == "France" & year >= 2002)
data(two_readings) + line * smooth + x(year) + y(life)

Error:
! gog: `smooth` fits a curve through the rows and needs at least 3 of them. This data has 2. Plot what you have with `point` and no transform, which draws the rows as rows rather than as a curve through them.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

The count that matters is the count inside each group, not in the table. A statistic runs within every group you split by, and faceting splits the rows before any of that. So a table with plenty of rows can still be asked for a fit through two:

recent <- subset(gm_all, year >= 2002)
data(recent) + line * smooth + x(year) + y(life) + group(country) |
  facet(continent)

Error:
! gog: `smooth` fits a curve through the rows and needs at least 3 of them. Split by `continent` or `country`, the smallest group has 2. Drop the split so the fit reads the rows together, or plot what you have with `point` and no transform, which draws the rows as rows rather than as a curve through them.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

There are 284 rows there and the refusal still stands, because each country in each panel is its own fit and each has two readings. The message names both splits, so you know which one to remove. Drop group(country) and each panel fits one curve through a continent’s rows, which is plenty.

This matters more than a plot from few rows usually would, because of what the picture would otherwise show. Below three rows the curve cannot be computed, so what would be drawn is the rows themselves, connected in order. A two-point group would draw a straight segment beside a hundred-point curve, with nothing to say that one is a fit and the other is the data. A plot that looks finished and is not is worse than one that refuses.

25.7 Chaining transforms

Some questions need two transforms. To show life expectancy across income bands, cut income into bands and take a mean of life inside each one.

Two transforms on one mark have to divide the work, and there are only four jobs to divide. A transform can say where the cells are, it can say what is in them, it can say what scale the answer is read on, and it can say where the marks sit. bounds names the sides of a rectangle. count tallies into the cells your categories already own, and says nothing about where those cells are. mean and its five siblings reduce a column you name, and say nothing about where the cells are either. proportion does the third job and only the third: it divides by a total, which is why it can stand after almost anything. dodge, stack, jitter and repel do the fourth, moving marks that would otherwise land on top of one another.

Two transforms compose when they do different jobs, and contradict when they do the same one. A cell holds one extent, one measurement, one scale and one arrangement. A second transform doing a job the first already did therefore leaves no reading that keeps both. The engine refuses, rather than drawing one and quietly throwing the other away. Combinations has the whole grid, generated from the same rule.

Because each job is filled once, the order you write them in does not change the picture. Write them in the order they happen and the sentence reads the way it runs.

bin is the one transform that does two of the four jobs, and the only one that can drop one of them. Cutting an axis is what makes it a bin; the tally it hands back is a by-product of the cut. So when you compose it with a statistic, bin keeps the cut and the statistic keeps the measurement:

data(gm_all) + bar * bin(20) * mean + x(gdp, scale = "log") + y(life) +
  title("Mean life expectancy by income band")

(data(gm_all) + bar * bin(20) * mean + x(col.gdp, scale = "log") + y(col.life) +
  title("Mean life expectancy by income band"))

data(gm_all) + bar * bin(20) * mean + x(:gdp, scale = "log") + y(:life) +
  title("Mean life expectancy by income band")

plot(data(gm_all), layer(bar, bin(20), mean),
  x(col.gdp, { scale: "log" }), y(col.life),
  title("Mean life expectancy by income band"))

“Given all the gapminder years: bars derived by bin into 20 and mean, x is gdp on a log scale, y is life.”

Read the sentence as three separate claims. bin(20) cuts income into twenty bands. mean reduces life within each one. bar draws the result as a length from zero. The bars touch because the bands tile the axis, exactly as a histogram’s do, and the y axis is now labeled for the column that was actually reduced.

Any mark that takes both transforms takes the pair, so the same three claims read as points instead:

data(gm_all) + point * bin(20) * median + x(gdp, scale = "log") + y(life) +
  title("Median life expectancy by income band")

(data(gm_all) + point * bin(20) * median + x(col.gdp, scale = "log") + y(col.life) +
  title("Median life expectancy by income band"))

data(gm_all) + point * bin(20) * median + x(:gdp, scale = "log") +
  y(:life) + title("Median life expectancy by income band")

plot(data(gm_all), layer(point, bin(20), median),
  x(col.gdp, { scale: "log" }), y(col.life),
  title("Median life expectancy by income band"))

“Given all the gapminder years: points derived by bin into 20 and median, x is gdp on a log scale, y is life.”

25.7.1 What cannot be chained, and why

Having seen bin * mean work, you will want to know which other pairs do.

The other two inventing transforms cannot give their measurement up, because measuring is all they do. Take it away from count and nothing is left of it:

data(gm_all) + bar * count * mean + x(continent) + y(life)

data(gm_all) + bar * count * mean + x(col.continent) + y(col.life)

data(gm_all) + bar * count * mean + x(:continent) + y(:life)

plot(data(gm_all), layer(bar, count, mean), x(col.continent), y(col.life))

Error:
! gog: `bar * count * mean` measures each cell twice — `count` supplies only a measurement: its cells are the slots the positions already own, so with `mean` measuring them too the cell is measured twice and `count` has nothing left to contribute. Keep whichever you meant: `bar * count` to measure what `count` computes, or `bar * mean` to reduce the column you name. To cut a continuous axis into cells and reduce a column inside each, `bin` is the transform that cuts without keeping the measurement: `bar * bin * mean + x(<number>)`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

density fails a different test. Its cells are the points where the estimate was sampled, and the estimate exists between your observations rather than inside cells that hold rows, so there is nothing inside one for a statistic to reduce:

data(gm_all) + bar * density * mean + x(gdp) + y(life)

data(gm_all) + bar * density * mean + x(col.gdp) + y(col.life)

data(gm_all) + bar * density * mean + x(:gdp) + y(:life)

plot(data(gm_all), layer(bar, density, mean), x(col.gdp), y(col.life))

Error:
! gog: `bar * density * mean` measures each cell twice — a `density` cell is a point where the estimate was sampled, not a bucket holding rows — the estimate exists *between* your observations — so there is nothing inside one for `mean` to reduce. Keep whichever you meant: `bar * density` to measure what `density` computes, or `bar * mean` to reduce the column you name. To cut a continuous axis into cells and reduce a column inside each, `bin` is the transform that cuts without keeping the measurement: `bar * bin * mean + x(<number>)`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

smooth is refused against all three, including bin, for a reason none of them share. It fits a curve through the rows and already averages locally while it fits, so cutting them into cells first changes nothing it was not doing:

data(gm_all) + bar * bin * smooth + x(gdp) + y(life)

data(gm_all) + bar * bin * smooth + x(col.gdp) + y(col.life)

data(gm_all) + bar * bin * smooth + x(:gdp) + y(:life)

plot(data(gm_all), layer(bar, bin, smooth), x(col.gdp), y(col.life))

Error:
! gog: `bar * bin * smooth` asks one question twice — `smooth` fits a curve through the rows and already averages locally as it goes, so cutting them into cells first changes nothing it was not doing. Keep whichever you meant: `bar * smooth + x(<a>) + y(<b>)` for the fitted curve, or `bar * bin` for the shape `bin` measures. For a summary per cell rather than a fitted curve, name the statistic: `bar * bin * mean + x(<a>) + y(<b>)`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

And two transforms that each invent their own measurement are the same contradiction, and neither can drop its measurement, so a cell would be measured twice:

data(gm_all) + bar * bin * count + x(life)

data(gm_all) + bar * bin * count + x(col.life)

data(gm_all) + bar * bin * count + x(:life)

plot(data(gm_all), layer(bar, bin, count), x(col.life))

Error:
! gog: `bar * bin * count` measures each cell twice — `bin` and `count` each invent their own measurement from the rows, and neither was handed a column to give way to, so there is no reading that keeps both. Keep whichever you meant: `bar * bin` or `bar * count`. To cut an axis into cells and measure something else inside them, the second transform has to be one you hand a column: `bar * bin * mean + x(<number>) + y(<column>)`. To read either as shares of the whole rather than as counts, `proportion` rescales whichever you keep: `bar * bin * proportion`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

Each refusal names both transforms you could drop, and the first two also name a third option: bin is the transform that cuts without keeping the measurement.

The same rule reaches the other three jobs, and it reads the same way each time. Two transforms that both produce a low and a high measure the cell twice. So range and confidence cannot stand together. A minimum and a maximum are not a 95% confidence interval, and one of the two would have to be discarded:

data(gm_all) + interval * range * confidence + x(continent) + y(life)

data(gm_all) + interval * range * confidence + x(col.continent) + y(col.life)

data(gm_all) + interval * range * confidence + x(:continent) + y(:life)

plot(data(gm_all), layer(interval, range, confidence), x(col.continent),
  y(col.life))

Error:
! gog: `interval * range * confidence` measures each cell twice — `range` and `confidence` both reduce the column you named, and a cell holds one answer, so drawing both would mean drawing one of them and discarding the other. Keep whichever you meant: `interval * range` or `interval * confidence`. To show both, draw them as two layers: `interval * range + interval * confidence`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

The same holds when one of the pair is a single value and the other is two of them:

data(gm_all) + line * sum * range + x(year) + y(population)

data(gm_all) + line * sum * range + x(col.year) + y(col.population)

data(gm_all) + line * sum * range + x(:year) + y(:population)

plot(data(gm_all), layer(line, sum, range), x(col.year),
  y(col.population))

Error:
! gog: `line * sum * range` measures each cell twice — `sum` and `range` both reduce the column you named, and a cell holds one answer, so drawing both would mean drawing one of them and discarding the other. Keep whichever you meant: `line * sum` or `line * range`. To show both, draw them as two layers: `line * sum + line * range`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

And two arrangements are the same contradiction on the fourth job. A bar cannot be set beside its neighbor and piled on top of it at once, because it sits in one place:

data(gm_all) + bar * count * dodge * stack + x(continent) + color(continent)

data(gm_all) + bar * count * dodge * stack + x(col.continent) + color(col.continent)

data(gm_all) + bar * count * dodge * stack + x(:continent) +
  color(:continent)

plot(data(gm_all), layer(bar, count, dodge, stack), x(col.continent),
  color(col.continent))

Error:
! gog: `bar * dodge * stack` arranges the same marks twice — `dodge` and `stack` each decide where colliding groups go, and a mark sits in one place, so one of the two would be discarded. Keep whichever you meant: `bar * dodge` or `bar * stack`. To show both readings, draw them as two plots side by side rather than as one layer.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

A transform can also turn out to be unnecessary rather than contradictory, and that is a different answer. proportion divides a tally by its total, so it already counts the rows, which leaves count beside it with nothing to add. The plot is still exactly the plot you asked for, so the engine draws it:

data(gm_all) + bar * count * proportion + x(continent) +
  title("Share of countries by continent")

(data(gm_all) + bar * count * proportion + x(col.continent) +
  title("Share of countries by continent"))

data(gm_all) + bar * count * proportion + x(:continent) +
  title("Share of countries by continent")

plot(data(gm_all), layer(bar, count, proportion), x(col.continent),
  title("Share of countries by continent"))

“Given all the gapminder years: bars derived by count and proportion, x is continent.”

It also prints a warning on your console. This page cannot show you that warning, because only refusals appear in a rendered book. It reads: “gog: bar * count * proportion draws the same plot as bar * proportion — a share is a share of a tally, so proportion already counts the rows and count adds nothing. Drop it, or keep bar * count on its own to read the tallies themselves rather than their shares.”

That is the difference between the two answers. When keeping both transforms would mean throwing one away, the engine refuses. When both can stay and one of them simply changes nothing, it draws and says so. Neither answer is silence, which is the rule underneath both: a word you wrote is never read and then ignored.

25.8 Output columns

Every transform that computes a value writes its answer into a named column. This table gives those names.

Transform	Reads from	Writes to
`bin`	`x(field)`	`x(field)` (bin centers), `y(your_name)` (counts)
`count`	`x(field)`	`x(field)` (unique values), `y(your_name)` (counts)
`proportion`	whatever measured, or `x(field)` if nothing did	the same column, divided by its total (sums to 1)
`smooth`	`x(field)`, `y(field)`	same `x`, same `y` (smoothed values)
`density`	`x(field)`	`x(field)` (eval points), `y(your_name)` (density)
`sum` / `mean` / `median` / `max` / `min`	`x(field)`, `y(field)`	`x(field)` (unique), `y(field)` (aggregated)
`range`	`x(field)`, `y(field)`	`x(field)` (each group twice), `y(field)` (low, then high)
`confidence`	`x(field)`, `y(field)`	`x(field)` (each group twice), `y(field)` (low, then high), `center` (the mean)

For bin, count, proportion, and density, the output y column name is whatever you bind with y() in the specification, or an empty string when y() is omitted (the renderer finds it either way). The y-axis label defaults to the transform name (“Count”, “Proportion”, “Density”) when y() is absent.

One row of that table changes when bin is composed with a statistic. Cutting is then all it does: it writes the bin centers to x(field) as before and leaves y(field) alone for the statistic to reduce, so the y column holds your column’s summary rather than a count.

25.9 `smooth`: LOESS trend line

The inventing transforms made new measurements. The reading transforms are the other half of the sort: each one reads a column you name and answers with a summary of it, and smooth is the first.

Income and life expectancy rise together across the scatter, and one curve through the cloud states that relationship more plainly than 142 separate points.

smooth fits a locally weighted linear regression (LOESS) through the (x, y) points and evaluates it at 100 evenly spaced x positions (Cleveland, 1979). The curve is the engine’s own, and it makes no second pass to reduce the pull of outliers, so it is identical in all four languages and differs from geom_smooth(), whose default fit is a local quadratic.

Combine it with line to overlay a trend line on a scatter plot:

data(gapminder_2007) + x(gdp) + y(life) +
  point + color(continent) +
  line * smooth +
  title("GDP vs Life Expectancy with LOESS Smooth")

(data(gapminder_2007) + x(col.gdp) + y(col.life) +
  point + color(col.continent) +
  line * smooth +
  title("GDP vs Life Expectancy with LOESS Smooth"))

data(gapminder_2007) + x(:gdp) + y(:life) + point + color(:continent) +
  line * smooth + title("GDP vs Life Expectancy with LOESS Smooth")

plot(data(gapminder_2007), x(col.gdp), y(col.life), point,
  color(col.continent), layer(line, smooth),
  title("GDP vs Life Expectancy with LOESS Smooth"))

“Given gapminder 2007: x is gdp, y is life, points colored by continent, and also a line derived by smooth.”

Span defaults to 0.75: 75% of the data is used for each local regression. A larger span gives a smoother curve; a smaller span follows the data more closely. smooth requires both x and y to be continuous columns, and at least three rows in each group it fits; both refusals are earlier in this chapter, under what these three refuse. It preserves the column names, so no extra y() binding is needed.

25.10 Aggregation family: `sum`, `mean`, `median`, `max`, `min`, `quantile`

The six summaries are the reading family at its plainest. When a table has multiple rows for the same x value (for example, medals from different years, all labeled “USA”), bar alone draws one bar per row, and the bars for one country are drawn over each other in one slot.

The aggregation family groups rows by x and reduces y to a single value per group. All six follow exactly the same grammar:

bar * sum     + x(country) + y(gold)
bar * mean    + x(country) + y(gold)
bar * median  + x(country) + y(gold)
bar * max     + x(country) + y(gold)
bar * min     + x(country) + y(gold)
bar * quantile(0.9) + x(country) + y(gold)

sum, total y per group, the most common aggregation for counts and amounts:

`medal_repeats`: all 5 rows
country	gold
USA	10
USA	5
GBR	8
GBR	4
JPN	6

data(medal_repeats) + bar * sum + x(country) + y(gold) +
  order(gold, desc = TRUE) +
  y_label("Total gold medals") + title("Sum per country: bar * sum")

(data(medal_repeats) + bar * sum + x(col.country) + y(col.gold) +
  order(col.gold, desc = True) +
  y_label("Total gold medals") + title("Sum per country: bar * sum"))

data(medal_repeats) + bar * sum + x(:country) + y(:gold) +
  order(:gold, desc = true) + y_label("Total gold medals") +
  title("Sum per country: bar * sum")

plot(data(medal_repeats), layer(bar, sum), x(col.country), y(col.gold),
  order(col.gold, { desc: true }), y_label("Total gold medals"),
  title("Sum per country: bar * sum"))

“Given the medal repeats table: bars derived by sum, x is country, y is gold, ordered by gold, largest first.”

mean, average y per group, useful for comparing typical values:

data(medal_repeats) + bar * mean + x(country) + y(gold) +
  order(gold, desc = TRUE) +
  y_label("Mean gold medals") + title("Mean per country: bar * mean")

(data(medal_repeats) + bar * mean + x(col.country) + y(col.gold) +
  order(col.gold, desc = True) +
  y_label("Mean gold medals") + title("Mean per country: bar * mean"))

data(medal_repeats) + bar * mean + x(:country) + y(:gold) +
  order(:gold, desc = true) + y_label("Mean gold medals") +
  title("Mean per country: bar * mean")

plot(data(medal_repeats), layer(bar, mean), x(col.country), y(col.gold),
  order(col.gold, { desc: true }), y_label("Mean gold medals"),
  title("Mean per country: bar * mean"))

median, the middle value, robust to outliers:

data(medal_repeats) + bar * median + x(country) + y(gold) +
  order(gold, desc = TRUE) +
  y_label("Median gold") + title("Median per country: bar * median")

(data(medal_repeats) + bar * median + x(col.country) + y(col.gold) +
  order(col.gold, desc = True) +
  y_label("Median gold") + title("Median per country: bar * median"))

data(medal_repeats) + bar * median + x(:country) + y(:gold) +
  order(:gold, desc = true) + y_label("Median gold") +
  title("Median per country: bar * median")

plot(data(medal_repeats), layer(bar, median), x(col.country), y(col.gold),
  order(col.gold, { desc: true }), y_label("Median gold"),
  title("Median per country: bar * median"))

max, the largest value within each group:

data(medal_repeats) + bar * max + x(country) + y(gold) +
  order(gold, desc = TRUE) +
  y_label("Max gold") + title("Max per country: bar * max")

(data(medal_repeats) + bar * max + x(col.country) + y(col.gold) +
  order(col.gold, desc = True) +
  y_label("Max gold") + title("Max per country: bar * max"))

data(medal_repeats) + bar * max + x(:country) + y(:gold) +
  order(:gold, desc = true) + y_label("Max gold") +
  title("Max per country: bar * max")

plot(data(medal_repeats), layer(bar, max), x(col.country), y(col.gold),
  order(col.gold, { desc: true }), y_label("Max gold"),
  title("Max per country: bar * max"))

“Given the medal repeats table: bars derived by max, x is country, y is gold, ordered by gold, largest first.”

min, the smallest value within each group:

data(medal_repeats) + bar * min + x(country) + y(gold) +
  order(gold, desc = TRUE) +
  y_label("Min gold") + title("Min per country: bar * min")

(data(medal_repeats) + bar * min + x(col.country) + y(col.gold) +
  order(col.gold, desc = True) +
  y_label("Min gold") + title("Min per country: bar * min"))

data(medal_repeats) + bar * min + x(:country) + y(:gold) +
  order(:gold, desc = true) + y_label("Min gold") +
  title("Min per country: bar * min")

plot(data(medal_repeats), layer(bar, min), x(col.country), y(col.gold),
  order(col.gold, { desc: true }), y_label("Min gold"),
  title("Min per country: bar * min"))

“Given the medal repeats table: bars derived by min, x is country, y is gold, ordered by gold, largest first.”

All six aggregations work the same whether x is continuous or categorical. For a categorical x, groups preserve first-appearance order (before any order); for a continuous x, groups are sorted ascending.

Transform	What it computes per group
`sum`	total of y values
`mean`	arithmetic average of y
`median`	middle value of y
`max`	largest y value
`min`	smallest y value
`quantile(p)`	the y at probability p

25.10.1 The same six, grouping by a pair

Mean wind speed by direction is one bar chart. Mean wind speed by direction and season is a grid of cells, with one mean in each.

count follows a bar onto a zone and tallies two axes instead of one, and these six follow it, for the same reason and by the same rule. A bar has a length to show the answer along, so it groups by one position and measures along the other; a zone measures nothing by length, so both its positions are free to be grouping columns and the answer goes to color:

data(winds) + zone * mean + x(direction) + y(season) + color(speed) +
  title("The same mean, grouped by a pair")

(data(winds) + zone * mean + x(col.direction) + y(col.season) + color(col.speed) +
  title("The same mean, grouped by a pair"))

data(winds) + zone * mean + x(:direction) + y(:season) + color(:speed) +
  title("The same mean, grouped by a pair")

plot(data(winds), layer(zone, mean), x(col.direction), y(col.season),
  color(col.speed), title("The same mean, grouped by a pair"))

What tells the transform which column to reduce is the same channel the answer comes back on. That is not a special case for the zone: a summary reduces in place everywhere, and on bar * mean + x(country) + y(gold) it is y that both names gold and receives the average. A zone measures by color, so color does both jobs there.

So the rule is one sentence, and it is the dimensional rule read a second time: a summary groups by every position the mark does not measure with, and reduces the column named on the one it does.

mark	positions	measures with	groups by	reduces
`bar`	`x`, `y`	`y`	`x`	`y`
`bar` in `space()`	`x`, `y`, `z`	`z`	`x`, `y`	`z`
`surface`	`x`, `y`, `z`	`z`	`x`, `y`	`z`
`zone`	`x`, `y`	`color`	`x`, `y`	`color`

The middle two rows are identical, and that is the rule working rather than a coincidence: a 3-D bar and a surface have the same three positions and measure along the same one, so they group and reduce alike. What differs is only what each lays on the cell the pair names, a standing bar, or a flat plateau.

The division from count, bin and density is not about how many axes anything can reach: all of them read a mesh. It is about what each was handed. Those three invent their measurement, so they need no column named and write their answer into a column of their own name; these six are given a column, so the sentence has to say which one, and the answer replaces it. proportion sits outside the division because it is handed neither: it takes whatever measurement the sentence already made and divides it.

25.10.2 Why these names?

One naming question is worth settling before the six are used. count, sum, mean, median, max and min each name an operation on y, not a data type and not a chart name. The * operator makes this explicit: bar * sum reads as “a bar that sums”. Every aggregation is the word a statistics course already uses (Law 3: Plain Names).

25.11 `quantile`: one point in the distribution

sum, mean, median, max and min each answer a fixed question. quantile takes the question as its argument: quantile(0.9) is the value nine tenths of the group falls below, which is the 90th percentile.

Many reported numbers have exactly that form. A service level is the 95th percentile of response time. A growth chart is a set of percentiles by age. A pay band is usually quoted at the 10th, 50th and 90th percentiles. Here are three of them across every country, year by year:

data(gm_all) + x(year) + y(life) +
  line * quantile(0.9) + style(color = "steelblue") +
  line * median + style(color = "gray") +
  line * quantile(0.1) + style(color = "tomato") +
  y_label("Life expectancy") +
  title("The 90th percentile, the median, and the 10th")

(data(gm_all) + x(col.year) + y(col.life) +
  line * quantile(0.9) + style(color = "steelblue") +
  line * median + style(color = "gray") +
  line * quantile(0.1) + style(color = "tomato") +
  y_label("Life expectancy") +
  title("The 90th percentile, the median, and the 10th"))

data(gm_all) + x(:year) + y(:life) + line * quantile(0.9) +
  style(color = "steelblue") + line * median + style(color = "gray") +
  line * quantile(0.1) + style(color = "tomato") +
  y_label("Life expectancy") +
  title("The 90th percentile, the median, and the 10th")

plot(data(gm_all), x(col.year), y(col.life), layer(line, quantile(0.9)),
  style({ color: "steelblue" }), layer(line, median),
  style({ color: "gray" }), layer(line, quantile(0.1)),
  style({ color: "tomato" }), y_label("Life expectancy"),
  title("The 90th percentile, the median, and the 10th"))

“Given all the gapminder years: x is year, y is life, a line derived by quantile at 0.9, colored steelblue, and also a line derived by median, colored gray, and also a line derived by quantile at 0.1, colored tomato.”

The three lines are three sentences over one table, and they say different things about the same 142 countries. The middle rose steadily. The top rose more slowly, because it was already high. The bottom rose and then declined for a decade, which neither of the others shows.

quantile has no default. Every transform whose parameter is a number works bare, and this one is refused:

data(gm_all) + line * quantile + x(year) + y(life)

data(gm_all) + line * quantile + x(col.year) + y(col.life)

data(gm_all) + line * quantile + x(:year) + y(:life)

plot(data(gm_all), layer(line, quantile), x(col.year), y(col.life))

Error:
! gog: `quantile` needs the probability it reduces to, e.g. `quantile(0.9)` for the 90th percentile. There is no default, because the only sensible one is the middle and that already has a plain name: `median`.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

The only sensible default is the middle, and the middle already has a plain name. That is also why three particular values get a warning rather than a refusal:

data(gm_all) + line * quantile(0.5) + x(year) + y(life) +
  y_label("Life expectancy") + title("The median, written as a quantile")

(data(gm_all) + line * quantile(0.5) + x(col.year) + y(col.life) +
  y_label("Life expectancy") + title("The median, written as a quantile"))

data(gm_all) + line * quantile(0.5) + x(:year) + y(:life) +
  y_label("Life expectancy") + title("The median, written as a quantile")

plot(data(gm_all), layer(line, quantile(0.5)), x(col.year), y(col.life),
  y_label("Life expectancy"), title("The median, written as a quantile"))

gog: `quantile(0.5)` is the median — write `median` for the plain name. Drawn as asked.

The plot draws, and gog says that median is the plain name for it. quantile(0) and quantile(1) say the same about min and max. They are not refused, because a program stepping through the deciles would stop at 0.5 for no reason. They are not silent either, because Orthogonality says no atom is redundant.

25.12 `range`: a low/high pair

A mean per species says where each group sits and nothing about how far its values reach.

Every transform above writes a single measured value per group. range is the first to write two: the minimum and the maximum of y within each group, emitted as a low then a high. That pair is exactly what an interval spans:

`iris_flowers`: first 5 of 150 rows
sepal_length	sepal_width	petal_length	species
5.1	3.5	1.4	setosa
4.9	3.0	1.4	setosa
4.7	3.2	1.3	setosa
4.6	3.1	1.5	setosa
5.0	3.6	1.4	setosa

data(iris_flowers) + interval * range + x(species) + y(petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("Petal length range per species")

(data(iris_flowers) + interval * range + x(col.species) + y(col.petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("Petal length range per species"))

data(iris_flowers) + interval * range + x(:species) + y(:petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("Petal length range per species")

plot(data(iris_flowers), layer(interval, range), x(col.species),
  y(col.petal_length), x_label("Species"), y_label("Petal length (cm)"),
  title("Petal length range per species"))

“Given the iris flowers: intervals derived by range, x is species, y is petal length.”

range reads a y and groups by x exactly like the aggregation family: it needs a y() in scope and works the same whether x is continuous or categorical. What differs is the output: two rows per group instead of one. This low/high-rows shape is reused elsewhere: the box mark encodes its own five-number summary the same way (min and max as the pair of rows, the quartiles alongside), and the ribbon mark fills between the same pair as a continuous band. See Interval for the whisker, and for why its extents are invented here rather than bound to two extra channels the kernel would never otherwise have.

25.12.1 A band between two quantiles

The minimum and the maximum each come from a single row, so one unusual row moves the whole span. To show the middle of a group instead of its extremes, give range two quantiles:

data(iris_flowers) + interval * range(0.25, 0.75) + x(species) + y(petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("Middle half of petal length per species")

(data(iris_flowers) + interval * range(0.25, 0.75) + x(col.species) + y(col.petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("Middle half of petal length per species"))

data(iris_flowers) + interval * range(0.25, 0.75) + x(:species) +
  y(:petal_length) + x_label("Species") + y_label("Petal length (cm)") +
  title("Middle half of petal length per species")

plot(data(iris_flowers), layer(interval, range(0.25, 0.75)),
  x(col.species), y(col.petal_length), x_label("Species"),
  y_label("Petal length (cm)"),
  title("Middle half of petal length per species"))

“Given the iris flowers: intervals derived by range from 0.25 to 0.75, x is species, y is petal length.”

The two numbers are quantile probabilities, so range(0.25, 0.75) spans the first quartile to the third. That span is the interquartile range, and it is the same one a box draws as its body, computed by the same rule. Any other pair works the same way: range(0.1, 0.9) keeps the middle 80%, and range(0.05, 0.95) the middle 90%.

Bare range is range(0, 1), because the quantile at 0 is the minimum and the quantile at 1 is the maximum. So interval * range is unchanged, and the two arguments only make its default visible. Naming one end leaves the other at its extreme, so range(high = 0.9) runs from the smallest value to the 90th percentile.

Both ends are probabilities. A number below 0 or above 1 is refused, not corrected:

data(iris_flowers) + interval * range(0.25, 75) + x(species) + y(petal_length)

data(iris_flowers) + interval * range(0.25, 75) + x(col.species) + y(col.petal_length)

data(iris_flowers) + interval * range(0.25, 75) + x(:species) +
  y(:petal_length)

plot(data(iris_flowers), layer(interval, range(0.25, 75)), x(col.species),
  y(col.petal_length))

Error:
! gog: `range(high = 75)` is not a probability — the band's ends are quantiles, so each is between 0 and 1. `range(0.25, 0.75)` is the middle half, `range(0.1, 0.9)` the middle 80 percent, and bare `range` the whole group.

For a low and a high you computed yourself, in the data’s own units, use bounds instead.

25.13 `confidence`: the interval of the mean

range shows the full spread; often you want the uncertainty of the mean instead: an error bar. confidence computes the mean’s confidence interval per group (mean ± t·se, with se the standard error: the t-interval), emitting a low, a high, and a center. That center is what turns the whisker into a pointrange:

data(iris_flowers) + interval * confidence(0.95) + x(species) + y(petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("95% CI of mean petal length")

(data(iris_flowers) + interval * confidence(0.95) + x(col.species) + y(col.petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("95% CI of mean petal length"))

data(iris_flowers) + interval * confidence(0.95) + x(:species) +
  y(:petal_length) + x_label("Species") + y_label("Petal length (cm)") +
  title("95% CI of mean petal length")

plot(data(iris_flowers), layer(interval, confidence(0.95)),
  x(col.species), y(col.petal_length), x_label("Species"),
  y_label("Petal length (cm)"), title("95% CI of mean petal length"))

“Given the iris flowers: intervals derived by confidence at 95%, x is species, y is petal length.”

The level is a parameter, exactly like bin(30) and density(2): confidence alone is 95%, confidence(0.99) is wider. It uses the t-distribution (not a fixed 1.96), so a small group’s interval is honestly wider, and a group of one, no spread, collapses to a point.

The center dot appears because the statistic has a center; range has none, so it draws a bare whisker. That is the rule for the whole family: the transform decides whether there is a center, and the mark draws what it is given. So the difference between an error bar and a pointrange is which statistic you ask for, not another mark to remember.

25.14 `deviation`: the spread of the data

confidence says how precisely a mean is known. deviation says how spread the values behind it are, which is a different question with the same picture. It emits the same low, high and center, so it also draws a pointrange:

data(iris_flowers) + interval * deviation + x(species) + y(petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("Mean petal length, with one standard deviation")

(data(iris_flowers) + interval * deviation + x(col.species) + y(col.petal_length) +
  x_label("Species") + y_label("Petal length (cm)") +
  title("Mean petal length, with one standard deviation"))

data(iris_flowers) + interval * deviation + x(:species) +
  y(:petal_length) + x_label("Species") + y_label("Petal length (cm)") +
  title("Mean petal length, with one standard deviation")

plot(data(iris_flowers), layer(interval, deviation), x(col.species),
  y(col.petal_length), x_label("Species"), y_label("Petal length (cm)"),
  title("Mean petal length, with one standard deviation"))

“Given the iris flowers: intervals derived by deviation, x is species, y is petal length.”

The multiplier is a parameter, like confidence’s level: deviation alone reaches one standard deviation on each side of the mean, and deviation(2) reaches two, so that band is four standard deviations wide. The standard deviation is the sample one, dividing by n − 1, so a group of one has no spread to show and collapses to a point exactly as confidence does.

Neither transform makes the other redundant, and neither does range. On the iris flowers the confidence band is under a third as long as the deviation band: fifty measurements locate a mean precisely, and their spread stays as wide as it was. Adding rows narrows the first band and does not change the second. Interval draws the two side by side and says which sentence each one supports.

range(0.25, 0.75) answers a third version of the question. It reads the sorted values, so it reports where the middle half of the values actually lie. deviation starts from the mean instead, and reports a distance out from it. On a skewed group they disagree, and the disagreement is informative rather than a defect.

25.15 `bounds`: a pre-computed low/high pair

range and confidence compute the pair from raw values. bounds is their counterpart that computes nothing. You already have the low and the high in two columns, computed before the plot: a model’s standard error, a psychometric standard error of measurement, a bootstrap interval. bounds(lower, upper) reshapes those two columns into the very same low/high pair, so a ribbon or interval draws them with no new atom and no new rule:

`score_band`: first 5 of 21 rows
score	expected	lower	upper
0	0	-1.610304	1.610304
1	1	-1.865557	3.865557
2	2	-1.944424	5.944424
3	3	-1.694803	7.694803
4	4	-1.259232	9.259232

data(score_band) + ribbon * bounds(lower, upper) + x(score) +
  y_label("Expected true score") + x_label("Raw score") +
  title("A pre-computed band: ribbon with bounds")

(data(score_band) + ribbon * bounds(col.lower, col.upper) + x(col.score) +
  y_label("Expected true score") + x_label("Raw score") +
  title("A pre-computed band: ribbon with bounds"))

data(score_band) + ribbon * bounds(:lower, :upper) + x(:score) +
  y_label("Expected true score") + x_label("Raw score") +
  title("A pre-computed band: ribbon with bounds")

plot(data(score_band), layer(ribbon, bounds(col.lower, col.upper)),
  x(col.score), y_label("Expected true score"), x_label("Raw score"),
  title("A pre-computed band: ribbon with bounds"))

“Given the score band: a ribbon from lower to upper, x is score.”

It differs from every transform above in two ways that both follow from reshaping, not computing. Its arguments are column names, not a number: it has to be told which columns hold the bounds. And it needs no y(): those two columns are the extents, so (like count, which invents its own y) the mark takes its low and high from bounds and the y-axis fits them. This is exactly where gog stops. It computes a deliberate few statistics; everything else you compute yourself, and bounds is how a band you already have gets drawn.

It is legal on the five marks that can draw a low/high pair, each a different geometry over the same statistic, the No Exceptions rule that gives one bin three histograms:

ribbon fills the pair (a filled band),
interval draws a whisker across it at each x,
line traces its two boundaries (the unfilled band),
step traces them as staircases (stepped control limits), and
zone shades the region between them, the band that names its own sides rather than tracking a curve.

So line * bounds(lower, upper) + style(pattern = "dashed") is the unfilled, dashed error band (the two ± curves with no fill between them) and it is the same pair, drawn by a different mark, that ribbon * bounds fills. On any mark that draws neither a pair nor a single value, bounds is refused with direction.

25.16 `partition`: a whole among nested parts

Nearly every transform so far reads a column and writes a summary of it; bounds alone reshapes what you already had. partition reads a hierarchy (two or three columns spelling the path down to each row) and writes the region each node of that hierarchy occupies: four edges and a center, which is exactly what bin writes for a cell. That is what makes it a transform rather than a coordinate space, and it is why zone needs nothing new to draw the result.

The levels can run two ways, and the difference is one parameter. Nested is the default: every level divides the same axis, and the other axis steps out one ring per level. On a flat plot that reading is the icicle. In polar() it is the sunburst, and that chapter works it through in full.

Crossed is the other, and it is the one this chapter draws, because it stays flat. The levels alternate: the first divides the width, the second divides the height within each of those columns, a third would divide the width again. The plot it makes is the mosaic.

25.16.1 The mosaic

A cross-tabulation raises two questions at once: how the two categories relate, and how many people each cell counts. A mosaic answers both in one picture.

commutes is a survey of how people get to work in four cities, one row per city and mode, with a count:

`commutes`: first 6 of 16 rows
city	mode	people
Ashford	Car	980
Ashford	Bus or tram	340
Ashford	Bicycle	110
Ashford	On foot	110
Brightwell	Car	420
Brightwell	Bus or tram	380

data(commutes) + zone * partition(city, mode, cross = TRUE) + x(people) +
  color(mode) + style(border_color = "white", border_size = 2) +
  x_label("People surveyed") + title("How four cities get to work")

(data(commutes) + zone * partition(col.city, col.mode, cross = True) + x(col.people) +
  color(col.mode) + style(border_color = "white", border_size = 2) +
  x_label("People surveyed") + title("How four cities get to work"))

data(commutes) + zone * partition(:city, :mode, cross = true) +
  x(:people) + color(:mode) +
  style(border_color = "white", border_size = 2) +
  x_label("People surveyed") + title("How four cities get to work")

plot(data(commutes),
  layer(zone, partition(col.city, col.mode, { cross: true })),
  x(col.people), color(col.mode),
  style({ border_color: "white", border_size: 2 }),
  x_label("People surveyed"), title("How four cities get to work"))

“Given the commutes: zones derived by partition through city and mode, crossed, x is people, color by mode.”

Two variables, and the plot reads them in two directions at once. A column’s width is that city’s share of everyone surveyed, so Ashford’s column is three times Millbrook’s; a cell’s height is that mode’s share within its own city, so every column fills the panel whatever its width. The consequence is the one a mosaic is drawn for: a cell’s area is the number of people in it, and the eye compares areas across the whole panel without being told to.

That is also what makes Millbrook readable rather than misleading. Its cycling share is the largest of the four and its column is the narrowest, so the plot reports the finding and how many people it rests on at once, which four equal-width stacked bars cannot do at all.

The two axes therefore carry different quantities, which is worth looking at before reading anything off them. The horizontal runs 0 to the total in the data’s own units, 3,790 people, exactly as the icicle’s measure axis does, so composing with proportion turns it into shares. The vertical runs 0 to 1 and can do nothing else, because what a height means is a share of its own column and the grand total is not the thing being divided.

partition writes a name column holding each node’s own label, and a text layer reads it. The sunburst uses the same layer, for the same reason: a shallower partition of the same table lands its nodes in exactly the same places, so the second layer needs no filtering and no arithmetic of its own.

data(commutes) + x(people) +
  zone * partition(city, mode, cross = TRUE) + color(mode) +
  style(border_color = "white", border_size = 2) +
  text * partition(city, cross = TRUE) + label(name) +
  x_label("People surveyed") + title("The same plot, with its columns named")

(data(commutes) + x(col.people) +
  zone * partition(col.city, col.mode, cross = True) + color(col.mode) +
  style(border_color = "white", border_size = 2) +
  text * partition(col.city, cross = True) + label(col.name) +
  x_label("People surveyed") + title("The same plot, with its columns named"))

data(commutes) + x(:people) +
  zone * partition(:city, :mode, cross = true) + color(:mode) +
  style(border_color = "white", border_size = 2) +
  text * partition(:city, cross = true) + label(:name) +
  x_label("People surveyed") +
  title("The same plot, with its columns named")

plot(data(commutes), x(col.people),
  layer(zone, partition(col.city, col.mode, { cross: true })),
  color(col.mode), style({ border_color: "white", border_size: 2 }),
  layer(text, partition(col.city, { cross: true })), label(col.name),
  x_label("People surveyed"),
  title("The same plot, with its columns named"))

“Given the commutes: x is people, zones derived by partition through city and mode, crossed, color by mode, and also text derived by partition through city, crossed, label by name.”

The labels sit at the middle of each column because that is where the center of a column is, and style(nudge = ) is what moves them if you want them along the top. A mosaic’s column names belong on an axis, but gog’s category axis gives every slot the same width, which is the one thing a mosaic’s columns are not. So the names arrive as a layer.

The cell edges are a style() setting like any other, and they matter more here than anywhere else in the book. Without them, two neighboring cells that share a color merge into one shape and the column boundary disappears:

data(commutes) + zone * partition(city, mode, cross = TRUE) + x(people) +
  color(mode) + x_label("People surveyed") +
  title("The same plot with no edges")

(data(commutes) + zone * partition(col.city, col.mode, cross = True) + x(col.people) +
  color(col.mode) + x_label("People surveyed") +
  title("The same plot with no edges"))

data(commutes) + zone * partition(:city, :mode, cross = true) +
  x(:people) + color(:mode) + x_label("People surveyed") +
  title("The same plot with no edges")

plot(data(commutes),
  layer(zone, partition(col.city, col.mode, { cross: true })),
  x(col.people), color(col.mode), x_label("People surveyed"),
  title("The same plot with no edges"))

Both plots draw the same numbers. The second is the reason a zone carries style(border_color = , border_size = ) at all: the mark is both a highlight behind a line, where a frame would distract from the data, and the cells that are the data.

One level is legal and means something: the spine plot, columns of varying width and nothing dividing them, which is the marginal distribution on its own.

data(commutes) + zone * partition(city, cross = TRUE) + x(people) +
  color(city) + style(border_color = "white", border_size = 2) +
  x_label("People surveyed") + title("The margin alone: a spine plot")

(data(commutes) + zone * partition(col.city, cross = True) + x(col.people) +
  color(col.city) + style(border_color = "white", border_size = 2) +
  x_label("People surveyed") + title("The margin alone: a spine plot"))

data(commutes) + zone * partition(:city, cross = true) + x(:people) +
  color(:city) + style(border_color = "white", border_size = 2) +
  x_label("People surveyed") + title("The margin alone: a spine plot")

plot(data(commutes), layer(zone, partition(col.city, { cross: true })),
  x(col.people), color(col.city),
  style({ border_color: "white", border_size: 2 }),
  x_label("People surveyed"), title("The margin alone: a spine plot"))

Everything partition refuses nested it refuses crossed, because the refusals are about the tree and not about the layout. A level must be a name rather than a number. A branch may stop early, but it may not have a hole in the middle. And an interior node carrying a value of its own is the one genuine ambiguity, so it is refused rather than guessed. The sunburst chapter meets each of them where they are easiest to see.

25.17 `flow`: a magnitude through its stages

Passengers are counted by class, then by whether they survived. The question is how many of each class reached each outcome.

flow is partition’s neighbor. Both read categorical columns named in the atom, both place their own marks, and both feed more than one mark from one computation. Where partition divides a whole among nested parts, flow carries a magnitude through a sequence of stages. One row of the table is one path through every stage the atom names.

`titanic`: first 5 of 32 rows
class	sex	age	survived	n
1st	Male	Child	No	0
2nd	Male	Child	No	0
3rd	Male	Child	No	35
Crew	Male	Child	No	0
1st	Female	Child	No	0

data(titanic) + y(n) +
  ribbon * flow(class, survived) + color(class) +
  zone * flow(class, survived) +
  style(color = "gray") +
  title("Who survived, by class")

(data(titanic) + y(col.n) +
  ribbon * flow(col["class"], col.survived) + color(col["class"]) +
  zone * flow(col["class"], col.survived) +
  style(color = "gray") +
  title("Who survived, by class"))

data(titanic) + y(:n) + ribbon * flow(:class, :survived) + color(:class) +
  zone * flow(:class, :survived) + style(color = "gray") +
  title("Who survived, by class")

plot(data(titanic), y(col.n),
  layer(ribbon, flow(col.class, col.survived)), color(col.class),
  layer(zone, flow(col.class, col.survived)), style({ color: "gray" }),
  title("Who survived, by class"))

“Given titanic: y is n, ribbons derived by flow through class and survived, color by class, and also zones derived by flow through class and survived.”

The ribbon layer draws the bands, one per path, as thick as the path’s total at both ends. The zone layer draws each stage’s slots, stacked with no padding, so the measure axis is drawn with its usual ticks and reads true counts. A text layer names the slots. The Flow chapter builds the full diagram and gives the rules; this section only places the atom in the family.

25.18 `layout`: a graph placed from an edge table

partition, flow, layout and cluster each compute a whole picture rather than a column. layout is the third of them, and the first whose output draws in a space of its own. layout(from, to) reads the two endpoint columns of an edge table and computes a position for every distinct name, in the engine, the same way in every language. Three marks read the placement inside network(): edge the connections, point the nodes, text their names. Its positions mean nothing as quantities, so its space draws no axes, and the Network chapter is where the family lives.

25.19 `cluster`: the closest leaves joined first

cluster is the fourth of the picture-computing transforms, and the question it answers is which things are most alike. It treats each category of an axis as a leaf of a tree, joins the two closest leaves, then the next two, and keeps joining until one tree holds them all. Each leaf is described by a profile: cluster(amount, over = nutrient) says that one leaf’s profile is its amount in every category of nutrient.

`nutrients`: first 5 of 40 rows
food	nutrient	amount
salmon	protein	25.4
salmon	fat	12.4
salmon	carbs	0.0
salmon	fiber	0.0
salmon	iron	0.5

data(nutrients) + path * cluster(amount, over = nutrient) + x(food) +
  title("Eight foods, joined by what they are made of")

(data(nutrients) + path * cluster(col.amount, over=col.nutrient) + x(col.food) +
  title("Eight foods, joined by what they are made of"))

data(nutrients) + path * cluster(:amount, over = :nutrient) + x(:food) +
  title("Eight foods, joined by what they are made of")

plot(data(nutrients),
  layer(path, cluster(col.amount, { over: col.nutrient })), x(col.food),
  title("Eight foods, joined by what they are made of"))

“Given nutrients: paths derived by cluster on amount, over nutrient, x is food.”

path draws the tree, and the height of a join says how different its two branches are. The same transform on zone reorders a tile plot’s slots to the tree’s leaf order. The Cluster chapter grows the tree, orders the tiles, and builds the clustered heatmap; this section only places the atom in the family.

25.20 `dodge`: side by side instead of on top

Every transform above answers a question about the data. The four that remain answer none. dodge is a different kind of atom: a collision modifier. It writes no new values; it moves the marks that a color split would otherwise stack at one position, setting them side by side instead.

When color splits a bar, a box or an interval into groups, the groups share their category’s slot and pile up. A histogram reads well that way, because the overlapping bars are translucent. A grouped bar chart does not, because there you want the groups set side by side. dodge does exactly that:

`gm_eras`: first 5 of 284 rows
country	continent	year	life	population	gdp	era
Afghanistan	Asia	1957	30.332	9240934	820.8530	1957
Afghanistan	Asia	2007	43.828	31889923	974.5803	2007
Albania	Europe	1957	59.280	1476505	1942.2842	1957
Albania	Europe	2007	76.423	3600523	5937.0295	2007
Algeria	Africa	1957	45.685	10270856	3013.9760	1957

data(gm_eras) + bar * mean * dodge + x(continent) + y(life) + color(era) +
  y_label("Mean life expectancy") +
  title("Mean life expectancy by continent, 1957 vs 2007")

(data(gm_eras) + bar * mean * dodge + x(col.continent) + y(col.life) + color(col.era) +
  y_label("Mean life expectancy") +
  title("Mean life expectancy by continent, 1957 vs 2007"))

data(gm_eras) + bar * mean * dodge + x(:continent) + y(:life) +
  color(:era) + y_label("Mean life expectancy") +
  title("Mean life expectancy by continent, 1957 vs 2007")

plot(data(gm_eras), layer(bar, mean, dodge), x(col.continent),
  y(col.life), color(col.era), y_label("Mean life expectancy"),
  title("Mean life expectancy by continent, 1957 vs 2007"))

“Given the gapminder eras: bars derived by mean and dodge, x is continent, y is life, color by era.”

The two eras sit next to each other in every continent, each bar half as wide, together filling exactly the slot one bar would; the gap between continents is untouched. There is no width to set: dodge derives it from the number of groups (G groups, each 1/G of the slot). And because the offset resolves the overlap outright, dodged bars draw solid: the translucent fill exists only to see through overlapping bars, and none overlap now.

It is written after * like any transform, so it composes cleanly: bar * mean * dodge aggregates first and dodges the result; box * dodge dodges a box’s own summary; interval * range * dodge dodges grouped whiskers. Here it sets each continent’s two distributions side by side:

data(gm_eras) + box * dodge + x(continent) + y(life) + color(era) +
  y_label("Life expectancy") +
  title("Life expectancy distribution, 1957 vs 2007")

(data(gm_eras) + box * dodge + x(col.continent) + y(col.life) + color(col.era) +
  y_label("Life expectancy") +
  title("Life expectancy distribution, 1957 vs 2007"))

data(gm_eras) + box * dodge + x(:continent) + y(:life) + color(:era) +
  y_label("Life expectancy") +
  title("Life expectancy distribution, 1957 vs 2007")

plot(data(gm_eras), layer(box, dodge), x(col.continent), y(col.life),
  color(col.era), y_label("Life expectancy"),
  title("Life expectancy distribution, 1957 vs 2007"))

“Given the gapminder eras: boxes derived by dodge, x is continent, y is life, color by era.”

Which marks dodge serves is decided by what a mark is. It subdivides a width (a bar’s thickness, a box’s body, a whisker’s slot) so it is defined for bar, box and interval. A point has no width to subdivide (its overlap answer is jitter, below); a line or area is offset by accumulating rather than subdividing (that is stack, next). Ask for dodge on one of those and gog refuses, naming the collision modifier that fits. That is the discipline the rest of the kernel follows: an atom combines with the marks it was defined for, and the refusal names the atom that fits rather than drawing something wrong.

dodge also needs a split to separate: a color (or group) binding. Without one there is a single mark per slot and nothing to set beside anything, so bar * dodge on its own is refused toward adding the color, never accepted as a word that quietly does nothing.

25.21 `stack`: piled up, not side by side

Side by side, two eras’ populations never show what they add up to, and when the total means something the pile is the better picture.

stack is dodge’s sibling: the other way to resolve a color split’s overlap. Where dodge sets the groups beside each other, stack piles them on top of each other along the measure axis, each group starting where the one below it ended. It is the collision modifier for the marks that accumulate: bar and area pile heights, and point piles rows.

Take the dodge example and change one atom:

data(gm_eras) + bar * sum * stack + x(continent) + y(population) + color(era) +
  y_label("Population") +
  title("Population by continent, 1957 and 2007 stacked")

(data(gm_eras) + bar * sum * stack + x(col.continent) + y(col.population) + color(col.era) +
  y_label("Population") +
  title("Population by continent, 1957 and 2007 stacked"))

data(gm_eras) + bar * sum * stack + x(:continent) + y(:population) +
  color(:era) + y_label("Population") +
  title("Population by continent, 1957 and 2007 stacked")

plot(data(gm_eras), layer(bar, sum, stack), x(col.continent),
  y(col.population), color(col.era), y_label("Population"),
  title("Population by continent, 1957 and 2007 stacked"))

“Given the gapminder eras: bars derived by sum and stack, x is continent, y is population, color by era.”

Notice the measure also changed, from mean life to sum population. You stack quantities that add up: two populations sum to a total that means something, so the full bar reads as “everyone, both eras”. Two mean life expectancies do not add (a bar reaching 130 years is nonsense), so that comparison wants dodge, side by side. The grammar writes either; the data decides which is honest. Because the pile resolves the overlap, the bars draw solid, and the axis grows to the tallest stack: Asia passes five billion, the sum of its two eras.

25.21.1 Stacked areas: the case `stack` was built for

A split area is where overlap costs the most: filled regions hide one another, and no opacity setting recovers them (the area chapter shows the warning gog raises). stack is the answer. Each region sits on the cumulative height of the ones below, so they abut into one band instead of hiding each other, the classic “parts of a whole, over time”:

data(gm_all) + area * sum * stack + x(year) + y(population) + color(continent) +
  y_label("Population") +
  title("World population by continent, stacked")

(data(gm_all) + area * sum * stack + x(col.year) + y(col.population) + color(col.continent) +
  y_label("Population") +
  title("World population by continent, stacked"))

data(gm_all) + area * sum * stack + x(:year) + y(:population) +
  color(:continent) + y_label("Population") +
  title("World population by continent, stacked")

plot(data(gm_all), layer(area, sum, stack), x(col.year),
  y(col.population), color(col.continent), y_label("Population"),
  title("World population by continent, stacked"))

“Given all the gapminder years: areas derived by sum and stack, x is year, y is population, color by continent.”

Asia sits at the foot; Europe, Africa, the Americas and Oceania pile on top, and the band’s upper edge is the world total climbing from 2.4 billion to 6.25. Read any one color’s thickness for that continent’s share.

25.21.2 `stack(share = TRUE)`: every pile filled to one

Reading a thickness is the hard part of the plot above, because each band’s baseline moves under it. When the composition is the question and the total is not, stack(share = TRUE) divides every pile by its own slot’s total, so all of them reach 1 and only the proportions are left. That is filling the pile:

data(gm_all) + area * sum * stack(share = TRUE) + x(year) + y(population) +
  color(continent) + title("Every year's composition, filled to one")

(data(gm_all) + area * sum * stack(share = True) + x(col.year) + y(col.population) +
  color(col.continent) + title("Every year's composition, filled to one"))

data(gm_all) + area * sum * stack(share = true) + x(:year) +
  y(:population) + color(:continent) +
  title("Every year's composition, filled to one")

plot(data(gm_all), layer(area, sum, stack({ share: true })), x(col.year),
  y(col.population), color(col.continent),
  title("Every year's composition, filled to one"))

“Given all the gapminder years: areas derived by sum and stack as shares, x is year, y is population, color by continent.”

Now Asia’s band holds about three fifths and Europe’s visibly narrows across those 55 years, both of which the plot above hides inside a rising total. The axis says Share, because whatever the numbers were before, they are fractions of one now.

This is not a second name for proportion. The two divide by different totals. proportion divides by the whole plot’s, so its bars still say how big each slot is. This one divides by the slot’s own, which deliberately throws that away. And only this one composes with a measurement it was not given: sum of a population column is exactly the case proportion has no way to say. Filling is a position adjustment: it changes where the marks sit and what the scale reads, never what was counted, which is why it belongs to stack and not to a statistic.

A bar fills the same way, and that reading is the chart the name comes from: the 100% stacked bar, worked through in the bar chapter. One parameter, both geometries, exactly as plain stack hands both marks the same span and lets each draw it: No Exceptions.

Which marks stack serves is decided by what a mark is: the same rule dodge follows. What stack computes is one thing for every mark, a span from a foot to a top along the measure axis, and what differs is how a geometry draws that span. A bar fills it, an area fills it across x, and a point, having no length to stretch across it, spends it on how many dots there are (the dot plot, just below). A box or interval is set beside its neighbors rather than piled (that is dodge); an unfilled line has nothing to fill and stack (use area); a ribbon already spans a low to a high, so it measures no height from a baseline. Ask for stack on one of those and gog refuses, naming the collision modifier that fits. For bar and area it also needs a color (or group) split to pile: bar * stack alone is refused toward adding the color, never accepted as a word that quietly does nothing.

25.21.3 `stack(baseline = )`: where the pile sits

Two things about a pile can be changed. Its heights are one, and share is how you change them. Where the pile sits is the other, and baseline is how you change that.

The default is "zero". Every pile sits on the axis, which is both plots above. That layout has a known weakness. Only the bottom band has a shape you can read. Every band above it rests on the sum of the bands below, so its shape carries all of their movement as well as its own.

The weakness is easiest to see when each series peaks at a different time. Here are four music genres over half a year, in the listening table:

`listening`: first 5 of 104 rows
week	genre	plays
1	Folk	38.0
2	Folk	36.4
3	Folk	38.6
4	Folk	39.5
5	Folk	38.3

data(listening) + area * stack + x(week) + y(plays) + color(genre) +
  y_label("Plays") + title("Stacked on zero")

(data(listening) + area * stack + x(col.week) + y(col.plays) + color(col.genre) +
  y_label("Plays") + title("Stacked on zero"))

data(listening) + area * stack + x(:week) + y(:plays) + color(:genre) +
  y_label("Plays") + title("Stacked on zero")

plot(data(listening), layer(area, stack), x(col.week), y(col.plays),
  color(col.genre), y_label("Plays"), title("Stacked on zero"))

Techno is the red band. It looks like it grows to about week 17 and then collapses. Part of that shape is Techno, and part of it is Folk and Jazz shrinking underneath it. You cannot tell which part is which.

"center" places each pile so that its middle is at zero:

data(listening) + area * stack(baseline = "center") + x(week) + y(plays) +
  color(genre) + title("Centered: the ThemeRiver")

(data(listening) + area * stack(baseline = "center") + x(col.week) + y(col.plays) +
  color(col.genre) + title("Centered: the ThemeRiver"))

data(listening) + area * stack(baseline = "center") + x(:week) +
  y(:plays) + color(:genre) + title("Centered: the ThemeRiver")

plot(data(listening), layer(area, stack({ baseline: "center" })),
  x(col.week), y(col.plays), color(col.genre),
  title("Centered: the ThemeRiver"))

“Given the listening table: areas derived by stack on the center baseline, x is week, y is plays, color by genre.”

That symmetric layout is the ThemeRiver (Havre et al., 2000), named for the plot it was built to draw: how much each theme is written about, over time.

"wiggle" instead picks the foot that makes the bands as flat as it can. Thick bands are held steadiest, because a reader looks at them first. The movement is pushed into the thin bands instead:

data(listening) + area * stack(baseline = "wiggle") + x(week) + y(plays) +
  color(genre) + title("Flattest: the streamgraph")

(data(listening) + area * stack(baseline = "wiggle") + x(col.week) + y(col.plays) +
  color(col.genre) + title("Flattest: the streamgraph"))

data(listening) + area * stack(baseline = "wiggle") + x(:week) +
  y(:plays) + color(:genre) + title("Flattest: the streamgraph")

plot(data(listening), layer(area, stack({ baseline: "wiggle" })),
  x(col.week), y(col.plays), color(col.genre),
  title("Flattest: the streamgraph"))

“Given the listening table: areas derived by stack on the wiggle baseline, x is week, y is plays, color by genre.”

Now Techno is one clean shape, widening and then narrowing, which is what the numbers actually say. Folk, the blue band, is a steady decline rather than a wedge. No band changed thickness between these three plots. Only the place each pile sits changed.

The last plot is a streamgraph (Byron & Wattenberg, 2008). The middle one is a ThemeRiver. The two are often confused and they are not the same layout: one is symmetric, the other is chosen to reduce movement. The streamgraph’s designers argue readers compare values more accurately on it, and that argument is the reason this option exists (Byron & Wattenberg, 2008).

Notice also that these two plots have no numbers on the measure axis. Once a pile has moved off zero, no value on that axis means anything. A band drawn from 12 to 20 says “this group is 8 here”, and the 12 and the 20 only record where the pile was placed. So gog draws no ticks and no axis name, for the same reason it leaves an empty heatmap cell unpainted. The scale is unchanged, so thicknesses stay comparable everywhere in the plot, and thickness is what a streamgraph asks you to read.

baseline is not an area-only parameter. A bar has a pile too, so it is placed the same way:

data(listening) + bar * stack(baseline = "center") + x(week) + y(plays) +
  color(genre) + title("The same baseline, drawn as bars")

(data(listening) + bar * stack(baseline = "center") + x(col.week) + y(col.plays) +
  color(col.genre) + title("The same baseline, drawn as bars"))

data(listening) + bar * stack(baseline = "center") + x(:week) +
  y(:plays) + color(:genre) + title("The same baseline, drawn as bars")

plot(data(listening), layer(bar, stack({ baseline: "center" })),
  x(col.week), y(col.plays), color(col.genre),
  title("The same baseline, drawn as bars"))

One parameter, both geometries. share follows the same rule, and so does plain stack, which hands both marks one span and lets each draw it: No Exceptions.

Moving a pile gives up the origin of the measure axis. Only a flat plot can give that up, so the other spaces refuse:

data(listening) + area * stack(baseline = "wiggle") + x(week) + y(plays) +
  color(genre) + polar()

(data(listening) + area * stack(baseline = "wiggle") + x(col.week) + y(col.plays) +
  color(col.genre) + polar())

data(listening) + area * stack(baseline = "wiggle") + x(:week) +
  y(:plays) + color(:genre) + polar()

plot(data(listening), layer(area, stack({ baseline: "wiggle" })),
  x(col.week), y(col.plays), color(col.genre), polar())

Error:
! gog: `stack(baseline = "wiggle")` moves every pile off the origin of the measure axis, and `polar()` has no origin to spare. A displaced pile is a statement about a plane: in `polar()` the measure is an angle or a radius or a height the space has already fixed, so moving the foot asks for a place that is not on the plot. Keep the space and drop the baseline (`stack` alone, or `stack(share = TRUE)` for composition), or keep the baseline and draw it flat.
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

In polar() the measure is an angle or a radius. A radius of zero is the center of the circle, so moving a pile off it asks for a negative radius, which is not a place on the plot.

25.21.4 A pile has one direction

stack has a third condition. The first two are about the sentence. This one is about the numbers.

In a table of sales and returns, the returns arrive as negative numbers. The question is then whether a pile can grow downward at all.

A pile puts its groups end to end. Every member must therefore point the same way.

Negative values are allowed. A pile of them grows downward from zero, exactly as a positive pile grows up:

`drawdown`: first 5 of 6 rows
quarter	cost	kind
Q1	-5	refunds
Q2	-4	refunds
Q3	-6	refunds
Q1	-3	chargebacks
Q2	-2	chargebacks

data(drawdown) + bar * stack + x(quarter) + y(cost) + color(kind) +
  y_label("Cost") + title("A pile that grows down")

(data(drawdown) + bar * stack + x(col.quarter) + y(col.cost) + color(col.kind) +
  y_label("Cost") + title("A pile that grows down"))

data(drawdown) + bar * stack + x(:quarter) + y(:cost) + color(:kind) +
  y_label("Cost") + title("A pile that grows down")

plot(data(drawdown), layer(bar, stack), x(col.quarter), y(col.cost),
  color(col.kind), y_label("Cost"), title("A pile that grows down"))

A pile whose members disagree has no correct drawing. Suppose one group is 5 and another is -3 at the same position. The negative band would begin where the first band ended, at 5, and reach back down to 2. That region already belongs to the first group. A reader would see a solid block of length 3 where the number is negative. So gog refuses:

`mixed_signs`: first 5 of 6 rows
quarter	amount	kind
Q1	5	sales
Q2	5	sales
Q3	5	sales
Q1	2	returns
Q2	-3	returns

data(mixed_signs) + bar * stack + x(quarter) + y(amount) + color(kind)

data(mixed_signs) + bar * stack + x(col.quarter) + y(col.amount) + color(col.kind)

data(mixed_signs) + bar * stack + x(:quarter) + y(:amount) + color(:kind)

plot(data(mixed_signs), layer(bar, stack), x(col.quarter), y(col.amount),
  color(col.kind))

Error:
! gog: `bar * stack` piles the groups at one position on top of one another, and at quarter = Q2 they disagree in sign — `sales` is 5 and `returns` is -3. A pile has one direction: the band going the other way is drawn *inside* the ones below it, so the plot would show a block of 3 where the number is negative and read its sign backwards. Either keep the sign and drop the pile — `bar * dodge` sets the groups side by side instead, and a dodged bar keeps its own baseline, so a negative one hangs below the axis where a reader can see that it is negative — or keep the pile and drop the sign, stacking magnitudes you took the absolute value of in the host. (A pile whose members agree stays legal both ways: all-negative grows downward from zero exactly as all-positive grows up.)
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

The message names the position, both groups, and both numbers. A message that said only “some values are negative” would make you search the whole table for a pile you cannot see on the page.

There are two ways to fix this, and you choose by what you want the plot to show. If the sign matters, use dodge. Each bar then keeps its own baseline, so a negative bar hangs below the axis where a reader can see that it is negative. If the total matters, take the absolute value in the host language first. You are then stacking magnitudes, and the sentence says so.

stack(share = TRUE) fails for the same reason, and the failure is easier to see. A share is a fraction of the pile’s total. Here the total is 5 plus -3, which is 2, so the two shares come out as 2.5 and -1.5. No rescaling fixes that. The problem is in the pile itself, not in how the pile is drawn.

Piles at different positions may still point different ways. Each pile is read on its own. One quarter can be positive and the next negative in the same plot.

25.21.5 The dot plot

A histogram’s bars hide the rows inside them. When every country should stay visible, and countable, the mark to pile is the point.

A point has no length. Give it the same span every other mark gets and the only thing it can spend it on is itself: one dot per unit, piled up from the foot. The unit of a count is one row, so a pile of dots is a count you can count:

data(gapminder_2007) + point * bin * stack + x(life) +
  x_label("Life expectancy (years)") + title("One dot per country")

(data(gapminder_2007) + point * bin * stack + x(col.life) +
  x_label("Life expectancy (years)") + title("One dot per country"))

data(gapminder_2007) + point * bin * stack + x(:life) +
  x_label("Life expectancy (years)") + title("One dot per country")

plot(data(gapminder_2007), layer(point, bin, stack), x(col.life),
  x_label("Life expectancy (years)"), title("One dot per country"))

“Given gapminder 2007: points derived by bin and stack, x is life.”

That is a dot plot (Wilkinson, 1999), and it is the same sentence as the histogram at the top of this chapter with one atom changed and one added. Compare the two: bar * bin draws each tally as a length, and you read the shape off the outline; point * bin * stack draws each tally as that many dots, and every country in the table is on the page. Count the tallest pile and you get its bin’s count exactly, which no bar can offer.

Nothing here is a color split, and none is needed. bar and area pile groups, so they ask for one; a point piles rows, which every table already has. What it asks for instead is a transform that counts them, because a pile of 3.7 dots means nothing:

data(gapminder_2007) + point * mean * stack + x(continent) + y(life)

data(gapminder_2007) + point * mean * stack + x(col.continent) + y(col.life)

data(gapminder_2007) + point * mean * stack + x(:continent) + y(:life)

plot(data(gapminder_2007), layer(point, mean, stack), x(col.continent),
  y(col.life))

Error:
! gog: `point * stack` piles one dot per observation along the measure axis — the dot plot — so the measure has to be a count of rows, and nothing here counts them. Add the transform that does: `point * bin * stack` for a continuous axis (dots per interval), `point * count * stack` for a categorical one (dots per category).
gog: nothing was rendered. Fix the above, or set GOG_STRICT=0 to draw anyway.

The refusal names the two sentences that do count: bin for a continuous axis, and count for a categorical one. The second draws the bar chart’s abstraction with the unit left visible, one dot per row:

data(gapminder_2007) + point * count * stack + x(continent) +
  title("One dot per country, piled by continent")

(data(gapminder_2007) + point * count * stack + x(col.continent) +
  title("One dot per country, piled by continent"))

data(gapminder_2007) + point * count * stack + x(:continent) +
  title("One dot per country, piled by continent")

plot(data(gapminder_2007), layer(point, count, stack), x(col.continent),
  title("One dot per country, piled by continent"))

The dots pack tightly here and loosely on a small table, and that is not a setting. The rung is the fixed vertical gap between stacked dots, and it is one count unit on the y axis. A taller pile therefore subdivides the panel more finely, and a pile of ten dots in a tall panel leaves wide gaps. style(size = ) will enlarge the dots, but making them touch means fixing the panel’s height to the tallest pile, and no setting does that. The count axis is real either way, and that choice is deliberate: the alternative is to draw the dots touching and leave the axis with no meaning.

25.21.6 How many rows is too many

One dot per country works at 142 rows. A table twelve times that size is where the picture starts to fail.

Push the same sentence far enough and the dots stop being dots. Every year of gapminder rather than one is 1704 rows, and the tallest pile is 314 of them:

data(gm_all) + point * bin * stack + x(life) +
  x_label("Life expectancy (years)") + title("Every year, every country: 1704 dots")

(data(gm_all) + point * bin * stack + x(col.life) +
  x_label("Life expectancy (years)") + title("Every year, every country: 1704 dots"))

data(gm_all) + point * bin * stack + x(:life) +
  x_label("Life expectancy (years)") +
  title("Every year, every country: 1704 dots")

plot(data(gm_all), layer(point, bin, stack), x(col.life),
  x_label("Life expectancy (years)"),
  title("Every year, every country: 1704 dots"))

gog: `point * bin * stack` piled 314 dots into one column, and at that height they overlap instead of counting — a dot plot says what it says by showing every observation separately, so this is a histogram drawn dot by dot. For this many rows, `bar * bin` reads the same shape as a summary, and `line * density` as a curve. `style(size = )` shrinks the dots, which buys a little room.

gog draws it, and says what happened. The piles have become solid sticks: at 314 rungs the gap between two of them is under a pixel, so the dots overlap and nothing can be counted, which was this plot’s entire claim. What you are looking at is a histogram drawn dot by dot, and a costly one: it draws 1704 circles where bar * bin draws twelve rectangles.

The warning is not a row limit, and that matters. It compares one rung’s gap against one dot’s diameter, both of which gog knows once the panel is laid out. So the warning appears exactly when the picture degrades, and not when it does not. Shrink the dots with style(size = ) and the same table draws no warning; widen the piles and a smaller table draws one. A number like “more than 500 rows” would be wrong at some panel size or dot size, and there is no need to guess one when the real condition is available.

Which also means the warning is advice, not a limit. Two atoms usually answer it without giving up a single dot, and the first is the one people forget: more bins spread the same rows over more piles, so every pile gets shorter without anything getting smaller. Then the dots themselves:

data(gm_all) + point * bin(40) * stack + x(life) + style(size = 1.5) +
  x_label("Life expectancy (years)") + title("The same 1704 rows, cut finer and drawn smaller")

(data(gm_all) + point * bin(40) * stack + x(col.life) + style(size = 1.5) +
  x_label("Life expectancy (years)") + title("The same 1704 rows, cut finer and drawn smaller"))

data(gm_all) + point * bin(40) * stack + x(:life) + style(size = 1.5) +
  x_label("Life expectancy (years)") +
  title("The same 1704 rows, cut finer and drawn smaller")

plot(data(gm_all), layer(point, bin(40), stack), x(col.life),
  style({ size: 1.5 }), x_label("Life expectancy (years)"),
  title("The same 1704 rows, cut finer and drawn smaller"))

“Given all the gapminder years: points derived by bin into 40 and stack, x is life, with size 1.5.”

The columns are made of visible dots again, and every one of the 1704 is still on the page. Faceting is the third way, since fewer rows per panel means shorter piles in each.

What none of them changes is the arithmetic. Counting needs at least one dot-diameter of axis per row, so the tallest honest pile is about the panel’s height divided by a dot: a few hundred with dots you can see, a few dozen if you mean to count them. Past that a summary is the better answer, for the reason that makes the dot plot worth having at all. A bin width or a bandwidth is a guess about structure when there are thirty rows. It is a reliable description when there are ten thousand. So the dot plot stops working at the same size where a summary starts working well.

25.21.7 One column, split into its parts

Not every question comes with a category to stand bars on. The 264 winds are one total, and the only question is how it divides by direction.

Every stacked bar so far has stood at a position, one pile per category on the x-axis. Take the position away and the split still has something to divide: a single column, showing how one total breaks up.

data(winds) + bar * count * stack + color(direction)

data(winds) + bar * count * stack + color(col.direction)

data(winds) + bar * count * stack + color(:direction)

plot(data(winds), layer(bar, count, stack), color(col.direction))

“Given the winds: bars derived by count and stack, color by direction.”

There is no x() in that sentence, and none is needed. A bar normally requires a position because it has to stand somewhere. Here the color split is what divides the bar into parts, and there is one slot for those parts to share, so the position is no longer required. A required position may be dropped in only a few places in the grammar, and this is one of them. Bend this same plot into a circle and it is a pie, the subject of Polar.

Without a split there is nothing to divide, so the position is required again and the refusal says so. The statistic has to be one that means something with nothing to group by, too: count and sum reduce a set of rows to one number, while bin and density describe how values spread along an axis, and there is no axis here.

25.22 `jitter`: spread sideways, along the category

dodge and stack resolve the overlap of grouped marks. The third collision modifier resolves a different overlap: not groups colliding, but a mark’s own points landing on top of each other. A strip plot, every country as a dot above its continent, stacks its points into a single vertical line, and heavy overlap hides how many there are:

data(gapminder_2007) + point + x(continent) + y(life) + style(opacity = 0.5) +
  y_label("Life expectancy") + title("Strip plot: the points pile onto one line")

(data(gapminder_2007) + point + x(col.continent) + y(col.life) + style(opacity = 0.5) +
  y_label("Life expectancy") + title("Strip plot: the points pile onto one line"))

data(gapminder_2007) + point + x(:continent) + y(:life) +
  style(opacity = 0.5) + y_label("Life expectancy") +
  title("Strip plot: the points pile onto one line")

plot(data(gapminder_2007), point, x(col.continent), y(col.life),
  style({ opacity: 0.5 }), y_label("Life expectancy"),
  title("Strip plot: the points pile onto one line"))

jitter nudges each point a little to the side, so the pile spreads and the shape of each group shows:

data(gapminder_2007) + point * jitter + x(continent) + y(life) + style(opacity = 0.5) +
  y_label("Life expectancy") + title("point * jitter: the same points, spread to show density")

(data(gapminder_2007) + point * jitter + x(col.continent) + y(col.life) + style(opacity = 0.5) +
  y_label("Life expectancy") + title("point * jitter: the same points, spread to show density"))

data(gapminder_2007) + point * jitter + x(:continent) + y(:life) +
  style(opacity = 0.5) + y_label("Life expectancy") +
  title("point * jitter: the same points, spread to show density")

plot(data(gapminder_2007), layer(point, jitter), x(col.continent),
  y(col.life), style({ opacity: 0.5 }), y_label("Life expectancy"),
  title("point * jitter: the same points, spread to show density"))

“Given gapminder 2007: points derived by jitter, x is continent, y is life.”

Africa’s long tail of low values and Europe’s tight cluster near the top were both there before; the spread is what lets you read them.

jitter moves the category, never the value. Look again: every dot sits at exactly the same height as before; the spread is entirely sideways. This is the rule, not a coincidence. The horizontal axis is a category, and a category has no magnitude, so nudging a point along it changes nothing you could misread. The vertical axis is a measurement, and moving a point off its measured value would misreport the data, so jitter never touches it. (Spreading in both directions is the other way to do this. It moves a point off its measured value, which is why gog spreads along one axis only.) On a horizontal strip, with the category on y, the spread is vertical instead. jitter always spreads the categorical axis and leaves the measured one exact.

The spread is deterministic: it is computed from the data itself, not from a random-number generator or the clock, so the same plot drawn twice is pixel-for-pixel the same picture. A specification always means one plot.

25.22.1 How much to spread: `jitter(amount)`

The default spread is one guess at what reads well, and on your figure it may be too wide or too narrow.

The default spread fills a fraction of each category’s slot: wide enough to show the pile, narrow enough to leave a gap between categories so no point strays into a neighbor. When you want more or less, jitter(amount) scales it: a plain number multiplying the default, so jitter(0.5) is half the spread and jitter(2) twice it (a bare jitter is jitter(1)).

data(gapminder_2007) + point * jitter(0.5) + x(continent) + y(life) +
  style(opacity = 0.5) + y_label("Life expectancy") +
  title("point * jitter(0.5): a tighter spread")

(data(gapminder_2007) + point * jitter(0.5) + x(col.continent) + y(col.life) +
  style(opacity = 0.5) + y_label("Life expectancy") +
  title("point * jitter(0.5): a tighter spread"))

data(gapminder_2007) + point * jitter(0.5) + x(:continent) + y(:life) +
  style(opacity = 0.5) + y_label("Life expectancy") +
  title("point * jitter(0.5): a tighter spread")

plot(data(gapminder_2007), layer(point, jitter(0.5)), x(col.continent),
  y(col.life), style({ opacity: 0.5 }), y_label("Life expectancy"),
  title("point * jitter(0.5): a tighter spread"))

“Given gapminder 2007: points derived by jitter at 0.5, x is continent, y is life.”

The parameter has the same shape as density’s adjust, a dimensionless multiple of an automatic value. It is there for the same reason density and bin take a parameter while dodge does not. The jitter amount is a legibility choice with no single right value, whereas dodge’s width is determined by the group count. A value the writer must choose takes a parameter; a value gog can work out does not. (jitter(0) is legal and draws the plain strip plot, un-spread: gog never forbids the harmless.)

Two things set jitter apart from its siblings. It needs no color split: dodge and stack separate groups, but jitter separates a mark’s own coincident points, so a plain point * jitter with nothing but x and y is the whole expression. And it needs a categorical position axis to spread within: ask for jitter when both x and y are continuous and gog refuses, because there is no band to spread across and no axis it could move without misplacing a value. The answer for overplotting on two continuous axes is style(opacity = ), which reveals density without moving any point.

Which marks jitter serves is decided by what a mark is: the rule that divides the whole trio. jitter spreads a scatter of individual points, so it is defined for point alone. A bar, box or interval has a width, and its groups are set apart by dodge; a line or area is one connected shape, not a cloud of points. Ask for jitter on any of those and gog refuses, naming the collision modifier that fits.

Three of the four collision modifiers divide the marks by geometry and by axis: dodge subdivides a width, stack accumulates along the measure axis, jitter spreads along the categorical one. Every mark has an honest answer to overlap, and point has two, because it is the one mark that can collide on either axis: piled into a countable stack of dots along a measure, nudged apart within a slot along a category. The modifier follows the axis, which is why neither reading needs a third name.

25.23 `repel`: move the labels off each other

The fourth collision modifier answers a collision the first three cannot see. They all resolve marks that landed on one position, which is a fact about the data. A label is as wide as the word it draws, so two labels overlap at positions their points never shared, and no modifier computed from a position can find them. Thirty European countries, each named:

`gm_europe`: first 5 of 30 rows
country	continent	year	life	population	gdp
Albania	Europe	2007	76.423	3600523	5937.030
Austria	Europe	2007	79.829	8199783	36126.493
Belgium	Europe	2007	79.441	10392226	33692.605
Bosnia and Herzegovina	Europe	2007	74.852	4552198	7446.299
Bulgaria	Europe	2007	73.005	7322858	10680.793

data(gm_europe) + point + style(color = "#9e9e9e") +
  text + x(gdp) + y(life) + label(country) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("The names overlap")

(data(gm_europe) + point + style(color = "#9e9e9e") +
  text + x(col.gdp) + y(col.life) + label(col.country) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("The names overlap"))

data(gm_europe) + point + style(color = "#9e9e9e") + text + x(:gdp) +
  y(:life) + label(:country) + x_label("GDP per capita") +
  y_label("Life expectancy") + title("The names overlap")

plot(data(gm_europe), point, style({ color: "#9e9e9e" }), text,
  x(col.gdp), y(col.life), label(col.country), x_label("GDP per capita"),
  y_label("Life expectancy"), title("The names overlap"))

text * repel measures the space each label will take, and moves the labels until no two words overlap:

data(gm_europe) + point + style(color = "#9e9e9e") +
  text * repel + x(gdp) + y(life) + label(country) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("text * repel: the same names, moved apart")

(data(gm_europe) + point + style(color = "#9e9e9e") +
  text * repel + x(col.gdp) + y(col.life) + label(col.country) +
  x_label("GDP per capita") + y_label("Life expectancy") +
  title("text * repel: the same names, moved apart"))

data(gm_europe) + point + style(color = "#9e9e9e") + text * repel +
  x(:gdp) + y(:life) + label(:country) + x_label("GDP per capita") +
  y_label("Life expectancy") +
  title("text * repel: the same names, moved apart")

plot(data(gm_europe), point, style({ color: "#9e9e9e" }),
  layer(text, repel), x(col.gdp), y(col.life), label(col.country),
  x_label("GDP per capita"), y_label("Life expectancy"),
  title("text * repel: the same names, moved apart"))

“Given gapminder Europe: points, and also text derived by repel, x is gdp, y is life, label by country.”

Every label ends up outside its own dot, and one that moved far keeps a thin line back to its point. Two rules set where each label rests. First, a label rests beside its dot, on whichever side has room. repel pulls the label toward its dot and pushes it away from every word and every dot, and the label stops where the two balance. A label with no neighbors sits just above its dot. One with neighbors moves to whichever side is free. Second, when a label is pushed by another label and by its own dot at the same time, it moves off the other label and may rest over its dot. A word over a dot is still readable. Two words over each other are not.

Like jitter, the placement is deterministic: it depends only on the labels and the rows, never on a random-number generator, so one specification always draws the same picture. Like jitter, it needs no color split, since what it separates is the layer’s own labels.

repel takes no parameter, and jitter takes one, for the same reason. A jitter’s spread has no single right value, so the writer chooses it. A label moves exactly as far as the overlap requires and no further. That distance is computed, the way dodge computes a width, so there is nothing to set. The gap left between two labels is set by the font size, as the nudge’s distance is.

Above some number of labels, no arrangement fits at all. repel draws every one of them anyway and reports how many still overlap. A plot missing the names it could not fit would read as a plot with fewer rows, and nobody looking at it would know.

repel is text-only. Each of the other three serves the marks whose geometry fits it, and a label is the only glyph made of a word. Ask for it on a point and gog refuses, naming jitter; on a bar it names dodge. Its counterpart among the settings is style(nudge = ), a fixed shift in a direction you name, described in Text. The two compose: the nudge says which side a label prefers, and repel resolves the overlaps that remain.

The four collision modifiers are one family, and two questions divide it. Three of them ask what geometry does this mark have, and answer with a width, a measure or a band. The fourth asks what is this mark made of, and only one mark is made of words.

25.1 One variable, or two?

25.2 bin: histogram

25.2.1 Choosing the bin count

25.2.2 Why bar * bin, not a histogram() shortcut?

25.2.3 The same bin, cutting two axes

25.2.4 One of each: the mixed mesh

25.2.5 Which mesh: tiling

25.3 count: frequency aggregation

25.3.1 The same count, tallying two axes

25.4 proportion: relative frequency

25.4.1 The total is the whole plot, always

25.4.2 Normalizing something other than a tally

25.5 density: kernel density estimate

25.5.1 Choosing the bandwidth

25.5.2 One transform, four readings

25.6 What bin, density and smooth refuse

25.6.1 A fit needs rows to fit through

25.7 Chaining transforms

25.7.1 What cannot be chained, and why

25.8 Output columns

25.9 smooth: LOESS trend line

25.10 Aggregation family: sum, mean, median, max, min, quantile

25.10.1 The same six, grouping by a pair

25.10.2 Why these names?

25.11 quantile: one point in the distribution

25.12 range: a low/high pair

25.12.1 A band between two quantiles

25.13 confidence: the interval of the mean

25.14 deviation: the spread of the data

25.15 bounds: a pre-computed low/high pair

25.16 partition: a whole among nested parts