29  Where this ends, and gog begins

god manipulates tables. It does not draw them, and it is not going to, because drawing is a different job with a different vocabulary. A grammar that tried to do both would stop being small enough to hold in your head, which is the one thing this project will not trade.

That other grammar exists. gog is this project’s sibling: a grammar of graphics, by the same author, built on the same discipline and holding the same promise about day one and day two. Most real questions use both, because a question usually starts as a table that is the wrong shape and ends as a picture.

This chapter is one question answered end to end. Nothing else is loaded.

29.1 The table

Gapminder’s country panel: 142 countries, twelve observations each from 1952 to 2007, 1,704 rows in all. It is a real table rather than a demonstration, which is the point of using it here.

nations |> take(5)
country continent year life population gdp
Afghanistan Asia 1952 28.801 8425333 779.4453
Afghanistan Asia 1957 30.332 9240934 820.8530
Afghanistan Asia 1962 31.997 10267083 853.1007
Afghanistan Asia 1967 34.020 11537966 836.1971
Afghanistan Asia 1972 36.088 13079460 739.9811 
nations >> take(5)
country continent year life population gdp
Afghanistan Asia 1952 28.801 8425333 779.445314
Afghanistan Asia 1957 30.332 9240934 820.853030
Afghanistan Asia 1962 31.997 10267083 853.100710
Afghanistan Asia 1967 34.020 11537966 836.197138
Afghanistan Asia 1972 36.088 13079460 739.981106

Five rows of 1,704. Every table on this page is shown that way, because a chapter that prints 1,704 rows is not showing you anything.

29.2 The question

Which countries gained the most life expectancy, and what did the world look like while it happened?

The first half is the table’s job. Each country has twelve rows and the answer needs two of them, the earliest and the latest, which is what first and last are for. They answer by position, so a sort in front of them is what decides which position that is.

gains <- nations |>
  sort(year) |>
  summarize(started = first(life), ended = last(life), by = c(country, continent)) |>
  add(gained = ended - started) |>
  sort(descending(gained))

gains |> take(5)
country continent started ended gained
Oman Asia 37.578 75.640 38.062
Vietnam Asia 40.412 74.249 33.837
Indonesia Asia 37.468 70.650 33.182
Saudi Arabia Asia 39.875 72.777 32.902
Libya Africa 42.723 73.952 31.229 
gains = (nations
  >> sort(col.year)
  >> summarize(started = first(col.life), ended = last(col.life),
               by = [col.country, col.continent])
  >> add(gained = col.ended - col.started)
  >> sort(descending(col.gained)))

gains >> take(5)
country continent started ended gained
Oman Asia 37.578 75.640 38.062
Vietnam Asia 40.412 74.249 33.837
Indonesia Asia 37.468 70.650 33.182
Saudi Arabia Asia 39.875 72.777 32.902
Libya Africa 42.723 73.952 31.229

Oman gained more than thirty years of life expectancy in half a century. That is one sentence of god: sort, summarize by two columns, subtract, sort again.

The other end of the same table is a different question and the same pipeline with one word changed.

gains |> sort(gained) |> take(5)
country continent started ended gained
Zimbabwe Africa 48.451 43.487 -4.964
Swaziland Africa 41.407 39.613 -1.794
Zambia Africa 42.038 42.384 0.346
Lesotho Africa 42.138 42.592 0.454
Botswana Africa 47.622 50.728 3.106 
gains >> sort(col.gained) >> take(5)
country continent started ended gained
Zimbabwe Africa 48.451 43.487 -4.964
Swaziland Africa 41.407 39.613 -1.794
Zambia Africa 42.038 42.384 0.346
Lesotho Africa 42.138 42.592 0.454
Botswana Africa 47.622 50.728 3.106

29.3 When you need collect

Nothing above called collect, and the tables still appeared. That is worth explaining, because it is the question everyone asks once.

A pipeline is a plan, and it does not run when you write it. Printing one runs it. That is why every example so far has shown a table: this book prints each pipeline, so each one is asked for its answer.

collect is how you ask for the answer when something other than printing needs it. Handing a table to another function is exactly that case, and gog is about to be handed one.

top <- gains |> take(12) |> collect()
class(top)
[1] "data.frame"
top = collect(gains >> take(12))
type(top).__name__
'DataFrame'

So the rule is short. If you want to look at it, print it. If you want to use it, collect it.

29.4 The picture

Now gog. The table goes in, a mark and some channels say what to draw, and the sentence reads the way a god pipeline does.

data(top) + bar + x(gained) + y(country) + color(continent)
0 10 20 30 Jordan Egypt Gambia Yemen, Rep. West Bank and Gaza Nicaragua Korea, Rep. Libya Saudi Arabia Indonesia Vietnam Oman Country Gained Continent Asia Africa Americas 
gog.data(top) + gog.bar + gog.x(gog.col.gained) + gog.y(gog.col.country) + gog.color(gog.col.continent)
0 10 20 30 Jordan Egypt Gambia Yemen, Rep. West Bank and Gaza Nicaragua Korea, Rep. Libya Saudi Arabia Indonesia Vietnam Oman Country Gained Continent Asia Africa Americas

29.5 What the whole world was doing

The bar chart answers the question. This next one is why the pairing is worth having: the same panel, unsummarized, with one more channel.

play turns a column into time. Every country is a point, income runs along the bottom on a log scale, life expectancy up the side, and the year becomes the animation rather than another axis.

data(nations) + point + x(gdp, scale = "log") + y(life) +
  color(continent) + size(population) + play(year)
1K 10K 100K 40 60 80 1952 1957 1962 1967 1972 1977 1982 1987 1992 1997 2002 2007 Life Gdp Continent Asia Europe Africa Americas Oceania Population 60K 659.4M 1.3B 
(gog.data(nations) + gog.point + gog.x(gog.col.gdp, scale = "log") + gog.y(gog.col.life)
   + gog.color(gog.col.continent) + gog.size(gog.col.population) + gog.play(gog.col.year))
1K 10K 100K 40 60 80 1952 1957 1962 1967 1972 1977 1982 1987 1992 1997 2002 2007 Life Gdp Continent Asia Europe Africa Americas Oceania Population 60K 659.4M 1.3B

That is the Rosling chart, and neither package needed a special case to draw it. play is a channel like any other, so the year binds the way a color does.

29.6 What it replaced

Written in the tools this book keeps comparing itself against, those two steps need a reshaping function from one package and a plotting system from another, each with its own vocabulary, its own idea of what a column reference is, and its own rules about which arguments are quoted. The animation usually needs a third.

The saving is not the line count. It is that summarize and play come from the same design, so the second half of the analysis does not ask you to change how you are thinking. by means in gog what it means here. So does color.

29.7 The one thing to know about using both

In R, load them and go. R looks a column name up in your data before it looks anywhere else, and the two packages take care not to collide.

In Python, one name is claimed by both. col is how each of them names a column, so from god import * followed by from gog import * would leave you with one of them and no warning. Import gog by name instead, which is what the examples above do:

from god import *
import gog

Then col.life is god’s and gog.col.life is gog’s, and nothing is ambiguous. Appendix A says the same thing about every other name god takes over.

29.8 Where to go next

The graphics grammar has its own manual, at the gog book, written the way this one is: every plot on the page was drawn by the engine while the page was built.

Read it in the order you would use them. You start in god, with a table that is the wrong shape. You end in gog, looking at the answer.

The table on this page is the Gapminder panel, by way of the gapminder R package. It is public domain, and credited anyway: the package is Jennifer Bryan’s, and the figures are the Gapminder Foundation’s.