| country | continent | year | life | population | gdp |
|---|---|---|---|---|---|
| Afghanistan | Asia | 2007 | 43.828 | 31889923 | 974.5803 |
| Albania | Europe | 2007 | 76.423 | 3600523 | 5937.0295 |
| Algeria | Africa | 2007 | 72.301 | 33333216 | 6223.3675 |
| Angola | Africa | 2007 | 42.731 | 12420476 | 4797.2313 |
| Argentina | Americas | 2007 | 75.320 | 40301927 | 12779.3796 |
6 The book’s data
Where do gapminder_2007, medals and winds come from, and how do you get them?
Every chapter of this book uses the same shared tables, listed in full at the end of this page. They are published beside the book as CSV files, one per table, so you can run any example in this book yourself. Each file has a fixed address:
https://psychometrician.github.io/gog-book/data/<name>.csvgapminder_2007.csv holds the 142 countries the first chapter plots. medals.csv holds the medal counts the reading chapter draws, and winds.csv holds the wind observations the polar chapter draws.
Reading one takes a single call, gog_table(), which comes with the package in all four languages. Nothing else needs installing. You do not have to use it. The files are ordinary CSVs at a fixed address, and any reader that takes a web address reads them.
6.1 Reading a table
To run the first chapter’s plot yourself, you need its table in your session. One line fetches it. gog_table() reads any of these tables by name:
gapminder_2007 <- gog_table("gapminder_2007")
data(gapminder_2007) + point + x(gdp) + y(life)gapminder_2007 = gog_table("gapminder_2007")
data(gapminder_2007) + point + x(col.gdp) + y(col.life)gapminder_2007 = gog_table("gapminder_2007")
data(gapminder_2007) + point + x(:gdp) + y(:life)const gapminder_2007 = await gog_table("gapminder_2007");
plot(data(gapminder_2007), point, x(col.gdp), y(col.life));Those four calls draw the same picture, because the table is the same file and the engine is the same program.
gog_table() also gives each column the right type. A CSV is text, so numbers arrive as text and have to be converted. A column becomes numbers when every value in it is a number, and stays text otherwise. The text argument forces a column to stay text. Three columns in these tables need that, and Labels that look like numbers, below, explains which and why.
Why does a graphics package carry a table reader at all? R reads a CSV in one call, and Python and Julia in a few. JavaScript has no CSV reader in its standard library, so the same work there takes dozens of lines. One country in gapminder_2007.csv is "Congo, Dem. Rep.", and its name holds a comma, so splitting each line on commas gives that row seven fields where the header has six. Nobody should have to paste dozens of lines of parsing before drawing, so each package carries gog_table() instead.
R’s read.csv(), urllib in Python, Downloads in Julia and fetch in JavaScript are all standard, so gog_table() needs nothing beyond the package you already installed.
Read a file with a reader of your own, and the rest of this page still applies, because data() takes any table your session holds. You then do two things yourself. Your reader decides each column’s type. The three columns named below then need whatever it offers for keeping a column as text. And a declared order is never in the file, so you declare it after reading, whichever reader you use. Two things a CSV cannot record covers both.
Reading a different table is the same call with a different name: gog_table("winds"), gog_table("medals"). A misspelled name is refused, and the refusal suggests the nearest published name, or points to the list at the end of this page.
6.2 Two things a CSV cannot record
A CSV records what a value is, and never what kind of thing it is. Two kinds of information are therefore lost when the file is written. You restore them when you read the file back. Both change the plot, and this page takes each in turn.
6.2.1 A declared order
The eight compass directions have a fixed order, and a plot of them is correct only when it keeps that order. The winds table records one wind observation per row:
| direction | bearing | speed | season |
|---|---|---|---|
| N | 19.612260 | 10.1 | Winter |
| N | 357.049499 | 11.0 | Winter |
| N | 349.709737 | 13.8 | Winter |
| N | 2.602499 | 7.4 | Winter |
| N | 338.210715 | 3.7 | Summer |
The direction column of winds holds the eight points of the compass. Their order is N, NE, E, SE, S, SW, W, NW, clockwise from north. The Data chapter shows how a declared order sets the axis, the color assignment and the legend. A CSV cannot record that declaration. gog_table("winds") therefore returns direction as plain text, and the declaration is yours to make after reading.
Until you declare it, only the file’s own row order keeps the plot correct. The winds file lists its rows in compass order, so a plot made without the declaration reads correctly. Sort the rows by speed, and the same plot changes. The directions now follow the sorted rows, and that order has nothing to do with the compass. The plot still draws, and nothing on it shows that the compass is in the wrong order.
The example below reads the table, sorts it by speed, and then declares the compass order:
winds <- gog_table("winds")
# `base::order`, because gog's `order()` masks it
winds <- winds[base::order(winds$speed), ]
compass <- c("N", "NE", "E", "SE", "S", "SW", "W", "NW")
winds$direction <- factor(winds$direction, levels = compass)
data(winds) + bar * count + x(direction)winds = gog_table("winds")
by_speed = sorted(enumerate(winds["speed"]), key=lambda pair: pair[1])
winds = {name: [column[i] for i, _ in by_speed]
for name, column in winds.items()}
compass = ["N", "NE", "E", "SE", "S", "SW", "W", "NW"]
winds["direction"] = ordered(winds["direction"], compass)
data(winds) + bar * count + x(col.direction)winds = gog_table("winds")
by_speed = sortperm(winds["speed"])
winds = Dict{String,Any}(name => column[by_speed] for (name, column) in winds)
compass = ["N", "NE", "E", "SE", "S", "SW", "W", "NW"]
winds["direction"] = ordered(winds["direction"], compass)
data(winds) + bar * count + x(:direction)const winds = await gog_table("winds");
const by_speed = winds.speed.map((_, i) => i)
.sort((a, b) => winds.speed[a] - winds.speed[b]);
for (const name in winds) winds[name] = by_speed.map((i) => winds[name][i]);
const compass = ["N", "NE", "E", "SE", "S", "SW", "W", "NW"];
winds.direction = ordered(winds.direction, compass);
plot(data(winds), layer(bar, count), x(col.direction));“Given the winds: bars derived by count, x is direction.”
The rows are in speed order, and the axis still reads N to NW. The declared order decides the axis, not the order of the rows. The polar chapter draws this same count as a wind rose, and the change is one word. Add polar(), and the axis bends into a circle, with N at the top and the rest clockwise from it. The declared order goes with it:
data(winds) + bar * count + x(direction) + polar()data(winds) + bar * count + x(col.direction) + polar()data(winds) + bar * count + x(:direction) + polar()plot(data(winds), layer(bar, count), x(col.direction), polar())“Given the winds: bars derived by count, x is direction, in polar.”
The polar chapter explains that plot. Here it shows that a declared order survives a change of space. The circle starts at the first category and runs clockwise through the rest, so the compass reads correctly in both.
The sort exists only to undo the file’s compass order. The declaration is the part you keep. Declare it once after each gog_table("winds"), and every plot drawn from winds in the rest of the book reads the compass in order.
6.2.2 Labels that look like numbers
The second thing a CSV cannot record is which columns are text. Three columns in these tables hold text that gog_table() would read as numbers unless you name them. census.csv has an age column of 0, 5, 10, which are the names of age bands rather than quantities. sessions.csv has a session column of 01, 02, 03, and a number cannot keep a leading zero, so 01 comes back as 1. gm_eras.csv has an era column of 1957 and 2007. Those two values name the eras the rows compare; they do not measure anything. That is the whole reason gog_table() takes a text argument:
census <- gog_table("census", text = "age")
sessions <- gog_table("sessions", text = "session")
gm_eras <- gog_table("gm_eras", text = "era")census = gog_table("census", text=("age",))
sessions = gog_table("sessions", text=("session",))
gm_eras = gog_table("gm_eras", text=("era",))census = gog_table("census"; text = ["age"])
sessions = gog_table("sessions"; text = ["session"])
gm_eras = gog_table("gm_eras"; text = ["era"])const census = await gog_table("census", ["age"]);
const sessions = await gog_table("sessions", ["session"]);
const gm_eras = await gog_table("gm_eras", ["era"]);Without it, age becomes numbers, and an axis that should show one slot per age band is drawn as a numeric scale from 0 to 85. The plot looks reasonable, so nothing tells you the axis is wrong.
Each call above names one column, and a text argument can name more than one. gm_eras holds the same values in two columns: year as a number, era as a label. The book reads only era as text, so year stays a number. To read the year as a label too, name both columns in one text argument:
gm_eras <- gog_table("gm_eras", text = c("year", "era"))gm_eras = gog_table("gm_eras", text=("year", "era"))gm_eras = gog_table("gm_eras"; text = ["year", "era"])const gm_eras = await gog_table("gm_eras", ["year", "era"]);Each language writes more than one name in its own way. R puts them inside c(), Python inside parentheses, and Julia and JavaScript inside square brackets. Only R’s spelling changes from the block above. The other three already put one name inside parentheses or brackets, so a second name joins it there.
6.3 The tables
Those three tables need a column named in a text argument, and no other shared table does. The list below names every shared table this book reads, with its rows and columns. The preface introduces the eight families of table that most plots read. A family can cover more than one name below, so the list is longer than eight. Every other table is introduced on the page that first draws it. Each name goes in place of <name> in the address at the top of this page. The same name is what you give gog_table():
| Table | Rows | Columns |
|---|---|---|
| actuals | 5 | 2 |
| banded | 1 | 4 |
| botswana_arrow | 2 | 2 |
| botswana_label | 1 | 3 |
| capitals | 5 | 3 |
| cashflow | 6 | 3 |
| census | 36 | 3 |
| channel_sales | 6 | 3 |
| cities | 8 | 3 |
| coefs | 5 | 3 |
| commutes | 16 | 3 |
| day_cycle | 25 | 2 |
| decay | 6 | 2 |
| departments | 5 | 2 |
| depth_readings | 4 | 2 |
| drawdown | 6 | 3 |
| equator | 1 | 1 |
| far_north | 4 | 2 |
| flight | 2 | 2 |
| forecast | 3 | 2 |
| gapminder_2007 | 142 | 6 |
| gapminder_asia | 60 | 6 |
| gdp_threshold | 1 | 1 |
| gm_all | 1704 | 6 |
| gm_continents | 1056 | 6 |
| gm_eras | 284 | 7 |
| gm_europe_cdf | 30 | 2 |
| gm_europe | 30 | 6 |
| healthy_band | 2 | 3 |
| income_note | 1 | 3 |
| inventory | 16 | 3 |
| iris_flowers | 150 | 4 |
| life_bands | 3 | 2 |
| listening | 104 | 3 |
| maunga_whau | 1364 | 3 |
| medal_repeats | 5 | 2 |
| medals | 5 | 4 |
| milestones | 2 | 3 |
| mixed_signs | 6 | 3 |
| monitoring | 37 | 2 |
| nutrients | 40 | 3 |
| octaves | 6 | 2 |
| policy_rates | 6 | 2 |
| population_spikes | 120 | 6 |
| prevailing_winds | 2 | 1 |
| quakes_fiji | 1000 | 5 |
| quarterly | 19 | 2 |
| receipts | 4 | 2 |
| recessions | 2 | 3 |
| revenue | 26 | 2 |
| ripples | 5400 | 5 |
| routes | 14 | 4 |
| sales_box | 1 | 4 |
| score_band | 21 | 4 |
| scrambled | 5 | 2 |
| sessions | 14 | 5 |
| six_weeks | 42 | 4 |
| slump | 1 | 4 |
| span_early | 1 | 2 |
| span_late | 1 | 2 |
| span_middle | 1 | 2 |
| speed_target | 1 | 1 |
| spending | 12 | 4 |
| spiral | 90 | 2 |
| target_band | 1 | 2 |
| target_edges | 2 | 1 |
| team_trend | 12 | 3 |
| tenure | 150 | 2 |
| thermal_marks | 34 | 5 |
| thermals | 340 | 5 |
| tide | 8 | 2 |
| titanic | 32 | 5 |
| trade_partners | 22 | 3 |
| winds | 264 | 4 |
| world_borders | 4150 | 5 |
| world_median | 2 | 2 |
Not every table here holds real data. The tables whose names begin with gapminder or gm hold real figures from the Gapminder Foundation, released into the public domain. iris_flowers, maunga_whau, quakes_fiji and titanic are reshaped from tables included with R, and carry R’s own license. world_borders is Natural Earth’s public-domain outline of the countries. Everything else was written for this book, each table to show one thing; census is two plausible city profiles, not a real census. The terms for each are in data/LICENSE.md.