| god | dplyr | pandas |
|---|---|---|
keep where [region] is "West" |
sales |> filter((region == "West")) |
(sales .loc[lambda d: (d["region"] == "West")]) |
pick [product, revenue] |
sales |> select(product, revenue) |
(sales .loc[:, ["product", "revenue"]]) |
add [margin] as [revenue] - [cost] |
sales |> mutate(margin = (revenue - cost)) |
(sales .assign(margin=lambda d: (d["revenue"] - d["cost… |
summarize [total] as total([revenue]) by [product] |
sales |> summarise(total = sum(revenue, na.rm = TRUE), .b… |
(sales .groupby(["product"], as_index=False).agg(total=… |
sort [revenue] descending |
sales |> arrange(desc(revenue)) |
(sales .sort_values(["revenue"], ascending=[False])) |
take 3 |
sales |> head(3) |
(sales .head(3)) |
drop_duplicates |
sales |> distinct() |
(sales .drop_duplicates() .sort_values(["region", "… |
keep where [revenue] > 150 and [cost] < 100 |
sales |> filter(((revenue > 150) & (cost < 100))) |
(sales .loc[lambda d: ((d["revenue"] > 150) & (d["cost"… |
Appendix C — The same sentence in six dialects
If you already write dplyr, pandas, polars, PySpark or SQL, this appendix is the translation. Each row is one god sentence and what it becomes in each of them.
Nothing here is written by hand. Every cell is produced by asking god to print the pipeline in that language when this page is built, which is the same machinery Chapter 25 describes. A translation table maintained by a person goes out of date; this one cannot, because it is generated from the pipeline it describes.
C.1 Row by row
| god | polars | pyspark |
|---|---|---|
keep where [region] is "West" |
(sales .filter((pl.col("region") == "West"))) |
(sales .filter((F.col("region") == "West"))) |
pick [product, revenue] |
(sales .select(["product", "revenue"])) |
(sales .select("product", "revenue")) |
add [margin] as [revenue] - [cost] |
(sales .with_columns((pl.col("revenue") - pl.col("cost"… |
(sales .withColumn("margin", (F.col("revenue") - F.col(… |
summarize [total] as total([revenue]) by [product] |
(sales .group_by(["product"]).agg(pl.col("revenue").sum… |
(sales .groupBy("product").agg(F.sum(F.col("revenue")).… |
sort [revenue] descending |
(sales .sort(["revenue"], descending=[True])) |
(sales .orderBy(F.col("revenue").desc())) |
take 3 |
(sales .head(3)) |
(sales .limit(3)) |
drop_duplicates |
(sales .unique() .sort(["region", "product", "reven… |
(sales .dropDuplicates() .orderBy(["region", "produ… |
keep where [revenue] > 150 and [cost] < 100 |
(sales .filter(((pl.col("revenue") > 150) & (pl.col("co… |
(sales .filter(((F.col("revenue") > 150) & (F.col("cost… |
C.2 One sentence in full
The table above shortens the long lines so that six languages fit across a page. Here is one sentence written out completely in each, so you can see the shape rather than the abbreviation.
C.2.1 sql
WITH step0 AS (SELECT * FROM "sales"),
step1 AS (SELECT * FROM step0 WHERE ("region" = 'West')),
step2 AS (SELECT "product", sum("revenue") AS "total" FROM step1 GROUP BY "product" ORDER BY "product")
SELECT * FROM step2C.2.2 spark
WITH step0 AS (SELECT * FROM `sales`),
step1 AS (SELECT * FROM step0 WHERE (`region` = 'West')),
step2 AS (SELECT `product`, sum(`revenue`) AS `total` FROM step1 GROUP BY `product` ORDER BY `product`)
SELECT * FROM step2C.2.3 dplyr
sales |>
filter((region == "West")) |>
summarise(total = sum(revenue, na.rm = TRUE), .by = product)C.2.4 pandas
(sales
.loc[lambda d: (d["region"] == "West")]
.groupby(["product"], as_index=False).agg(total=("revenue", "sum"))
.sort_values(["product"]))C.2.5 polars
(sales
.filter((pl.col("region") == "West"))
.group_by(["product"]).agg(pl.col("revenue").sum().alias("total"))
.sort(["product"]))C.2.6 pyspark
(sales
.filter((F.col("region") == "West"))
.groupBy("product").agg(F.sum(F.col("revenue")).alias("total"))
.orderBy("product"))C.3 What the table is showing
Read down a column and you are reading one library’s idea of how a pipeline is written. Read across a row and you are reading one idea, six ways.
The lengths differ, and the differences are the honest result rather than a selection. polars and PySpark have a method per verb, so a pipeline maps onto a chain and stays short. pandas has no single way to chain, so the idiom that does is .assign and .loc[lambda d: ...], and every column inside one of those names the frame again. SQL puts the steps in an order that is not the order they happen in.
This is also the way out. A small vocabulary covers most of what people do and never all of it, so when you reach the end of what god can say, you can ask it what your pipeline would have been in the tool you already use, and carry on there.