Appendix C — The same sentence in six dialects

If you already write dplyr, pandas, polars, PySpark, SQL or Spark SQL, this appendix is the translation. Each row is one god sentence and what it becomes in each of them.

Nothing here is written by hand. Every cell is produced by asking god to print the pipeline in that language when this page is built, which is the same machinery What it wrote describes. A translation table maintained by a person goes out of date; this one cannot, because it is generated from the pipeline it describes.

C.1 Row by row

`god`	`dplyr`	`pandas`
`keep where [region] is "West"`	`sales \|> filter((region == "West"))`	`(sales .loc[lambda d: (d["region"] == "West")])`
`pick [product, revenue]`	`sales \|> select(product, revenue)`	`(sales .loc[:, ["product", "revenue"]])`
`add [margin] as [revenue] - [cost]`	`sales \|> mutate(margin = (revenue - cost))`	`(sales .assign(margin=lambda d: (d["revenue"] - d["cost…`
`summarize [total] as total([revenue]) by [product]`	`sales \|> summarise(total = sum(revenue, na.rm = TRUE), .b…`	`(sales .groupby(["product"], as_index=False).agg(total=…`
`sort [revenue] descending`	`sales \|> arrange(desc(revenue))`	`(sales .sort_values(["revenue"], ascending=[False]))`
`take 3`	`sales \|> head(3)`	`(sales .head(3))`
`drop_duplicates`	`sales \|> distinct()`	`(sales .drop_duplicates() .sort_values(["date", "re…`
`keep where [revenue] > 150 and [cost] < 100`	`sales \|> filter(((revenue > 150) & (cost < 100)))`	`(sales .loc[lambda d: ((d["revenue"] > 150) & (d["cost"…`

`god`	`polars`	pyspark
`keep where [region] is "West"`	`(sales .filter((pl.col("region") == "West")))`	`(sales .filter((F.col("region") == "West")))`
`pick [product, revenue]`	`(sales .select(["product", "revenue"]))`	`(sales .select("product", "revenue"))`
`add [margin] as [revenue] - [cost]`	`(sales .with_columns((pl.col("revenue") - pl.col("cost"…`	`(sales .withColumn("margin", (F.col("revenue") - F.col(…`
`summarize [total] as total([revenue]) by [product]`	`(sales .group_by(["product"]).agg(pl.col("revenue").sum…`	`(sales .groupBy("product").agg(F.sum(F.col("revenue")).…`
`sort [revenue] descending`	`(sales .sort(["revenue"], descending=[True], nulls_last…`	`(sales .orderBy(F.col("revenue").desc_nulls_last()))`
`take 3`	`(sales .head(3))`	`(sales .limit(3))`
`drop_duplicates`	`(sales .unique() .sort(["date", "region", "product"…`	`(sales .dropDuplicates() .orderBy(["date", "region"…`
`keep where [revenue] > 150 and [cost] < 100`	`(sales .filter(((pl.col("revenue") > 150) & (pl.col("co…`	`(sales .filter(((F.col("revenue") > 150) & (F.col("cost…`

`god`	sql	spark
`keep where [region] is "West"`	`WITH step0 AS (SELECT * FROM "sales"), step1 AS (SELEC…`	WITH step0 AS (SELECT * FROM `sales`), step1 AS (SELEC…
`pick [product, revenue]`	`WITH step0 AS (SELECT * FROM "sales"), step1 AS (SELEC…`	WITH step0 AS (SELECT * FROM `sales`), step1 AS (SELEC…
`add [margin] as [revenue] - [cost]`	`WITH step0 AS (SELECT * FROM "sales"), step1 AS (SELEC…`	WITH step0 AS (SELECT * FROM `sales`), step1 AS (SELEC…
`summarize [total] as total([revenue]) by [product]`	`WITH step0 AS (SELECT * FROM "sales"), step1 AS (SELEC…`	WITH step0 AS (SELECT * FROM `sales`), step1 AS (SELEC…
`sort [revenue] descending`	`WITH step0 AS (SELECT * FROM "sales"), step1 AS (SELEC…`	WITH step0 AS (SELECT * FROM `sales`), step1 AS (SELEC…
`take 3`	`WITH step0 AS (SELECT * FROM "sales"), step1 AS (SELEC…`	WITH step0 AS (SELECT * FROM `sales`), step1 AS (SELEC…
`drop_duplicates`	`WITH step0 AS (SELECT * FROM "sales"), step1 AS (SELEC…`	WITH step0 AS (SELECT * FROM `sales`), step1 AS (SELEC…
`keep where [revenue] > 150 and [cost] < 100`	`WITH step0 AS (SELECT * FROM "sales"), step1 AS (SELEC…`	WITH step0 AS (SELECT * FROM `sales`), step1 AS (SELEC…

C.2 One sentence in full

The tables above shorten the long lines, and take the six languages two at a time, because six columns do not fit across a page. Here is one sentence written out completely in each, so you can see the shape rather than the abbreviation.

C.2.1 `sql`

WITH step0 AS (SELECT * FROM "sales"),
     step1 AS (SELECT * FROM step0 WHERE ("region" = 'West')),
     step2 AS (SELECT "product", sum("revenue") AS "total" FROM step1 GROUP BY "product" ORDER BY "product" NULLS LAST)
SELECT * FROM step2

C.2.2 `spark`

WITH step0 AS (SELECT * FROM `sales`),
     step1 AS (SELECT * FROM step0 WHERE (`region` = 'West')),
     step2 AS (SELECT `product`, sum(`revenue`) AS `total` FROM step1 GROUP BY `product` ORDER BY `product` NULLS LAST)
SELECT * FROM step2

C.2.3 `dplyr`

sales |>
  filter((region == "West")) |>
  summarise(total = sum(revenue, na.rm = TRUE), .by = product)

C.2.4 `pandas`

(sales
    .loc[lambda d: (d["region"] == "West")]
    .groupby(["product"], as_index=False).agg(total=("revenue", "sum"))
    .sort_values(["product"]))

C.2.5 `polars`

(sales
    .filter((pl.col("region") == "West"))
    .group_by(["product"]).agg(pl.col("revenue").sum().alias("total"))
    .sort(["product"], nulls_last=True))

C.2.6 `pyspark`

(sales
    .filter((F.col("region") == "West"))
    .groupBy("product").agg(F.sum(F.col("revenue")).alias("total"))
    .orderBy("product"))

C.3 A whole pipeline, all seven

The sentence above is two steps, chosen to stay readable. The translation does not thin as a pipeline grows, and a technical reader deciding whether to trust that should see it on a hard case. Six steps this time: a join, a computed column, a grouped total, a sort, a rank, and a cut. The seventh answer is the grammar’s own text form, and it is printed first. It is what show_as gives when you ask for "god", and it is the one spelling that runs anywhere.

C.3.1 `god`

sales
  then join products by [product]
  then add [margin] as ([revenue] - [cost])
  then summarize [margin] as total([margin]) by [maker]
  then sort [margin] descending
  then add [place] as rank([margin] descending)
  then take 3

C.3.2 `sql`

WITH step0 AS (SELECT * FROM "sales"),
     step1 AS (SELECT step0.*, "products".* EXCLUDE ("product") FROM step0 LEFT JOIN "products" ON step0."product" = "products"."product"),
     step2 AS (SELECT *, ("revenue" - "cost") AS "margin" FROM step1),
     step3 AS (SELECT "maker", sum("margin") AS "margin" FROM step2 GROUP BY "maker" ORDER BY "maker" NULLS LAST),
     step4 AS (SELECT * FROM step3 ORDER BY "margin" DESC NULLS LAST),
     step5 AS (SELECT *, RANK() OVER (ORDER BY "margin" DESC NULLS LAST) AS "place" FROM step4 ORDER BY "margin" DESC NULLS LAST),
     step6 AS (SELECT * FROM step5 LIMIT 3)
SELECT * FROM step6

C.3.3 `spark`

WITH step0 AS (SELECT * FROM `sales`),
     step1 AS (SELECT step0.*, `products`.* EXCEPT (`product`) FROM step0 LEFT JOIN `products` ON step0.`product` = `products`.`product`),
     step2 AS (SELECT *, (`revenue` - `cost`) AS `margin` FROM step1),
     step3 AS (SELECT `maker`, sum(`margin`) AS `margin` FROM step2 GROUP BY `maker` ORDER BY `maker` NULLS LAST),
     step4 AS (SELECT * FROM step3 ORDER BY `margin` DESC NULLS LAST),
     step5 AS (SELECT *, RANK() OVER (ORDER BY `margin` DESC NULLS LAST) AS `place` FROM step4 ORDER BY `margin` DESC NULLS LAST),
     step6 AS (SELECT * FROM step5 LIMIT 3)
SELECT * FROM step6

C.3.4 `dplyr`

sales |>
  left_join(products, by = join_by(product)) |>
  mutate(margin = (revenue - cost)) |>
  summarise(margin = sum(margin, na.rm = TRUE), .by = maker) |>
  arrange(desc(margin)) |>
  mutate(place = min_rank(desc(margin))) |>
  head(3)

C.3.5 `pandas`

(sales
    .merge(products, on=["product"], how="left")
    .assign(margin=lambda d: (d["revenue"] - d["cost"]))
    .groupby(["maker"], as_index=False).agg(margin=("margin", "sum"))
    .sort_values(["maker"])
    .sort_values(["margin"], ascending=[False])
    .assign(place=lambda d: d["margin"].rank(method="min", ascending=False))
    .head(3))

C.3.6 `polars`

(sales
    .join(products, on=["product"], how="left")
    .with_columns((pl.col("revenue") - pl.col("cost")).alias("margin"))
    .group_by(["maker"]).agg(pl.col("margin").sum().alias("margin"))
    .sort(["maker"], nulls_last=True)
    .sort(["margin"], descending=[True], nulls_last=True)
    .with_columns(pl.col("margin").rank("min", descending=True).alias("place"))
    .sort(["margin"], descending=[True], nulls_last=True)
    .head(3))

C.3.7 `pyspark`

(sales
    .join(products, on=["product"], how="left")
    .withColumn("margin", (F.col("revenue") - F.col("cost")))
    .groupBy("maker").agg(F.sum(F.col("margin")).alias("margin"))
    .orderBy("maker")
    .orderBy(F.col("margin").desc_nulls_last())
    .withColumn("place", F.rank().over(Window.orderBy(F.col("margin").desc_nulls_last())))
    .orderBy(F.col("margin").desc_nulls_last())
    .limit(3))

C.4 What the table is showing

Read down a column and you are reading one library’s idea of how a pipeline is written. Read one god sentence across all three tables and you are reading one idea, six ways.

The lengths differ, and the differences are the honest result rather than a selection. polars and PySpark have a method per verb, so a pipeline maps onto a chain and stays short. pandas has no single way to chain, so the idiom that does is .assign and .loc[lambda d: ...], and every column inside one of those names the frame again. SQL puts the steps in an order that is not the order they happen in.

This is also the way out. A small vocabulary covers most of what people do and never all of it. When you reach the end of what god can say, ask it what your pipeline would have been in the tool you already use, and carry on there.

C.1 Row by row

C.2 One sentence in full

C.2.1 sql

C.2.2 spark

C.2.3 dplyr

C.2.4 pandas

C.2.5 polars

C.2.6 pyspark

C.3 A whole pipeline, all seven

C.3.1 god

C.3.2 sql

C.3.3 spark

C.3.4 dplyr

C.3.5 pandas

C.3.6 polars

C.3.7 pyspark