Why it matters

Koalas → pyspark.pandas is common migration.

Advertisement

The architecture

Rename: import pyspark.pandas as ps.

Mostly API-compatible.

Koalas → Spark migrationKoalas 1.xstandaloneSpark 3.2+integrated as pyspark.pandasSame APImostly compatibleMinor API changes for imports; most code works unchanged
Migration path.
Advertisement

How it works end to end

Migration: change import.

Deprecated in Koalas standalone.

Same features + performance in integrated.