Alex Garbiak
← Lib

Polars: Reading & Exploring Data

Updated pythonpolarsdatacheatsheet

A general-purpose baseline for now — the plan is to trim this down to the calls I actually reach for, and drop the rest.

uv add polars

Reading Parquet

CallDescription
pl.read_parquet("file.parquet")Read eagerly into a DataFrame.
pl.scan_parquet("file.parquet")Read lazily into a LazyFrame — nothing loads until .collect().
pl.read_parquet("data/*.parquet")Read and concatenate multiple files matching a glob.
pl.read_parquet("file.parquet", columns=["a", "b"])Read only specific columns (projection pushdown).
pl.read_parquet("file.parquet", n_rows=1000)Read only the first n rows.

Prefer scan_parquet over read_parquet for anything beyond a quick look — Polars pushes column selection and row filters down into the scan itself, so you never materialize data you’re about to drop:

df = (
    pl.scan_parquet("data.parquet")
    .select(["id", "category", "amount"])
    .filter(pl.col("amount") > 0)
    .collect()
)

Shape & basic info

CallDescription
df.shape(n_rows, n_columns) tuple.
len(df)Row count on its own.
df.widthColumn count on its own.
df.estimated_size("mb")Approximate in-memory size.
df.glimpse()A vertical, per-column summary — dtype and first few values — good for wide frames.

Schema & dtypes

CallDescription
df.schemaAn ordered mapping of column name → dtype.
df.dtypesJust the list of dtypes, in column order.
df.columnsJust the list of column names.
df[col].dtypeThe dtype of a single column.

First look at the data

CallDescription
df.head(n)First n rows (default 5).
df.tail(n)Last n rows.
df.sample(n)n random rows.
df.describe()Summary stats per column (count, null count, mean, std, min, max, quantiles for numeric columns).

Missing values

CallDescription
df.null_count()A one-row DataFrame with the null count per column.
df.select(pl.all().is_null().sum())Equivalent, written as an expression.
df.select(pl.col("col").is_null().mean())Null rate (0–1) for a single column.

Unique values for categorical-ish columns

CallDescription
df["col"].n_unique()Count of distinct values in one column.
df["col"].unique()The distinct values themselves, as a Series.
df["col"].value_counts()Distinct values with their frequency, sorted by count.
df.select(pl.col(pl.String, pl.Categorical).n_unique())Distinct-value counts for every string/categorical column at once.
df.select(pl.col("col").cast(pl.Categorical))Cast a low-cardinality string column to Categorical — cheaper comparisons and groupbys.

For a quick cardinality scan across every column when you don’t yet know which ones are categorical:

df.select(pl.all().n_unique()).transpose(include_header=True, column_names=["n_unique"])

Putting it together: a first-look script

import polars as pl

df = pl.read_parquet("data.parquet")

print(df.shape)
print(df.schema)
print(df.null_count())
print(df.describe())

for col, dtype in df.schema.items():
    if dtype in (pl.String, pl.Categorical) and df[col].n_unique() < 50:
        print(col, df[col].value_counts())