Partition Filter Mapping

Tables on Hadoop-family engines are often partitioned on a technical column — an epoch integer, a lowercased region key — that no analyst would ever filter on. Unless a query carries a predicate on that column, the engine scans every partition.

Partition filter mapping makes this a dataset setting instead of a per-chart chore. A dataset owner names the partition column, the business column whose filters should be mirrored onto it, and a value transform. Superset then appends an equivalent predicate on the partition column to every query. Chart authors change nothing; queries prune.

Experimental

This feature is behind the PARTITION_FILTER_MAPPING feature flag and is off by default.

Enabling it

FEATURE_FLAGS = {
    "PARTITION_FILTER_MAPPING": True,
}

Configure it as a static boolean. FEATURE_FLAGS also accepts per-request callables, but a flag that resolves differently per user or tenant would let a user with the feature off read a cached chart result that was produced from pruned SQL by a user with it on.

Configuring a mapping

In the dataset editor's Columns tab, under Default Column Settings, pick a Partition column. By default the mapping follows the dataset's default datetime column, so re-pointing that column moves the mapping with it; set an explicit override if you want it pinned to a different column.

Expand the mapped column's row in Column Settings and set the value transform: a SQL expression containing a :value placeholder, which stands for the filter value being mirrored. The Transform preserves ordering checkbox sits directly beneath it.

Mapped column	Partition column	Transform
event_time (TIMESTAMP)	dt_epoch (BIGINT)	unix_timestamp(:value)
country (VARCHAR)	region_key (VARCHAR)	lower(:value)

A filter of event_time >= '2026-01-01' then adds dt_epoch >= 1767225600 to the query. The added predicate is an ordinary WHERE clause and shows up in View query.

Transform preserves ordering

Range filters — including the Explore time range, the most important case — are only mirrored when you check Transform preserves ordering.

Monotonicity is a property of the transform, not of the column's data type. unix_timestamp(:value) preserves ordering. hour(:value), date_format(:value, 'dd') and dayofweek(:value) are all perfectly reasonable partition transforms on a TIMESTAMP column and none of them do: hour('2026-01-01 23:00') is greater than hour('2026-01-02 01:00') even though the first instant is earlier. Mirroring a range through one of those would silently return wrong numbers, so Superset asks you to declare it rather than guessing.

A bucketing transform preserves ordering and is the common case rather than an exception: to_char(:value, 'YYYYMMDD'), date(:value) and cast(unix_timestamp(:value) / 86400 as bigint) each map a whole day onto one partition key, which is exactly what a daily-partitioned table wants. Check the box for these.

When the box is unchecked, = and IN filters still mirror; ranges do not.

What is and isn't mirrored

Filter	Mirrored
=, IN	Always, except on a text column whose engine does not compare text exactly, or when the engine compares less of the value than the filter carries
>, >=, <, <=, time ranges	Only when the transform preserves ordering — and a strict bound mirrors non-strictly
!=, NOT IN, LIKE, ILIKE, IS NULL, IS TRUE	Never

Strict bounds. Preserving ordering means non-decreasing, not strictly increasing, so event_time < '2026-07-06 12:00' mirrors to day_key <= '20260706' rather than day_key < '20260706'. Under a day key both sides of a mid-day boundary share a key, and the strict form would prune away the partition the boundary rows live in. The mirror reads one extra partition and keeps every row the filter keeps.

Negations are never safe. A transform need not be injective: lower(:value) with country != 'US' would mirror to region_key != 'us', which excludes rows whose country is already lowercase 'us' — rows the original filter keeps.

Text comparison. Mirroring col = v onto partition_col = T(v) assumes the engine compares col the way the transform was written against. Under a case-insensitive collation it does not: a stored country of 'us' satisfies a filter for 'US', while the mirror is derived from 'US' and excludes the row. So = and IN on a text mapped column only mirror on engines whose default text comparison is exact — Hive, Impala, Trino, Presto, Spark, Databricks, PostgreSQL, Snowflake, BigQuery and SQLite. On MySQL and SQL Server, whose default collations are case-insensitive, a text mapping is stored and shown but no = or IN filter mirrors through it; map a numeric or temporal column instead. Numeric and temporal comparison is exact everywhere, and ranges are unaffected either way — checking Transform preserves ordering is already a statement about the column's own ordering.

One case Superset cannot check for you: a column that declares its own collation, such as country COLLATE NOCASE, on an engine that is otherwise exact. Nothing in the metadata exposes that, so it falls under the same pipeline assumption as the partition key itself.

Value resolution. Mirroring also assumes the engine compares the value your filter carries. It does not always. On a DATE mapped column, event_date = '2026-07-06 10:08:11' is compared on the date part alone — Postgres casts the string, Presto and Trino render DATE '2026-07-06' — so the filter keeps the whole of July 6 while a mirror derived from 10:08:11 keeps one instant of it. Superset detects what your engine drops, from the column's type rather than from a list of engines, and responds by operator: a range widens to the day, so event_date >= '2026-07-05 10:00' mirrors as if the bound were '2026-07-05 00:00' — one extra partition read, every row the filter keeps — while = and IN have nowhere to widen to and do not mirror at all. A = or IN whose value carries no time of day, such as '2026-07-06' or a date picked in the filter popover, mirrors normally. The same applies one grain finer on engines that compare timestamps only to the second.

Reading the value is what makes all of that possible, and Superset reads ISO 8601 only — 2026-07-06, 2026-07-06 10:08:11, 2026-07-06T10:08:11. It is the one format whose meaning is not a guess: 06/07/2026 is July 6 to a US reader and June 7 to most of the world, and a mirror built from the wrong one drops rows silently. So on a column the engine compares coarsely, a datetime typed in any other format — 07/06/2026 10:08:11, 2026/07/06 10:08:11, July 6 2026 10:08:11 — does not mirror, for any operator, and no pruning indicator appears. The filter itself is unaffected; the query is correct and simply reads the whole table. Write the bound in ISO 8601, or pick it from the filter popover, and the mirror comes back. None of this applies where the engine compares the whole value: on a TIMESTAMP column any format your database accepts mirrors, because there is nothing left for Superset to resolve.

If = and IN on a date column matter to you, map a TIMESTAMP column instead: the engine then compares the whole value and every operator mirrors.

Time grains. A filter that carries a time grain — drill-to-detail, mostly — compares the truncated column, so the raw bounds it carries do not describe the rows it keeps: a row in the final partial bucket satisfies DATE_TRUNC(...) < until while col < until excludes it. Grained ranges still mirror, with both bounds widened by one bucket so the mirror stays no narrower than the real filter. A P1D drill therefore reads three days of partitions rather than one, instead of scanning the table. Grained = and IN filters do not mirror.

Widening is only applied to grains whose bucket width Superset knows, which means the built-in ones. A grain you added through TIME_GRAIN_ADDONS, or a built-in grain whose SQL you replaced through TIME_GRAIN_ADDON_EXPRESSIONS, has no width Superset can rely on and simply does not mirror.

Known gaps, all of which are out of scope rather than bugs:

• Filter-value dropdowns do not prune. Populating a filter's value list runs its own SELECT DISTINCT, which never goes through the chart query path. There is no filter to mirror from.
• Row-level security predicates do not mirror. They are stored as raw SQL and appended downstream of the structured filters.
• Custom SQL WHERE clauses do not mirror, for the same reason.
• Columns with an active advanced data type do not mirror. Those build their own predicate shape from translated values, so there is no operator/value pair to mirror.
• Dashboard native filters and cross-filters do mirror — they arrive as ordinary filters — they just carry no visual indicator in the filter bar.

The assumption this rests on

Superset emits a predicate on the partition column that stands in for one on the mapped column. That substitution is only valid if, for every row in the table:

partition_column = <transform>(mapped_column)

Superset cannot verify this. It is a property of whatever ETL populates the partition column. If that job lags, backfills with different logic, or writes the partition key in a different timezone than the transform resolves, mirrored predicates silently drop real rows and charts show quietly wrong numbers. Confirm the invariant with whoever owns the pipeline before enabling a mapping on a production dataset.

Rows in a NULL partition are the one case Superset does defend against. A predicate like dt_epoch >= X is NULL — and so drops the row — wherever dt_epoch is NULL, even when the original filter matches it. That is reachable: a dynamic-partition insert on Hive or Impala parks rows whose partition key was NULL in the default partition (__HIVE_DEFAULT_PARTITION__), and the column reads back as NULL when you query it. A transform that returns NULL for an input it cannot convert produces the same thing on any engine.

The mirrored predicate is therefore emitted as:

(dt_epoch >= X AND dt_epoch <= Y) OR dt_epoch IS NULL

The mirror only has to be no narrower than the filter it stands in for, so admitting the NULL partition costs one extra partition read and keeps those rows. Engines still prune everything else.

How the transform is evaluated

The transform is evaluated against the engine — pinned to the dataset's database, catalog and schema — and the result is emitted as a literal constant. Results are cached (PARTITION_TRANSFORM_PROBE_CACHE_TIMEOUT, 24 hours by default), which matters because this adds a round trip to the chart query path. Day-aligned ranges like "Last month" hit the cache constantly; second-granularity relative ranges like "Last 24 hours" essentially never do.

If the evaluation fails for any reason, no predicate is added: the query still runs and is still correct, it just scans more partitions.

The same goes for a transform that evaluates but answers with something the partition column cannot hold — a number against a text key, or text against a numeric one. Not every engine accepts a comparison across those types: Postgres, Trino and BigQuery refuse it outright, and SQLite compares it as never equal rather than erroring, which would drop every row the filter keeps. So Superset declines the mirror instead. The preview panel names the column and the value it got, and the fix is to cast the result to the key's own type — cast(unix_timestamp(:value) as string) for a text partition key.

Because the evaluation happens in a different session from the chart query, transforms that call non-deterministic functions are rejected when you save. That includes now(), current_date, current_timestamp, rand() and the zero-argument unix_timestamp(), which means "now" on Hive and Impala. The one-argument unix_timestamp(:value) is fine.

Session-dependent behaviour that Superset cannot detect is still your responsibility: unix_timestamp(:value) is timezone-dependent on Hive and Impala, so if the evaluating session and the query session resolve different timezones the emitted bounds will disagree with the timestamp bounds they mirror. Prefer explicitly-anchored transforms.

Jinja templating is not supported in a transform. The template would render in a different context and at a different time from the chart query.

What a transform may be

A transform has to be one scalar SQL expression around :value and nothing more. These are refused when you save:

• A clause of its own, or a sub-query — password || :value FROM users, (SELECT secret FROM vault LIMIT 1) || :value. Superset splices the transform into the probe as SQL text and the engine runs it, so a transform carrying its own FROM reads a table the mapping has no business reading.
• A :value the engine would never evaluate, because it sits inside a string literal (':value') or inside a comment (1 -- :value). Neither is a transform of the filter value: the first is the constant text :value, and the second comments out the rest of the probe's own query.
• More than 1,024 characters. A transform is one hand-written expression, and the stored value is parsed on every Explore load.

Related configuration

Setting	Default	Purpose
PARTITION_TRANSFORM_PROBE_CACHE_TIMEOUT	24 hours	How long an evaluated transform stays cached
PARTITION_TRANSFORM_PREVIEW_RATE_LIMIT	30	Per-user, per-dataset preview requests per minute; the editor's preview panel runs a real query

Three things need a working cache backend

None of them does anything on its own. The probe cache, the preview budget and the pruning indicator all live in CACHE_CONFIG, whose default is {"CACHE_TYPE": "NullCache"} — so out of the box:

• every chart load on a mapped dataset re-probes the warehouse, because nothing is cached;
• the preview budget is never enforced, because a null cache reports every request as the first one in its window; and
• the pruning indicator stays optimistic. A transform the database rejects parses fine, so Superset cannot tell it apart from a working one until something probes — and with nothing to record that answer in, the editor's banner and the glyph on a filter chip go on promising a speed-up the query gives up. Charts stay correct either way; what is lost is the UI's ability to stop claiming otherwise.

Configuring DATA_CACHE_CONFIG does not help; all three read CACHE_CONFIG.

The backend also has to be shared across processes. A per-process SimpleCache or a FileSystemCache on a local disk gives each gunicorn worker its own counter and its own probe cache, so the effective preview budget is the configured one multiplied by the number of workers. Use Redis or Memcached if you run more than one.

Throttling fails open by design: if the cache backend errors, or cannot count, the preview is allowed rather than refused, so a cache outage does not take the dataset editor down with it. That is also why an unconfigured cache leaves previews unthrottled rather than blocked.

Mappings travel with the dataset in import/export.