A Delta table that flew at launch and crawls six months later is one of the most common things teams bring us, and the cause is almost never the thing they suspect. They look at cluster size and query logic, because that is what the getting-started material talks about, when the real problem is in how the table is physically laid out and what statistics Delta is keeping about it. None of these issues raise an error. The table works; it is just doing far more work per query than it should, and that overhead grows quietly until someone notices the morning reports are late. Here are the ones worth knowing before they find you.
The small-files problem is probably your slowdown
If a Delta table has gotten slow and you only investigate one thing, investigate this. Every write to a Delta table creates files, and a pipeline that writes frequently in small batches, streaming ingestion, regular micro-batches, lots of small MERGE operations, leaves behind thousands upon thousands of tiny files. When a query runs, a large share of its time goes not to reading data but to the overhead of opening and listing all those files. The table is the same size it always was; it is just shattered into too many pieces.
The fix is OPTIMIZE, which compacts the many small files into fewer large ones, and the difference on a fragmented table is often dramatic, because you are removing per-file overhead that had been multiplying with every write. Better than cleaning up after the fact is preventing the build-up in the first place, which is what optimized writes and auto-compaction do, nudging the engine to produce reasonably-sized files as it writes rather than leaving a mess for OPTIMIZE to sweep later. The practical posture is to enable the write-time options so fragmentation does not accumulate, and to run OPTIMIZE on a schedule for tables that churn. Small files are the entropy of a lakehouse; left alone, every table drifts toward them.
Statistics stop at the first columns, so column order is a performance decision
This is the gotcha that surprises even experienced engineers, because it is invisible unless you know to look for it. Delta Lake speeds up queries through data skipping, using per-file statistics, the min and max values in each file, to skip files that cannot contain the rows your filter is looking for. The catch is that, by default, Delta only collects these statistics on the first 32 columns of the table. Columns beyond that boundary get no statistics, which means data skipping cannot help them, which means a filter on a late column scans every file in the table no matter how selective it is.
Two consequences follow, and both are about column order being a decision rather than an accident. First, the columns you actually filter on need to live within that early window, or you forfeit data skipping on exactly the predicates that matter most. Second, long free-text or large columns sitting early in the table are actively wasteful, because they consume slots in that statistics window and the statistics on them are not useful for skipping anyway, pushing the columns you do filter on out past the boundary. The fix is to order the table so that the columns you filter and join on come first and the bulky, rarely-filtered columns come later. It is the kind of decision that costs nothing at design time and is annoying to change once a table is large, which is exactly why it is worth getting right early.
Z-ordering only helps the queries it matches
Z-ordering is a clustering technique that co-locates related data in the same files so that queries filtering on the clustered columns read far fewer files. It is genuinely powerful, and it is also routinely applied in a way that buys nothing, because the clustering only helps queries whose filters match the columns you clustered on.
The mistake is Z-ordering on columns that seemed important rather than on the columns your queries actually filter and join by. If you cluster on a field nobody filters on, you have paid for the OPTIMIZE run that did the Z-ordering and gained nothing, because no query's predicate lines up with the layout. So the prerequisite for Z-ordering well is knowing your real query patterns: which columns appear in the WHERE clauses and joins that matter, on the queries that run often enough to care about. Cluster on those, not on the columns that feel like natural keys. And keep it focused; Z-ordering on many columns dilutes the benefit, so the few columns that dominate your real filters are the ones to choose. Clustering is a bet that pays off only when it matches how the table is actually queried.
VACUUM reclaims storage and quietly limits how far back you can go
The last one is a trade-off people make accidentally because the command sounds purely beneficial. Delta keeps old versions of files around, which is what powers time travel, the ability to query or restore the table as it was at an earlier point, and what lets you roll back a bad write or audit a historical state. VACUUM removes those old files to reclaim storage, and it is necessary, because without it the retained history grows without bound.
The subtlety is that VACUUM sets the floor on time travel. Once you vacuum away the files for older versions, you can no longer travel back to them, so an aggressive retention window that reclaims storage quickly also quietly removes your ability to roll back or audit beyond that window. This matters most exactly when you need it least often: the day you discover a bad write from last week and want to restore the prior state, and find that VACUUM already cleared it. The retention window is therefore a deliberate decision that balances storage cost against how far back you might need to recover or audit, and in regulated domains the audit side usually wins. Set it on purpose; do not let a default decide how much of your history you are allowed to keep.
How to keep a Delta table fast
The maintenance posture that keeps these from creeping up on you is not complicated, it just has to be deliberate. Keep file sizes healthy by enabling optimized writes and auto-compaction and running OPTIMIZE on tables that churn, so the small-files problem never accumulates. Order columns so the ones you filter and join on fall within the statistics window and the bulky columns sit later, so data skipping actually fires. Z-order only on the columns your real queries filter by, and only where the query volume justifies the OPTIMIZE cost. And set a VACUUM retention window that reflects how far back you genuinely need to recover or audit, rather than accepting whatever the default gives you.
None of these show up as errors, which is exactly why they are worth knowing. A Delta table degrades silently, one small write and one unindexed filter at a time, and the teams whose tables stay fast are the ones treating physical layout and maintenance as a design concern rather than something to think about after the queries get slow.
If your lakehouse tables are slowing down with no obvious cause
Queries that have gotten slower while the logic and the cluster stayed the same are almost always a physical-layout problem: fragmentation, statistics that do not cover your filters, or clustering that does not match your queries. The cause is rarely the thing the dashboards point at, and it is fixable without throwing hardware at it.
Sapota's data team tunes Delta and lakehouse layouts as part of the platforms we build, the same care that went into the medallion-on-AWS platform we delivered for a regulated fintech. Getting layout and maintenance right is usually cheaper and more durable than scaling compute to outrun a fragmented table.
Reach out via the custom software page with a description of which tables have slowed down and how they are written and queried. The fix is often in the layout, not the cluster.
