A while back I inherited a nightly batch that pulled account data out of a legacy core-banking system and pushed a summarized view into Salesforce so relationship managers had a portfolio dashboard waiting for them each morning. The transformation logic lived entirely in DataWeave, and on paper it was simple: read a few hundred accounts, drop the closed ones, total the balances, and group everything by account type.
It worked fine in the demo. Then one morning the dashboard came up showing a total balance of "0125000000125000..." — a forty-digit string instead of a number. Another morning the flow fell over on a StackOverflowError from a recursive helper somebody had written to walk a date range. And as the integration grew to a dozen APIs, the same money-formatting snippet had been copy-pasted into ten DataWeave files, three of which were already out of sync.
None of these were map-and-filter problems. They were the harder, second-tier stuff: aggregation with reduce, grouping at scale, and the discipline of turning repeated logic into functions and modules. That second tier is where DataWeave quietly punishes you, so it's worth going through carefully.
reduce is where the silent bugs live
reduce collapses an array into a single value, and the shape is always the same lambda: (item, accumulator) -> newAccumulator. The thing that trips people up is the accumulator's initial value. If you write [] reduce ((item, acc) -> acc + item) with no initial value, DataWeave defaults the accumulator to the first element and starts iterating from the second — which means an empty array either errors out or returns null depending on the runtime version. That matters enormously in real pipelines because you almost always reduce after a filter, and a filter can perfectly legitimately produce an empty array. The fix is non-negotiable: always initialize explicitly.
{
totalBalance: payload.accounts
filter ((acc) -> acc.status_code == "A")
reduce ((acc, total = 0) -> total + (acc.balance default 0))
}
That total = 0 is what guarantees you get 0 instead of null when every account got filtered out. The acc.balance default 0 guards against the source system omitting the field entirely, which the core-banking system did for freshly opened accounts.
The forty-digit-string bug came from the other classic reduce trap: type. The balance field arrived as a JSON string in some records, and total + "125000000" is string concatenation, not addition. DataWeave will not warn you. The only defense is to coerce at the point of aggregation whenever the input isn't fully trusted: total + (acc.balance as Number default 0).
Once you're comfortable with the mechanics, reduce becomes more than a summing tool. You can carry an object as the accumulator and compute several statistics in a single pass instead of iterating three times:
payload.accounts reduce ((acc, stats = {totalBalance: 0, count: 0, maxBalance: 0}) -> {
totalBalance: stats.totalBalance + (acc.balance default 0),
count: stats.count + 1,
maxBalance: if (acc.balance > stats.maxBalance) acc.balance else stats.maxBalance
})
That's the right instinct when you need summary stats. The one caution: if you find yourself building an array accumulator with acc ++ [newItem] inside a reduce, remember DataWeave is fully immutable, so ++ clones the whole array every iteration. On ten thousand elements that's O(N²) and your transform crawls. If the operation is genuinely one-to-one, use map; only reach for reduce when you actually need state to flow between iterations, like a running total.
groupBy is convenient until the dataset is real
groupBy turns an array into an object keyed by whatever the lambda returns, with each value being the list of elements in that group. For the morning dashboard it was exactly the right tool — group active accounts by type, then mapObject over the result to compute per-group statistics:
payload.accounts
filter ((acc) -> acc.status_code == "A")
groupBy ((acc) -> acc.account_type)
mapObject ((accounts, accountType) -> {
(accountType as String): {
count: sizeOf(accounts),
totalBalance: accounts reduce ((a, total = 0) -> total + a.balance),
accountNos: accounts map ((a) -> a.account_no)
}
})
When you need to group on more than one dimension, the cleanest trick is to concatenate the key — acc.branch_code ++ "_" ++ acc.account_type — rather than nesting groups.
The thing nobody tells you in the tutorials is that groupBy materializes. It loads the entire array into memory and builds the result object before anything downstream runs. For a few hundred accounts that's invisible. For a batch of a million records grouped across two hundred branches, the JVM heap simply explodes. The same is true of orderBy and of any reduce that accumulates into an array. So when Mule is streaming a large CSV or a big database result set, map stays lazy and behaves, but the moment you introduce groupBy, orderBy, or an array-building reduce, you force the whole stream into RAM.
The honest fix is usually not in DataWeave at all. Push the grouping down to the database with SELECT ... ORDER BY branch_code and aggregate incrementally, or move the work into a Mule Batch Job designed for high-volume records. DataWeave is for shaping data, not for holding a million rows in heap.
While we're on the subject of things DataWeave shouldn't do: never call an external API from inside a lambda. I have seen payload.accounts map ((acc) -> HttpRequest::get(...)) in a code review, and it does exactly what you'd fear — one synchronous HTTP call per element, N calls, guaranteed timeout. Enrichment belongs in the flow, using a Parallel For Each with an HTTP Request connector, with the results fed into DataWeave afterward to transform.
The pipeline idiom
Once the helpers click — distinctBy to drop duplicates by key (the core-banking batch occasionally resent records on retry), orderBy with a negated key for descending sort, take and drop for slicing — the natural way to write a transform is as a top-to-bottom pipeline, one verb per line, the way you'd read a Unix grep | sort | uniq | head:
payload.accounts
filter ((acc) -> acc.status_code == "A") // drop closed
distinctBy ((acc) -> acc.account_no) // dedupe retries
orderBy ((acc) -> -acc.balance) // largest first
take 50 // top 50
groupBy ((acc) -> acc.account_type) // group by type
mapObject ((items, type) -> { ... }) // summarize each group
This filter → distinctBy → orderBy → take → groupBy → mapObject shape became our standard for batch sync work. It reads cleanly, and each line does exactly one thing, which makes review and debugging far easier than a single dense expression.
Functions and modules: stop copy-pasting
The money-formatting saga is the reason the second half of this matters. Ten APIs each formatted an amount like 1234567 into "1.234.567 ₫", the logic copy-pasted into ten files. When the formatting rule changed, someone had to edit ten places, missed three, and shipped a production bug. That's a DRY violation with teeth.
The fix is a named function, declared in the header section between %dw 2.0 and the ---, ideally with type annotations so you get type safety and autocomplete:
fun formatVND(amount): String =
if (amount == null or amount == 0) "0 ₫"
else (
var rounded = round(amount as Number)
var sign = if (rounded < 0) "-" else ""
---
sign ++ (abs(rounded) as String {format: "#,###"} replace "," with ".") ++ " ₫"
)
Reach for an anonymous lambda when the logic is used once inside a map or filter; reach for a named fun when it's reused, when it needs a clear name, or when it's recursive. The shorthand $ and $$ are tempting, but they cost readability, so they belong in a quick proof of concept, not production. Pattern matching with match deserves a mention too — it's the cleanest way to express routing or classification, binding a value and adding a guard like case { amount: a } if a > 100000000 -> ..., and you should order the most frequently matched cases first.
One real caution on recursion: DataWeave has no strong tail-call optimization, so a recursive function deeper than roughly a thousand levels throws StackOverflowError. That was our date-range bug. If you're iterating a large range, rewrite it as a reduce over (1 to n) rather than recursing.
Functions solve copy-paste within a file. Once you accumulate more than three utilities used across flows — formatting, phone parsing, card masking, validation — gather them into a module. A module is just a .dwl file under src/main/resources/dwl/, where subfolders become the namespace (TCB/Common.dwl becomes TCB::Common). The detail that bites everyone the first time: a module file contains only %dw 2.0 plus fun, var, and type declarations. It has no output directive and no ---. Get that wrong, or get the file's casing wrong, and you'll see Cannot resolve reference. DataWeave is case-sensitive about module paths.
For imports, prefer the explicit form — import formatVND, maskCard from TCB::Common — over wildcards. A wildcard import * can silently shadow both your own local functions and dw::Core built-ins like capitalize or sizeOf, and the output will look plausible while being wrong. Explicit imports make the dependency obvious and the shadowing impossible.
The principle underneath all of it
Every one of these patterns points the same direction: make the transformation say exactly what it means and fail loudly rather than quietly. Initialize your accumulators so empty input is a defined case, coerce types so a stray string can't masquerade as a number, keep memory-hungry operations out of streamed paths, and pull shared logic into modules so one edit propagates everywhere instead of leaving you to remember nine other files. Invest in that structure early; six months into a growing integration, it pays back many times over.
Building or operating MuleSoft integrations? Our Salesforce team designs API-led architectures, builds Mule flows, and runs them in production. Get in touch ->
See our full platform services for the stack we cover.
