Paying one person is a transaction. Paying four thousand is an operation. The difference is that a batch will contain failures — stale account numbers, closed accounts, name mismatches, a beneficiary bank having a bad afternoon — and a design that treats the batch as a single unit will handle every one of them badly.
A batch is a collection, not a transaction
The most common structural mistake is modelling a batch as something that either succeeds or fails. It will do neither. Some transfers settle, some fail immediately, some sit pending for hours, and a few settle and are returned days later.
- Give every line its own identity, status and retry history.
- Treat the batch as a grouping for approval and reporting, not as the unit of success.
- Make the line, not the file, the thing your support team can search for.
- Expect the batch to be in several states at once, and design the status display around that rather than a single progress bar.
The states a line actually moves through
Writing these down before you write code is the cheapest hour you will spend on a payout system, because it forces the awkward transitions into the open while they are still easy to handle.
| State | Means | Can move to |
|---|---|---|
| Draft | In the file, not yet approved | Approved, Cancelled |
| Approved | Released for submission | Submitted, Cancelled |
| Submitted | Sent, no outcome yet | Processing, Failed |
| Processing | Accepted by the rail | Settled, Failed |
| Settled | Funds delivered | Returned |
| Failed | Terminal for this attempt | Submitted (on retry) |
| Returned | Settled then sent back | Submitted (after correction) |
The row people forget is the last but one. Settled is not terminal, and a model that assumes it is will either crash on the return event or discard it silently — which is worse, because the money is back and nobody knows.
What the approval step is for
The batch is still a useful unit for one thing: approval. Somebody signs off on a total and a count, and that is the control point where an error in the file is cheapest to catch.
- Show the total, the line count, and the change against the previous comparable run. A payroll that jumped 40% is worth a second look before it goes.
- Flag lines that are new, changed, or materially different from that beneficiary's history.
- Separate the person who uploads from the person who approves. This is the highest-value control in the whole flow.
- Freeze the file at approval, so what was approved is what is submitted.
Validate before you submit
The cheapest failure is the one that never reaches the rail. A meaningful share of batch failures are catchable locally, which turns a failed transfer into a correctable row before anything is attempted.
What to check locally
- Account number format and IFSC structure, both of which have rules you can apply without a network call.
- Duplicate lines within the same file — the same beneficiary and amount appearing twice is far more often an error than an intention.
- Amounts that are zero, negative, or implausibly large relative to that beneficiary's history.
- Beneficiaries that have never been verified, which should not be in a payment run at all.
- The file total against the funding balance, so the run does not stop halfway for want of money.
What to check once, not every time
Verifying that the destination account exists and belongs to the payee is worth doing properly, but it belongs at the point the beneficiary is created rather than inside the payment run. Cache the result against the beneficiary and invalidate it when the account details change.
The methods and their trade-offs are covered in bank account verification. The relevant point here is placement: a batch is the worst possible moment to discover a destination is wrong, because there is no one present to correct it.
The funding check people skip
A run that exhausts the balance at line 2,800 leaves a mess that is worse than not starting: some people are paid, some are not, and the finance team now has to work out which. Checking the total against available funds before submission costs one call.
- Compare the batch total against the available balance, not the ledger balance.
- Account for other runs scheduled in the same window.
- Decide in advance whether a short balance blocks the run or releases it in priority order — and make that choice visible to whoever approves.
- If the run can proceed partially, make the ordering deliberate rather than whatever order the file happened to be in.
Idempotency, or how to avoid paying twice
If a submission times out, you cannot know whether it was received. Without an idempotency key your only options are to retry and risk duplicate payment, or not retry and risk non-payment. Both are bad; the key removes the choice.
- Derive the key from your own domainSomething stable like the payroll run identifier plus the beneficiary. A random value generated at call time defeats the mechanism, because the retry generates a different one.
- Generate it before the first attemptThe key must be identical across every retry of the same logical payment, which means creating it when the line is created, not when it is sent.
- Store it with the lineIt is the handle you use to ask the provider what happened to a payment whose response you never received.
- Check the validity windowMake sure the provider's idempotency window is longer than your longest retry schedule, or a late retry becomes a new payment.
- Never reuse a key for a different paymentIf a line is corrected and resubmitted as a genuinely different payment, it needs a new key — otherwise the provider returns the original result and the correction silently does nothing.
Retry narrowly, and only what is retryable
Resubmitting a whole file because nine rows out of four thousand failed is the classic route to duplicate payments. Retry the failed lines, never the batch, and only where another attempt could plausibly produce a different result.
| Failure | Retryable? | What to do |
|---|---|---|
| Request timeout | Yes | Retry with backoff using the same idempotency key |
| Provider 5xx | Yes | Retry with backoff; alert if it persists |
| Rate limited | Yes | Back off and reduce concurrency, do not just retry harder |
| Insufficient balance | Yes, later | Hold the line and retry after funding; do not fail it |
| Invalid account number | No | Route to an operator; the data has to change first |
| Account closed | No | Route to an operator and flag the beneficiary record |
| Beneficiary name mismatch | No | Stop. This needs a human decision, not another attempt |
Backoff that does not make things worse
A thousand lines failing at once and retrying on a fixed interval produces a thousand simultaneous retries, which is how a recovering upstream gets knocked over a second time.
- Back off exponentially rather than on a fixed interval.
- Add jitter, so lines that failed together do not retry together.
- Cap the total attempts and then stop, leaving the line visible in a pending state.
- Reduce concurrency on rate-limit responses rather than only delaying — the signal is that you are sending too fast, not too soon.
- Alert on the retry rate crossing a threshold; it is your earliest warning of an upstream problem.
Make the failure list the product
The team running payroll does not want a success rate. They want the specific rows that did not go through, why, and what to do about each. That list is the actual deliverable of a payout system.
- Group failures by cause, so one upstream problem reads as one problem rather than four hundred.
- Use language an operations person can act on, not the provider's raw error code.
- Make it exportable, because the correction usually happens in a spreadsheet or an HR system.
- Allow a corrected subset to be resubmitted as its own run, with its own idempotency keys.
- Keep the original batch linked, so the full history of an individual payment survives the correction.
Translating provider errors into actions
| What the provider says | What the operator needs to read |
|---|---|
| BENE_ACC_INVALID | Account number not recognised by the bank — check with the employee |
| IMPS_LIMIT_EXCEEDED | Above the per-transfer limit for this rail — will be sent by another route |
| BENE_BANK_OFFLINE | The receiving bank is temporarily unavailable — retrying automatically |
| NAME_MISMATCH | The account holder name does not match our record — needs review before sending |
| INSUFFICIENT_BALANCE | Not enough funds to complete the run — add funds and resume |
This mapping is boring to write and it is the difference between a payout tool an operations team can run and one that needs an engineer every month.
Watch the money, not the API
A batch that submits cleanly has not paid anyone. Until funds leave the account and settle, nothing has happened — and your database records what you intended, not what occurred.
- Reconcile against the account statement rather than against the API's acknowledgements.
- Alert on lines still pending well past their expected settlement window; those become support tickets.
- Watch for returns, which arrive after apparent success and will not appear unless you look for them.
- Check both directions: a debit on the statement with no matching line is the more alarming case.
- Run the reconciliation daily, so a discrepancy is hours old rather than weeks old when you find it.
The same principle applies to collections, where virtual accounts make the matching tractable by putting identity into the destination rather than the narration.
Frequently asked questions
Should a failed batch be resubmitted in full?
No. Resubmit only the failed lines, each carrying the same idempotency key it had originally. Resubmitting the whole file is the most common cause of duplicate payments.
What makes a good idempotency key?
Something derived from stable data in your own system — a payroll run identifier plus the beneficiary, for example. It must be generated before the first attempt and reused unchanged on every retry.
Does an idempotency key stop me paying someone twice?
Only for duplicate submissions of the same payment. If the same person appears twice in the file, those are two different logical payments with two different keys, and both will go. Duplicate detection within the file is a separate check.
How long should a pending payout stay pending before it is investigated?
Set the threshold from the rail's normal settlement window rather than a fixed number, and alert on anything materially past it. Those are the lines that turn into support contacts.
Can a payout fail after it has succeeded?
Yes. A transfer can be returned by the beneficiary bank days later. Your state model has to allow a settled payment to move to returned, or the event will either crash the handler or be silently dropped.
What happens if the balance runs out mid-run?
Decide in advance. Either check the total against available funds before submitting and block the run, or define a deliberate priority order for partial release. Discovering it at line 2,800 leaves a reconciliation problem nobody wants.
Should verification happen inside the batch?
No. Verify beneficiaries when they are created, while someone is present to fix a typo. A batch is the worst moment to discover a destination is wrong, because there is nobody to ask.
