For anyone about to sign off an integration

Integration Testing and Acceptance: Proving It Works Before You Trust It

Almost every integration is demonstrated the same way. Someone places a test order, everyone watches it appear in the other system, and the room agrees it works. That demonstration proves one path, on one clean record, on a quiet day. It tells you nothing about the records that will actually break it.

This page is about the testing in between the build finishing and the business relying on it: what environments you need, how to get realistic test data without mishandling customer information, which awkward cases to deliberately throw at it, and what acceptance evidence should look like so sign off means something.

Book a Testing Review (03) 9003 0111

Realistic ROI

1 to 5%

Of real records are awkward in some way

Which is exactly the population a three record demo will never contain

Volume worth rehearsing before peak

A rough working figure, set it from your own seasonal history rather than a rule of thumb

Every retry

Must be provably safe to run twice

Untested idempotency is the most common cause of duplicate invoices and orders

Once is not enough

Vendors change APIs after you go live

Regression testing is the reason an integration is still working in two years

Four Things That Separate Real Testing From a Demo

Integrations almost never fail on the happy path. They fail on the cases nobody thought to try, so the testing has to be designed around finding those.

Test data has to be realistic and lawfully handled

Testing on three invented records proves nothing, because the failures live in your real data: the apostrophe in a surname, the address with no street number, the customer whose account was merged in 2019, the product with a code containing a slash. But copying a production database into a test environment moves real customer information somewhere with weaker controls, which engages your obligations under the Privacy Act 1988 and the Australian Privacy Principles. The workable answer is a masked or synthesised extract that preserves the shapes and the awkwardness without carrying identifiable personal information.

The retry path needs testing more than the first attempt

Networks drop and APIs time out, so every integration retries. The question that decides whether you get duplicate orders and duplicate invoices is whether a repeated action updates the existing record or creates a second one. This cannot be verified by inspection or promised in a meeting. It has to be tested by deliberately sending the same message twice, interrupting a write halfway, and replaying a batch that already succeeded, and then checking the receiving system for duplicates.

Sandboxes lie in specific and predictable ways

Vendor test environments often have different rate limits, smaller data sets, features disabled, and behaviour that quietly differs from production. They are still worth using, but you need to know where yours diverges. The practical approach is to test logic in the sandbox, then run a limited, closely watched test in production on records you control before opening the flow up, because some faults will only ever appear against the real system.

Acceptance needs evidence, not a demonstration

Sign off should rest on a documented set of cases with recorded outcomes, including the ones that were expected to fail and did, plus a reconciliation showing that the number of records that went in matches the number that arrived. A demonstration shows that the software can work. Evidence shows how it behaves across the range of inputs your business actually produces, which is the question the person signing is really being asked.

What a Proper Integration Test Plan Covers

Six layers. The first two are usually done, the rest are usually skipped, and the rest are where the faults are.

Somewhere safe

Environment strategy

Deciding which systems have usable sandboxes, which do not, and what to do about the ones that do not. Where no test environment exists, the answer is normally a controlled production test against records created for the purpose, with a documented cleanup. What matters is that this is a decision made in the open rather than discovered on go live day.

Realistic, not personal

Test data preparation

A masked or synthesised extract that keeps the difficult characteristics of your real data: long names, unusual characters, missing fields, historical records with obsolete codes, accounts that have been merged. Realistic enough to find faults, handled so that customer personal information is not sitting in a weakly controlled environment.

Known edge cases

The awkward record corpus

A deliberately assembled set of the cases that break things: the zero quantity line, the negative amount, the refund against a deleted product, the duplicate email across two accounts, the order placed at the moment a price changed. Assembled once from your real history, reused for every future change, and added to whenever something new goes wrong.

Safe to repeat

Failure and replay testing

Deliberately breaking things to see how the integration behaves. Sending the same message twice, cutting the connection mid write, throttling responses, taking the destination system offline and bringing it back. Then confirming that nothing was duplicated, nothing was silently dropped, and the failed items can be replayed successfully once the cause is fixed.

Holds at peak

Volume and timing rehearsal

Running realistic volumes through before the business does, at the shape peak actually takes, which for most Australian retailers is a short intense burst rather than a steady increase. Watching for rate limiting, queue backlogs and processing times that grow non linearly, and confirming that a backlog drains rather than compounding.

Evidence and a suite

Acceptance and ongoing regression

Documented cases with recorded results, a reconciliation of records in against records out, and a set of tests kept and rerun whenever a vendor changes an API, a system is upgraded or a rule is amended. The suite is what keeps the integration working long after everyone who built it has moved on.

Cases Worth Testing That Usually Are Not

Task	Traditional	Tested Properly	Notes
The same message arrives twice	Assumed to be handled	Sent deliberately, checked for duplicates	The single highest value test in integration work, and among the quickest to run.
Destination system is offline	Never tried	Taken down mid run, then recovered	Reveals whether records queue and replay or simply vanish into a log nobody reads.
A record with unusual characters	Fails in week three	In the test corpus from day one	Apostrophes, ampersands and accented characters in names and addresses are ordinary in Australia.
Price changes mid order	Found by a customer complaint	Tested as a timing case	Matters most around promotion start and end times, so test at the boundary deliberately.
Refund against a deleted product	Errors silently	Routed to the exception queue	Historical data always contains references to things that no longer exist.
Ten times normal volume in an hour	Discovered on the day	Rehearsed weeks earlier	Sale events and marketplace campaigns produce bursts, not gentle curves.
The vendor changes a field name	Integration stops quietly	Regression suite catches it	This is the case that justifies keeping the tests rather than discarding them at go live.
Reconciling what went in against what arrived	Not done at all	A counted, evidenced comparison	The only test that proves nothing was lost, as opposed to proving something worked.

Testing Mistakes That Cost Real Money

Production data is copied into a test environment

It is the fastest way to get realistic data and one of the easier ways to create a privacy problem. Test environments usually have broader access, weaker logging and less scrutiny than production, and a copy of your customer database sitting in one is a genuine exposure under the Privacy Act 1988 and the Australian Privacy Principles. Mask names, contact details and payment information on extraction, keep the structure and the awkwardness, and delete the extract on a schedule rather than leaving it indefinitely.

Testing stops at go live

The tests you built are most valuable after launch, because that is when vendors change APIs, systems get upgraded and rules get amended. Discarding the suite means every future change is verified by hope. Keep the cases, keep the data, and rerun them whenever anything in the chain changes. This is the practical difference between an integration that still works in three years and one that gets rebuilt.

Only the happy path is signed off

Acceptance criteria that say orders flow to the accounting system are not testable, because they do not say what should happen to the orders that cannot. Every acceptance case needs a defined expected outcome including the failures: this record should be rejected with this reason, this one should be queued for review, this one should be corrected automatically and noted. Sign off then means something specific rather than a general impression that it seems fine.

Nobody from the business is in the testing

Developers test whether the integration does what the specification says. Only the people who process orders, chase payments or answer customers know whether the result is right, and they are the ones who recognise the awkward cases from experience. Testing without them produces an integration that is technically correct and operationally wrong, and the gap is usually found in the first fortnight of live running.

Test transactions reach real customers or real ledgers

Testing against production without a clear boundary has sent confirmation emails to real customers, created real invoices and pushed test products onto live storefronts. Where you must test in production because no sandbox exists, agree the boundary in advance: dedicated test accounts, a suppressed notification path, a documented cleanup and a person watching while it runs. Treat it as a controlled exercise rather than a quick check.

The rehearsal uses the wrong shape of load

Averaging your peak day over twenty four hours and testing that rate proves very little. Australian retail peaks arrive as short bursts around a sale launch, a marketplace promotion or a payday, and rate limits are hit by the burst rather than the average. Rehearse the actual shape, watch whether the queue drains between bursts, and confirm that a backlog recovers rather than growing until it stops.

How Yes AI Approaches Integration Testing

We build the corpus from your real history

Rather than inventing test cases, we take a sample of your actual records and mine it for the shapes that cause trouble. That corpus becomes a reusable asset you keep, and it is the reason the second and third integrations go live more smoothly than the first.

Realistic data without the privacy exposure

We mask personal information on extraction while preserving the characteristics that break integrations, so testing is meaningful without moving customer records into an environment with weaker controls. Extracts are held for the life of the project and then removed.

Failure testing as standard, not as an extra

Duplicate messages, interrupted writes, offline destinations and throttled responses are part of the normal test plan on every integration we build, along with the replay of anything that failed. If a flow cannot survive being run twice we would rather find that in testing than in your ledger.

Acceptance evidence you can actually sign

You receive documented cases with recorded outcomes and a reconciliation of records in against records out, in language a finance or operations manager can check. The regression suite stays in place afterwards and is rerun when vendors change things.

From Built to Genuinely Trusted

Five steps. On a single integration this is typically one to three weeks, and it is the cheapest insurance in the project.

Agree what correct looks like

Testable acceptance criteria for each flow, including the expected outcome for the records that should fail. Written so the people who will use the integration can check them, and signed off before testing starts rather than negotiated afterwards.

Prepare environments and safe data

Sandbox availability confirmed per system, the gaps identified, and a masked extract prepared that keeps the difficult characteristics of your real records without carrying identifiable personal information.

Run the corpus and the failure cases

The awkward record set, the duplicate message tests, interrupted writes, offline destinations and throttled responses. Every result recorded, and anything unexpected fixed and retested rather than noted as a known issue.

Rehearse volume, then run a watched pilot

Realistic peak shaped load through the flow, then a limited live release on a narrow segment with a person watching and a defined rollback, before the integration is opened to everything.

Hand over evidence and keep the suite

Documented acceptance evidence, reconciliation of records in against records out, and the regression tests retained and scheduled so vendor changes and upgrades are caught rather than discovered by your customers.

FAQ

What is integration testing, as distinct from testing the software?

Testing the software asks whether each system behaves correctly on its own. Integration testing asks whether the connection between them behaves correctly, which is a different question with different failure modes: records that do not match, fields that mean different things in each system, messages that arrive twice or out of order, and the behaviour when one side is unavailable. A system can pass all its own tests and still be part of an integration that loses records daily.

Can we test using a copy of our production data?

You can, but not by copying it unchanged. Realistic data is genuinely necessary because the faults live in your real records, so the answer is a masked extract that preserves the structure and the awkwardness while removing identifiable personal information. Moving a full copy of customer records into a test environment with weaker access controls creates an exposure under the Privacy Act 1988 and the Australian Privacy Principles that is easy to avoid and awkward to explain afterwards.

What if a system has no sandbox environment?

This is common with older and specialised systems, and it changes the approach rather than preventing testing. The usual answer is a controlled test against production using accounts and records created for the purpose, with notifications suppressed, a documented cleanup and someone watching while it runs. What matters is agreeing that approach in advance with the business owner, because the risk is real and it should be accepted deliberately rather than encountered by accident.

How do we test that retries do not create duplicates?

By deliberately causing them. Send the same message twice and confirm the receiving system holds one record. Interrupt a write halfway and confirm the retry completes it rather than starting a second one. Replay a batch that already succeeded and confirm nothing is duplicated. This requires every write to carry a stable reference the destination can recognise, and it is worth verifying rather than trusting, because untested idempotency is behind most duplicate invoice and duplicate order incidents.

How long should testing take?

For a single well defined integration, typically one to three weeks including the volume rehearsal and the watched pilot. For a programme spanning several systems it is proportionally longer, though much of the test data preparation is reused. The temptation is to compress it because the build looks finished, but this is the phase that prevents the expensive class of problem, and the cost of a fortnight of testing is small next to a month of reconciling duplicate transactions.

Who should be involved from our side?

The people who will live with the result: whoever processes the orders, reconciles the accounts or answers the customers. They recognise the awkward cases from experience, they know what the output should look like, and their sign off is worth having because they will be the ones raising the issues later. A test plan approved only by technical staff routinely produces integrations that satisfy the specification and still do not work for the business.

Do we need to keep testing after go live?

Yes, and this is the part most often skipped. Vendors change and deprecate APIs, systems get upgraded, and your own rules change, usually without a warning that reaches the right person. Keeping the acceptance cases as a regression suite and rerunning them on a schedule and after any change is what turns a working integration into one that is still working in two years. It also makes the next change cheaper, because you can prove you have not broken anything.

Sign Off With Evidence, Not a Demonstration

Book a call. We review how your integrations are being tested, build the case corpus from your real data, and give you an acceptance plan with prices. The plan is yours either way.