Our integration has been down for weeks. What should we do first?

Three things, in this order. Stop it from creating any further wrong data, which may mean pausing it entirely and agreeing who handles the affected process manually in the meantime. Secure administrative access to every system and credential involved, including anything held only by whoever built it. Then measure the gap properly, comparing record counts and totals on both sides across the whole affected period. Resist restarting until you have done the third step, because a restart can push stale records into live systems or overwrite corrections your team has already made by hand.

Should we repair the existing integration or start again?

It comes down to whether the design was sound. Repair is usually right when the logic is visible and documented, the fault is identifiable, there is an agreed source of truth for each data type, and the writes are safe to repeat. Rebuild is usually right when there is no monitoring, no exception handling, no documentation, no agreed data ownership, or when the same integration has failed repeatedly in different places. That last pattern is the clearest signal, because it means each fix is addressing a symptom of a missing foundation rather than a defect. We give the recommendation in writing with the reasoning and a cost against each path.

The original developer is not responding. Can you take it over?

Usually yes, and it is a common situation rather than an unusual one. What we need is administrative access to the systems involved, whatever documentation exists even if it is fragmentary, and access to the environment the integration runs in if it is reachable. Where nothing was documented, we reconstruct the behaviour from the logs and from the data itself, which is slower but reliable. The urgent part is securing access before it lapses, so if there is any prospect of a difficult conversation, do the access audit now rather than after.

How do we know how much damage has been done?

By counting rather than sampling. We compare record counts and financial totals on both sides of every affected flow across the full period, then categorise what we find: records that never arrived and can be replayed, records that arrived wrong such as duplicates or incorrect prices, and consequential damage such as oversells, incorrect tax figures or decisions made on stale reports. The output is a written report with counts, dollar values and dates. Spot checks are the main reason businesses underestimate this, because failures are often intermittent or limited to one category of record, which is exactly the pattern a sample misses.

Will you help us deal with the supplier who built it?

We will give you the factual basis to have that conversation, which is usually what is missing. The damage report, the reconstruction of what the integration was doing, and an honest assessment of whether the design met a reasonable professional standard are all things you can put in front of them or in front of your own advisers. What we will not do is act as your legal representative or write a report designed to reach a predetermined conclusion. Our findings are the same whoever is reading them, which is what makes them useful to you.

How long does a rescue take?

Triage, meaning containment, the damage report and the access recovery, is normally a week or less. Backfilling and correcting depends entirely on how much data is affected and how tangled it is, and can be anything from a few days to several weeks. Stabilisation, meaning monitoring, alerting, exception handling and safe repeatable writes, is usually two to four weeks and then needs to run through a full month end before anyone should call it done. A full rebuild, if that is the recommendation, follows the normal timeline for a new integration but with the advantage that the rules have already been reconstructed.

How do we stop this happening again?

Four things prevent almost all repeat occurrences. Monitoring that alerts a named person the same day, rather than a dashboard nobody opens. Reconciliation that proves what left one system arrived in the other, at record level, every day. Writes that are safe to repeat, so a retry updates rather than duplicates. And documentation that is a deliverable kept current, so the integration is repairable by somebody other than its author. None of these are expensive relative to a rescue, and their absence is what turns a small technical fault into the situation that brought you to this page.

For businesses living with an integration that no longer works

When the Integration You Paid For Has Stopped Working

Integrations rarely announce their failure. More often somebody notices at month end that the numbers do not agree, or a customer is charged twice, or the warehouse has been picking against stock figures that stopped updating three weeks ago. By the time it is obvious, the problem is not just technical. There is a data mess to quantify, a supplier relationship to navigate, and a decision to make about whether any of it is worth saving.

This page is the sequence we use. Stop the bleeding first, measure the damage second, recover the knowledge and credentials third, and only then decide between repairing what exists and replacing it. Rebuilding is not automatically the right answer, and neither is persevering. What matters is making that call on evidence rather than on frustration.

Book an Integration Triage Call (03) 9003 0111

Realistic ROI

Days to weeks

Typical gap between failure and discovery

Which is why the damage assessment matters more than the fault itself

3 questions

Decide repair versus rebuild

Is it documented, is it reachable, and is the design sound

Stabilise first

Before any improvement

Adding features to an unstable integration is how a bad month becomes a bad quarter

Every record

Should be accounted for, not sampled

A spot check that looks fine is the most common reason damage is underestimated

Four Things to Establish Before You Change Anything

The instinct is to start fixing. The four questions below determine whether fixing is even the right move, and answering them takes days rather than weeks.

What has actually been happening, and for how long

Before anything is repaired, you need a factual account of what moved, what did not and when it changed. That means comparing record counts and totals on both sides across a period, not sampling a handful and concluding it looks fine. The common discoveries are that the failure started well before anybody noticed, that it is intermittent rather than total, or that it only affects one category of record such as orders with a particular payment method. Each of those points to a different fix and a very different remediation effort.

What the failure has cost so far

Damage has three parts and they need separate counting. Missing records are the ones that never arrived and can usually be replayed. Wrong records are more dangerous, because duplicates, incorrect prices and misapplied payments have already been acted on by people and downstream systems. Consequential damage is the oversells, incorrect BAS figures, unhappy customers and decisions made on stale reports. Only the first is fixed by restarting the integration, and treating a restart as the end of the job is the most common mistake in this situation.

What you actually control

The uncomfortable audit is of access rather than code. Do you hold the credentials for every system involved, or does the original builder? Do you have the field mappings, the business rules, the specification and the logs, or are they in somebody else’s account? Can you see what the integration has been doing, and can you switch it off yourself? A business that cannot answer these has a commercial problem alongside a technical one, and resolving the access question usually has to come before anything else.

Whether the design was ever right

Some integrations fail because something broke. Others fail because they were built on assumptions that were never sound: no agreed source of truth, writes that duplicate on retry, no exception handling, real time chosen where a scheduled batch belonged, or no monitoring at all. The first kind is worth repairing and often quickly. The second will keep failing in new ways however carefully each individual fault is patched, and continuing to pay for repairs on it is the more expensive path even though each individual invoice looks small.

The Rescue Sequence

In this order. Skipping ahead to the rebuild is tempting and it is how businesses end up with a second integration that inherits the first one’s data problems.

Stop the damage

Contain

If the integration is actively creating wrong data, pausing it is better than letting it run while you investigate. That means agreeing quickly what manual process fills the gap, telling the people affected, and making sure the pause is genuine rather than a flow that keeps retrying quietly in the background. Containment is unglamorous and it is what stops a fortnight of duplicate invoices turning into a quarter of them.

Damage report

Measure

A record level comparison across the affected period: what exists on each side, what is missing, what is duplicated and what is wrong. The output is a written damage report with counts and dollar values, split into what can be replayed automatically, what needs a person, and what has consequences that cannot be undone. This document is what makes every later decision, including any commercial conversation with the original supplier, a matter of fact rather than assertion.

Access and knowledge

Recover

Credentials brought under your control and rotated, logs and configuration exported while you still have access, and whatever specification, mapping or documentation exists gathered in one place. Where nothing was written down, the behaviour is reconstructed from the logs and from the data itself. This step is time critical because access can disappear at exactly the moment a relationship becomes difficult.

Repair or rebuild

Decide

With the damage report and the access position established, the choice becomes answerable. Repair suits an integration whose design is sound, whose logic is visible, and where the fault is identifiable. Rebuild suits one that has no monitoring, no exception handling, no agreed data ownership or no documentation, because those are not defects to be patched but the absence of a foundation. We put the recommendation in writing with the reasoning, including the case for keeping what you have.

Data restored

Backfill

Missing records replayed in the right order, duplicates identified and removed or merged under a rule your finance team has approved, and incorrect records corrected with an audit trail so the change is explicable later. This is done deliberately and reconciled at each step rather than in one large run, because a backfill that itself goes wrong is considerably harder to unpick than the original failure.

Then improve

Stabilise

Monitoring, alerting to a named person, an exception queue, idempotent writes and a documented recovery path, all in place and proven over a full month end before anyone adds a new feature. The discipline of finishing stabilisation before starting improvement is what separates a rescue that holds from one that becomes the next rescue.

Common Failure Patterns and What They Mean

Task	Traditional	The Right Response	Notes
Sync stopped weeks ago	Restart and hope	Replay the gap, then fix the cause	Usually an expired credential or a changed vendor interface. The restart is easy, the missing period is the work.
Duplicate invoices or orders	Delete them manually	Rule based cleanup plus idempotent writes	Non idempotent writes plus retries. It will recur at peak unless the write is fixed, not just the data.
Records vanish with no error	Nobody knows how many	Counted, queued and replayed	Silent skips on unmatched data. The count is almost always larger than the estimate.
Two systems overwrite each other	Argue about which is right	Declare one owner per data type	A design fault, not a bug. Patching it field by field does not hold.
Works except at month end	Blamed on the systems	Volume and rate limits modelled	Throttling under load. A throttled integration looks exactly like a stopped one.
The builder is unreachable	Wait and escalate	Recover access first, then decide	Secure credentials and export logs early. Access is what you lose first in a soured relationship.
Nobody can explain the rules	Reverse engineer under pressure	Reconstruct and document	The logs and the data usually reveal the rules. Writing them down is half the rebuild anyway.
Fixed repeatedly, keeps breaking	Another patch each time	Assess the design honestly	Recurring faults in new places is the signal that the foundation is the problem.

Mistakes That Turn a Problem Into a Crisis

Restarting before measuring

Switching a failed integration back on can push a backlog of stale records into live systems, overwrite corrections somebody made by hand, or replay transactions that were already dealt with manually. Establish what is missing and what is wrong first, decide deliberately what should be replayed, and only then restart. The urge to get it running again is understandable and it is how a contained problem becomes an uncontained one.

Losing access while the relationship sours

When a supplier relationship deteriorates, access tends to disappear before anybody has agreed a handover. Export the logs, configuration and any documentation early, get administrative control of every credential the integration uses, and rotate anything you cannot account for. Do this while conversations are still cordial. It is a normal precaution rather than an accusation, and businesses that leave it until the dispute stage frequently cannot recover their own configuration.

Rebuilding on the same unexamined assumptions

A rebuild that repeats the original design, with no agreed source of truth, no exception handling and no monitoring, produces the same failure in eighteen months with a newer invoice attached. Before rebuilding, write down which system owns each data type, what happens to records that cannot be matched, what frequency each flow genuinely needs, and who is alerted when something stops. If the rescue does not produce those decisions, it has not addressed the reason the first one failed.

Cleaning data without a rule and an audit trail

Merging duplicates and correcting records changes financial history, and doing it manually under time pressure creates errors that are far harder to trace than the original problem. Agree the merge and correction rules with whoever owns the numbers, apply them programmatically so they are consistent, keep a record of every change with its reason, and reconcile after each batch. Where a correction has tax consequences, involve your accountant before the change rather than at the next BAS.

Under-reporting the impact

If the failure touched customer orders, payments or personal information, the consequences reach beyond operations. Customers charged twice or shipped nothing have rights under Australian Consumer Law and the practical remedy is to find them proactively rather than wait for complaints. If personal information was exposed, sent to the wrong recipient or lost, the Privacy Act 1988 and the notifiable data breaches scheme set out assessment and notification obligations with time expectations attached. Establish early whether either applies, because both get harder to handle the longer they sit.

Adding improvements during the rescue

There is always a request to add the extra field or the new channel while somebody is in there anyway. Doing it during a rescue means you can no longer tell whether a new symptom comes from the original fault, the repair or the addition, and it extends the period of instability. Stabilise, prove it across a full month end, then take the improvement list as a separate piece of work with its own specification.

How Yes AI Runs a Rescue

Triage in days, not weeks

A short engagement that establishes what is failing, how long it has been failing, what it has cost and what you control. You get a written damage report and a recommendation, including where that recommendation is to keep what you have and fix it.

A damage report you can act on commercially

Counts, dollar values and dates, separated into recoverable and consequential. It is what you need for an internal decision, an insurance question or a conversation with the original supplier, and it is yours whether or not we do any further work.

Stabilise before improving

Monitoring, alerting, an exception queue and idempotent writes go in first and are proven over a full month end. New capability comes afterwards as a separate piece of work with its own specification, so cause and effect stay legible.

No lock in, on principle

Documentation, mappings and credentials are yours from the start. We take over integrations built by others regularly and we know how much harder the situation is when the previous arrangement was opaque, so we do not create that situation for the next person.

From Broken to Trustworthy

Five steps. Triage is normally complete within a week, and stabilisation within four to eight depending on what the damage report finds.

Contain and take stock

Pause anything actively creating wrong data, agree the manual fallback and who is doing it, tell the people affected, and secure administrative access to every system and credential the integration touches before anything else moves.

Measure the damage at record level

A full comparison across the affected period rather than a sample, producing counts and dollar values split into missing, duplicated, incorrect and consequential. This is the document everything else is decided from.

Recover knowledge and decide

Logs, configuration and any documentation exported and consolidated, behaviour reconstructed where nothing was written down, then a written repair or rebuild recommendation with the reasoning and a cost for each option.

Backfill and correct deliberately

Missing records replayed in order, duplicates resolved under rules your finance team approved, corrections applied with an audit trail, and reconciliation after each batch rather than one large run and a hope.

Stabilise, prove, then improve

Monitoring, alerting to a named person, exception queue and idempotent writes in place, proven across a full month end, and only then a separate scoped piece of work for whatever you actually wanted to add.

FAQ

Get an Honest Assessment of What You Have

Book a triage call. We establish what is failing, how long it has been failing and what it has cost, then tell you plainly whether to repair or replace it. The damage report is yours either way.