How a Reliable Sell-Through Pipeline Works
Six building blocks. Each one fixes a specific way that retailer data goes wrong between the retailer’s system and your decision.
Every file captured, on time Collect from every source
Each retailer is set up with its own collection method. Portal reports are downloaded on schedule where the portal allows automated access, or uploaded by a person to a monitored drop location where it does not. Emailed spreadsheets go to a dedicated mailbox and are picked up automatically. EDI sales and inventory reports, such as EANCOM SLSRPT and INVRPT messages where a retailer supports them, arrive through your EDI provider. Online retailers and marketplaces are polled by API. Each expected file has a due time, and a missing one raises an alert.
One table, every retailer Normalise into one model
Every source is translated into the same structure: retailer, store, week ending date, your SKU, units sold, sales value and stock on hand, plus the original file and row it came from. Columns are mapped per retailer and per file version, because retailers change report layouts without notice. Values are recorded with their basis, whether at retail or at cost and whether GST inclusive or exclusive, so nothing is added together that should not be. Raw files are kept so any figure can be traced back.
Retailer codes become your SKUs Map item codes through GTIN
Retailers report against their own article numbers, and sometimes against a barcode. The match uses the GTIN wherever it is present, because it is the one identifier both sides share, then a maintained cross reference of retailer item code to your SKU. Inner and outer carton barcodes, superseded GTINs after a pack change, and retailer specific bundles are all handled explicitly. Anything that does not match goes to an unmatched queue with the units and value at stake, so the biggest gaps are fixed first.
Maintain the store master
Each retailer’s stores are held in a store master with the retailer’s store code, name, state, format, opening date, closing date and any grade or cluster you use. When a new store code appears it is queued for setup rather than guessed. Closures stop being expected in the data, renames and code changes are linked to the same physical store, and like for like comparisons exclude stores that were not trading in both periods. State is needed for rep territories and for any regional promotion analysis.
Align the calendars
Retailers report in their own retail weeks, which may end on Saturday, Sunday or another day, grouped into 4-4-5 or similar periods, with an occasional 53 week year. Your finance calendar is probably months. Every row is tagged with the retailer’s week, a standard week in your own calendar, and the period and financial year it falls into. Where week end days differ, the data is compared at the week level with the difference noted, rather than split into days that the source never provided.
Check quality before anyone uses it
Each load runs checks before it is published: missing weeks or stores compared with the last period, duplicate files or restated weeks that would double count, negative sales from returns or corrections, stock on hand that jumps implausibly, values that do not reconcile to the file total, and totals that move sharply against prior weeks. Restated weeks replace the earlier version rather than adding to it. Failed checks hold that retailer’s data and tell the owner, while the other retailers publish normally.