Data migration done right

Data migration is the most underestimated workstream in a warehouse system project, and the one that produces the most visible failures at go-live. A correctly configured system started with incorrect data looks exactly like a faulty system.

This guide explains what needs to be done, in what order and with whom. It assumes no technical knowledge.

What to migrate, and what not to migrate

The first decision, often made by default: migrating everything is the wrong instinct.

A new system is a rare opportunity to get rid of clutter. An item created in 2014 and not ordered since 2019 has no reason to be migrated. Nor does a customer that has been inactive for five years.

---------------------- ----------------------- ----------------------- Migrate Decide Do not migrate

Active items Dormant but strategic Items with no movement (moved within 12–24 items for 3 years months)

Actual locations Planned locations not Deleted locations yet built

Verified physical stock Stock in dispute Stock already provided for destruction

Active customers and Recently inactive Old inactive accounts suppliers accounts

Open orders Old orders not fully Full order history closed ----------------------- ----------------------- -----------------------

History deserves particular mention. People often want to migrate it ‘so nothing is lost’. This is almost always a bad idea: history burdens the new system, arrives in structures that do not match and is only consulted exceptionally. Keep the old system available in read-only mode for a year — it is cheaper and more reliable.

This is the longest task, and it does not depend on any technical choice: you can start it today, even before selecting your vendor.

What you are looking for:

duplicates — the same product under two item numbers, often created by two different people;

inconsistent labels — varying abbreviations, random capitalisation and information mixed into the name;

incorrect units — the costliest error: an item whose stock unit is a case but whose sales unit is an individual piece, without the conversion ratio being recorded;

missing attributes — weight, dimensions and storage conditions, without which no slotting rule can work.

A practical method: sort your items by number of movements over twelve months. Give absolute priority to the top of the list — those items account for most of your daily activity and therefore most of your future incidents. The rest can be handled later, or even after go-live.

Who: someone from procurement or sales administration who knows the products, not an IT specialist.

The new system needs an exact list of your locations, with each location’s address, type, usable dimensions, capacity and any restrictions (maximum weight, temperature range, prohibited products).

Watchpoint: this list must reflect the physical reality on go-live day, not the original layout. If racking has been moved, an area has been repurposed or locations cannot be used, that must appear in the data.

Method: walk the entire warehouse with the layout in hand and the person who operates it. Allow one day for a medium-sized warehouse. This is also an opportunity to identify missing or illegible labels.

Three options, in decreasing order of reliability:

A complete stocktake with operations stopped. Activity is frozen, everything is counted and the system starts from that snapshot. This is the safest method. It costs one to three days of downtime and requires labour.

A zone-by-zone stocktake over several days. Each zone is counted while movements in that zone are frozen. Less disruptive, but it requires strict discipline: any movement in a counted zone invalidates the count.

Migrate theoretical stock, followed by intensive cycle counting. Start with the stock recorded in the current system, accepting that it is imperfect, and correct it during the first few weeks. Reserve this approach for situations where the initial variance is small and documented.

Our recommendation: whichever option you choose, use blind counts — the counter does not know the expected quantity — and plan an independent second count for significant variances. A variance confirmed by two independent people is information; one observed by just one person is a hypothesis.

A decision to make before starting: what happens to recorded variances? Who approves them? Who posts them to the accounts? This decision must come from finance management, not the warehouse.

Lower in volume, and often forgotten until the final week:

active customers and suppliers, with their actual delivery addresses;

carriers and their constraints;

packaging and logistics units by item;

customer-specific rules (special labelling, required documents, time slots);

users, their roles and their permissions.

The last point deserves attention: access permissions are almost always defined in a rush the day before go-live, then never reviewed. That is how everyone ends up as an administrator.

---------------------- ----------------------- ----------------------- Stage When Typical duration

Item-data cleaning From project launch 4 to 10 weeks, in parallel

Location survey After layout approval 1 to 3 days

Extraction and After main configuration 1 to 2 weeks transformation

Test load + 4 to 6 weeks before 1 week verification go-live

Second test load 2 to 3 weeks before 3 days

Migration stocktake Go-live weekend 1 to 3 days

Final load Immediately before opening A few hours ----------------------- ----------------------- -----------------------

The most important point in this table is the second test load. Many projects perform only one. Yet the first is used to discover problems; the second proves that they have been solved. Never go live with a load that has been tested only once.

Do not merely confirm that the load completed without a technical error. Check that the data makes sense:

The total. Do the value and number of stock units migrated match your accounts, allowing for the variance you approved?

The extremes. List the ten items with the largest quantities and the ten with the smallest. Anomalies are immediately visible — confusing an individual unit with a case produces a very obvious factor of 12 or 24.

A floor sample. Select thirty locations at random, inspect them physically and compare them with the screen. This is the only test that truly matters.

A complete journey. Create a test order, pick it in real life and ship it in the system. If the end-to-end process works with migrated data, the migration is usable.

Treating migration as a technical task. This is not an IT issue: it is about product knowledge and management decisions. IT carries the data; it does not decide what is correct.

Cleaning after migration. Cleaning in the old system is ten times faster than in the new one, where every correction must be carried through.

Not deciding how variances will be handled before counting. The count then produces figures that no one dares approve, and go-live slips.

Starting without a baseline. Record your stock-accuracy rate and current lead times before cutover. Without this, no honest assessment will be possible in six months.

Underestimating the teams’ time. Migration involves the people who know the operation — in other words, the people who keep it running. Their time must be planned and covered, not simply added to their workload.

Checklist before giving the go-ahead

☐ Active items have been cleaned and deduplicated, with units and attributes verified

☐ The location list matches verified physical reality

☐ The migration stocktake method has been chosen and scheduled

☐ The variance-handling rule is written down and approved by finance management

☐ Two test loads have been completed, the second with no blocking anomaly

☐ The four sense checks have been completed (total, extremes, floor sample, complete journey)

☐ Access permissions are defined by role, not individually

☐ Baseline indicators have been measured and archived

☐ The old system remains available for consultation

☐ A fallback procedure exists in case go-live has to be stopped