Using ETL to simplify a complex data migration

Runnymede Borough Council were about to undertake a complex data migration project, but the high volume of data and tight deadlines meant they needed a little help.

4 weeks saved using FME

Shift to OpenSource GIS completed

The Challenge

The Council needed to perform a complex data migration project as part of a council-wide drive to make significant cost savings. The decision was made to shift their GIS over to OpenSource software offerings including QGIS, PostGIS and Geoserver. Not only would this reduce cost, it would enable the spread of GIS throughout the council by expanding to additional machines.

To make this happen, huge volumes of spatial data would need to be migrated between new and legacy systems. Teams were using a complicated and bespoke Python script for the migration process, but project deadlines were at risk as they struggled to work with this.

The Solution

FME allowed the Council to:

Extract data from various applications/systems including LLPG, Planning and Land Charges.
Consolidate datasets as required.
Transform data into the required format.
Load data into the new PostGIS database.

In turn, the workflow processes enabled the Council to efficiently automate the migration of GIS desktop software to QGIS, increasing the number of desktop users at the Council from 6 to 14.

Runnymede Borough Council perform complex data migration with FME

FME® automates data manipulation and transformation

Data rarely arrives in your environment in a state that’s ready to use. The repetitive tasks required to get it into a usable form are laborious and costly. FME® automates these data manipulation and transformation tasks, leaving you to work much more efficiently with your newly optimised data set.

Here’s the problem; you have lots of data from lots of different internal and external sources, but managing it all can be difficult. You’re faced with a unique set of challenges that have a large impact on the way you work:

Driving value for your organisation by making the most of your business intelligence.
Managing the complex combination of data sources, formats and destinations.
Dealing with evolving data, resources and requirements.