Case Studies

NSW Spatial Services: Leap from Legacy to Microservices

Holly Hudson
February 24, 2022

When the Spatial Services Division went to open market via a competitive tender process, they were seeking to replace two of their important systems.

One system provided the Department and all of the Councils across NSW with a platform that managed the process of road naming – which is a seemingly simple function of government, but that is of critical importance if you are waiting for an ambulance to arrive at your address to help with an emergency.

The other system has a primary function of providing the master data record for all geographic locations across the State, this includes parks, lakes, mountains, rivers, and beaches – as well as their indigenous name and history of the location.

Both systems were tools for processing the legislative and regulatory requirements to managing these important datasets and any additions or changes to the records in these systems were subject to a multitude of business logic and procedural controls.

Spatial Services recognised that over time these two systems were preventing the team from providing the service they wanted to offer their stakeholders.

The underlying technologies had become aged and unsupported, which at best made them unstable and costly to operate, and at worst presented an increasing risk of irrecoverable failure, data loss and severe reputational damage. Components of the system also depended on proprietary technology that was costly.

The functional capabilities of these two systems were challenging too, with pain points relating to:

Regulatory and legislative functions were not being fully supported
Poor user experiences were inhibiting users and creating unsustainable support demands on the team
The systems lacked features such as workflow support, adequate validation of inputs, and reporting
Processes for user creation and management were labour-intensive
The systems lacked self-service features and required multiple instances of offline processing
Legacy data structures made it difficult to reflect the Machinery of Government changes
User interface accessibility did not meet the required standards (WCAG) and the system did not meet the businesses file format support needs

“We needed a service provider to gain a deep understanding of the challenges we were facing, and to assist us to build a new service that would firstly, overcome those challenges so we could do a lot more than we were previously able, and then provide a platform upon which we could execute some of the key strategic objectives we had for the near and mid-term future. Crucially all of this needed to happen without any business interruption. The team achieved that with aplomb, and we are incredibly appreciative of the role their AWS skills and expertise played in the success of the project.”

Christian Dowling, Senior Project Manager.

Our approach

Experience Digital, an AWS Advanced Tier Services Partner, are often engaged by clients for application modernisation and legacy data migration.

It was clear from early on that the Spatial Services project would require a modern cloud architecture to host the replacement applications and to effectively overcome the infrastructure challenges being faced.

AWS Lambda services power an API layer that connects a Vue. js front end, a JIRA workflow engine, AWS Cognito, AWS S3 for storage and SendGrid for communications. This all sat within a well-architected framework supported by a new modernised PostgresSQL database.

The Experience Digital team worked collaborative with the team at Spatial Services to map out a plan for designing and building an entirely new solution and to migrate the master data from the old platforms onto the new. This process required creating a “migration as a service” – a comprehensive migration script that could be run and re-run multiple times upon validation of the data by the customer from the preexisting Informix data structure. In one instance this entailed the Experience Digital team reauthoring a no-longer supported legacy database driver to effectively extract and transform the data.

Using an Agile approach, the initial sprints were dedicated to detailing the use cases and validating the technology requirements to support those use cases.

The following sprints included a comprehensive design-driven process to ensure the user interfaces were suited to the intended users, with the business outcome being the focus – not basic replication of the current archaic interface.

With the design of the front-end completed, subsequent sprints focussed on building the front-end interfaces, whilst building the back-end to support all the business functions through JIRA as a workflow engine in a framework that accommodates future needs. The final sprints focused on getting the project ready for cutover. We also facilitated the training material prepared for all user types. The data migration was finalised. The support channels were initiated. Fully developed Continuous Deployment and Continuous Integration pipelines were engaged to support simple and fool-proof future releases.

AWS Lambda’s Role in the Approach

One of the main requirements of the client was a highly available and scalable solution and as a result, a micro-services architecture was chosen.

Using Lambdas to run the Node.js functions we picked off the shelf AWS Services such as API gateway, Cognito and SQS. AWS Cognito was the primary authentication service while AWS SQS was used for queuing the messages being emailed and document generation.

The first deployment and proof of concepts were created using AWS Amplify, as it allowed us to quickly provision services and test out our deployment and CI/CD strategy. Eventually, a custom cloud formation template took over as requirements became more complex and needed to be catered for.

The outcome

Experience Digital delivered a responsive, WCAG compliant public-facing web application that provided detailed information to various user types and offered a pleasant interactive user experience on the back of an Atlassian Jira powered workflow engine.

The Spatial Services Road and Place Naming application project was successfully delivered and has gained a welcome reception as one of the Departments of Spatial Services’ most successful projects in collaboration with the private sector. A specific highlight from the project is that one of NSW’s largest councils, that previously had refused to even use the legacy system, was the first to embrace the new one. We conducted a thorough handover for the platform and it is now maintained in-house at the DFSI headquarters in Bathurst, NSW.