Over the past decade, there’s been a lot of discussion and emphasis on open data across the world and in the UK in particular. The importance of reaching data maturity and optimal ways of doing so are hot topics for a range of public sector organisations looking to streamline operations, enhance service delivery, and boost performance.

Despite the ongoing discourse and efforts, the reality of open data often falls short of its idealistic vision. Data that may at first glance seem ‘open’ still isn’t. This disparity came to the forefront as we explored Scotland’s transport network data. 

With Scotland investing significantly into its data infrastructure, we decided to take a look at the country’s frontline services and investigate how operational data across various settings could create meaningful efficiencies and service improvements. We started with the transport sector and what unfolded was a revealing journey into the challenges that hinder the realisation of truly open, interoperable, mature data.

This article marks the beginning of a comprehensive study presented in a two-part series exploring Scotland’s data infrastructure across diverse sectors. Below, we uncover some of the challenges to achieving data maturity that are applicable to organisations across the world. From addressing issues of accessibility and standardisation concerns, to exploring the broader impacts on service improvements, join us as we navigate the intricacies of the data maturity journey, using Scotland’s transport network as a lens to comprehend the broader challenges and opportunities.

During our exploration of the Scottish transport network’s data, we examined 14 datasets, including public transport access points, road network details, as well as train, ferry, bus routes and timetables. 

We found several issues with the available datasets that were out in the open, but not easy to operate. Some datasets were outdated, and others had unstructured data or were missing information entirely, which is not in line with ODI’s characterisation of good open data.

In fact, of the 14 datasets we explored only Network Rail’s open data was in good condition. It was easily accessible via an API and provided wiki documentation with clear instructions that were short and understandable.

The issues we encountered are real roadblocks to achieving data maturity and interoperability. They are also not unique to the Scottish transport network. So, if you recognise your organisation as we outline the top challenges below, it might be time to innovate your data infrastructure.

So, what is the effect of these data issues on residents in Scotland?

After digitising Scotland’s transport schedule data internally, it became evident that residents in rural Scotland face significant challenges when attempting certain public transport journeys during the work week, particularly if their departure time is not precisely timed.

This issue stems from the lack of accessible open data and established data practices, making it challenging to integrate information across different transport systems.

You can find the full list of journeys we investigated here. Below we explore a few key examples.

The discrepancies we encountered in Scotland's transport data directly impact the efficiency of public transport journeys, particularly in rural areas. The lack of seamless integration between various transport systems results in extended waiting times, making it evident that data interoperability is not just a theoretical concept but a tangible necessity for enhancing the lives of individuals relying on these services. 

To address these challenges and build a more efficient and connected transport system, a strategic approach to data engineering is imperative. Cleaning and organising datasets, ensuring they are updated regularly, and promoting standardised formats are crucial steps toward achieving true data maturity.

Of course, Scotland’s geography presents a natural challenge, especially for reaching remote areas, particularly those in the West of the country. However, rather than accepting the current status quo, we should strive to enhance the transportation system, leveraging modern technologies and practices to overcome these obstacles.

Contributors:

Charles Roadnight, Lead Data Engineer at Zühlke
Sherri Chuah, Professional Data Engineer at Zühlke
Tabitha Day, Professional Data Engineer at Zühlke
Neelesh Sonawane, Principal UX Consultant at Zühlke
Anna Ronco, Former Expert UX Designer at Zühlke

Acknowledgements:

Giuseppe Sollazzo, Head of Data Products and Services at the Department for Work and Pensions (former Head of Data at the Department for Transport)