5 challenges in utilizing maritime data
As a context broker in the maritime world, Teqplay has accumulated a deep well of knowledge on both the powerful potential of data, as well as the challenges of utilizing it in the maritime context.
In this article, we outline 5 major challenges that need to be overcome order to make the most use of maritime data.
Challenge in definitions
When working with data from multiple sources, one of the key challenges is dealing with varying data formats and definitions.
For instance, an ETA (estimated time of arrival) provided by one source might not have the same meaning or accuracy as an ETA from another. Additionally, update rates can differ significantly between sources, further complicating data integration.
However, the most difficult aspect often lies in the discrepancies between how these definitions are applied, making it essential to carefully interpret and standardize the data to ensure consistency.
Managing identifiers and data sensitivity
Another challenge involves handling identifiers. Each data source assigns its own unique identifier to the same piece of information. We gather data from various sources—such as agents, terminals, and port authorities—which results in the same information being labeled with different identifiers.
It’s crucial to ensure these identifiers are presented consistently to the appropriate parties. While much of the data we work with is open, some of it, like planning-related information from our customers, is sensitive. A key challenge is ensuring that this sensitive data is protected and not exposed to other customers or platform participants. Managing data sensitivity from the outset is critical to maintaining trust and security.
Managing noise in maritime data
One of the key aspects of handling maritime data is dealing with noise—irrelevant or unwanted information that can skew results and reduce accuracy.
Noise can take many forms. Irrelevant data, for instance, might include information about sailing ships when the focus is on cargo movements. Sensor data can also introduce noise, as some sensors occasionally produce incorrect measurements. Additionally, the use of open data sources carries the risk of receiving spoofed or falsified data, which needs to be identified and eliminated.
The process of filtering noise typically occurs early in the data collection phase. By detecting and removing unwanted data from the outset, the system ensures only reliable information is processed. Even later in the data pipeline, after calculations are complete, outliers may emerge. For example, if a vessel docks unexpectedly during its port stay, it could skew the turnaround time, polluting the results presented to customers. Detecting and filtering these outliers is crucial to delivering accurate and trustworthy data insights.
Manual data vs. AIS data
It is a common perception in the industry that data entered manually is more trustworthy than data from AIS or other technologies.
While operators have the discerning eyes and experience for entering important information, the risk of human error still cannot be neglected. For instance, when it is close to shift closing time, operators might sometimes start to guess the information and lose accuracy.
In such instances, technology proves to be incredibly useful thanks to its stability and tirelessness. Moreover, manually entered data can be difficult to process, and the use of technology and digitization will increase the efficiency of the process quite significantly.
The use of spreadsheets
CTO Richard van Klaveren asserted that spreadsheets are valuable as first steps toward information sharing and transparency, but they also have their fair shares of problems.
When the goal is continuous improvement with real-time updates, spreadsheets either need very dedicated people to keep it up-to-date, or there need to be an automated process on top.
This manual way of updating information often only gets people halfway into their operational processes. It is important to move away from overreliance on spreadsheets and start embedding real-time information into the work flow.
Hear more from maritime experts and industry thought leaders in our video podcast here.