AI is a broad label covering different methods and levels of capability. A learned fuel model, an object detector and a route optimiser solve different problems. Evidence that one works well does not validate the others, and no single performance number establishes that an integrated navigation system is suitable for every vessel, passage or operating condition.
Define the domain in which evidence applies
Evaluation should describe the vessel types, environmental conditions, sensors, geographical coverage and tasks represented in testing. A model developed for open-water passages may encounter different demands near a pilot station. A camera detector tested in clear daylight may behave differently in glare, precipitation or darkness. These differences are not minor footnotes if they affect the intended use.
Distribution shift occurs when operational data differ from the data used to develop or evaluate a model. Changes in hull condition, traffic patterns or sensor installation can all matter. A system may continue producing plausible outputs without recognising that its evidence base no longer fits. Detecting unfamiliar inputs is itself a capability that needs evaluation, not an automatic property of AI.
Read validation beyond headline accuracy
Average accuracy can conceal rare but consequential failures. Evaluation should distinguish false alarms, missed detections and errors in the situations that matter to the task. Test data should be meaningfully separate from development data. If near-identical voyages appear in both, a result may overstate how well the system generalises to new conditions.
Simulation allows controlled exploration of difficult scenarios, but simulated evidence depends on the realism and coverage of the simulator. Sea trials contribute operational observations while also having limits in duration and encountered conditions. Neither approach alone justifies unlimited claims. Credible assurance combines evidence, stated scope, change control and a clear account of unresolved limitations.
Separate availability from integrity
A display that remains online is not necessarily receiving trustworthy information. Sensor faults, incorrect configuration, corrupted updates or malicious interference can affect integrity while leaving the interface responsive. GNSS interference is one example of why a functioning position display should not automatically be treated as proof of a correct position.
Cyber resilience therefore includes understanding dependencies and failure behaviour. Which services require connectivity? What happens when a feed stops? Does a cached forecast remain visibly dated? Can a failed component contaminate other outputs? A degraded mode should be recognisable and compatible with vessel procedures rather than presenting stale information as an unchanged live picture.
Keep learning systems under change control
A software update can alter a recommendation even when the input data remain the same. Changes to feature processing, model weights or default objectives may affect behaviour. Operators need an appropriate way to identify versions, understand material changes and evaluate their consequences. “Continuously improving” is not a substitute for a controlled release and assurance process.
Post-deployment monitoring should also distinguish a genuine improvement from a change in the voyages being assessed. If a fleet begins operating on easier routes, aggregate performance may improve without any better modelling. Good evaluation records context and adverse outcomes, not only accepted recommendations or favourable fuel comparisons.
Understand the boundary between advice and authority
The master’s and bridge team’s responsibilities are not transferred by an attractive interface or confident recommendation. The applicable legal framework, safety management system, equipment requirements and standing orders remain relevant. The International Safety Management Code provides an important context for procedures, responsibilities and learning within company operations.
Likewise, the International Convention for the Safety of Life at Sea, or SOLAS, and the STCW framework address matters that cannot be reduced to a software score. A product’s description as “AI-powered” does not establish statutory acceptance, equipment approval or suitability for a particular function. Specific approval claims must be checked against their precise scope and authoritative documentation.
Ask for a usable account of failure
A useful procurement or review discussion asks what the system will not do, how users will know it is struggling and what evidence supports its claimed boundaries. It should include foreseeable misuse, training needs, maintainability and the possibility that users become over-reliant on a normally helpful output.
The right conclusion is not that decision support has no value. Its value is strongest when the task is bounded, the evidence is inspectable and the team can recognise when to question it. A system that communicates uncertainty clearly can support better judgment than one that offers a reassuring answer to every question.
Primary context: IMO: International Safety Management Code, SOLAS and IMO maritime cyber risk resources.