Key Takeaways
- Shippy’s design separates soul, skills, and config for auditable and versioned agent deployment.
- Deterministic CLI tools reduce errors from nondeterministic LLM behavior.
- Agent-specific evaluation on live data catches failures that static benchmarks miss.
How Ai2 Engineered an Agent for Uncompromising Maritime Security
On July 15, 2026, the Skylight team at Ai2 published the detailed architecture behind Shippy, an AI agent purpose-built for real-time maritime domain awareness. The system, already serving hundreds of government agencies and NGOs, tackles the core challenge of agent reliability in high-stakes operations where a wrong answer can misdirect patrol vessels and endanger personnel.
Table of Contents
Agent Anatomy: Soul, Skills, and Config
Shippy’s architecture decomposes an agent into three distinct components. The ‘soul’ is the system prompt that defines behavioral boundaries and persona. ‘Skills’ are plain markdown files following the agent-skills spec used by Codex and Claude Code, each encoding a specific workflow such as querying the Skylight API or interpreting vessel tracks. Together, soul and skills are baked into a versioned Docker image, while ‘config’ covers runtime settings like the harness (OpenClaw) and LLM (Claude Opus 4.6). This separation allows model swaps and skill updates without rebuilding the core image.
Critical to the design is that boundaries are auditable. The soul explicitly refuses to make legal determinations or speculate beyond data, keeping the agent’s role as a decision-support tool rather than an autonomous authority.
Deterministic Tools for Nondeterministic Agents
To combat the subtle bugs that arise from letting an LLM craft complex API calls directly, Shippy uses a purpose-built CLI that wraps the Skylight API. The CLI collapses dozens of input types into typed filter flags, handles authentication and pagination, and always outputs to a local JSON file. This deterministic layer significantly reduces errors that slipped through in early prototypes, such as malformed pagination or geometry encoding mistakes.
Agent skills reference these CLI commands, creating a layered system where each layer narrows the failure surface. The API, CLI, and skills can each be tested independently, vastly improving reliability over monolithic implementations.
Sandboxed Hosting with Mothership
Every user interaction with Shippy occurs inside an ephemeral, isolated Kubernetes session managed by a platform called Mothership. When a conversation opens, pods are provisioned that package the agent runtime, skills, and CLI, with the user’s JWT injected to scope data access. This architecture ensures that conversation history and analysis files remain private, and network access is restricted to only required services. Mothership was built as a general-purpose agent hosting platform, already slated for use with Ai2’s wildlife conservation platform EarthRanger.
Evaluating an Agent, Not a Model
Traditional benchmarks rank static QA performance, but Shippy required a new evaluation paradigm that scores the entire agent system against live data. Using the Harbor framework, subject-matter experts create scenarios with weighted rubrics, and an LLM judge grades each criterion with written reasoning. A scenario like ‘show fishing activity in Panama’s EEZ’ weights data accuracy most heavily, while guardrail tasks test refusal of inappropriate requests. The suite runs in parallel against versioned builds, catching regressions before deployment.
In latest runs, Shippy consistently scores high on data retrieval and guardrail tasks, but failures in overstepping into tactical recommendations, geometry simplification, and hallucinated CLI commands directly inform skill improvements.
Why Shippy’s Design Principles Signal a Shift in Agent Engineering
As explained in the official Ai2 blog post, Shippy’s architecture arrives amid a flurry of research rethinking how agents should be built and deployed. Recent work from IBM Research, titled ‘Model Routing Is Not Classification‘, exposes hidden cost and latency traps in naive routing strategies — a problem Shippy plans to address with dynamic model routing that sends simple lookups to smaller models. Similarly, NVIDIA NeMo’s ‘Agent-Led RL Research’ automates end-to-end agent experiments, reflecting the industry’s push toward treating agents as systems that require continuous, automated improvement.
Shippy’s approach of separating deterministic tooling from the nondeterministic LLM core is echoed in emerging best practices. The use of the agent-skills spec, shared by coding agents, suggests a convergence around standardized skill formats. As enterprises deploy agents in regulated domains, Shippy’s auditable soul and versioned builds provide a template for compliance and safety.
The real innovation, however, lies in evaluating the whole agent stack on live data. Most organizations still benchmark models in static environments, but Shippy’s live eval suite, using Harbor and subject-matter expert rubrics, sets a new bar for production readiness. This technique directly addresses the ‘last mile’ problem of agent reliability.
The Road Ahead for Reliable AI Agents
Shippy is already in early adopter access, and the Skylight team is extending its capabilities with agent-driven UI control, model routing, and cross-thread memory. More importantly, Mothership is being generalized to host agents across domains, starting with wildlife conservation and Earth observation. The lessons from maritime — where reliability is non-negotiable — are directly transferable to any field where AI agents must operate under real-world constraints.
Staying ahead in the rapidly shifting landscape of AI requires precision. To future-proof your digital strategy and scale effortlessly, you need a foundation built on precision. Optimize your site with advanced speed engineering, secure your infrastructure in high-performance hosting environments, and streamline your entire workflow through autonomous AI pipelines. If you are ready to elevate your systems, Connect with Andres at Andres SEO Expert to build your ultimate architecture.
Frequently Asked Questions
What is Shippy and who developed it?
Shippy is an AI agent built by the Skylight team at Ai2 for real-time maritime domain awareness. It serves government agencies and NGOs, focusing on agent reliability in high-stakes operations.
How does Shippy’s architecture decompose an agent?
Shippy decomposes an agent into three components: the soul (system prompt defining behavior), skills (markdown files encoding workflows), and config (runtime settings like harness and LLM). Soul and skills are versioned in a Docker image, allowing model swaps without rebuilding.
What is the purpose of the deterministic CLI in Shippy?
The CLI wraps the Skylight API, collapsing inputs into typed filter flags, handling authentication and pagination, and outputting to a local JSON file. This reduces errors from LLM-generated API calls, creating a layered system where each layer narrows failure surfaces.
How does Shippy ensure data privacy and isolation for each user?
Every user interaction occurs in an ephemeral, isolated Kubernetes session managed by Mothership. Pods package the agent runtime, skills, and CLI with the user’s JWT to scope data access, ensuring conversation history remains private and network access is restricted.
How is Shippy evaluated differently from traditional AI models?
Shippy uses the Harbor framework where subject-matter experts create scenarios with weighted rubrics, and an LLM judge grades each criterion. This evaluates the entire agent system against live data, catching regressions before deployment, unlike static QA benchmarks.
What future capabilities are planned for Shippy and Mothership?
The Skylight team is extending Shippy with agent-driven UI control, model routing, and cross-thread memory. Mothership is being generalized to host agents across domains, starting with wildlife conservation via EarthRanger, transferring reliability lessons from maritime.
