Runtime Verification

Internal runtime verification feedback loop

Overview

camel-verify is the internal runtime verification skill that checks whether generated integrations actually work. Through a 3-phase feedback loop with error classification and targeted fixes, it runs the applicable build or startup smoke checks and integration tests, while reporting every skipped or failed check.

The output is runtime-verification evidence returned to /camel-execute, with a PASS, PARTIAL, FAIL, or NOT_RUN outcome. Verification failures do not block the execute completion summary. A chained pipeline still continues through Phase 4 static /camel-validate before it is complete.

When It Runs

/camel-execute dispatches camel-verify when generated code is ready for runtime checks. It handles:

  • Want to validate a generated integration works at runtime
  • Encounter build failures, runtime errors, or test failures
  • Want automated diagnosis and fixing of common issues

It is not exposed as a command stub. Use /camel-debug for ad-hoc troubleshooting outside an active pipeline run.

The Verification Loop

The verification process runs three phases in sequence. Maven compilation and Citrus testing each have a 15-attempt ceiling; the Camel Main startup smoke test has a 6-attempt ceiling. Repeated error classes can promote to re-planning before those limits. An environment probe runs before verification (as the first step of camel-execute) to catch dependency, service, and startup issues before code is generated.

Error Classification System

The source taxonomy uses specific error families rather than four generic uppercase buckets. Representative routing is:

Error familyFix target
Missing Camel or third-party dependency; version incompatibilitySelf-repair with runtime-aware dependency coordinates and BOM alignment
Route creation, unknown component, missing bean, or injection failurecamel-implement for the affected flow
Wrong endpoint option or YAML schema failurecamel-validate for diagnosis, then camel-implement for the correction
Expression, type-conversion, transformation, or assertion mismatchcamel-implement
Timeout, external-service, or container-startup failureSelf-repair first; route logic goes to camel-implement when indicated
Test syntax or incorrect test expectationscamel-test
MCP-confirmed unavailable component/pattern or a persistent error familyBounded re-plan of affected design-spec flow sections, followed by affected-task execution and re-verification
Missing build plugin, Quarkus augmentation failure, iteration limit, or unclassified errorEscalate with the raw error and attempted fix

Retry Budget

Maven compilation and Citrus testing each have a ceiling of 15 attempts; Camel Main startup smoke has a ceiling of 6. Promotion rules can end a local fix loop sooner.

Retry Strategy:

1. Classify the current error and route one targeted fix.
2. If the same error remains, short-circuit and evaluate promotion.
3. Trigger immediate Tier 1 re-planning when the MCP catalog confirms the required runtime feature does not exist.
4. Trigger Tier 2 re-planning after three failed fixes for the same error class.
5. Escalate unclassified errors or a phase that reaches its attempt limit.

Pipeline Re-entry

Resume /camel-execute after an approved manual correction so it can dispatch runtime verification again. If a previously working route breaks outside the pipeline, use /camel-debug; it preserves local state, diagnoses the root cause, applies a targeted fix, and recommends a recurrence guard.

Environment-in-the-Loop Concept

camel-verify is “environment-in-the-loop” verification: it doesn’t just check code. Citrus runs the integration and exercises its endpoints, while Testcontainers provides required external databases and brokers.

Why This Matters:

  • Catches real issues: Code might compile but fail at runtime
  • Validates integrations: Test endpoints are exercised and messages actually flow
  • Tests behavior: Not just unit tests, but integration tests with required databases or brokers and mocked external APIs
  • Prevents surprises: Find issues now, not in production
  • Self-contained: Where external databases or brokers are required and Docker is available, Testcontainers manage their lifecycle; external APIs use mocks – no manual Docker Compose setup needed

Contrast with traditional testing:

  • Unit tests: Mock everything (no environment)
  • Integration tests: Exercise the application with required databases or brokers managed by Testcontainers and external APIs mocked (environment-in-the-loop)

Camel-Kit uses Citrus YAML integration tests for verification. Tests use Testcontainers when required external databases or brokers and Docker are available, and mocks for external APIs. Camel-Kit generates Docker Compose as a local-development artifact only when the integration needs external services; verification does not consume that Compose file.

Graceful Degradation

If tools are unavailable, the AI adapts:

No Docker

Warning: Docker not available.

Running container-free and mock-only Citrus tests.
Skipping only tests that declare Testcontainers.

Note: Without Docker, integration tests cannot start required external databases or brokers.
Consider running on a system with Docker installed.

Proceeding to report phase with dependent checks recorded as skipped...

No Maven (Spring Boot or Quarkus)

Warning: neither ./mvnw nor system mvn is available. Skipping build verification phase.

Proceeding to test verification...

No Camel Test CLI

Warning: camel test command not available.

Skipping test verification phase.

Note: Without the Camel JBang test plugin, we cannot run integration tests.
After installing Camel JBang if needed, run: camel plugin add test

No Citrus Tests

Warning: No Citrus YAML tests found (*.it.yaml)

Skipping test verification phase.

Note: Without tests, we cannot verify integration behavior.
The execute orchestrator can dispatch `camel-test` to generate missing tests.

The AI continues with available tools, warning about limitations.

Summary

camel-verify collects runtime evidence through a 3-phase feedback loop:

  1. Build / Startup Smoke - Compile Spring Boot or Quarkus with the wrapper or system Maven; start and inspect Camel Main instead
  2. Test Verification - Run Citrus YAML tests via camel test run, using Testcontainers only for tests that declare required external services
  3. Report Generation - Summarize results and provide insights

Key Features:

  • Error Classification - Specific build, startup, runtime, and test error families with explicit fix targets
  • Targeted Fix Loops - Up to 15 compile/test attempts or 6 Camel Main startup attempts, with earlier promotion for persistent errors
  • Fix Routing - Errors routed to self-repair, camel-validatecamel-implement, camel-implement, camel-test, bounded re-plan, or escalation
  • Environment-in-the-Loop - Testcontainers manage required databases and brokers for the tests that declare them
  • Graceful Degradation - Reports unavailable tools and explicitly skips dependent checks
  • Internal Dispatch - Runs within /camel-execute; /camel-debug handles ad-hoc failures

The result is an explicit PASS, PARTIAL, FAIL, or NOT_RUN record rather than a silent claim that the integration works.

After runtime verification finishes, regardless of outcome, a chained pipeline continues from Execute to /camel-validate; standalone /camel-execute reports completion and stops.