Files
inter-hub/workplans/IHUB-WP-0012-ihf-phase11-advanced-ai-federation.md
Bernd Worsch 133dae3d23
Some checks failed
Test / test (push) Has been cancelled
feat(WP-0012): IHF Phase 11 — Advanced AI Federation
- Schema: AgentRegistration, ModelRoutingPolicy, AgentDelegation,
  CollectiveProposal, CollectiveProposalContribution, AiGovernancePolicy,
  AgentPerformanceRecord + ALTER TABLE agent_proposals
  (migration 1744156800; CHECK constraints on trust_level, status,
  consensus_status — GAAF compliant)

- Bridge: scripts/llm_bridge.py (llm-connect subprocess seam) +
  Application/Helper/AgentBridge.hs (callBridge, callAgent,
  checkGovernancePolicy, jsonArrayTexts)

- Routing: Application/Helper/ModelRouter.hs (resolveAgent,
  resolveAllAgents) + ModelRoutingPolicies CRUD

- Registry: AgentRegistrations CRUD (Index/Show/New/Edit/Performance),
  DeactivateAgentAction, ComputeAgentPerformanceAction

- Delegation: AgentDelegations controller + views, DelegateSubtaskAction
  with token budget enforcement at bridge call time

- Collective: CollectiveProposals controller + views,
  CreateCollectiveProposalAction (fan-out → synthesis → consensus detection)

- Governance: AiGovernancePolicies CRUD + ToggleAiGovernancePolicyAction;
  checkGovernancePolicy enforced at all 4 Phase 5 invocation points

- Phase 5 wiring: replaced callClaudeApi in Widgets, DecisionRecords,
  RequirementCandidates with resolveAgent + callAgent + token tracking

- llm-connect feature requests: ~/llm-connect/FEATURE_REQUESTS.md
  (FR-1 HTTP serve, FR-2 RoutingPolicy, FR-3 async, FR-4 BudgetTracker)

- GAAF scorecard: 3.61 (up from 3.56); Functional 3.4→3.6, Extensions 3.8→3.9

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 20:57:17 +00:00

23 KiB
Raw Blame History

id, type, title, domain, repo, status, owner, topic_slug, created, updated, state_hub_sync, state_hub_workstream_id
id type title domain repo status owner topic_slug created updated state_hub_sync state_hub_workstream_id
IHUB-WP-0012 workplan IHF Phase 11 — Advanced AI Federation inter_hub inter-hub done custodian inter_hub 2026-04-01 2026-04-01 done b6d8058b-b786-4cb8-81d4-19067b2684f8

IHF Phase 11 — Advanced AI Federation

Goal

Move AI assistance from single-model, single-agent operation (Phase 5) to a federated, multi-model, multi-agent architecture. Phase 11 introduces agent specialisation, model routing, inter-agent delegation, collective proposals, and AI governance policies — while keeping humans firmly in control of final decisions.

Background

Phases 110 and IHUB-WP-0012 entry gates are satisfied:

  • Phase 5 AgentProposal, AgentReviewRecord, ConfidenceAnnotation operational ✓
  • HubCapabilityManifest and type registries in place (governance policies attach per-hub) ✓
  • /api/v2/ surface live (agent registry will expose an API endpoint) ✓
  • GAAF scorecard at 3.56 (Strong) ✓
  • llm-connect available at ~/llm-connect as a Python subprocess library ✓

Reference: specs/InteractionHubFrameworkSpecification_v0.2.md §Phase 11.

llm-connect Integration Architecture

llm-connect (~/llm-connect) is a Python library with pluggable LLM adapters for OpenRouter, Gemini, OpenAI, and Claude Code CLI. It is not a Haskell library; the integration is via a thin Python subprocess bridge.

Bridge pattern

IHP controller action
  → AgentBridge.callLlmBridge (Haskell)
      → System.Process: python3 scripts/llm_bridge.py
          → llm_connect.create_adapter(provider, model, api_key, system_prompt)
          → adapter.execute_prompt(prompt, RunConfig(...))
          → JSON stdout: {content, model, tokens_in, tokens_out, finish_reason}
  ← AgentResponse | AgentError

scripts/llm_bridge.py is the sole integration seam. Each AgentRegistration record stores the provider, model name, and system prompt that are passed to the bridge. The API key is resolved from environment variables at bridge call time (following llm-connect's key resolution order: env var → key file → error).

Supported providers via llm-connect

Provider llm-connect adapter Notes
openrouter OpenRouterAdapter Multi-model routing; recommended for production
gemini GeminiAdapter Free tier available; Google AI Studio key
openai OpenAIAdapter GPT-4o and family
claude-code ClaudeCodeAdapter Shells out to claude --print CLI

Known llm-connect limitations (feature requests raised in T10)

Gap Impact on Phase 11 Feature request
No HTTP serve mode Process spawn overhead on every agent call FR-1: --serve mode (local JSON-RPC HTTP server)
No routing policy API ModelRoutingPolicy implemented in Haskell only FR-2: RoutingPolicy class for declarative provider/model selection
Synchronous only Collective proposals invoke agents sequentially FR-3: async_execute_prompt() for concurrent execution
No token budget ledger Delegation budget is checked by Haskell, not enforced by bridge FR-4: BudgetTracker that spans a delegation chain

These limitations do not block Phase 11 — they affect performance and architectural elegance. The feature requests are filed against llm-connect as a separate deliverable (T10).

GAAF Architectural Constraints

  1. agent_registrations.trust_level must carry a CHECK constraint (advisory, elevated, autonomous) — no bare TEXT discriminator.
  2. agent_delegations.status must carry a CHECK constraint (pending, completed, failed, cancelled).
  3. collective_proposals.consensus_status must carry a CHECK constraint (pending, consensus, divergent).
  4. ai_governance_policies.allowed_actions is a JSONB array — validated at the controller layer (each element must be one of: read, propose, delegate, auto_apply).
  5. Core tables remain frozen — agent_proposals gains agent_registration_id and token columns via ALTER TABLE, with a corresponding update to /contracts/core/.
  6. Append-only invariant on interaction_events and outcome_signals is permanent and unaffected by this phase.

Data Artifacts Introduced

AgentRegistration, ModelRoutingPolicy, AgentDelegation, CollectiveProposal, CollectiveProposalContribution, AiGovernancePolicy, AgentPerformanceRecord

Schema additions

-- agent_registrations: named, versioned AI agents backed by llm-connect providers
CREATE TABLE agent_registrations (
    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
    hub_id UUID NOT NULL REFERENCES hubs(id),
    name TEXT NOT NULL,
    slug TEXT NOT NULL UNIQUE,
    description TEXT,
    provider TEXT NOT NULL,        -- openrouter | gemini | openai | claude-code
    model_name TEXT NOT NULL,
    trust_level TEXT NOT NULL DEFAULT 'advisory',
    capabilities JSONB NOT NULL DEFAULT '[]',
    system_prompt TEXT,
    is_active BOOLEAN NOT NULL DEFAULT TRUE,
    version INTEGER NOT NULL DEFAULT 1,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
    CHECK (trust_level IN ('advisory', 'elevated', 'autonomous'))
);

-- model_routing_policies: task_type → agent selection rules per hub
CREATE TABLE model_routing_policies (
    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
    hub_id UUID NOT NULL REFERENCES hubs(id),
    task_type TEXT NOT NULL,
    agent_registration_id UUID NOT NULL REFERENCES agent_registrations(id),
    priority INTEGER NOT NULL DEFAULT 0,
    is_active BOOLEAN NOT NULL DEFAULT TRUE,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
    UNIQUE (hub_id, task_type, priority)
);

-- agent_delegations: auditable inter-agent subtask delegation records
CREATE TABLE agent_delegations (
    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
    delegating_agent_id UUID NOT NULL REFERENCES agent_registrations(id),
    receiving_agent_id UUID NOT NULL REFERENCES agent_registrations(id),
    parent_proposal_id UUID REFERENCES agent_proposals(id),
    scope TEXT NOT NULL,
    token_budget INTEGER NOT NULL DEFAULT 1000,
    tokens_used INTEGER,
    status TEXT NOT NULL DEFAULT 'pending',
    result JSONB,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
    completed_at TIMESTAMP WITH TIME ZONE,
    CHECK (status IN ('pending', 'completed', 'failed', 'cancelled'))
);

-- collective_proposals: multi-agent proposals with attribution
CREATE TABLE collective_proposals (
    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
    title TEXT NOT NULL,
    summary TEXT,
    task_type TEXT NOT NULL,
    consensus_status TEXT NOT NULL DEFAULT 'pending',
    final_content JSONB,
    source_widget_id UUID REFERENCES widgets(id),
    source_candidate_id UUID REFERENCES requirement_candidates(id),
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
    CHECK (consensus_status IN ('pending', 'consensus', 'divergent'))
);

-- collective_proposal_contributions: per-agent contribution records
CREATE TABLE collective_proposal_contributions (
    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
    collective_proposal_id UUID NOT NULL REFERENCES collective_proposals(id),
    agent_registration_id UUID NOT NULL REFERENCES agent_registrations(id),
    content JSONB NOT NULL,
    tokens_in INTEGER,
    tokens_out INTEGER,
    model_used TEXT,
    contributed_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL
);

-- ai_governance_policies: per-hub rules controlling agent scope
CREATE TABLE ai_governance_policies (
    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
    hub_id UUID NOT NULL REFERENCES hubs(id),
    agent_registration_id UUID NOT NULL REFERENCES agent_registrations(id),
    artifact_type TEXT NOT NULL,
    allowed_actions JSONB NOT NULL DEFAULT '["read"]',
    is_active BOOLEAN NOT NULL DEFAULT TRUE,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL
);

-- agent_performance_records: periodic snapshots of per-agent metrics
CREATE TABLE agent_performance_records (
    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
    agent_registration_id UUID NOT NULL REFERENCES agent_registrations(id),
    hub_id UUID NOT NULL REFERENCES hubs(id),
    period_start TIMESTAMP WITH TIME ZONE NOT NULL,
    period_end TIMESTAMP WITH TIME ZONE NOT NULL,
    proposals_generated INTEGER NOT NULL DEFAULT 0,
    proposals_accepted INTEGER NOT NULL DEFAULT 0,
    proposals_rejected INTEGER NOT NULL DEFAULT 0,
    proposals_revised INTEGER NOT NULL DEFAULT 0,
    mean_confidence DOUBLE PRECISION,
    calibration_score DOUBLE PRECISION,
    computed_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL
);

-- Extend agent_proposals with agent_registration_id and token tracking
ALTER TABLE agent_proposals
    ADD COLUMN agent_registration_id UUID REFERENCES agent_registrations(id),
    ADD COLUMN tokens_in INTEGER,
    ADD COLUMN tokens_out INTEGER;

Tasks

T01 — Schema: all Phase 11 tables + migration

id: IHUB-WP-0012-T01
status: done
priority: high
state_hub_task_id: "d86574f0-f0a4-4217-b4de-d020442de7e4"

Add all Phase 11 tables to Application/Schema.sql and write migration Application/Migration/<timestamp>-ihf-phase11-ai-federation.sql.

Includes the ALTER TABLE on agent_proposals to add agent_registration_id, tokens_in, tokens_out. Run migrate after writing.


T02 — llm-connect bridge: scripts/llm_bridge.py + Application/Helper/AgentBridge.hs

id: IHUB-WP-0012-T02
status: done
priority: high
state_hub_task_id: "404d1a89-aae2-49ea-a565-90261001a633"

scripts/llm_bridge.py — thin wrapper around llm-connect:

#!/usr/bin/env python3
# Usage: echo '{"provider":"openrouter","model":"...","prompt":"..."}' | python3 scripts/llm_bridge.py
import sys, json, os
sys.path.insert(0, os.path.expanduser("~/llm-connect"))
from llm_connect import create_adapter, RunConfig
from llm_connect.exceptions import LLMError

def main():
    req = json.load(sys.stdin)
    try:
        adapter = create_adapter(
            provider=req.get("provider", "openrouter"),
            model=req.get("model"),
            api_key=req.get("api_key"),
            system_prompt=req.get("systemPrompt"),
        )
        config = RunConfig(
            model_name=req.get("model", ""),
            temperature=req.get("temperature", 0.7),
            max_tokens=req.get("maxTokens", 2000),
        )
        resp = adapter.execute_prompt(req["prompt"], config)
        print(json.dumps({
            "content": resp.content,
            "model": resp.model,
            "tokensIn": resp.usage.get("prompt_tokens", 0),
            "tokensOut": resp.usage.get("completion_tokens", 0),
            "finishReason": resp.finish_reason,
        }))
    except LLMError as e:
        json.dump({"error": str(e), "errorType": type(e).__name__}, sys.stdout)
        sys.exit(1)

if __name__ == "__main__":
    main()

Application/Helper/AgentBridge.hs — Haskell wrapper:

module Application.Helper.AgentBridge where

import IHP.Prelude
import Data.Aeson (object, (.=), encode, decode)
import qualified Data.ByteString.Lazy as LBS
import System.Process (readProcessWithExitCode)
import System.Exit (ExitCode(..))
import Generated.Types

data BridgeRequest = BridgeRequest
    { provider    :: !Text
    , model       :: !Text
    , systemPrompt :: !(Maybe Text)
    , prompt      :: !Text
    , maxTokens   :: !Int
    , temperature :: !Double
    }

data BridgeResponse = BridgeResponse
    { content      :: !Text
    , modelUsed    :: !Text
    , tokensIn     :: !Int
    , tokensOut    :: !Int
    , finishReason :: !Text
    } deriving (Show)

data BridgeError = BridgeError
    { errorMessage :: !Text
    , errorType    :: !Text
    } deriving (Show)

callBridge :: BridgeRequest -> IO (Either BridgeError BridgeResponse)
callBridge req = do
    let input = cs . LBS.toStrict . encode $ object
            [ "provider"     .= req.provider
            , "model"        .= req.model
            , "systemPrompt" .= req.systemPrompt
            , "prompt"       .= req.prompt
            , "maxTokens"    .= req.maxTokens
            , "temperature"  .= req.temperature
            ]
    (exitCode, stdout, stderr) <-
        readProcessWithExitCode "python3" ["scripts/llm_bridge.py"] input
    let outBytes = LBS.fromStrict (cs stdout)
    case exitCode of
        ExitSuccess -> case decode outBytes of
            Just v  -> pure (Right v)
            Nothing -> pure (Left (BridgeError "Unparseable bridge output" "ParseError"))
        ExitFailure _ -> case decode outBytes of
            Just v  -> pure (Left v)
            Nothing -> pure (Left (BridgeError (cs stderr) "BridgeError"))

-- | Call bridge using an AgentRegistration record.
callAgent :: AgentRegistration -> Text -> IO (Either BridgeError BridgeResponse)
callAgent agent prompt =
    callBridge BridgeRequest
        { provider     = agent.provider
        , model        = agent.modelName
        , systemPrompt = agent.systemPrompt
        , prompt
        , maxTokens    = 2000
        , temperature  = 0.7
        }

T03 — Agent Registry: AgentRegistration CRUD + capability/trust UI

id: IHUB-WP-0012-T03
status: done
priority: high
state_hub_task_id: "3ae5b3b1-e644-444d-ae72-8eef07318c49"

Controller: Web/Controller/AgentRegistrations.hs

Actions:

  • AgentRegistrationsAction — list all agents with hub name, trust level, provider, active/inactive
  • ShowAgentRegistrationAction { agentRegistrationId } — detail with routing policies, recent proposals, performance record
  • NewAgentRegistrationAction / CreateAgentRegistrationAction — form with hub selector, provider dropdown (openrouter / gemini / openai / claude-code), model name, trust level selector, capabilities multi-tag input, system prompt textarea
  • EditAgentRegistrationAction / UpdateAgentRegistrationAction
  • DeactivateAgentAction { agentRegistrationId } — set is_active = False

Provider dropdown options map to the four llm-connect adapters.

Views: Web/View/AgentRegistrations/{Index,Show,New,Edit}.hs

Add nav link ("Agents") next to existing "Agent" proposals link.


T04 — Model routing: ModelRoutingPolicy CRUD + Application/Helper/ModelRouter.hs

id: IHUB-WP-0012-T04
status: done
priority: high
state_hub_task_id: "20bcff74-1923-4c11-95ef-2b37a7a10dd8"

Controller: Web/Controller/ModelRoutingPolicies.hs

Actions:

  • ModelRoutingPoliciesAction — list by hub, grouped by task_type
  • NewModelRoutingPolicyAction / CreateModelRoutingPolicyAction — hub, task_type selector (requirement_draft / triage / synthesis / policy_check / implementation), agent selector, priority
  • DeleteModelRoutingPolicyAction { modelRoutingPolicyId }

Application/Helper/ModelRouter.hs:

module Application.Helper.ModelRouter where

import IHP.Prelude
import IHP.ControllerPrelude
import Generated.Types

-- | Resolve the AgentRegistration to use for a hub + task type.
-- Selects the highest-priority active policy for the hub.
-- Returns Nothing if no policy matches (caller should fall back or error).
resolveAgent ::
    (?modelContext :: ModelContext) =>
    Id Hub -> Text -> IO (Maybe AgentRegistration)
resolveAgent hubId taskType = do
    mPolicy <- sqlQuery
        "SELECT mrp.agent_registration_id \
        \ FROM model_routing_policies mrp \
        \ WHERE mrp.hub_id = ? AND mrp.task_type = ? AND mrp.is_active = TRUE \
        \ ORDER BY mrp.priority DESC LIMIT 1"
        (hubId, taskType)
    case mPolicy of
        [Only agentId] -> fetchOneOrNothing agentId
        _              -> pure Nothing

T05 — Agent invocation: route + invoke via bridge, replace Phase 5 hard-coded calls

id: IHUB-WP-0012-T05
status: done
priority: high
state_hub_task_id: "45fec21a-47d3-4381-99bc-88dcf0f117cb"

Update the four existing agent invocation points to use the routing + bridge:

  1. SummarizeClusterAction in Web/Controller/Widgets.hs
  2. DraftRequirementAction in Web/Controller/Widgets.hs
  3. ProposeImplementationAction in Web/Controller/DecisionRecords.hs
  4. DetectDuplicatesAction in Web/Controller/RequirementCandidates.hs

Pattern for each:

mAgent <- resolveAgent hubId "requirement_draft"
case mAgent of
    Nothing    -> setErrorMessage "No routing policy for this task type"
    Just agent -> do
        -- Check AI governance policy before invocation
        allowed <- checkGovernancePolicy hubId agent.id "requirement_candidate"
        if not allowed
            then do
                newRecord @AgentProposal
                    |> set #status "blocked_by_policy"
                    |> set #agentRegistrationId (Just agent.id)
                    |> createRecord
                setErrorMessage "Blocked by AI governance policy"
            else do
                result <- callAgent agent prompt
                case result of
                    Left err -> setErrorMessage err.errorMessage
                    Right resp -> do
                        newRecord @AgentProposal
                            |> set #content resp.content
                            |> set #agentRegistrationId (Just agent.id)
                            |> set #tokensIn (Just resp.tokensIn)
                            |> set #tokensOut (Just resp.tokensOut)
                            |> createRecord

checkGovernancePolicy defined in Application/Helper/AgentBridge.hs: queries ai_governance_policies for the hub + agent + artifact_type combination.


T06 — Inter-agent delegation: AgentDelegation records + bounded sub-task invocation

id: IHUB-WP-0012-T06
status: done
priority: medium
state_hub_task_id: "9fe1c284-ecb7-4777-98f3-253224df704c"

Controller: Web/Controller/AgentDelegations.hs

Actions:

  • AgentDelegationsAction — list delegations for a proposal
  • ShowAgentDelegationAction { agentDelegationId } — detail with result
  • DelegateSubtaskAction { agentProposalId } — form: pick receiving agent, scope description, token budget; on submit: create AgentDelegation record, call bridge with maxTokens = delegation.tokenBudget, store result + tokensUsed, set status = completed | failed

Delegation tree on proposal Show page: recursive query over parent_proposal_id to render the full provenance chain.


T07 — Collective proposals: multi-agent attribution + consensus detection

id: IHUB-WP-0012-T07
status: done
priority: medium
state_hub_task_id: "37782cec-2d42-44b8-8bda-2049a7bf4898"

Controller: Web/Controller/CollectiveProposals.hs

Actions:

  • CollectiveProposalsAction — list by hub
  • ShowCollectiveProposalAction { collectiveProposalId } — side-by-side agent contributions + consensus/divergent status + human review buttons
  • CreateCollectiveProposalAction — invokes all is_active agents for the task_type in model_routing_policies priority order; creates a CollectiveProposalContribution per agent; then runs a synthesis step (invoke the highest-priority agent with all contributions as context) to produce final_content

Consensus detection:

detectConsensus :: [CollectiveProposalContribution] -> Text
detectConsensus contribs =
    -- Simple heuristic: if all contributions contain the same top-level
    -- recommendation key, mark as consensus; otherwise divergent
    if allSameRecommendation contribs then "consensus" else "divergent"

Views: Web/View/CollectiveProposals/{Index,Show}.hs


T08 — AI governance policies: AiGovernancePolicy CRUD + enforcement at invocation

id: IHUB-WP-0012-T08
status: done
priority: medium
state_hub_task_id: "50d05787-9629-4f94-ac55-50a6e417f730"

Controller: Web/Controller/AiGovernancePolicies.hs

Actions:

  • AiGovernancePoliciesAction — list by hub
  • NewAiGovernancePolicyAction / CreateAiGovernancePolicyAction — form: hub, agent, artifact_type, allowed_actions checkboxes (read / propose / delegate / auto_apply)
  • ToggleAiGovernancePolicyAction { aiGovernancePolicyId } — flip is_active

checkGovernancePolicy in Application/Helper/AgentBridge.hs:

checkGovernancePolicy ::
    (?modelContext :: ModelContext) =>
    Id Hub -> Id AgentRegistration -> Text -> IO Bool
checkGovernancePolicy hubId agentId artifactType = do
    mPolicy <- query @AiGovernancePolicy
        |> filterWhere (#hubId, hubId)
        |> filterWhere (#agentRegistrationId, agentId)
        |> filterWhere (#artifactType, artifactType)
        |> filterWhere (#isActive, True)
        |> fetchOneOrNothing
    case mPolicy of
        Nothing -> pure True  -- no policy = permit by default
        Just p  -> pure ("propose" `elem` jsonArrayTexts p.allowedActions)

Governance compliance panel added to existing GovernanceDashboardAction: count of blocked invocations in the last 7 days per hub.


T09 — Agent performance dashboard

id: IHUB-WP-0012-T09
status: done
priority: medium
state_hub_task_id: "6ef63612-f913-4a13-b683-f44929cb1b2d"

Action added to AgentRegistrationsController: ComputeAgentPerformanceAction { agentRegistrationId } — computes a 30-day snapshot and writes AgentPerformanceRecord:

SELECT
    COUNT(*) FILTER (WHERE ap.status = 'accepted') AS accepted,
    COUNT(*) FILTER (WHERE ap.status = 'rejected') AS rejected,
    COUNT(*) FILTER (WHERE ap.status NOT IN ('accepted','rejected')) AS other,
    AVG(ca.score) AS mean_confidence
FROM agent_proposals ap
LEFT JOIN confidence_annotations ca ON ca.proposal_id = ap.id
WHERE ap.agent_registration_id = ?
  AND ap.created_at >= NOW() - INTERVAL '30 days'

Calibration score = Pearson correlation between ca.score and the binary accept/reject outcome.

View: Web/View/AgentRegistrations/Performance.hs — panel per agent with progress bars for acceptance/rejection ratio, mean confidence gauge, calibration indicator.


T10 — llm-connect feature requests + scorecard + CLAUDE.md

id: IHUB-WP-0012-T10
status: done
priority: medium
state_hub_task_id: "41a64c6c-0ce8-4c63-9175-6b8d28143f81"

File feature requests against ~/llm-connect (as GitHub issues or local FEATURE_REQUESTS.md in that repo):

# Title Why needed Proposed API
FR-1 HTTP/JSON-RPC serve mode Avoid process spawn per call in production python -m llm_connect.server --port 9999; IHP calls POST localhost:9999/execute
FR-2 RoutingPolicy class Declarative provider/model selection beyond model_name in RunConfig RoutingPolicy(rules=[{task_type: "triage", prefer: [{provider: "openrouter", model: "claude-haiku-4-5"}], max_cost_per_1k: 0.5}])
FR-3 async_execute_prompt() Collective proposals need parallel agent invocation Standard asyncio coroutine interface matching execute_prompt signature
FR-4 BudgetTracker for delegation chains Token budget enforcement across nested delegations BudgetTracker(total=4000) passed through RunConfig; raises LLMBudgetExceededError

ARCHITECTURE-LAYERS.md scorecard update:

  • Functional: 3.4 → 3.6 (multi-agent federation formalises AI collaboration)
  • Extensions: 3.8 → 3.9 (agent registry exposes an API surface)
  • Target overall: ≥3.7

CLAUDE.md: move IHUB-WP-0012 to completed; set active → IHUB-WP-0013 (Phase 12 — Platform Memory and Continuous Learning).

Commit all changes and mark workplan status: done.