feat(infospace,llm): agent ergonomics — entity lookup, model fallback, better errors

- `markitect infospace entity <name>`: single-entity lookup tolerating
  hyphens/underscores/case, with substring matching, ambiguity listing,
  and near-match hints. Prints slug, source path, domain, chapter, word
  count, VSM system, overall score, evaluator, and evaluation file path.
- `markitect infospace evaluate --model-fallback <model>`: if any
  entities fail with a rate-limit error, retry just those with a fresh
  adapter on the fallback model (different free-tier models have
  separate quota buckets).
- `markitect llm-check`: advisory when `OPENROUTER_API_KEY` is set but
  not used by the resolved provider; targeted hint when OpenRouter
  returns 401 (almost always a stale env key).
- `build_state`: raises `TypeError` with actionable message if passed a
  path instead of an `InfospaceConfig` — prior failure mode was a
  confusing `AttributeError` deep in the stack.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-04-22 01:07:25 +02:00
parent c0615c2d50
commit d44a4cd3df
3 changed files with 172 additions and 2 deletions

View File

@@ -240,8 +240,14 @@ def llm_catalog(output_format):
)
def llm_check(provider, model):
"""Send a minimal prompt to verify a provider is reachable and responding."""
import os
from markitect.llm import create_adapter
from markitect.llm.exceptions import LLMConfigurationError, LLMError
from markitect.llm.exceptions import (
LLMAPIError,
LLMConfigurationError,
LLMError,
)
from markitect.prompts.execution.models import RunConfig
resolved = resolve_llm(cli_provider=provider, cli_model=model)
@@ -252,6 +258,17 @@ def llm_check(provider, model):
f" model from: {resolved.model_source}"
)
# Advisory: OPENROUTER_API_KEY is set but this call won't use it. Common
# source of "works for me, fails for agents" when the env var holds a
# stale key that overrides a clean config entry.
if resolved.provider != "openrouter" and os.environ.get("OPENROUTER_API_KEY"):
click.echo(
" note: OPENROUTER_API_KEY is set but won't be used for this "
"provider. If OpenRouter calls fail elsewhere with 401, the env "
"var may be stale — unset or update it.",
err=True,
)
try:
adapter = create_adapter(
provider=resolved.provider,
@@ -273,6 +290,19 @@ def llm_check(provider, model):
except LLMError as exc:
elapsed = time.monotonic() - start
click.echo(f"ERROR \u2014 LLM error after {elapsed:.1f}s: {exc}", err=True)
# Targeted hint: 401 on openrouter almost always means a stale key.
if (
resolved.provider == "openrouter"
and isinstance(exc, LLMAPIError)
and exc.status_code == 401
):
click.echo(
" hint: OpenRouter returned 401 (unauthorized). Check whether "
"OPENROUTER_API_KEY is stale (`unset OPENROUTER_API_KEY` to "
"fall back to the key in ~/.config/markitect/config.toml, or "
"update the env var).",
err=True,
)
sys.exit(1)
except Exception as exc:
elapsed = time.monotonic() - start