Spanlens

What It Does

Spanlens is an open-source (MIT) LLM observability and monitoring platform that provides complete visibility into your AI application's behavior. By replacing just one line of code in your existing OpenAI, Anthropic, or Gemini SDK calls, Spanlens automatically logs every request with full cost, latency, token usage, and request/response bodies—with under 3ms p99 overhead.

Who It's For & The Problem It Solves

Spanlens is built for engineering teams deploying LLM-powered applications who need to understand cost drivers, debug multi-step agent workflows, catch quality regressions, and prevent budget overruns before the bill arrives. It solves the "black box" problem: knowing what your LLM calls are doing, why costs spiked, and whether your latest prompt version actually improved quality.

Key Features & Capabilities

• Request Logging: Every call captured with model, tokens, cost, latency, and full request/response body. Filter, group, and export.
• Cost Tracking: Per-request breakdown, daily rollups, budget alerts, and projected monthly spend with delta vs. last month.
• Agent Tracing: Multi-step workflows visualized as waterfall span trees with critical path highlighting, cost attribution per LLM/tool, and automatic retry/error span detection.
• Anomaly Detection: Automatic 3σ deviation alerts for latency or cost vs. your 7-day rolling baseline.
• PII & Security Scanning: Regex detection for SSN, credit cards, email, IBAN, passports at log time; API keys auto-masked before storage; injection and jailbreak pattern flagging.
• Model Recommendations: Evidence-based suggestions to swap models (e.g., "Your gpt-4o calls look like classification, try gpt-4o-mini") with projected dollar savings.
• Evals & Experiments: LLM-as-judge scoring (0 to 1) per prompt version; replay fixed datasets across prompt versions and models to compare quality, cost, and latency side by side.
• User Analytics: Per end-user and per-session cost, volume, and error tracking to identify budget-burning customers.
• Self-Hostable: Run in your own VPC with Docker Compose or a single binary; prompts and completions never leave your network.
• Team Collaboration: Projects for workload isolation, role-based access control, audit logs, webhooks, and Slack alerts.

Typical Workflows

Developers instrument their code with Spanlens' drop-in SDK replacement, then monitor live traffic in the unified dashboard. When costs spike or latency degrades, teams drill from aggregate charts to exact failing requests in two clicks. For continuous improvement, capture real traffic into datasets, score with LLM judges or human review, then run experiments to validate new prompt versions before deployment. The platform integrates with OpenTelemetry for existing observability stacks.

What Differentiates It

Spanlens requires zero agents, no SDK rewrites, and no architectural changes—just swap the baseURL. It's MIT-licensed, self-hostable, and designed as a single source of truth: every screen reads the same span store, so cost dashboards, trace waterfalls, evals, and security scans all reference the same underlying data. Pricing is flat monthly (not per-seat), with a generous free tier for up to 50K requests/month.