LangSmith vs Portkey
LLM observability, testing & evaluation — by LangChain
vs. Enterprise AI gateway + observability + guardrails + prompt mgmt
Pricing tiers
LangSmith
Developer (Free)
Free forever. 5,000 traces/month. 14-day retention. 1 seat. Basic evaluations.
Free
Plus
$39/seat/month. 10k base traces included ($2.50 per 1k overage). Full evaluations, custom dashboards, email support.
$39/mo
Enterprise
Custom. Self-host option, SSO, custom retention, dedicated support.
Custom
Portkey
Developer (Free)
Free forever. 10k logs/month. Universal API + key management. 3 prompt templates. Basic observability.
Free
Gateway (OSS)
MIT-licensed gateway only (no observability UI). Self-host for routing/fallbacks.
$0 base (usage-based)
Production
$49/month. 100k logs ($9 per additional 100k). Fallbacks, load balancing, retries, semantic caching. Unlimited prompts. RBAC.
$49/mo
Enterprise
Custom. 10M+ logs/month. Custom guardrails, advanced evals, SSO, budget controls, VPC + on-prem, SOC2, HIPAA, GDPR.
Custom
Free-tier quotas head-to-head
Comparing developer on LangSmith vs free on Portkey.
| Metric | LangSmith | Portkey |
|---|---|---|
| No overlapping quota metrics for these tiers. | ||
Features
LangSmith · 14 features
- Alerts — Threshold alerts on latency, cost, eval metrics.
- Annotation Queues — Human-review workflows for trace quality rating.
- Custom Dashboards — Aggregate metrics dashboards per project/tag.
- Datasets — Collect examples → use as eval sets or training data.
- Evaluations — LLM-as-judge, embedding similarity, custom Python evaluators, offline batch eval…
- LangChain Integration — Auto-trace any LangChain/LangGraph run with env var.
- LangGraph Integration — First-class trace + eval for LangGraph agents.
- LLM Tracing — Automatic trace every LLM call + tool call + chain step.
- OpenTelemetry Export — Export traces as OTLP to Datadog/Honeycomb/etc.
- Playground — Test prompts + models inline before deploying.
- Prompt Canvas — Visual prompt editor with live test + eval.
- Prompt Hub — Public + private prompt library with versioning.
- Self-Hosted (Enterprise) — Docker + k8s deployment in your infra.
- Threads + Sessions — Group traces into conversational sessions.
Portkey · 18 features
- AI Gateway — Unified OpenAI-compatible API to 250+ LLMs.
- Alerts — Thresholds on latency, error rate, cost, usage.
- Budget Controls — Per-key + per-team spending limits.
- Evaluations — Built-in evaluator templates + custom.
- Fallbacks — Config-driven provider fallback chains.
- Guardrails — Pre/post processors for safety + compliance.
- Load Balancing — Round-robin, weighted, least-latency across providers.
- MCP Support — Use MCP servers as tools through gateway.
- Observability — Logs, traces, feedback, alerts, cost tracking.
- OSS Gateway — Open-source gateway (portkey-ai/gateway).
- Prompt Library — Shared prompt library + public marketplace.
- Prompt Templates — Version + test + collaborate on prompts.
- Retries — Configurable retry policies per route.
- Role-Based Access Control — Team permissions on prompts + keys.
- Semantic Caching — Vector-based cache on query meaning.
- Simple Caching — Exact-match cache.
- Virtual Keys — Per-app keys with budget + rate limits + permissions.
- VPC Deployment (Ent) — Deploy in your own VPC for compliance.
Developer interfaces
| Kind | LangSmith | Portkey |
|---|---|---|
| CLI | LangSmith CLI | Portkey CLI |
| SDK | langsmith-js, langsmith-python | portkey-ai (Node), portkey-ai (Python) |
| REST | LangSmith REST API | Portkey API (OpenAI-compat) |
| MCP | LangSmith MCP | Portkey MCP |
| OTHER | LangSmith Dashboard | Portkey Dashboard |
Staxly is an independent catalog of developer platforms. Outbound links to LangSmith and Portkey are plain references to their official websites. Pricing is verified against vendor pages at publication time — reconfirm before buying.
Want this comparison in your AI agent's context? Install the free Staxly MCP server.