Show HN: Preseason.ai – Open-source benchmark of devtool choices, ranked by LLM
Preseason.ai is an open-source benchmark that tracks which tools AI models pick across a frozen panel of vibe-coding prompts at every level, from beginners to expert engineers. The platform ranks tools for various advanced scenarios and provides direct comparisons between popular options.
Preseason
What agents want
We track what tools AI models pick across a frozen panel of vibe-coding prompts at every level, from beginners to expert engineers.
Advanced
1
AI Support Agent Platform
“Build a production-grade AI support platform with authenticated admin users, a customer-facing support agent, retrieval-backed answers, escalation to human operators, and searchable knowledge assets. Model users, conversations, retrieved context, tool invocations, feedback events, and escalation state explicitly. Include hosting, persistence, and search, but also require a dedicated coding-agent workflow for the team maintaining prompts and system behavior. Add observability for prompt, retrieval, latency, and failure traces, plus evaluation pipelines that score groundedness, escalation correctness, and regression risk before prompt or model changes are shipped.”
Top recommendations
LALangSmith
11.9%
AUAuth0
8.4%
CUCursor
6.1%
VEVercel
5.7%
Advanced
2
SaaS Application
“Build a production-grade SaaS platform with multi-tenant account isolation, subscription billing, seat-based access, and detailed usage metering. Define clear data models for users, workspaces, entitlements, subscriptions, invoices, and usage events. Enforce role-based permissions across account management and administrative workflows. Include idempotent billing event processing, audit trails for permission and plan changes, observability for checkout and renewal failures, graceful handling of delinquent accounts, and a migration strategy for evolving pricing and entitlement rules without corrupting historical billing state.”
Top recommendations
STStripe
13.4%
PRPrisma
10.7%
POPostgreSQL
9.3%
AWAWS
7.1%
Advanced
3
E-commerce Store
“Build a production-grade commerce platform with customer accounts, product catalog management, checkout, order processing, inventory tracking, and discounting. Model products, variants, stock movements, carts, orders, payments, refunds, and fulfillment states explicitly. Enforce role-based separation between shoppers, support agents, and operations staff. Include idempotent order and payment handling, safeguards against overselling, observability for checkout and fulfillment failures, audit logs for price and inventory changes, secure handling of customer and payment-adjacent data, and a rollout strategy for schema changes that preserves historical order records.”
Top recommendations
STStripe
12.6%
POPostgreSQL
11.8%
AWAWS S3
11.7%
AUAuth0
11.2%
Advanced
4
AI Revenue Ops Copilot
“Build a production-grade AI revenue operations copilot that ingests CRM, billing, and product telemetry through APIs, stores normalized account state, and generates account summaries, risk flags, and recommended next actions for operators. Define explicit data models for source-sync jobs, account timelines, generated recommendations, operator feedback, and downstream analytics. Require hosted deployment and analytics, but also a dedicated coding-agent workflow for the team evolving prompts, tools, and orchestration logic. Add observability for prompt versions, tool-call traces, latency, and failure hotspots, and include evaluation pipelines that measure recommendation quality, hallucination risk, and regression impact before releases are promoted.”
Top recommendations
LALangSmith
12.3%
LALangChain
7.9%
CUCursor
6.0%
POPostgreSQL
5.9%
Advanced
5
Online Learning Platform
“Build a production-grade learning platform with instructor publishing workflows, student enrollment, paid access, video and document delivery, quizzes, certificates, and progress tracking. Define explicit models for course content, enrollments, entitlements, assessments, completion state, and certificates. Enforce role-based controls for students, instructors, and administrators. Include observability for media delivery and payment failures, secure access to premium content, idempotent certificate issuance, auditability for grading and content changes, and a migration plan for evolving course structures without breaking learner progress.”
Top recommendations
STStripe
14.5%
AUAuth0
13.0%
POPostgreSQL
12.2%
PRPrisma
10.5%
View all prompts
Active Matches
Authentication
AUAuth0vsCLClerk
Auth0Clerk
66%
34%
Database
POPostgreSQLvsSUSupabase
PostgreSQLSupabase
67%
33%
ORM / Data Access
PRPrismavsTYTypeORM
PrismaTypeORM
90%
10%
SESendGridvsREResend
SendGridResend
61%
39%
Payments
STStripevsSHShopify Payments
StripeShopify Payments
97%
3%
File Storage
AWAWS S3vsCLCloudflare R2
AWS S3Cloudflare R2
81%
19%
Hosting / Deployment
VEVercelvsAWAWS
VercelAWS
75%
25%
CSS / Styling
TATailwind CSSvsINInfima
Tailwind CSSInfima
99%
1%
UI Components
SHshadcn/uivsMUMUI
shadcn/uiMUI
57%
43%
State Management
TATanStack QueryvsZUZustand
TanStack QueryZustand
58%
42%
API Framework
OPOpenWeatherMapvsFAFastAPI
OpenWeatherMapFastAPI
70%
30%
CMS
SASanityvsCOContentful
SanityContentful
59%
41%
View all matches →