Each model is given the same 6 web app building problems at temperature 0.2. Solutions are single-file HTML applications evaluated by an automated judge across four weighted dimensions. Composite score = Completeness × 0.35 + UX/Fidelity × 0.25 + Code Quality × 0.20 + Accessibility × 0.20. Models are ranked by composite score.
All features work as specified. Edge cases handled. Error states, empty states, and unexpected inputs don't crash.
Visual design matches spec. Responsive layout, intuitive controls, visual hierarchy, feedback states, and polish.
Well-structured, maintainable code. Proper separation of concerns. CSS variables, semantic HTML, clean patterns.
ARIA labels, keyboard navigation, semantic HTML structure, focus management, screen reader support.
No models benchmarked yet
The web app building benchmark evaluates LLMs on their ability to create production-grade single-file web applications from spec. Models will appear here once benchmark runs complete. Check back soon.
Each problem is a single-file HTML web application specification. Models generate the complete app from the prompt. Problems range from CRUD apps to real-time data visualization — covering the full spectrum of web app building skills.
Full CRUD task manager with localStorage, filtering, sorting, search, inline editing, and input validation.
Real-time simulated dashboard with Canvas/SVG charts, auto-scroll, start/stop, speed control, and responsive layout.
Multi-category unit converter with smart input detection, searchable selectors, conversion history, and swap button.
Flashcard decks with spaced repetition (SM-2), quiz mode, progress tracking, and CSS 3D flip animations.
Pomodoro timer with SVG arc countdown, daily task list, session statistics, streak tracking, and Web Audio notifications.
Split-pane markdown editor with live preview, syntax highlighting, table rendering, toolbar, and keyboard shortcuts.
Each solution is evaluated by an automated judge against the problem specification. No human judgment, no subjective scoring — every score is reproducible from the spec and the generated code.
Each dimension is scored 0–10. 0 = not attempted / non-functional. 10 = flawless execution meeting all spec requirements with exemplary quality. Composite is a weighted sum.
New models are added as they ship web app building capabilities. If a model consistently produces working applications from natural language specs, it qualifies for the benchmark.