Crawde Review Standard v2
The open standard for automated AI website reviews.
Overview
The Crawde Review Standard defines how AI agents evaluate websites across 8 engineering-focused dimensions. Every review produces a score out of 100 and a letter grade. The v2 standard emphasizes measurable engineering metrics — performance, security headers, SEO elements, and accessibility are measured directly from page data, not estimated. Reviews are delivered as human-readable markdown reports with a dedicated engineering metrics table.
Websites can request reviews via the Crawde API and optionally receive results via a webhook callback. The review report is always available as markdown at /api/reviews/{id}/markdown.
Scoring Breakdown
| Category | Points | Weight | Focus |
|---|---|---|---|
| Functionality | 25 | 25% | Do all features actually work? Forms, buttons, integrations, error handling tested. |
| Performance | 15 | 15% | Load time benchmarks (<2s excellent, >4s poor), compression, caching headers, content size. |
| Security | 10 | 10% | HTTPS, CSP, HSTS, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy. |
| SEO & Discoverability | 10 | 10% | Title, meta, canonical, OG tags, Twitter Card, robots.txt, sitemap.xml, JSON-LD, viewport. |
| UI/UX Quality | 15 | 15% | Visual consistency, navigation, responsiveness, accessibility (alt text, ARIA, contrast). |
| Feature Completeness | 10 | 10% | Auto-discover features from nav/CTAs/pricing. Cross-reference against working features. |
| Code & Technical Health | 10 | 10% | Console errors, broken links, HTML structure, no exposed API keys or env variable leaks. |
| Content & Completeness | 5 | 5% | No placeholders or dead ends. Privacy policy and terms present. |
| Total | 100 | 100% |
Grading
Category Details
1. Functionality (25 pts)
The highest-weighted category. The review engine tests whether features actually work, not just whether they look like they should. This includes:
- • Forms are submittable and produce expected responses
- • Buttons trigger actions (not just visual targets)
- • Navigation leads to real pages (no 404s)
- • API endpoints respond correctly
- • Error states are handled gracefully
2. Performance (15 pts)
Scored using measured data, not estimates. Load time is benchmarked: <2s excellent, 2-4s acceptable, >4s poor.
- • Page load time (measured in ms)
- • Content compression (gzip/brotli detection)
- • Cache headers (Cache-Control, ETag)
- • Content size optimization
3. Security (10 pts)
Security is scored by detecting actual HTTP headers. HTTPS is baseline — 0 points without it. Each security header earns credit:
- • Content-Security-Policy (CSP)
- • X-Frame-Options
- • X-Content-Type-Options
- • Strict-Transport-Security (HSTS)
- • Referrer-Policy
- • Permissions-Policy
4. SEO & Discoverability (10 pts)
Each SEO element is checked from page source and HTTP requests. Missing elements reduce the score proportionally:
- • Title tag & meta description
- • Canonical URL
- • Open Graph tags
- • Twitter Card meta
- • robots.txt accessible
- • sitemap.xml present
- • JSON-LD structured data
- • Viewport meta & lang attribute
5. UI/UX Quality (15 pts)
Visual design consistency, navigation clarity, responsive layout, loading states, and accessibility basics: alt text coverage, ARIA labels, semantic HTML, and color contrast.
6. Feature Completeness (10 pts)
Auto-discovers every feature from navigation items, headings, CTAs, button labels, pricing tiers, and page content. Each discovered feature is cross-referenced against what's verifiably functional. A verification percentage is computed.
7. Code & Technical Health (10 pts)
Console errors, broken links (4xx/5xx pages), proper HTML structure (doctype, lang, charset), and checks for exposed API keys or environment variables in page source.
8. Content & Completeness (5 pts)
Clarity, spelling, grammar. No placeholder or TODO content. Privacy policy and terms of service presence is verified.
Engineering Metrics (New in v2)
Every v2 review includes a dedicated Engineering Metrics table with measured data points:
- • Load Time (ms) — measured from request to response
- • HTTPS — yes/no
- • Compression — gzip/brotli detected from Content-Encoding
- • Cache Headers — Cache-Control or ETag present
- • Security Headers — count of 6 key headers detected
- • SEO Elements — count of 9 key elements detected
- • Alt Text Coverage — images with alt text vs total images
- • ARIA Labels — count found in page source
- • Privacy Policy / Terms of Service — link detected on page
Review Process
- 1. Submission — A URL is submitted via the API or web form.
- 2. Crawling — The engine fetches the main page and discovers up to 10 subpages via link analysis.
- 3. Page Analysis — Each page is analyzed for structure, content, forms, buttons, links, headings, load time, HTTPS, console errors, security headers, SEO elements, compression, caching, and accessibility indicators.
- 4. Feature Discovery — Features are auto-discovered from navigation, headings, CTAs, pricing, and page content. Each is cross-referenced against actual functionality.
- 5. AI Evaluation — All collected data (including measured engineering metrics) is evaluated by GPT-4o following the v2 scoring rubric.
- 6. Report Generation — A structured markdown report is generated with scores, findings, and recommendations.
- 7. Delivery — Results are available via API, web view, and optionally delivered via webhook callback.
API
Submit a Review
POST https://crawde.com/api/reviews
Content-Type: application/json
{
"url": "https://yoursite.com",
"callbackUrl": "https://yoursite.com/api/crawde-review",
"callbackSecret": "your-secret",
"email": "you@example.com"
}
Response (202 Accepted):
{
"id": "review-uuid",
"status": "pending",
"checkUrl": "https://crawde.com/api/reviews/{id}",
"viewUrl": "https://crawde.com/reviews/{id}"
}Callback Webhook Payload
POST {callbackUrl}
Content-Type: application/json
X-Crawde-Event: review.completed
X-Crawde-Signature: hmac-sha256-hex (if secret provided)
{
"event": "review.completed",
"reviewId": "...",
"url": "https://yoursite.com",
"status": "completed",
"overallScore": 85,
"grade": "B",
"verdict": "One-line summary",
"markdownReport": "# Full markdown report...",
"sections": [...],
"progressAnalysis": {
"claimedFeatures": [...],
"testedFeatures": [...],
"untestedFeatures": [...],
"accuracyPercent": 80
},
"commonSenseCheck": {
"passed": true,
"notes": [...]
}
}Receive Reviews: Endpoint Implementation
To automatically receive reviews, implement a POST endpoint on your server. The endpoint should:
- • Accept POST requests with JSON body
- • Verify the
X-Crawde-Signatureheader using HMAC-SHA256 with your callback secret - • Return 200 to acknowledge receipt
Code Patch Protocol (CPP)
The Code Patch Protocol is an optional extension that allows Crawde to go beyond reviewing and actually generate code patches to fix identified issues. The protocol uses end-to-end encryption so that source code is never exposed in plain text during transit.
How It Works
- 1. Discovery — During review, Crawde checks for
/.well-known/crawde-reviewon your domain. - 2. Key Exchange — Crawde generates ephemeral ECDH P-256 keys and sends its public key to your code endpoint.
- 3. Code Request — Your server encrypts source files using the shared ECDH secret (AES-256-GCM) and signs the payload with ECDSA P-256.
- 4. Patch Generation — Crawde decrypts and verifies the code, then uses AI to generate targeted patches based on review findings.
- 5. Patch Delivery — Patches are encrypted with the same shared secret, signed by Crawde, and sent to your patch endpoint.
Cryptography
| Purpose | Algorithm |
|---|---|
| Key Agreement | ECDH P-256 |
| Encryption | AES-256-GCM |
| Signing | ECDSA P-256 (SHA-256) |
| Key Derivation | SHA-256(ECDH shared secret) |
1. Well-Known Configuration
Serve a JSON file at /.well-known/crawde-review:
{
"patchEnabled": true,
"codeEndpoint": "/api/crawde/code",
"patchEndpoint": "/api/crawde/patch",
"publicKey": "<your ECDH P-256 public key, base64>",
"signingPublicKey": "<your ECDSA P-256 public key, PEM>"
}2. Code Endpoint
Crawde sends a POST with its public keys. Your server should respond with encrypted source files:
// Request from Crawde
POST /api/crawde/code
{
"crawdeEcdhPublicKey": "<base64>",
"crawdeSigningPublicKey": "<PEM>"
}
// Your response
{
"ecdhPublicKey": "<your ECDH public key, base64>",
"signingPublicKey": "<your ECDSA public key, PEM>",
"payload": {
"ciphertext": "<AES-256-GCM encrypted JSON of files, base64>",
"iv": "<initialization vector, base64>",
"authTag": "<GCM auth tag, base64>",
"signature": "<ECDSA signature of ciphertext, base64>",
"signerPublicKey": "<PEM>",
"contentHash": "<SHA-256 hex of plaintext>"
}
}The encrypted plaintext should be a JSON array: [{"path": "src/app.ts", "content": "..."}]
3. Patch Endpoint
Crawde sends encrypted patches back to your server:
// Crawde sends
POST /api/crawde/patch
{
"crawdeEcdhPublicKey": "<base64>",
"crawdeSigningPublicKey": "<PEM>",
"payload": {
"ciphertext": "<encrypted patch array, base64>",
"iv": "<base64>",
"authTag": "<base64>",
"signature": "<base64>",
"signerPublicKey": "<PEM>",
"contentHash": "<hex>"
}
}
// Decrypted payload structure:
[
{
"path": "src/app.ts",
"patch": "--- a/src/app.ts\n+++ b/src/app.ts\n@@ -10,3...",
"explanation": "Fix: Added null check for user input"
}
]Transparency
When the Code Patch Protocol is used, the review detail page displays cryptographic fingerprints of both Crawde and the website's session keys, the list of files modified, and a summary of all patches. Humans can verify that the handshake occurred and what changes were proposed.