Crawler-Factory

Sessions/Session-2b-MockupGapAnalysis.md

Session 2b — Mockup-vs-Dashboard Gap Analysis

Why this file exists. Earlier in Session 2 we built 4 clickable HTML mockups (docs/workspace/ux-redesign/) before we'd captured the Algolia dashboard. Then we captured 32 dashboard screenshots and produced the Design Language Pack v1.1. Then Arijit committed to the embedded-in-Algolia-dashboard architecture. The mockups are now demonstrably wrong against both the design language and the embedded architecture. This document audits each mockup screen against both, lists every change needed, and feeds the Phase 1 Screen Inventory.

The mockups themselves are not thrown away. Their information architecture and interaction patterns (animated tree, live counters, per-category configurator, staggered crawler launches) are still the right ideas. What's wrong is the visual language and the outer chrome. The mockups need to be re-skinned, not redesigned.


Section A — Top-level violations across all 4 screens

These violations apply to every mockup and need to be fixed once, not four times.

A1. Outer chrome — wrong by definition

What we built: Each mockup has its own top bar with a < back arrow, custom Algolia-blue brand mark, "Crawler Factory" wordmark, ALPHA pill, and a 4-step horizontal breadcrumb (DISCOVER › REVIEW › CONFIGURE › CRAWL).

What Algolia has: When you're inside a focused workspace (operating one crawler), the outer chrome is replaced by: green-dot CRAWLER wordmark top-left, "Crawler / Wellington" scope indicator center-top, "Start Crawling" CTA + close X top-right. The 4-step horizontal breadcrumb does not exist anywhere in the Algolia dashboard.

What changes: - The outer chrome our mockups draw is wrong and is owned by Algolia (we don't draw it at all when embedded). - The 4-step horizontal breadcrumb gets dropped entirely. The wizard is a modal over the list view (per **2026-05-03-EmbeddedArchitecture**), not a routed flow with step navigation. The "step of 4" indicator becomes lighter — a small Step 1 of 4 label in the modal header, not a clickable breadcrumb. - The "← Back" / "Continue →" footer becomes a standard modal footer (Cancel + Back + Continue right-aligned).

A2. Color palette — too saturated, gradients banned

What we built: A blue-purple gradient on H1 accent text ("Turn any website into a searchable index"), a blue-purple gradient on the root tree node, a blue-purple gradient inside the progress bar.

What Algolia has: Flat fills only. Algolia blue (#003DFF) is the only accent. Status colors live exclusively inside small pill badges. No gradients anywhere.

What changes: - Remove the H1 gradient. The hook copy becomes plain --ink-primary weight 700. - Remove the root tree node gradient. Use a flat surface card with the root domain and URL count. - Remove the progress-bar gradient. Use a solid --accent fill.

A3. Typography — too marketing, too varied

What we built: "Turn any website into a searchable index — in minutes." 38px / 700, gradient accent, hero-eyebrow pill above. Reads like a marketing landing page.

What Algolia has: Page H1 is 24px / 700 / sentence case. No eyebrows, no marketing energy. "Crawler Configuration." "Suggestions." "Path Explorer." Plain.

What changes: - All H1s drop to 24px. Drop the eyebrow pill on screen 1. - Replace the hero copy with literal Algolia-style framing: "Add a new crawler" or "Create a smart crawler" — no "in minutes" promise. - Drop the gradient accent entirely.

A4. Card patterns — wrong shape

What we built: Each "branch" on the live discovery tree is a card with a colored icon tile (📦 with a green tint, 📰 with a purple tint, etc.), the category name with a confidence pill, and a URL count. The configure screen has a left rail of category rows + a right tab panel, but the rows have green status checkmarks in colored circles.

What Algolia has: The setting card is the workhorse — icon tile (light grey, monochrome), title (15px / 600), one-line description, optional value on the right, Edit button. No colored category icons. No bright status checkmarks. The icon tile is always grey.

What changes: - All branch cards on the live tree drop their colored icon tints. Use the standard light-grey icon tile pattern (Recipe 1 in the design pack). - Confidence pill stays (it's a status pill — Recipe 11 — and pills are allowed to use status color). - The configure screen's left-rail category rows lose the green-circle checkmarks. Use Algolia's quieter active-state treatment: 2px blue left bar + blue text on the active row, plain text on inactive rows.

A5. Buttons — wrong styling

What we built: Pill-shaped buttons with box-shadow: 0 4px 14px rgba(0, 61, 255, 0.25). Heavy shadow. Slight transform: translateY(-1px) on hover.

What Algolia has: Buttons are 12px-radius rectangles with no shadow. Hover changes color, not position. No transform animations.

What changes: - All buttons re-skinned per tokens.css .btn class. - Drop the lift animation. - Drop the heavy shadow.

A6. Animations — too many

What we built: Branches sliding in with branchIn keyframes and staggered delays. Progress bar shimmer. URL counter ramping with easeOut. Pulse dot on the status indicator. Shimmer background on the in-flight classifying branch. Crawler cards staggered start.

What Algolia has: Almost nothing. The dashboard is request-response. No entrance animations on lists. No counter ramping. No shimmer. The only motion is the soft expand/collapse of an Edit card and a discreet color transition on hover.

What changes: - Keep: the live-dot pulse on the Run-State pill (the only sanctioned animation in the design pack — opacity-only, 1.6s ease). - Keep: the inline expand/collapse of the wizard's per-category edit pattern (it's the standard Algolia card-edit pattern). - Drop: every other animation. Branches just appear. Counters just update. Progress bar fills without shimmer. The classifying branch is a static "pending" row with a 12px spinner glyph.

A7. Layout grid — too narrow

What we built: max-width: 1240px content centered with 32px side margins.

What Algolia has: The content area scales with the viewport but generally feels wider — the table on the Crawlers list spans almost the full viewport width. No hard 1240px cap.

What changes: - Use a wider max — 1440px or full viewport minus the left rail (240px). - Tables and lists especially benefit from the extra width.

A8. Voice — too marketing

What we built: "Turn any website into a searchable index — in minutes." "Tell us your domain. We'll map your site, discover every page…" "We only read public pages. No login, no scraping behind paywalls." "How it works: 1. We find your sitemap. 2. We walk the URL tree."

What Algolia has: "Customise how your crawler navigates and processes your site, ensuring efficient data collection while maintaining control over performance and security." Plain, professional, slightly clinical. No "we" voice. No promises. No marketing energy.

What changes: - All copy gets a re-pass. Default to Algolia's voice. - Hook copy on screen 1 drops the marketing paragraph. The wizard is launched from inside an authenticated dashboard — the user is already a customer; we don't need to sell. - Replace "How it works" with a more functional "What happens" or just embed the steps inline with the form.

A9. Empty / error / loading states — entirely missing

What we built: Happy path only on every screen. We hand-waved the "what if there are 0 sitemaps" case in the senior UX review.

What Algolia has: Three distinct empty patterns (data, gated, edit-empty) with consistent treatment per the design pack §3.7.

What changes: - Every screen needs a designed empty state, error state, and loading state. - The wizard especially needs gated-state design for: "this domain has no sitemap" (must use HTML link-graph instead — defer to Phase 2), "this domain blocks our crawler" (WAF detected, surface a domain-verification step), "this site has 0 pages we could classify."

A10. The screen we never built — the list view

What we built: The 4 wizard screens. No list view of crawlers.

What Algolia has: The first thing you see at Data sources → Crawler is a list of all crawlers in the application. The wizard is launched from a button on that list page.

What changes: - Add the list view as a fifth mockup screen. (See **2026-05-03-Phase1ScreenInventory** for the full spec.)


Section B — Per-screen audit

Screen 1: Discover (01-discover-screen.html)

What it is: The wizard's first step. URL input, hook copy, "How it works" rail, Start Discovery button.

Element Issue Severity Fix
Outer chrome (top bar with breadcrumb) Wrong — owned by Algolia Blocker Remove. Wizard is a modal over the list view. Replace top bar with modal header: title Add a new crawler + step indicator Step 1 of 4 + close X.
H1 with gradient accent Marketing voice, banned gradient High Replace with plain 24px H1: Tell us your website or What domain do you want to crawl?
Hook subtitle paragraph Marketing voice Medium Replace with a single functional line: We'll discover your content structure and create one crawler per content type.
URL input (url-input class) OK structurally; uses raw hex blue Low Restyle with tokens.css.input class, focus ring var(--accent-ring).
Inline scheme-prefix slot Missing Medium Add the https:// dropdown to the left of the URL input (matches Algolia's Start URLs edit pattern, Recipe 6).
Start discovery button Wrong styling — heavy shadow, lift animation, monumental width High Re-skin with .btn .btn--primary. Right-aligned in modal footer alongside Cancel + Back. Not full-width.
Example chips (algolia.com, shopify.com, stripe.com) OK idea, but filled-blue chips are too loud Medium Re-skin as ghost-style suggestion pills below the input. border: 1px solid var(--line-default); color: var(--ink-secondary).
"How it works" rail (right column) Wastes real estate; marketing tone Medium Drop the rail entirely. Move the 4 steps into a compact info banner under the input (Recipe 19). Or drop the explanation entirely — once you've used it once you don't need it.
Footer nav (Back / Continue) Custom layout Medium Replace with standard modal footer: Cancel (ghost left) + Continue (primary right).
Loading state on Continue Missing Medium Disabled state with spinner; modal doesn't dismiss until the API call returns.
Error state (invalid URL, unreachable site) Missing High Inline error below the input field per .input-error pattern.

Screen 2: Review (02-live-discovery.html)

What it is: The animated discovery screen. Progress bar with 4 stages, 4 KPI counters, animated tree of category branches, deselect-to-exclude interaction, Confirm Categories CTA.

Element Issue Severity Fix
Outer chrome Wrong — same fix as Screen 1 Blocker Modal header. Step 2 of 4.
Progress bar with 4 inline stage labels Algolia doesn't use this pattern High Replace with two stacked elements: (1) a thin neutral progress bar (8px, no shimmer), (2) a small descriptive line beneath: Classifying content branches… (3 of 4 stages complete). Drops the inline-stage-label pattern.
Pulse dot + "Classifying content branches…" Pulse OK if we limit it to the live indicator Low Keep but use the sanctioned 1.6s opacity-only pulse from Recipe 23.
4 counter cards (Sitemaps / URLs / Categories / Elapsed) Wrong styling — each card has a colored highlight when active Medium Re-skin as 4 KPI tiles per Recipe 12. Drop the per-card "active" highlight. The active stage is communicated by the descriptive line under the progress bar, not by counter highlight.
URL counter ramping animation Banned Medium Counter just updates to current count; no ramp.
Live tree with root + branches Right idea, wrong skin High Re-skin: drop the gradient root node, drop the colored category icons (use grey icon tiles), drop the entrance animation. New branches just appear as new rows.
Branch shimmer on the in-flight classifying branch Banned Medium Replace with a static row marked "Classifying…" with a 12px circular spinner where the icon would be.
Confidence pills (94% CONFIDENT) OK — these are status pills Low Restyle to match Recipe 11. pill--success for >85%, pill--warning for 70-85%, pill--neutral for <70%.
Branch click-to-toggle (deselect interaction) OK functionally but no visual contract Medium Add a leading checkbox to each branch row. Selected branches keep the checkbox checked. The whole row remains clickable.
"All 5 categories selected" hint Tone OK; placement OK Low Keep, in the design pack's t-description style.
Footer "Confirm categories →" Wrong styling Medium Standard modal footer per Recipe 14.
Edit / merge / rename branches (the senior UX review's biggest gap) Missing High Each branch row gets an inline edit (rename) action and a row overflow menu (Merge with… / Split into… / Exclude). Specced as part of Phase 1.

Screen 3: Configure (03-configure.html)

What it is: Per-category configurator. Left rail of categories with green checkmarks. Right tab panel: Index name / Fields / Test extractor.

Element Issue Severity Fix
Outer chrome Wrong Blocker Modal header. Step 3 of 4.
Summary bar with "5 of 5 ready" Lies (the senior UX review caught this) High Honest readiness check: ready = (index name set + ≥3 fields + extractor test passes). Otherwise show "3 of 5 ready" with the 2 incomplete categories highlighted.
Left rail with green-circle checkmarks Wrong active treatment High Re-skin per .nav-item pattern: 2px blue left bar + blue text on active. Status check on each row uses a tiny pill--success (12px) only when the category is genuinely ready.
Right pane with tabs (Index name / Fields / Test extractor) Tab pattern OK Low Re-skin per .tabs (bottom-border underline). Add count badges to Fields (Fields 5).
Index name input Missing validation High Live-validate against Algolia's index-name regex. Inline error below the input. Disable Continue until valid.
Description (optional) input Vague purpose Low Either remove or document where it appears (Algolia dashboard? Internal blueprint registry?). Default: keep but add help text.
Fields tab — 2-col grid of field cards OK pattern Low Re-skin as smaller selectable cards. Selected = blue border + blue checkmark.
Add custom field Ghost button OK Low Wire to a small modal (Recipe 9) for the new field's name + extractor expression.
Test extractor tab Single sample URL — the UX review caught this High Show 3-5 sample URLs minimum. Per-URL "Re-run" action. Aggregate "All passing" status.
Footer "Build & start crawlers" Wrong styling, fires immediately High Re-skin as standard modal footer. Clicking "Build crawlers" does not start crawling. It creates the crawlers, dismisses the modal, returns the user to the list view, where they can click Crawl now per row. (Or Build & crawl all if we want a one-click variant — TBD.)
Per-category bulk-edit (apply field to all categories) Missing Medium "Apply to all categories" affordance per field.

Screen 4: Crawl (04-crawl.html)

What it is: The status screen showing all crawlers running. Header + 5 crawler cards in a 2-col grid + live indexing feed + done banner.

Element Issue Severity Fix
Outer chrome Wrong Blocker This screen should not exist as a wizard step. The wizard ended at Screen 3 (creation). After creation the user is dropped back to the list view with N new crawlers visible. The "Crawl" status is the per-crawler focused workspace's Overview page (Algolia's existing pattern), not a 4th wizard step.
5 crawler cards in a 2-col grid Wrong — duplicates the list view Blocker Drop. The crawler list view shows all N crawlers as rows with their Run-State pills. Each crawler's run progress lives on its own focused workspace's Overview page.
Big stats header (RECORDS INDEXED / URLS TOTAL / ELAPSED) OK as a per-crawler summary Medium Move to the focused workspace's Overview page (matches Algolia's Overview's "Last complete crawl" card).
Per-card progress bar OK as per-crawler element Low Move to focused workspace Overview. Re-skin per design pack (no shimmer).
Per-card status pill OK Low Replace with Recipe 23 Run-State Pill.
Per-card actions (Open in Algolia / View log / Pause) Pause is the value-add Low Keep as the row's overflow menu (3-dot) per Recipe 22 / 23. Bulk-select supported.
Live indexing feed OK pattern Medium Lives on focused workspace's Logs Explorer page (Algolia already has this — just enable it via the Crawler Logs setting).
All-done banner Modal-style banner, marketing tone Medium Drop. The user knows it's done because the Run-State pill says Done. No celebration banner.
Footer "Open in Algolia ↗" / "Crawling…" Marketing tone Medium Drop the footer entirely. The user is already in Algolia.

Net for Screen 4: the screen as built is a duplicate of patterns Algolia already has (the list view + the per-crawler overview page). It should be removed from the wizard flow and its content distributed: - Per-crawler progress → focused workspace Overview page. - Per-crawler actions → Run-Control Cluster (top-right) + 3-dot row menu. - Cross-crawler progress → list view's Run-State pill column + bulk actions toolbar. - Indexing feed → Logs Explorer (Algolia native).


Section C — Patterns we keep

Despite the cosmetic re-skin, these mockup ideas are right and survive:

  1. The branches-not-leaves tree on the Review screen. This is a genuine improvement over Algolia's Path Explorer, which is a flat table. Tree visualization for category discovery is differentiated and useful.
  2. The 4-stage progress sequence (Sitemap → URLs → Grouping → Classifying). This is the actual discovery pipeline. Showing it explicitly is better than a generic spinner.
  3. The 4 KPI counters (Sitemaps / URLs / Categories / Elapsed). Matches Algolia's Path Explorer KPI tile pattern.
  4. The per-category configurator with index-name / fields / test extractor sub-tabs. Right structure; needs honest readiness check + multi-URL extractor test.
  5. The Run-Control Surface (Pause / Resume / Stop / Re-crawl). Our value-add per **2026-05-03-RunControlSurface**.
  6. The choice modal at "Add new crawler" (Build manually / Use Crawler Factory). Crisp surfacing of the differentiation.

Section D — Patterns we drop

  1. The 4-step horizontal breadcrumb at the top.
  2. The standalone outer chrome (brand mark + ALPHA + back arrow).
  3. The marketing hook copy on Screen 1.
  4. The "How it works" rail.
  5. All gradient accents.
  6. The colored category icon tints (📦 green, 📰 purple, etc.).
  7. The progress bar shimmer.
  8. Counter ramping animations.
  9. Branch entrance animations.
  10. Heavy button shadows + lift animations.
  11. Per-counter "active" highlight states.
  12. The all-done banner on Screen 4.
  13. Screen 4 as a wizard step (its content moves to focused workspace pages).

Section E — Implementation order for the React refactor

When Session 3 begins the React work, the order is:

  1. Apply tokens.css to the existing src/index.css so all global styles pick up the new tokens.
  2. Build the missing list view (currently no list view exists — see **2026-05-03-Phase1ScreenInventory** Screen 1).
  3. Refactor the wizard into a modal flow triggered from the list view. Drop the routed /admin/factory flow.
  4. Re-skin screens 1-3 of the wizard per Sections B above. This is the bulk of the work.
  5. Drop screen 4 of the wizard. Move its content to: - The list view's Run-State pill column (per-crawler status at a glance). - The focused workspace's Overview page (per-crawler progress + KPI tiles). - The Logs Explorer page (live indexing feed; Algolia native).
  6. Build the focused workspace shell for a crawler (3-section nav + persistent Run-Control Cluster). Reuses Algolia's pattern.
  7. Implement Run-Control Cluster + Run-State Pill components per Recipes 22-23.
  8. Add empty / error / loading states per Recipe 13.
  9. Replace Edit interactions with the inline-expand pattern from Recipe 2.

Section F — Output for next session

The deliverables of this gap analysis are:

Output File Purpose
The change list per screen This file (Section B) Reference during refactor
The patterns to keep This file (Section C) Don't lose what's right
The Embedded Architecture decision **2026-05-03-EmbeddedArchitecture** Why the chrome is gone
The full screen inventory for Phase 1 **2026-05-03-Phase1ScreenInventory** What we will actually build
The Run-Control Surface decision **2026-05-03-RunControlSurface** Our value-add
The Design Language Pack v1.1 **01-design-language** The skin

When Session 3 starts, read the inventory + the gap analysis + the design pack. The refactor work is then purely mechanical.