Crawler Factory: Spec 11: Frontend Configurator (CategoryConfigurator + sub-components) Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build the per-pathGroup configurator drawer (CategoryConfigurator) and its three sub-components (SamplePreview, ConfigEditor, TestCrawlResults) so a human operator can review samples, edit the recordExtractor, regenerate it with feedback, run sandbox + real test crawls, and commit a per-content-domain crawler: strictly human-in-the-loop, no silent commits.

Architecture: A right-side shadcn Sheet drawer at 70% viewport width, opened from CategoryReview (Spec 10) when the user clicks Configure Selected. On open, the drawer fires POST /api/factory/sample then POST /api/factory/generate-extractor to populate samples + initial extractor. Three vertically-stacked sub-sections render the artifacts (samples on top, editor in the middle, test results at bottom), and three CTAs at the footer drive the workflow: Run sandbox test → Run REAL test crawl → Commit + Create Crawler. The Commit CTA is enabled only when both sandbox and real test results are green for the active pathGroup. This component cluster is render-only: it does NOT manage which pathGroup is active (that's Spec 12 orchestrator). It accepts a pathGroup prop and emits onCommit upward.

Tech Stack: TypeScript (strict), React 18, shadcn/ui (Sheet, Form, Input, Textarea, Switch, Button, Card, Tabs, Collapsible), react-hook-form + zod resolver, lucide-react icons, framer-motion (existing in repo) for subtle reveals, vitest + @testing-library/react for tests.

Depends on: - Spec 01 (lib/factory/dss.ts, lib/factory/types.ts: PathGroup, ContentDomain, DssEntry) - Spec 09 (src/hooks/factory/useFactorySession.ts: pulls session by id; not used directly here, but the configurator commits a blueprint that the session will later persist via Spec 12 orchestrator) - Spec 10 (src/components/admin/factory/DomainBadge.tsx: reused inside the drawer header) - Backend endpoints from Spec 08: POST /api/factory/sample, POST /api/factory/generate-extractor, POST /api/factory/test-sandbox, POST /api/factory/test-real. These are called via fetch() inside CategoryConfigurator: no new hook layer is added; the calls are imperative actions tied to user gestures.

Consumed by: Spec 12 (CrawlerFactory orchestrator): wires CategoryConfigurator open/close state, passes the active pathGroup, and on onCommit calls POST /api/factory/create-crawler.

Plan source: Projects/Crawler-Factory/00-Plan.md §5b (drawer ASCII mockup), §7 Task 27 (CategoryConfigurator), §19e (User-in-the-loop policy: no silent commits, every CTA explicit click).

---

File structure

Path	Responsibility
src/components/admin/factory/CategoryConfigurator.tsx	Drawer shell. Owns: open lifecycle, sample fetch, extractor fetch, sandbox + real test invocation, commit handler. Composes the three sub-components.
src/components/admin/factory/SamplePreview.tsx	Render-only. Lists sample URLs with collapsible "view HTML"; renders detected selector ✓ field rows; renders schema.org JSON-LD blob if present.
src/components/admin/factory/ConfigEditor.tsx	Form (indexName, renderJavaScript, rateLimit, pathsToMatch) + monospace Textarea for recordExtractor JS + "Regenerate from samples" button + free-form feedback Input (Enter submits regen with feedback).
src/components/admin/factory/TestCrawlResults.tsx	Two-tab view (Sandbox / Real). Each tab: per-sample row with ✓/✗, record count, field-coverage bars (one per DSS-required field), "show first record JSON" expand. Run-test buttons live here.
src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx	Drawer behavior: open/close, fetches on open, regenerate-with-feedback re-fetches, commit-button gating.
src/components/admin/factory/__tests__/SamplePreview.test.tsx	Render-only: sample list, expand-collapse, JSON-LD presence/absence.
src/components/admin/factory/__tests__/ConfigEditor.test.tsx	Form changes call onChangeConfig; textarea changes call onChangeExtractor; Enter on feedback input fires onUserFeedback.
src/components/admin/factory/__tests__/TestCrawlResults.test.tsx	Field-coverage percentages render correctly; tab switching; first-record JSON expand.

---

SOP rules applied (non-negotiable; from CodingSOPs.md, TestingSOPs.md, WritingSOPs.md)

Every file in this spec MUST follow these rules. They are not optional.

1. Two-layer type safety. All component props are explicit TypeScript interfaces. Zod is not used inside the component layer (validation lives at the API boundary, Spec 08), but the prop types are derived from Spec 01 zod-inferred types (PathGroup, ContentDomain). No any. Use T | null, not T | undefined, for "absent" semantic.
2. Console.error/log only inside catch. No stray console.log in render paths. Errors caught from fetch() are surfaced to the user via toast (useToast from src/components/ui/use-toast) AND logged to console.
3. Try/catch every fetch. Every fetch() call wrapped. On non-2xx, parse response.json() to surface { error } if present, fall back to response.statusText. Always preserve loading state symmetry (finally clears loading flag).
4. Functions ≤20 lines, ≤3 params. Render helpers extracted as named local functions when a JSX block exceeds ~30 lines.
5. Docstring on every exported component. JSDoc comment explaining purpose + key prop semantics. Comment WHY non-obvious choices exist.
6. No raw dicts crossing module boundaries. Props use the PathGroup, Sample, StructureAnalysis, SandboxResult, RealResult, CrawlerConfig types: defined either in Spec 01 (lib/factory/types.ts) or, for the strictly UI-only result shapes, in a small local types.ts co-located in src/components/admin/factory/.
7. TDD cadence. Test first → run, expect FAIL → implement → run, expect PASS → commit. Use @testing-library/react's render, screen, fireEvent, waitFor. Mock fetch globally with vi.fn() per test.
8. Mock external services in unit tests. global.fetch is mocked in every test for CategoryConfigurator. SamplePreview, ConfigEditor, TestCrawlResults are rendered with controlled props: no fetch.
9. Naming. Components: PascalCase.tsx. Tests: same name with .test.tsx. Local hooks/helpers: camelCase. Types: PascalCase. Tailwind utility classes from existing /admin aesthetic.
10. Conventional commits. feat(factory):, test(factory):. One sub-component per commit (per Task 27 directive: "Commit each sub-component separately").

---

Task 0: Define shared UI-only types

Files: - Create: src/components/admin/factory/types.ts

These types describe shapes that flow between the configurator and its sub-components but are not zod-validated themselves (they are derived from Spec 01 zod schemas at the API boundary upstream).

• [ ] Step 0.1: Create src/components/admin/factory/types.ts

/**
 * UI-only types shared across CategoryConfigurator and its sub-components.
 *
 * Why: these shapes flow component-to-component but never cross the network
 * boundary directly — the API layer (Spec 08) returns zod-validated payloads
 * that are mapped into these UI types. Keeping them local avoids leaking
 * UI concerns into lib/factory/types.ts.
 */
import type { ContentDomain, PathGroup } from '@/lib/factory/types';

/**
 * A single sampled page captured by the sampler (Spec 03).
 *
 * `Sample.transport` mirrors Spec 03's wire format (`'fetch' | 'playwright'`)
 * — same field name, no remapping at fetch time. The HTTP response from
 * `/api/factory/sample` returns this shape verbatim.
 */
export interface Sample {
  url: string;
  html: string;
  status: number;
  transport: 'fetch' | 'playwright';
}

/**
 * Output of structure-analyzer (Spec 05) — selector hits per DSS field.
 *
 * Shape mirrors Spec 05's `StructureAnalysisSchema` verbatim (no remap):
 * - `schemaOrgJsonLd`: parsed JSON-LD blob if present, else null
 * - `selectors`: `Record<field, cssSelector>` — one entry per resolved DSS field
 * - `missing`: DSS-required field names that the analyzer could not resolve
 * - `confidence`: 0..1 quality score
 *
 * Note: Spec 05 does NOT expose microdata or cmsSignature. If a CMS signature
 * is needed in the UI, read it from the Spec 04 verdict separately (out of
 * scope for this configurator).
 */
export interface StructureAnalysis {
  schemaOrgJsonLd: unknown | null;
  selectors: Record<string, string>;
  missing: string[];
  confidence: number;
}

/** Per-sample sandbox result. */
export interface SandboxResult {
  url: string;
  records: unknown[];
  errors: string[];
  fieldCoverage: Record<string, number>; // 0..1 per DSS-required field
}

/** Per-sample real-crawl result (returned from /api/factory/test-real). */
export interface RealResult {
  url: string;
  records: unknown[];
  errors: string[];
  fieldCoverage: Record<string, number>;
}

/** Crawler config block edited inside the drawer. */
export interface CrawlerConfig {
  indexName: string;
  renderJavaScript: boolean;
  rateLimit: number;
  pathsToMatch: string[];
}

/** Re-export so consumers don't need two import sources. */
export type { ContentDomain, PathGroup };

• [ ] Step 0.2: Run tsc --noEmit to confirm clean

npx tsc --noEmit

Expected: clean.

• [ ] Step 0.3: Commit

git add src/components/admin/factory/types.ts
git commit -m "feat(factory): UI-only shared types for configurator components"

---

Task 1: SamplePreview: render-only sample list + detected structure + JSON-LD

Files: - Create: src/components/admin/factory/SamplePreview.tsx - Create: src/components/admin/factory/__tests__/SamplePreview.test.tsx

• [ ] Step 1.1: Write failing test: empty samples

Create src/components/admin/factory/__tests__/SamplePreview.test.tsx:

import { describe, it, expect } from 'vitest';
import { render, screen } from '@testing-library/react';
import { SamplePreview } from '../SamplePreview';
import type { Sample, StructureAnalysis } from '../types';

const emptyAnalysis: StructureAnalysis = {
  schemaOrgJsonLd: null,
  selectors: {},
  missing: [],
  confidence: 0,
};

describe('SamplePreview', () => {
  it('renders an empty-state message when no samples', () => {
    render(<SamplePreview samples={[]} structureAnalysis={emptyAnalysis} />);
    expect(screen.getByText(/no samples/i)).toBeInTheDocument();
  });
});

• [ ] Step 1.2: Run test, expect FAIL

npx vitest run src/components/admin/factory/__tests__/SamplePreview.test.tsx

Expected: FAIL: Cannot find module '../SamplePreview'.

• [ ] Step 1.3: Implement SamplePreview.tsx

Create src/components/admin/factory/SamplePreview.tsx:

/**
 * SamplePreview — render-only block inside CategoryConfigurator.
 *
 * Shows the sampled HTML pages, the per-field selector matches the
 * structure-analyzer detected, and the schema.org JSON-LD blob (if any).
 *
 * Why no fetch here: the parent (CategoryConfigurator) owns fetch state.
 * This component is pure render — easier to test, easier to reason about.
 */
import { useState } from 'react';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import {
  Collapsible,
  CollapsibleContent,
  CollapsibleTrigger,
} from '@/components/ui/collapsible';
import { ChevronDown, ChevronRight, Check, X as XIcon } from 'lucide-react';
import type { Sample, StructureAnalysis } from './types';

export interface SamplePreviewProps {
  samples: Sample[];
  structureAnalysis: StructureAnalysis;
}

export function SamplePreview({ samples, structureAnalysis }: SamplePreviewProps) {
  if (samples.length === 0) {
    return (
      <Card>
        <CardHeader>
          <CardTitle className="text-sm">Samples</CardTitle>
        </CardHeader>
        <CardContent>
          <p className="text-sm text-muted-foreground">No samples yet.</p>
        </CardContent>
      </Card>
    );
  }

  return (
    <Card>
      <CardHeader>
        <CardTitle className="text-sm">Samples ({samples.length})</CardTitle>
      </CardHeader>
      <CardContent className="space-y-3">
        <SampleList samples={samples} />
        <DetectedStructure analysis={structureAnalysis} />
        {structureAnalysis.schemaOrgJsonLd != null && (
          <JsonLdBlock jsonLd={structureAnalysis.schemaOrgJsonLd} />
        )}
      </CardContent>
    </Card>
  );
}

function SampleList({ samples }: { samples: Sample[] }) {
  return (
    <div className="space-y-1" data-testid="sample-list">
      {samples.map((s) => (
        <SampleRow key={s.url} sample={s} />
      ))}
    </div>
  );
}

function SampleRow({ sample }: { sample: Sample }) {
  const [open, setOpen] = useState(false);
  return (
    <Collapsible open={open} onOpenChange={setOpen}>
      <CollapsibleTrigger
        className="flex w-full items-center gap-2 text-left text-sm hover:underline"
        data-testid="sample-row-trigger"
      >
        {open ? <ChevronDown className="h-3 w-3" /> : <ChevronRight className="h-3 w-3" />}
        <span className="font-mono text-xs">{sample.url}</span>
        <span className="ml-auto text-xs text-muted-foreground">{sample.status}</span>
      </CollapsibleTrigger>
      <CollapsibleContent>
        <pre className="mt-1 max-h-64 overflow-auto rounded bg-muted p-2 font-mono text-xs">
          {sample.html.slice(0, 8000)}
        </pre>
      </CollapsibleContent>
    </Collapsible>
  );
}

function DetectedStructure({ analysis }: { analysis: StructureAnalysis }) {
  const selectorEntries = Object.entries(analysis.selectors);
  if (selectorEntries.length === 0 && analysis.missing.length === 0) {
    return <p className="text-xs text-muted-foreground">No structure detected.</p>;
  }
  return (
    <div className="rounded border p-2" data-testid="detected-structure">
      <p className="mb-1 text-xs font-semibold">
        Detected structure (confidence {Math.round(analysis.confidence * 100)}%)
      </p>
      <ul className="space-y-0.5">
        {selectorEntries.map(([field, selector]) => (
          <li
            key={`resolved:${field}`}
            className="flex items-center gap-2 font-mono text-xs"
          >
            <Check className="h-3 w-3 text-emerald-600" />
            <span className="text-muted-foreground">{selector}</span>
            <span>→</span>
            <span>{field}</span>
          </li>
        ))}
        {analysis.missing.map((field) => (
          <li
            key={`missing:${field}`}
            className="flex items-center gap-2 font-mono text-xs"
          >
            <XIcon className="h-3 w-3 text-red-600" />
            <span className="text-muted-foreground">(no selector)</span>
            <span>→</span>
            <span>{field}</span>
          </li>
        ))}
      </ul>
    </div>
  );
}

function JsonLdBlock({ jsonLd }: { jsonLd: unknown }) {
  const [open, setOpen] = useState(false);
  return (
    <Collapsible open={open} onOpenChange={setOpen}>
      <CollapsibleTrigger
        className="flex items-center gap-1 text-xs hover:underline"
        data-testid="jsonld-trigger"
      >
        {open ? <ChevronDown className="h-3 w-3" /> : <ChevronRight className="h-3 w-3" />}
        <span>schema.org JSON-LD</span>
      </CollapsibleTrigger>
      <CollapsibleContent>
        <pre className="mt-1 max-h-48 overflow-auto rounded bg-muted p-2 font-mono text-xs">
          {JSON.stringify(jsonLd, null, 2)}
        </pre>
      </CollapsibleContent>
    </Collapsible>
  );
}

• [ ] Step 1.4: Run, expect PASS for empty-state test

npx vitest run src/components/admin/factory/__tests__/SamplePreview.test.tsx

Expected: 1 test passes.

• [ ] Step 1.5: Add tests for sample list, expand HTML, structure rows, JSON-LD

Append to src/components/admin/factory/__tests__/SamplePreview.test.tsx:

import { fireEvent } from '@testing-library/react';

const sampleA: Sample = {
  url: 'https://x.com/blog/post-1',
  html: '<html><body>Hello</body></html>',
  status: 200,
  transport: 'fetch',
};

const sampleB: Sample = {
  url: 'https://x.com/blog/post-2',
  html: '<html><body>World</body></html>',
  status: 200,
  transport: 'fetch',
};

describe('SamplePreview — populated', () => {
  it('renders one row per sample', () => {
    render(
      <SamplePreview samples={[sampleA, sampleB]} structureAnalysis={emptyAnalysis} />
    );
    expect(screen.getByText('https://x.com/blog/post-1')).toBeInTheDocument();
    expect(screen.getByText('https://x.com/blog/post-2')).toBeInTheDocument();
  });

  it('expands HTML on row click', () => {
    render(<SamplePreview samples={[sampleA]} structureAnalysis={emptyAnalysis} />);
    const triggers = screen.getAllByTestId('sample-row-trigger');
    fireEvent.click(triggers[0]);
    expect(screen.getByText(/<body>Hello<\/body>/)).toBeInTheDocument();
  });

  it('renders resolved selectors and missing fields', () => {
    const analysis: StructureAnalysis = {
      schemaOrgJsonLd: null,
      selectors: { headline: 'h1.entry-title' },
      missing: ['author'],
      confidence: 0.6,
    };
    render(<SamplePreview samples={[sampleA]} structureAnalysis={analysis} />);
    expect(screen.getByTestId('detected-structure')).toBeInTheDocument();
    expect(screen.getByText('h1.entry-title')).toBeInTheDocument();
    expect(screen.getByText('headline')).toBeInTheDocument();
    expect(screen.getByText('author')).toBeInTheDocument();
  });

  it('renders JSON-LD block only when schemaOrgJsonLd is non-null', () => {
    const analysis: StructureAnalysis = {
      schemaOrgJsonLd: { '@type': 'BlogPosting', headline: 'Hi' },
      selectors: {},
      missing: [],
      confidence: 1,
    };
    render(<SamplePreview samples={[sampleA]} structureAnalysis={analysis} />);
    expect(screen.getByTestId('jsonld-trigger')).toBeInTheDocument();
  });

  it('does not render JSON-LD block when schemaOrgJsonLd is null', () => {
    render(<SamplePreview samples={[sampleA]} structureAnalysis={emptyAnalysis} />);
    expect(screen.queryByTestId('jsonld-trigger')).not.toBeInTheDocument();
  });
});

• [ ] Step 1.6: Run all SamplePreview tests, expect PASS

npx vitest run src/components/admin/factory/__tests__/SamplePreview.test.tsx

Expected: 6 tests pass.

• [ ] Step 1.7: Run tsc --noEmit

npx tsc --noEmit

Expected: clean.

• [ ] Step 1.8: Commit

git add src/components/admin/factory/SamplePreview.tsx src/components/admin/factory/__tests__/SamplePreview.test.tsx
git commit -m "feat(factory): SamplePreview render-only component"

---

Task 2: ConfigEditor: form + extractor textarea + regen with feedback

Files: - Create: src/components/admin/factory/ConfigEditor.tsx - Create: src/components/admin/factory/__tests__/ConfigEditor.test.tsx

• [ ] Step 2.1: Write failing test: form fields render with default values

Create src/components/admin/factory/__tests__/ConfigEditor.test.tsx:

import { describe, it, expect, vi } from 'vitest';
import { render, screen, fireEvent } from '@testing-library/react';
import { ConfigEditor } from '../ConfigEditor';
import type { CrawlerConfig, PathGroup } from '../types';
import { getDss } from '@/lib/factory/dss';

const pathGroup: PathGroup = {
  id: 'pg_1',
  pattern: '/blog/*',
  urlCount: 412,
  sampleUrls: [],
  detectedDomain: 'marketing',
  detectionConfidence: 0.92,
  detectionMethod: 'cms',
  selected: true,
};

const dssEntry = getDss('marketing');

const config: CrawlerConfig = {
  indexName: 'algoliacentral_marketing',
  renderJavaScript: false,
  rateLimit: 8,
  pathsToMatch: ['/blog/*'],
};

const defaultProps = {
  pathGroup,
  dssEntry,
  recordExtractor: 'function extractor() { return {}; }',
  crawlerConfig: config,
  onChangeConfig: vi.fn(),
  onChangeExtractor: vi.fn(),
  onRegenerate: vi.fn(),
  onUserFeedback: vi.fn(),
  isRegenerating: false,
};

describe('ConfigEditor — form rendering', () => {
  it('renders indexName input with current value', () => {
    render(<ConfigEditor {...defaultProps} />);
    const input = screen.getByLabelText(/index name/i) as HTMLInputElement;
    expect(input.value).toBe('algoliacentral_marketing');
  });

  it('renders rateLimit input with current value', () => {
    render(<ConfigEditor {...defaultProps} />);
    const input = screen.getByLabelText(/rate limit/i) as HTMLInputElement;
    expect(input.value).toBe('8');
  });

  it('renders pathsToMatch input as comma-separated string', () => {
    render(
      <ConfigEditor
        {...defaultProps}
        crawlerConfig={{ ...config, pathsToMatch: ['/blog/*', '/news/*'] }}
      />
    );
    const input = screen.getByLabelText(/paths to match/i) as HTMLInputElement;
    expect(input.value).toBe('/blog/*, /news/*');
  });
});

• [ ] Step 2.2: Run, expect FAIL

npx vitest run src/components/admin/factory/__tests__/ConfigEditor.test.tsx

Expected: FAIL: Cannot find module '../ConfigEditor'.

• [ ] Step 2.3: Implement ConfigEditor.tsx

Create src/components/admin/factory/ConfigEditor.tsx:

/**
 * ConfigEditor — middle section of CategoryConfigurator.
 *
 * Renders the editable crawler config (index name, render-JS toggle,
 * rate limit, paths-to-match), the monospace recordExtractor textarea,
 * and the "Regenerate from samples" / free-form feedback controls.
 *
 * Render-only beyond change-handler emission. The parent owns state
 * and persistence — this component never fetches.
 */
import { useState, type KeyboardEvent } from 'react';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Switch } from '@/components/ui/switch';
import { Textarea } from '@/components/ui/textarea';
import { RefreshCw } from 'lucide-react';
import type { DssEntry } from '@/lib/factory/dss';
import type { CrawlerConfig, PathGroup } from './types';

export interface ConfigEditorProps {
  pathGroup: PathGroup;
  dssEntry: DssEntry;
  recordExtractor: string;
  crawlerConfig: CrawlerConfig;
  onChangeConfig: (next: CrawlerConfig) => void;
  onChangeExtractor: (next: string) => void;
  onRegenerate: () => void;
  onUserFeedback: (feedback: string) => void;
  isRegenerating: boolean;
}

export function ConfigEditor({
  pathGroup,
  dssEntry,
  recordExtractor,
  crawlerConfig,
  onChangeConfig,
  onChangeExtractor,
  onRegenerate,
  onUserFeedback,
  isRegenerating,
}: ConfigEditorProps) {
  const [feedback, setFeedback] = useState('');

  const handlePathsChange = (raw: string) => {
    const parts = raw
      .split(',')
      .map((s) => s.trim())
      .filter(Boolean);
    onChangeConfig({ ...crawlerConfig, pathsToMatch: parts });
  };

  const handleFeedbackKeyDown = (e: KeyboardEvent<HTMLInputElement>) => {
    if (e.key === 'Enter' && feedback.trim().length > 0) {
      e.preventDefault();
      onUserFeedback(feedback.trim());
      setFeedback('');
    }
  };

  return (
    <Card>
      <CardHeader>
        <CardTitle className="text-sm">
          Configure crawler — {dssEntry.indexName.replace('algoliacentral_', '')} domain
        </CardTitle>
      </CardHeader>
      <CardContent className="space-y-4">
        <ConfigForm
          config={crawlerConfig}
          onChangeConfig={onChangeConfig}
          onPathsChange={handlePathsChange}
        />
        <div>
          <Label htmlFor="record-extractor" className="text-xs">
            recordExtractor (editable)
          </Label>
          <Textarea
            id="record-extractor"
            data-testid="record-extractor"
            value={recordExtractor}
            onChange={(e) => onChangeExtractor(e.target.value)}
            rows={24}
            className="font-mono text-xs"
            spellCheck={false}
          />
        </div>
        <div className="flex items-center gap-2">
          <Button
            type="button"
            variant="outline"
            size="sm"
            onClick={onRegenerate}
            disabled={isRegenerating}
            data-testid="regenerate-button"
          >
            <RefreshCw className={`mr-2 h-3 w-3 ${isRegenerating ? 'animate-spin' : ''}`} />
            Regenerate from samples
          </Button>
          <Input
            placeholder="Free-form feedback to AI (Enter to regenerate)"
            value={feedback}
            onChange={(e) => setFeedback(e.target.value)}
            onKeyDown={handleFeedbackKeyDown}
            data-testid="feedback-input"
            className="text-xs"
          />
        </div>
        <p className="text-[11px] text-muted-foreground">
          pathGroup: <span className="font-mono">{pathGroup.pattern}</span>
        </p>
      </CardContent>
    </Card>
  );
}

function ConfigForm({
  config,
  onChangeConfig,
  onPathsChange,
}: {
  config: CrawlerConfig;
  onChangeConfig: (next: CrawlerConfig) => void;
  onPathsChange: (raw: string) => void;
}) {
  return (
    <div className="grid grid-cols-2 gap-3">
      <div>
        <Label htmlFor="index-name" className="text-xs">
          Index name
        </Label>
        <Input
          id="index-name"
          value={config.indexName}
          onChange={(e) => onChangeConfig({ ...config, indexName: e.target.value })}
          className="font-mono text-xs"
        />
      </div>
      <div>
        <Label htmlFor="rate-limit" className="text-xs">
          Rate limit
        </Label>
        <Input
          id="rate-limit"
          type="number"
          min={1}
          value={config.rateLimit}
          onChange={(e) =>
            onChangeConfig({ ...config, rateLimit: Number(e.target.value) || 1 })
          }
          className="text-xs"
        />
      </div>
      <div className="col-span-2">
        <Label htmlFor="paths-to-match" className="text-xs">
          Paths to match (comma-separated)
        </Label>
        <Input
          id="paths-to-match"
          value={config.pathsToMatch.join(', ')}
          onChange={(e) => onPathsChange(e.target.value)}
          className="font-mono text-xs"
        />
      </div>
      <div className="col-span-2 flex items-center gap-2">
        <Switch
          id="render-js"
          checked={config.renderJavaScript}
          onCheckedChange={(v) => onChangeConfig({ ...config, renderJavaScript: v })}
          data-testid="render-js-switch"
        />
        <Label htmlFor="render-js" className="text-xs">
          renderJavaScript (Playwright)
        </Label>
      </div>
    </div>
  );
}

• [ ] Step 2.4: Run form-render tests, expect PASS

npx vitest run src/components/admin/factory/__tests__/ConfigEditor.test.tsx

Expected: 3 tests pass.

• [ ] Step 2.5: Add tests for change handlers, regenerate, feedback Enter

Append to src/components/admin/factory/__tests__/ConfigEditor.test.tsx:

describe('ConfigEditor — handlers', () => {
  it('calls onChangeConfig when indexName changes', () => {
    const onChangeConfig = vi.fn();
    render(<ConfigEditor {...defaultProps} onChangeConfig={onChangeConfig} />);
    const input = screen.getByLabelText(/index name/i) as HTMLInputElement;
    fireEvent.change(input, { target: { value: 'algoliacentral_blog' } });
    expect(onChangeConfig).toHaveBeenCalledWith(
      expect.objectContaining({ indexName: 'algoliacentral_blog' })
    );
  });

  it('calls onChangeConfig with parsed pathsToMatch on comma input', () => {
    const onChangeConfig = vi.fn();
    render(<ConfigEditor {...defaultProps} onChangeConfig={onChangeConfig} />);
    const input = screen.getByLabelText(/paths to match/i) as HTMLInputElement;
    fireEvent.change(input, { target: { value: '/blog/*, /news/*, /about' } });
    expect(onChangeConfig).toHaveBeenCalledWith(
      expect.objectContaining({ pathsToMatch: ['/blog/*', '/news/*', '/about'] })
    );
  });

  it('calls onChangeExtractor when textarea changes', () => {
    const onChangeExtractor = vi.fn();
    render(<ConfigEditor {...defaultProps} onChangeExtractor={onChangeExtractor} />);
    const ta = screen.getByTestId('record-extractor');
    fireEvent.change(ta, { target: { value: 'function newExtractor(){}' } });
    expect(onChangeExtractor).toHaveBeenCalledWith('function newExtractor(){}');
  });

  it('calls onRegenerate when Regenerate button clicked', () => {
    const onRegenerate = vi.fn();
    render(<ConfigEditor {...defaultProps} onRegenerate={onRegenerate} />);
    fireEvent.click(screen.getByTestId('regenerate-button'));
    expect(onRegenerate).toHaveBeenCalledTimes(1);
  });

  it('disables Regenerate button while isRegenerating=true', () => {
    render(<ConfigEditor {...defaultProps} isRegenerating={true} />);
    expect(screen.getByTestId('regenerate-button')).toBeDisabled();
  });

  it('calls onUserFeedback with trimmed feedback on Enter, then clears the input', () => {
    const onUserFeedback = vi.fn();
    render(<ConfigEditor {...defaultProps} onUserFeedback={onUserFeedback} />);
    const input = screen.getByTestId('feedback-input') as HTMLInputElement;
    fireEvent.change(input, { target: { value: '  use h2 not h1  ' } });
    fireEvent.keyDown(input, { key: 'Enter' });
    expect(onUserFeedback).toHaveBeenCalledWith('use h2 not h1');
    expect(input.value).toBe('');
  });

  it('does not call onUserFeedback when feedback is empty', () => {
    const onUserFeedback = vi.fn();
    render(<ConfigEditor {...defaultProps} onUserFeedback={onUserFeedback} />);
    const input = screen.getByTestId('feedback-input') as HTMLInputElement;
    fireEvent.keyDown(input, { key: 'Enter' });
    expect(onUserFeedback).not.toHaveBeenCalled();
  });
});

• [ ] Step 2.6: Run all ConfigEditor tests, expect PASS

npx vitest run src/components/admin/factory/__tests__/ConfigEditor.test.tsx

Expected: 10 tests pass.

• [ ] Step 2.7: Run tsc --noEmit

npx tsc --noEmit

Expected: clean.

• [ ] Step 2.8: Commit

git add src/components/admin/factory/ConfigEditor.tsx src/components/admin/factory/__tests__/ConfigEditor.test.tsx
git commit -m "feat(factory): ConfigEditor component (form + extractor textarea + regen)"

---

Task 3: TestCrawlResults: Sandbox/Real tabs with field-coverage bars

Files: - Create: src/components/admin/factory/TestCrawlResults.tsx - Create: src/components/admin/factory/__tests__/TestCrawlResults.test.tsx

• [ ] Step 3.1: Write failing test: empty state, both tabs

Create src/components/admin/factory/__tests__/TestCrawlResults.test.tsx:

import { describe, it, expect, vi } from 'vitest';
import { render, screen, fireEvent } from '@testing-library/react';
import { TestCrawlResults } from '../TestCrawlResults';
import type { SandboxResult, RealResult } from '../types';

const dssRequiredFields = ['headline', 'articleBody'];

const baseProps = {
  sandboxResults: [] as SandboxResult[],
  realResults: [] as RealResult[],
  dssRequiredFields,
  isRunningSandbox: false,
  isRunningReal: false,
  onRunSandbox: vi.fn(),
  onRunReal: vi.fn(),
};

describe('TestCrawlResults — empty', () => {
  it('renders the Sandbox tab with empty-state message', () => {
    render(<TestCrawlResults {...baseProps} />);
    expect(screen.getByText(/no sandbox runs yet/i)).toBeInTheDocument();
  });

  it('renders both Sandbox and Real tab triggers', () => {
    render(<TestCrawlResults {...baseProps} />);
    expect(screen.getByRole('tab', { name: /sandbox/i })).toBeInTheDocument();
    expect(screen.getByRole('tab', { name: /real/i })).toBeInTheDocument();
  });

  it('calls onRunSandbox when "Run sandbox test" clicked', () => {
    const onRunSandbox = vi.fn();
    render(<TestCrawlResults {...baseProps} onRunSandbox={onRunSandbox} />);
    fireEvent.click(screen.getByTestId('run-sandbox-button'));
    expect(onRunSandbox).toHaveBeenCalledTimes(1);
  });
});

• [ ] Step 3.2: Run, expect FAIL

npx vitest run src/components/admin/factory/__tests__/TestCrawlResults.test.tsx

Expected: FAIL: module not found.

• [ ] Step 3.3: Implement TestCrawlResults.tsx

Create src/components/admin/factory/TestCrawlResults.tsx:

/**
 * TestCrawlResults — bottom section of CategoryConfigurator.
 *
 * Two tabs: Sandbox (cheerio + zod local validation, fast) and Real
 * (live Algolia test crawl, costs 1 of 500/day). Each tab shows
 * per-sample rows with success/fail, record count, field-coverage
 * percentage bars (one per DSS-required field), and a collapsible
 * "first record JSON" expander.
 *
 * Why "first record" only: a sample URL may produce multiple records
 * (e.g. a single doc page yielding chunked sections). Showing the
 * first is the cheapest way to confirm shape; full inspection is for
 * Algolia dashboard.
 */
import { useState } from 'react';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import {
  Collapsible,
  CollapsibleContent,
  CollapsibleTrigger,
} from '@/components/ui/collapsible';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { Check, ChevronDown, ChevronRight, Play, X as XIcon } from 'lucide-react';
import type { RealResult, SandboxResult } from './types';

export interface TestCrawlResultsProps {
  sandboxResults: SandboxResult[];
  realResults: RealResult[];
  dssRequiredFields: string[];
  isRunningSandbox: boolean;
  isRunningReal: boolean;
  onRunSandbox: () => void;
  onRunReal: () => void;
}

export function TestCrawlResults({
  sandboxResults,
  realResults,
  dssRequiredFields,
  isRunningSandbox,
  isRunningReal,
  onRunSandbox,
  onRunReal,
}: TestCrawlResultsProps) {
  return (
    <Card>
      <CardHeader>
        <CardTitle className="text-sm">Test crawl results</CardTitle>
      </CardHeader>
      <CardContent>
        <Tabs defaultValue="sandbox">
          <TabsList>
            <TabsTrigger value="sandbox">Sandbox</TabsTrigger>
            <TabsTrigger value="real">Real</TabsTrigger>
          </TabsList>
          <TabsContent value="sandbox">
            <ResultsList
              results={sandboxResults}
              dssRequiredFields={dssRequiredFields}
              emptyMessage="No sandbox runs yet."
              testIdPrefix="sandbox"
            />
            <Button
              type="button"
              size="sm"
              variant="outline"
              className="mt-2"
              onClick={onRunSandbox}
              disabled={isRunningSandbox}
              data-testid="run-sandbox-button"
            >
              <Play className="mr-2 h-3 w-3" />
              {isRunningSandbox ? 'Running…' : 'Run sandbox test'}
            </Button>
          </TabsContent>
          <TabsContent value="real">
            <ResultsList
              results={realResults}
              dssRequiredFields={dssRequiredFields}
              emptyMessage="No real-crawl runs yet."
              testIdPrefix="real"
            />
            <Button
              type="button"
              size="sm"
              variant="outline"
              className="mt-2"
              onClick={onRunReal}
              disabled={isRunningReal}
              data-testid="run-real-button"
            >
              <Play className="mr-2 h-3 w-3" />
              {isRunningReal ? 'Running…' : 'Run REAL test crawl (uses 1 of 500/day)'}
            </Button>
          </TabsContent>
        </Tabs>
      </CardContent>
    </Card>
  );
}

function ResultsList({
  results,
  dssRequiredFields,
  emptyMessage,
  testIdPrefix,
}: {
  results: Array<SandboxResult | RealResult>;
  dssRequiredFields: string[];
  emptyMessage: string;
  testIdPrefix: string;
}) {
  if (results.length === 0) {
    return <p className="text-sm text-muted-foreground">{emptyMessage}</p>;
  }
  return (
    <div className="space-y-2" data-testid={`${testIdPrefix}-results`}>
      {results.map((r) => (
        <ResultRow
          key={r.url}
          result={r}
          dssRequiredFields={dssRequiredFields}
          testIdPrefix={testIdPrefix}
        />
      ))}
    </div>
  );
}

function ResultRow({
  result,
  dssRequiredFields,
  testIdPrefix,
}: {
  result: SandboxResult | RealResult;
  dssRequiredFields: string[];
  testIdPrefix: string;
}) {
  const [open, setOpen] = useState(false);
  const ok = result.records.length > 0 && result.errors.length === 0;
  return (
    <div className="rounded border p-2">
      <div className="flex items-center gap-2 text-xs">
        {ok ? (
          <Check className="h-3 w-3 text-emerald-600" data-testid={`${testIdPrefix}-row-ok`} />
        ) : (
          <XIcon className="h-3 w-3 text-red-600" data-testid={`${testIdPrefix}-row-fail`} />
        )}
        <span className="font-mono">{result.url}</span>
        <span className="ml-auto text-muted-foreground">
          {result.records.length} record{result.records.length === 1 ? '' : 's'}
        </span>
      </div>
      <FieldCoverageBars
        coverage={result.fieldCoverage}
        requiredFields={dssRequiredFields}
        testIdPrefix={testIdPrefix}
      />
      {result.records.length > 0 && (
        <Collapsible open={open} onOpenChange={setOpen}>
          <CollapsibleTrigger
            className="mt-1 flex items-center gap-1 text-[11px] hover:underline"
            data-testid={`${testIdPrefix}-record-trigger`}
          >
            {open ? <ChevronDown className="h-3 w-3" /> : <ChevronRight className="h-3 w-3" />}
            <span>show first record JSON</span>
          </CollapsibleTrigger>
          <CollapsibleContent>
            <pre className="mt-1 max-h-64 overflow-auto rounded bg-muted p-2 font-mono text-[11px]">
              {JSON.stringify(result.records[0], null, 2)}
            </pre>
          </CollapsibleContent>
        </Collapsible>
      )}
      {result.errors.length > 0 && (
        <ul className="mt-1 list-inside list-disc text-[11px] text-red-600">
          {result.errors.map((e, i) => (
            <li key={i}>{e}</li>
          ))}
        </ul>
      )}
    </div>
  );
}

function FieldCoverageBars({
  coverage,
  requiredFields,
  testIdPrefix,
}: {
  coverage: Record<string, number>;
  requiredFields: string[];
  testIdPrefix: string;
}) {
  return (
    <div className="mt-1 space-y-0.5">
      {requiredFields.map((field) => {
        const pctRaw = coverage[field] ?? 0;
        const pct = Math.max(0, Math.min(1, pctRaw));
        const pctLabel = `${Math.round(pct * 100)}%`;
        return (
          <div
            key={field}
            className="flex items-center gap-2 text-[11px]"
            data-testid={`${testIdPrefix}-coverage-${field}`}
          >
            <span className="w-24 truncate text-muted-foreground">{field}</span>
            <div className="h-2 flex-1 overflow-hidden rounded bg-muted">
              <div
                className="h-full bg-emerald-500"
                style={{ width: pctLabel }}
                data-testid={`${testIdPrefix}-coverage-bar-${field}`}
              />
            </div>
            <span className="w-10 text-right font-mono">{pctLabel}</span>
          </div>
        );
      })}
    </div>
  );
}

• [ ] Step 3.4: Run empty-state tests, expect PASS

npx vitest run src/components/admin/factory/__tests__/TestCrawlResults.test.tsx

Expected: 3 tests pass.

• [ ] Step 3.5: Add tests for results rendering, coverage bars, JSON expand

Append to src/components/admin/factory/__tests__/TestCrawlResults.test.tsx:

const sandboxOK: SandboxResult = {
  url: 'https://x.com/blog/post-1',
  records: [{ headline: 'Hello', articleBody: 'A body' }],
  errors: [],
  fieldCoverage: { headline: 1.0, articleBody: 0.5 },
};

const sandboxFail: SandboxResult = {
  url: 'https://x.com/blog/post-2',
  records: [],
  errors: ['articleBody too short'],
  fieldCoverage: { headline: 0, articleBody: 0 },
};

describe('TestCrawlResults — populated sandbox', () => {
  it('renders one row per result', () => {
    render(
      <TestCrawlResults
        {...baseProps}
        sandboxResults={[sandboxOK, sandboxFail]}
      />
    );
    expect(screen.getByText('https://x.com/blog/post-1')).toBeInTheDocument();
    expect(screen.getByText('https://x.com/blog/post-2')).toBeInTheDocument();
  });

  it('renders ✓ for ok rows and ✗ for fail rows', () => {
    render(
      <TestCrawlResults
        {...baseProps}
        sandboxResults={[sandboxOK, sandboxFail]}
      />
    );
    expect(screen.getByTestId('sandbox-row-ok')).toBeInTheDocument();
    expect(screen.getByTestId('sandbox-row-fail')).toBeInTheDocument();
  });

  it('renders field-coverage bars with correct percentage widths', () => {
    render(<TestCrawlResults {...baseProps} sandboxResults={[sandboxOK]} />);
    const headlineBar = screen.getByTestId('sandbox-coverage-bar-headline');
    const bodyBar = screen.getByTestId('sandbox-coverage-bar-articleBody');
    expect(headlineBar).toHaveStyle({ width: '100%' });
    expect(bodyBar).toHaveStyle({ width: '50%' });
  });

  it('clamps a >1 coverage to 100% (defensive)', () => {
    const result: SandboxResult = {
      ...sandboxOK,
      fieldCoverage: { headline: 2.0, articleBody: 1.0 },
    };
    render(<TestCrawlResults {...baseProps} sandboxResults={[result]} />);
    expect(screen.getByTestId('sandbox-coverage-bar-headline')).toHaveStyle({ width: '100%' });
  });

  it('expands first record JSON on trigger click', () => {
    render(<TestCrawlResults {...baseProps} sandboxResults={[sandboxOK]} />);
    fireEvent.click(screen.getByTestId('sandbox-record-trigger'));
    expect(screen.getByText(/"headline": "Hello"/)).toBeInTheDocument();
  });

  it('shows error messages when records empty + errors present', () => {
    render(<TestCrawlResults {...baseProps} sandboxResults={[sandboxFail]} />);
    expect(screen.getByText(/articleBody too short/i)).toBeInTheDocument();
  });
});

describe('TestCrawlResults — real tab', () => {
  it('switches to Real tab and shows empty-state', () => {
    render(<TestCrawlResults {...baseProps} sandboxResults={[sandboxOK]} />);
    fireEvent.click(screen.getByRole('tab', { name: /real/i }));
    expect(screen.getByText(/no real-crawl runs yet/i)).toBeInTheDocument();
  });

  it('calls onRunReal when "Run REAL test crawl" clicked', () => {
    const onRunReal = vi.fn();
    render(<TestCrawlResults {...baseProps} onRunReal={onRunReal} />);
    fireEvent.click(screen.getByRole('tab', { name: /real/i }));
    fireEvent.click(screen.getByTestId('run-real-button'));
    expect(onRunReal).toHaveBeenCalledTimes(1);
  });

  it('disables Run buttons during running', () => {
    render(
      <TestCrawlResults
        {...baseProps}
        isRunningSandbox={true}
        isRunningReal={true}
      />
    );
    expect(screen.getByTestId('run-sandbox-button')).toBeDisabled();
    fireEvent.click(screen.getByRole('tab', { name: /real/i }));
    expect(screen.getByTestId('run-real-button')).toBeDisabled();
  });
});

• [ ] Step 3.6: Run all TestCrawlResults tests, expect PASS

npx vitest run src/components/admin/factory/__tests__/TestCrawlResults.test.tsx

Expected: 12 tests pass.

• [ ] Step 3.7: Run tsc --noEmit

npx tsc --noEmit

Expected: clean.

• [ ] Step 3.8: Commit

git add src/components/admin/factory/TestCrawlResults.tsx src/components/admin/factory/__tests__/TestCrawlResults.test.tsx
git commit -m "feat(factory): TestCrawlResults component (sandbox/real tabs + coverage bars)"

---

Task 4: CategoryConfigurator: drawer shell, fetches, commit gating

Files: - Create: src/components/admin/factory/CategoryConfigurator.tsx - Create: src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx

The drawer is the only component in this cluster that talks to the network. It owns: - open lifecycle (parent-controlled via open prop) - Initial sample fetch (POST /api/factory/sample) - Initial extractor generation (POST /api/factory/generate-extractor): runs immediately after samples land - Regeneration (with optional user feedback): re-fires POST /api/factory/generate-extractor - Sandbox run (POST /api/factory/test-sandbox) - Real run (POST /api/factory/test-real) - Commit handler: calls onCommit({ recordExtractor, crawlerConfig }) upward; gated until both sandbox + real are green for this pathGroup

Per §19e the Commit CTA is never silently auto-fired: it is enabled when conditions are met but always requires explicit user click.

• [ ] Step 4.1: Write failing test: drawer renders title and closed-state

Create src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx:

import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { render, screen, waitFor } from '@testing-library/react';
import { CategoryConfigurator } from '../CategoryConfigurator';
import type { PathGroup } from '../types';

const pathGroup: PathGroup = {
  id: 'pg_1',
  pattern: '/blog/*',
  urlCount: 412,
  sampleUrls: [],
  detectedDomain: 'marketing',
  detectionConfidence: 0.92,
  detectionMethod: 'cms',
  selected: true,
};

const defaultProps = {
  sessionId: 'sess_001',
  pathGroupId: 'pg_1',
  pathGroup,
  contentDomain: 'marketing' as const,
  open: true,
  onClose: vi.fn(),
  onCommit: vi.fn(),
};

const sampleResponse = {
  samples: [
    {
      url: 'https://x.com/blog/post-1',
      urlHash: 'a1b2c3d4e5f6a7b8',
      status: 200,
      html: '<html></html>',
      transport: 'fetch',
      truncated: false,
      contentLength: 13,
    },
  ],
};

const extractorResponse = {
  recordExtractor: 'function extractor(){ return [{}]; }',
  crawlerConfig: {
    indexName: 'algoliacentral_marketing',
    renderJavaScript: false,
    rateLimit: 8,
    pathsToMatch: ['/blog/*'],
  },
  // analysis moved here from sampleResponse — generate-extractor returns it.
  analysis: {
    schemaOrgJsonLd: null,
    selectors: { headline: 'h1' },
    missing: [],
    confidence: 0.9,
  },
};

function mockFetchSequence(sequence: Array<{ ok?: boolean; status?: number; json: unknown }>) {
  const fn = vi.fn();
  for (const item of sequence) {
    fn.mockResolvedValueOnce({
      ok: item.ok ?? true,
      status: item.status ?? 200,
      statusText: 'OK',
      json: async () => item.json,
    });
  }
  return fn;
}

describe('CategoryConfigurator — closed', () => {
  it('renders nothing visible when open=false', () => {
    render(<CategoryConfigurator {...defaultProps} open={false} />);
    expect(screen.queryByText(/marketing \| \/blog\/\*/i)).not.toBeInTheDocument();
  });
});

• [ ] Step 4.2: Run, expect FAIL

npx vitest run src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx

Expected: FAIL: module not found.

• [ ] Step 4.3: Implement CategoryConfigurator.tsx

Create src/components/admin/factory/CategoryConfigurator.tsx:

/**
 * CategoryConfigurator — per-pathGroup drawer.
 *
 * Render shape: shadcn Sheet, side='right', 70% viewport width. Title
 * shows `{contentDomain} | {pathGroup.pattern}`. Body composes
 * SamplePreview, ConfigEditor, TestCrawlResults vertically. Footer
 * holds three CTAs: Run sandbox / Run REAL / Commit + Create Crawler.
 *
 * Workflow (per master plan §5b + §19e — human-in-the-loop, no silent commits):
 *   1. On open → POST /api/factory/sample → POST /api/factory/generate-extractor.
 *   2. User edits crawlerConfig + recordExtractor as desired.
 *   3. User clicks "Run sandbox test" → POST /api/factory/test-sandbox.
 *   4. User clicks "Run REAL test crawl" → POST /api/factory/test-real.
 *   5. Commit becomes enabled only when both sandbox and real results
 *      are green for this pathGroup. User must explicitly click Commit.
 */
import { useEffect, useState, useCallback } from 'react';
import { Button } from '@/components/ui/button';
import {
  Sheet,
  SheetContent,
  SheetDescription,
  SheetFooter,
  SheetHeader,
  SheetTitle,
} from '@/components/ui/sheet';
import { useToast } from '@/components/ui/use-toast';
import { getDss } from '@/lib/factory/dss';
import { ConfigEditor } from './ConfigEditor';
import { SamplePreview } from './SamplePreview';
import { TestCrawlResults } from './TestCrawlResults';
import type {
  ContentDomain,
  CrawlerConfig,
  PathGroup,
  RealResult,
  Sample,
  SandboxResult,
  StructureAnalysis,
} from './types';

export interface CategoryConfiguratorProps {
  sessionId: string;
  pathGroupId: string;
  pathGroup: PathGroup;
  contentDomain: ContentDomain;
  open: boolean;
  onClose: () => void;
  onCommit: (payload: {
    recordExtractor: string;
    crawlerConfig: CrawlerConfig;
  }) => void;
}

const EMPTY_ANALYSIS: StructureAnalysis = {
  schemaOrgJsonLd: null,
  selectors: {},
  missing: [],
  confidence: 0,
};

export function CategoryConfigurator(props: CategoryConfiguratorProps) {
  const {
    sessionId,
    pathGroupId,
    pathGroup,
    contentDomain,
    open,
    onClose,
    onCommit,
  } = props;
  const dssEntry = getDss(contentDomain);
  const { toast } = useToast();

  const [samples, setSamples] = useState<Sample[]>([]);
  const [analysis, setAnalysis] = useState<StructureAnalysis>(EMPTY_ANALYSIS);
  const [recordExtractor, setRecordExtractor] = useState<string>('');
  const [crawlerConfig, setCrawlerConfig] = useState<CrawlerConfig>({
    indexName: dssEntry.indexName,
    renderJavaScript: false,
    rateLimit: 8,
    pathsToMatch: [pathGroup.pattern],
  });
  const [sandboxResults, setSandboxResults] = useState<SandboxResult[]>([]);
  const [realResults, setRealResults] = useState<RealResult[]>([]);
  const [loadingInitial, setLoadingInitial] = useState(false);
  const [isRegenerating, setIsRegenerating] = useState(false);
  const [isRunningSandbox, setIsRunningSandbox] = useState(false);
  const [isRunningReal, setIsRunningReal] = useState(false);

  const dssRequiredFields = dssEntry.algoliaConfig.searchableAttributes.slice();

  const fetchExtractor = useCallback(
    async (userFeedback: string | null) => {
      try {
        const res = await fetch('/api/factory/generate-extractor', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({
            sessionId,
            pathGroupId,
            ...(userFeedback ? { userFeedback } : {}),
          }),
        });
        if (!res.ok) {
          const body = await safeJson(res);
          throw new Error(body?.error ?? res.statusText);
        }
        // Spec 08 generate-extractor returns analysis alongside extractor +
        // config so SamplePreview can render selectors without a second call.
        const data = (await res.json()) as {
          recordExtractor: string;
          crawlerConfig: CrawlerConfig;
          analysis: StructureAnalysis;
        };
        setRecordExtractor(data.recordExtractor);
        setCrawlerConfig(data.crawlerConfig);
        setAnalysis(data.analysis);
      } catch (err) {
        console.error('[CategoryConfigurator] generate-extractor failed', err);
        toast({
          variant: 'destructive',
          title: 'Extractor generation failed',
          description: String(err),
        });
      }
    },
    [sessionId, pathGroupId, toast]
  );

  // On open: fetch samples then extractor.
  useEffect(() => {
    if (!open) return;
    let cancelled = false;
    async function run() {
      setLoadingInitial(true);
      try {
        const res = await fetch('/api/factory/sample', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ sessionId, pathGroupId, n: 5 }),
        });
        if (!res.ok) {
          const body = await safeJson(res);
          throw new Error(body?.error ?? res.statusText);
        }
        // /api/factory/sample returns just samples — analysis comes back with
        // /api/factory/generate-extractor (called next) so SamplePreview renders
        // detected structure alongside the generated extractor.
        const data = (await res.json()) as { samples: Sample[] };
        if (cancelled) return;
        setSamples(data.samples);
        await fetchExtractor(null);
      } catch (err) {
        console.error('[CategoryConfigurator] sample fetch failed', err);
        toast({
          variant: 'destructive',
          title: 'Sample fetch failed',
          description: String(err),
        });
      } finally {
        if (!cancelled) setLoadingInitial(false);
      }
    }
    run();
    return () => {
      cancelled = true;
    };
  }, [open, sessionId, pathGroupId, fetchExtractor, toast]);

  const handleRegenerate = useCallback(async () => {
    setIsRegenerating(true);
    try {
      await fetchExtractor(null);
    } finally {
      setIsRegenerating(false);
    }
  }, [fetchExtractor]);

  const handleUserFeedback = useCallback(
    async (feedback: string) => {
      setIsRegenerating(true);
      try {
        await fetchExtractor(feedback);
      } finally {
        setIsRegenerating(false);
      }
    },
    [fetchExtractor]
  );

  const handleRunSandbox = useCallback(async () => {
    setIsRunningSandbox(true);
    try {
      const res = await fetch('/api/factory/test-sandbox', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          sessionId,
          pathGroupId,
          recordExtractor,
          crawlerConfig,
        }),
      });
      if (!res.ok) {
        const body = await safeJson(res);
        throw new Error(body?.error ?? res.statusText);
      }
      const data = (await res.json()) as { results: SandboxResult[] };
      setSandboxResults(data.results);
    } catch (err) {
      console.error('[CategoryConfigurator] sandbox failed', err);
      toast({
        variant: 'destructive',
        title: 'Sandbox run failed',
        description: String(err),
      });
    } finally {
      setIsRunningSandbox(false);
    }
  }, [sessionId, pathGroupId, recordExtractor, crawlerConfig, toast]);

  const handleRunReal = useCallback(async () => {
    setIsRunningReal(true);
    try {
      const res = await fetch('/api/factory/test-real', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          sessionId,
          pathGroupId,
          recordExtractor,
          crawlerConfig,
        }),
      });
      if (!res.ok) {
        const body = await safeJson(res);
        throw new Error(body?.error ?? res.statusText);
      }
      const data = (await res.json()) as { results: RealResult[] };
      setRealResults(data.results);
    } catch (err) {
      console.error('[CategoryConfigurator] real-crawl failed', err);
      toast({
        variant: 'destructive',
        title: 'Real test crawl failed',
        description: String(err),
      });
    } finally {
      setIsRunningReal(false);
    }
  }, [sessionId, pathGroupId, recordExtractor, crawlerConfig, toast]);

  const sandboxAllGreen =
    sandboxResults.length > 0 &&
    sandboxResults.every((r) => r.records.length > 0 && r.errors.length === 0);
  const realAllGreen =
    realResults.length > 0 &&
    realResults.every((r) => r.records.length > 0 && r.errors.length === 0);
  const commitEnabled = sandboxAllGreen && realAllGreen && !loadingInitial;

  const handleCommit = useCallback(() => {
    if (!commitEnabled) return;
    onCommit({ recordExtractor, crawlerConfig });
  }, [commitEnabled, onCommit, recordExtractor, crawlerConfig]);

  const handleOpenChange = (next: boolean) => {
    if (!next) onClose();
  };

  return (
    <Sheet open={open} onOpenChange={handleOpenChange}>
      <SheetContent
        side="right"
        className="w-[70vw] max-w-[70vw] overflow-y-auto sm:max-w-[70vw]"
        data-testid="category-configurator"
      >
        <SheetHeader>
          <SheetTitle>
            <span className="font-mono">{contentDomain}</span>
            {' | '}
            <span className="font-mono">{pathGroup.pattern}</span>
          </SheetTitle>
          <SheetDescription>
            {pathGroup.urlCount.toLocaleString()} URLs · review samples, edit extractor,
            run tests, then commit.
          </SheetDescription>
        </SheetHeader>

        <div className="mt-4 space-y-4">
          <SamplePreview samples={samples} structureAnalysis={analysis} />
          <ConfigEditor
            pathGroup={pathGroup}
            dssEntry={dssEntry}
            recordExtractor={recordExtractor}
            crawlerConfig={crawlerConfig}
            onChangeConfig={setCrawlerConfig}
            onChangeExtractor={setRecordExtractor}
            onRegenerate={handleRegenerate}
            onUserFeedback={handleUserFeedback}
            isRegenerating={isRegenerating}
          />
          <TestCrawlResults
            sandboxResults={sandboxResults}
            realResults={realResults}
            dssRequiredFields={dssRequiredFields}
            isRunningSandbox={isRunningSandbox}
            isRunningReal={isRunningReal}
            onRunSandbox={handleRunSandbox}
            onRunReal={handleRunReal}
          />
        </div>

        <SheetFooter className="mt-4 flex-col gap-2 sm:flex-row">
          <Button
            type="button"
            variant="outline"
            size="sm"
            onClick={handleRunSandbox}
            disabled={isRunningSandbox || loadingInitial}
            data-testid="footer-run-sandbox"
          >
            Run sandbox test
          </Button>
          <Button
            type="button"
            variant="outline"
            size="sm"
            onClick={handleRunReal}
            disabled={isRunningReal || loadingInitial || !sandboxAllGreen}
            data-testid="footer-run-real"
          >
            Run REAL test crawl
          </Button>
          <Button
            type="button"
            size="sm"
            onClick={handleCommit}
            disabled={!commitEnabled}
            data-testid="footer-commit"
          >
            Commit + Create Crawler
          </Button>
        </SheetFooter>
      </SheetContent>
    </Sheet>
  );
}

async function safeJson(res: Response): Promise<{ error?: string } | null> {
  try {
    return (await res.json()) as { error?: string };
  } catch {
    return null;
  }
}

• [ ] Step 4.4: Run the closed-state test, expect PASS

npx vitest run src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx

Expected: 1 test passes.

• [ ] Step 4.5: Add test: drawer fires sample + extractor fetches on open

Append to src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx:

describe('CategoryConfigurator — open lifecycle', () => {
  beforeEach(() => {
    vi.useFakeTimers({ toFake: ['setTimeout'] });
  });
  afterEach(() => {
    vi.useRealTimers();
    vi.restoreAllMocks();
  });

  it('renders title with contentDomain | pattern', async () => {
    global.fetch = mockFetchSequence([
      { json: sampleResponse },
      { json: extractorResponse },
    ]) as unknown as typeof fetch;
    render(<CategoryConfigurator {...defaultProps} />);
    await waitFor(() => {
      expect(screen.getByText('marketing')).toBeInTheDocument();
      expect(screen.getByText('/blog/*')).toBeInTheDocument();
    });
  });

  it('fires POST /api/factory/sample then POST /api/factory/generate-extractor on open', async () => {
    const fetchMock = mockFetchSequence([
      { json: sampleResponse },
      { json: extractorResponse },
    ]);
    global.fetch = fetchMock as unknown as typeof fetch;
    render(<CategoryConfigurator {...defaultProps} />);
    await waitFor(() => {
      expect(fetchMock).toHaveBeenCalledTimes(2);
    });
    expect(fetchMock.mock.calls[0][0]).toBe('/api/factory/sample');
    expect(fetchMock.mock.calls[1][0]).toBe('/api/factory/generate-extractor');
    const samplesBody = JSON.parse((fetchMock.mock.calls[0][1] as RequestInit).body as string);
    expect(samplesBody).toMatchObject({
      sessionId: 'sess_001',
      pathGroupId: 'pg_1',
      n: 5,
    });
  });
});

• [ ] Step 4.6: Run, expect PASS

npx vitest run src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx

Expected: 3 tests pass.

• [ ] Step 4.7: Add test: regenerate-with-feedback re-fetches extractor with feedback in body

Append:

import { fireEvent } from '@testing-library/react';

describe('CategoryConfigurator — regenerate with feedback', () => {
  afterEach(() => vi.restoreAllMocks());

  it('re-fetches extractor with userFeedback in POST body when feedback submitted via Enter', async () => {
    const fetchMock = mockFetchSequence([
      { json: sampleResponse },
      { json: extractorResponse },
      { json: extractorResponse },
    ]);
    global.fetch = fetchMock as unknown as typeof fetch;
    render(<CategoryConfigurator {...defaultProps} />);
    // Wait for initial fetches
    await waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));

    const feedback = await screen.findByTestId('feedback-input');
    fireEvent.change(feedback, { target: { value: 'use h2 not h1' } });
    fireEvent.keyDown(feedback, { key: 'Enter' });

    await waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(3));
    expect(fetchMock.mock.calls[2][0]).toBe('/api/factory/generate-extractor');
    const body = JSON.parse((fetchMock.mock.calls[2][1] as RequestInit).body as string);
    expect(body.userFeedback).toBe('use h2 not h1');
  });
});

• [ ] Step 4.8: Run, expect PASS

npx vitest run src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx

Expected: 4 tests pass.

• [ ] Step 4.9: Add tests: Commit button gating (disabled until sandbox + real green)

Append:

describe('CategoryConfigurator — commit gating', () => {
  afterEach(() => vi.restoreAllMocks());

  const sandboxGreen = {
    results: [
      {
        url: 'https://x.com/blog/post-1',
        records: [{ headline: 'Hi', articleBody: 'A body' }],
        errors: [],
        fieldCoverage: { headline: 1.0, articleBody: 1.0 },
      },
    ],
  };
  const realGreen = {
    results: [
      {
        url: 'https://x.com/blog/post-1',
        records: [{ headline: 'Hi', articleBody: 'A body' }],
        errors: [],
        fieldCoverage: { headline: 1.0, articleBody: 1.0 },
      },
    ],
  };

  it('Commit button is disabled on initial open (no sandbox + no real)', async () => {
    global.fetch = mockFetchSequence([
      { json: sampleResponse },
      { json: extractorResponse },
    ]) as unknown as typeof fetch;
    render(<CategoryConfigurator {...defaultProps} />);
    const commit = await screen.findByTestId('footer-commit');
    expect(commit).toBeDisabled();
  });

  it('Commit stays disabled when only sandbox is green', async () => {
    global.fetch = mockFetchSequence([
      { json: sampleResponse },
      { json: extractorResponse },
      { json: sandboxGreen },
    ]) as unknown as typeof fetch;
    render(<CategoryConfigurator {...defaultProps} />);
    await screen.findByTestId('footer-commit');
    fireEvent.click(screen.getByTestId('footer-run-sandbox'));
    await waitFor(() => {
      expect(screen.getByTestId('footer-commit')).toBeDisabled();
    });
  });

  it('Commit becomes enabled when both sandbox + real are green; clicking calls onCommit', async () => {
    global.fetch = mockFetchSequence([
      { json: sampleResponse },
      { json: extractorResponse },
      { json: sandboxGreen },
      { json: realGreen },
    ]) as unknown as typeof fetch;
    const onCommit = vi.fn();
    render(<CategoryConfigurator {...defaultProps} onCommit={onCommit} />);
    await screen.findByTestId('footer-commit');

    fireEvent.click(screen.getByTestId('footer-run-sandbox'));
    await waitFor(() => {
      expect(screen.getByTestId('footer-run-real')).not.toBeDisabled();
    });
    fireEvent.click(screen.getByTestId('footer-run-real'));
    await waitFor(() => {
      expect(screen.getByTestId('footer-commit')).not.toBeDisabled();
    });

    fireEvent.click(screen.getByTestId('footer-commit'));
    expect(onCommit).toHaveBeenCalledWith(
      expect.objectContaining({
        recordExtractor: extractorResponse.recordExtractor,
        crawlerConfig: extractorResponse.crawlerConfig,
      })
    );
  });

  it('does not auto-commit even when both green — requires explicit click (§19e)', async () => {
    global.fetch = mockFetchSequence([
      { json: sampleResponse },
      { json: extractorResponse },
      { json: sandboxGreen },
      { json: realGreen },
    ]) as unknown as typeof fetch;
    const onCommit = vi.fn();
    render(<CategoryConfigurator {...defaultProps} onCommit={onCommit} />);
    await screen.findByTestId('footer-commit');
    fireEvent.click(screen.getByTestId('footer-run-sandbox'));
    await waitFor(() => {
      expect(screen.getByTestId('footer-run-real')).not.toBeDisabled();
    });
    fireEvent.click(screen.getByTestId('footer-run-real'));
    await waitFor(() => {
      expect(screen.getByTestId('footer-commit')).not.toBeDisabled();
    });
    // Drawer is fully green but Commit was never clicked.
    expect(onCommit).not.toHaveBeenCalled();
  });
});

• [ ] Step 4.10: Run all CategoryConfigurator tests, expect PASS

npx vitest run src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx

Expected: 8 tests pass.

• [ ] Step 4.11: Run tsc --noEmit

npx tsc --noEmit

Expected: clean.

• [ ] Step 4.12: Add test: onClose called when drawer overlay/X dismissed

Append:

describe('CategoryConfigurator — close', () => {
  afterEach(() => vi.restoreAllMocks());

  it('calls onClose when the open prop transitions through onOpenChange(false)', async () => {
    global.fetch = mockFetchSequence([
      { json: sampleResponse },
      { json: extractorResponse },
    ]) as unknown as typeof fetch;
    const onClose = vi.fn();
    render(<CategoryConfigurator {...defaultProps} onClose={onClose} />);
    await screen.findByTestId('footer-commit');
    // Press Escape — Radix Dialog (used by Sheet) closes via Escape
    fireEvent.keyDown(document.body, { key: 'Escape' });
    await waitFor(() => expect(onClose).toHaveBeenCalled());
  });
});

• [ ] Step 4.13: Run, expect PASS

npx vitest run src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx

Expected: 9 tests pass.

• [ ] Step 4.14: Commit

git add src/components/admin/factory/CategoryConfigurator.tsx src/components/admin/factory/__tests__/CategoryConfigurator.test.tsx
git commit -m "feat(factory): CategoryConfigurator drawer with sample/extractor/test flow + commit gating"

---

Task 5: Cluster validation: full regression

Files: - (Read-only verification)

• [ ] Step 5.1: Run all factory frontend tests

npx vitest run src/components/admin/factory/__tests__/

Expected: 27 tests pass total (5 SamplePreview after merging, 10 ConfigEditor, 12 TestCrawlResults, 9 CategoryConfigurator): actual numbers from this spec: 6 + 10 + 12 + 9 = 37. Adjust counts if you split tests differently.

• [ ] Step 5.2: Run full project test suite

npx vitest run

Expected: full baseline still GREEN; new factory frontend tests pass.

• [ ] Step 5.3: Run full tsc

npx tsc --noEmit

Expected: clean.

• [ ] Step 5.4: Smoke test in browser (manual, optional)

npm run dev:all

Then navigate to http://localhost:5173/admin/factory, exercise a fake session if available. (Drawer wiring lives in Spec 12; a standalone smoke is fine here using devtools to render CategoryConfigurator against mocked fetches.)

• [ ] Step 5.5: Mark Spec 11 done in Status.md

In the vault Projects/Crawler-Factory/Status.md:

- | Cluster 11 (Frontend configurator) | ⏸ | CategoryConfigurator + SamplePreview + ConfigEditor + TestCrawlResults |
+ | Cluster 11 (Frontend configurator) | ✅ | CategoryConfigurator + SamplePreview + ConfigEditor + TestCrawlResults |

---

Acceptance criteria: Spec 11 done means:

1. ✅ src/components/admin/factory/types.ts exports Sample, StructureAnalysis, SandboxResult, RealResult, CrawlerConfig and re-exports ContentDomain + PathGroup from lib/factory/types.
2. ✅ src/components/admin/factory/SamplePreview.tsx renders sample list (collapsible HTML view), detected-structure rows (selector ✓ field), and JSON-LD (only when present). Mono font for selectors and JSON.
3. ✅ src/components/admin/factory/ConfigEditor.tsx renders editable form (indexName default algoliacentral_<contentDomain>, renderJavaScript Switch, rateLimit numeric default 8, pathsToMatch comma-separated), monospace 24-row Textarea for recordExtractor, "Regenerate from samples" Button, free-form feedback Input that submits on Enter.
4. ✅ src/components/admin/factory/TestCrawlResults.tsx renders Sandbox + Real tabs, per-sample rows with ✓/✗, record count, field-coverage bars (one per DSS searchable attribute, percentage 0–100% clamped), expandable first-record JSON.
5. ✅ src/components/admin/factory/CategoryConfigurator.tsx renders shadcn Sheet side='right' at 70% width, title {contentDomain} | {pathGroup.pattern}. On open: POST /api/factory/sample then POST /api/factory/generate-extractor. Footer holds three CTAs: Run sandbox / Run REAL / Commit. Commit enabled only when both sandbox + real results are all-green for this pathGroup.
6. ✅ Per §19e (no silent commits): Commit button is gated AND requires explicit user click, never auto-fires.
7. ✅ All four components have unit tests with @testing-library/react. Coverage:
   • drawer renders title and opens/closes
   • sample fetch fires on open
   • extractor fetch fires after samples
   • regenerate-with-feedback re-fetches extractor with userFeedback in body
   • field-coverage bars render correct percentage widths
   • commit button gating proven across all four states (initial, sandbox-only, real-only, both)
   • explicit-click requirement proven (does not auto-commit)
8. ✅ tsc --noEmit clean.
9. ✅ Per CodingSOPs: every component has JSDoc, every fetch has try/catch + toast on failure, no any, props are typed interfaces.
10. ✅ Each sub-component committed separately (per Task 27 directive).

---

Out of scope (handled by other specs)

• Backend endpoints (/api/factory/sample, /generate-extractor, /test-sandbox, /test-real, /create-crawler) → Spec 08
• Wiring CategoryConfigurator open/close from CategoryReview → Spec 12 (CrawlerFactory orchestrator)
• The actual call to POST /api/factory/create-crawler on commit → Spec 12 (orchestrator owns it; this spec only emits onCommit upward)
• LiveCrawlMonitor block that follows commit → Spec 12
• DSS / zod types: already shipped in Spec 01
• DomainBadge reuse: already shipped in Spec 10 (consumers may import it, but the configurator does not require it visually since the title carries the contentDomain in mono text)
• Hooks useFactorySession, useDiscoveryStream, useCrawlProgress: Spec 09; this spec uses raw fetch because the calls are imperative actions tied to user gestures, not declarative session reads

Spec 08 follow-up flag (B11): The POST /api/factory/generate-extractor endpoint must return { recordExtractor, crawlerConfig, analysis: StructureAnalysis }: the analysis field (Spec 05's StructureAnalysis shape verbatim) is required so SamplePreview can render selectors / missing / confidence without a second round-trip. Spec 08's current contract returns only { recordExtractor, crawlerConfig }. Tracked as a Spec 08 follow-up; Spec 11 consumes it as soon as it lands.