How to vibe codeImport.io

Prerequisites

• Node.jsfree

  Runtime environment for the Next.js application and scraping workers

• GitHubfree

  Source control and deployment pipeline integration

• Anthropic API Keypay-as-you-go

  Powers the LLM-assisted schema mapping, data cleaning, and layout drift detection

AI coding tools

• Claude Code

  Best-in-class agentic coding tool for scaffolding full-stack apps, database schemas, and background scraping workers.

• Cursor

  Ideal for fine-tuning frontend dashboard components, data visualization charts, and extraction configuration forms.

Build guide

1. 01Project Scaffolding & Database Schema

   Initialize the Next.js project with Tailwind CSS, configure better-auth with SQLite/Neon, and define database schemas for Extractors, Scraping Runs, Extracted Items, and Competitor Pricing Alerts.

   Scaffold a new Next.js 16 application using App Router and Tailwind CSS. Set up Neon serverless Postgres connection using Drizzle ORM. Implement better-auth with email/password authentication. Create database tables for: 'extractors' (id, name, target_url, selectors_json, cron_schedule, user_id), 'scraping_runs' (id, extractor_id, status, items_extracted_count, error_log, executed_at), 'extracted_items' (id, run_id, raw_data_json, normalized_data_json, created_at), and 'competitor_prices' (id, product_name, retailer_name, current_price, map_price, url, status, updated_at). Ensure all foreign keys, timestamps, and indexing are properly configured for efficient querying.

   ⧉copy promptopen in Claude CodeCopen in Codexopen in Cursor

2. 02Core Extraction Engine & Playwright Integration

   Build a background scraping worker using Playwright and Firecrawl API integration to fetch target URLs, parse HTML DOM structures, and handle basic anti-bot evasion.

   Build a robust data extraction service module in Node.js using Playwright and Firecrawl API. The module should accept a target URL and a JSON schema mapping (CSS selectors or XPath). Implement headless browser navigation with randomized user-agents, viewport scaling, and stealth plugins to avoid basic bot detection. Handle dynamic page rendering by waiting for network idle events before capturing DOM content. Return structured JSON objects mapping directly to the requested fields, and log detailed execution metrics or errors to the database.

   ⧉copy promptopen in Claude CodeCopen in Codexopen in Cursor

3. 03AI-Assisted Schema Mapper & Self-Healing Pipeline

   Integrate the Vercel AI SDK and Anthropic API to parse unstructured web pages, suggest optimal extraction schemas, and implement self-healing logic when selectors break.

   Implement an AI-assisted extraction and self-healing service using the Vercel AI SDK and Anthropic Claude. When a user inputs a target URL and describes the data they want (e.g., 'product title, price, and stock status'), use Claude to fetch the page HTML, analyze the DOM tree, and generate an optimal JSON extraction schema. Furthermore, if a scheduled scraping run returns zero items or missing fields due to layout drift, trigger an automated recovery function where Claude inspects the new HTML structure, re-maps the selectors, updates the extractor schema in the database, and re-runs the extraction pipeline successfully.

   ⧉copy promptopen in Claude CodeCopen in Codexopen in Cursor

4. 04Extractor Management Dashboard & Scheduler

   Develop the user dashboard for creating, testing, scheduling, and monitoring extraction pipelines with real-time status indicators.

   Create a comprehensive dashboard interface in Next.js App Router. Build pages for: (1) Extractor list and creation wizard with live URL preview, (2) Scraping run logs showing success/failure rates and item counts, (3) Data explorer table displaying extracted JSON rows with export options (CSV/JSON), and (4) Cron scheduler configuration. Use Tailwind CSS and Lucide icons for a clean, enterprise-grade SaaS aesthetic similar to Linear or Import.io. Include interactive charts showing query success rates over time.

   ⧉copy promptopen in Claude CodeCopen in Codexopen in Cursor

5. 05Aperture Pricing Intelligence & MAP Monitoring Module

   Build the pricing intelligence layer to track competitor pricing, detect Minimum Advertised Price (MAP) violations, and trigger alerts.

   Build the Aperture pricing intelligence module within the dashboard. Create a 'Competitor Tracking' view that aggregates product prices across multiple scraped retailers for identical items using fuzzy string matching. Implement a MAP compliance rules engine where users can set a minimum advertised price threshold; automatically flag violations where retailer price < MAP price, compute price gaps, and display an auditable evidence feed with timestamps and product screenshots. Add an alert configuration page to dispatch webhook notifications or email alerts via Resend when a MAP violation occurs.

   ⧉copy promptopen in Claude CodeCopen in Codexopen in Cursor

6. 06Webhook API Delivery & End-to-End Polish

   Implement outgoing webhook dispatchers for automated data feeds, write unit tests, and finalize error handling across all modules.

   Implement a secure webhook delivery system that pushes newly extracted JSON data payloads to user-configured destination endpoints upon successful scraping run completion. Include HMAC SHA-256 signature verification headers, automatic retry logic with exponential backoff for failed deliveries, and a delivery history log table in the dashboard. Perform end-to-end testing of the extractor creation, AI schema mapping, scheduled execution, and Aperture pricing alert workflows. Polish UI loading states, toast notifications, and error boundaries.

   ⧉copy promptopen in Claude CodeCopen in Codexopen in Cursor

Cost vs paying for Import.io