# How to Vibe Code Your Own Social Fetch (and Stop Paying for It)

> Social media scraper API for every major platform

- Site: https://socialfetch.dev
- Category: Developer Tools & APIs
- Platforms: Web app
- Verdict: **Don't bother** (18/100 vibecodeable)
- Estimated effort: 6+ months of full-time work to replicate all 21 platforms

## Verdict

Build a personal subset for a single platform, but keep paying for the commercial API if you need multi-platform coverage; the scraper maintenance treadmill across 21 networks makes a full clone a grueling engineering burden.

The core value of Social Fetch is not the CRUD wrapper or the API gateway—it is the full-time, unglamorous engineering treadmill of fixing broken DOM selectors, managing residential proxy pools, and dodging aggressive anti-bot protections on TikTok, Instagram, and X. While you can easily spin up a Next.js API gateway with API key authentication and a SQLite credit tracker, maintaining scrapers against platforms actively trying to break them will consume every weekend you have. For personal use, paying $14 for 1,000 credits is infinitely cheaper than buying proxies and debugging headless browser fingerprinting.

### What you can't replicate

- Maintained anti-bot evasion and proxy rotation across 21 live target platforms
- Stable downstream schema mapping amidst frequent upstream DOM mutations
- Guaranteed uptime across 163 distinct platform endpoints

## What it does

A unified, commercial social media scraper API that aggregates public data across 21 major platforms into a single normalized JSON schema via 163 REST endpoints.

### Core features

- API Gateway & Key Management: Secure routing with API key validation and request ID tracking
- Unified Schema Envelope: Normalized JSON structure ({ data, meta }) across all endpoint types
- Live Upstream Scrapers: Real-time fetching of profiles, posts, metrics, and transcripts
- Proxy & Anti-Bot Infrastructure: Rotation management to bypass platform rate limits and CAPTCHAs
- Credit Accounting Engine: PAYG balance tracking and billing guarantee rules (no charge for 503 errors)
- Scheduled Monitors & Webhooks: Periodic background polling with payload diffing and signed webhooks
- TypeScript SDK & MCP Server: Client libraries and Model Context Protocol integration for AI assistants
- x402 Micropayments: Crypto-native USDC settlement layer on Base

## The business

### Pricing

- Free: $0.00 — No card required; includes 100 free credits for real production endpoints.
- Starter: $14.00 — One-time credit pack for initial testing and small workflows.
- Growth: $47.00 — For projects or one-off batch jobs.
- Scale: $379.00 — For larger batch processing tasks.

Founded 2023.
Team size: 1-10.

## The hard parts

- Platform Anti-Bot Arms Race: Constantly maintaining scraper selectors against aggressive TLS fingerprinting and DOM changes on TikTok and Instagram
- Proxy Rotation & IP Reputation: Procuring and managing large residential proxy pools to prevent blocks across 21 distinct networks
- Live Latency & Concurrency Bounds: Handling ~3.2s average upstream request latency without cascading thread blockages during burst traffic
- Runtime Boundary Validation: Ensuring external payload shifts fail safely without breaking downstream consumer pipelines

## How to vibe code Social Fetch

### Prerequisites

- Node.js LTS (free): Runtime environment for the TypeScript API gateway and Effect framework.
- GitHub (free): Code repository and CI/CD deployment pipeline.

### Recommended AI tools

- Claude Code: Best-in-class agentic coding tool for scaffolding the Effect-based API routing layer and writing complex scraper modules.
- Cursor: Ideal for inspecting scraper diffs and fine-tuning schema validation types.

### Stack

- Frontend: Next.js
- Backend: Node.js with Effect
- Database: Turso
- Auth: better-auth
- Payments: Stripe
- Other: Playwright, Zod

### Hosting

- Cloudflare (Hosting the API gateway workers and edge routing.): $0-5/mo
- Fly.io (Running long-lived scraper worker nodes and proxy managers.): $5/mo

### Build guide

1. **Scaffold API Gateway & Database Schema** — Initialize a Next.js project with a Turso SQLite database using better-auth for dashboard management and an API key generation schema.

```
Scaffold a new Next.js project using TypeScript and Tailwind CSS. Set up Turso database integration using libSQL client. Create database tables for users, API keys (with sfk_ prefix generation), credit balances, and request logs (tracking requestId, endpoint, creditsCharged, and lookupStatus). Integrate better-auth for developer dashboard authentication. Ensure strict type safety and structured error handling throughout the application layout.
```

2. **Implement Effect-Based Request Pipeline** — Build the core request validation and middleware layer using the Effect framework for typed error management and boundary validation.

```
Implement a backend request pipeline using the Effect functional programming framework in TypeScript. Create middleware that extracts and validates the x-api-key header from incoming requests, checks the developer's credit balance in Turso, and enforces rate limits. Build schema-based runtime boundary validation using Effect Schema to guarantee that outgoing JSON responses conform strictly to the standard { data, meta } envelope.
```

3. **Build Single-Platform Scraper Module (TikTok Profile)** — Develop a robust Playwright-based scraper for TikTok profiles with fallback selectors and proxy rotation logic.

```
Develop a robust scraper module for TikTok profiles (GET /v1/tiktok/profiles/{handle}) using Playwright and Axios. Implement proxy rotation middleware supporting external proxy lists with automatic retry logic on failure. Handle upstream rate limits and anti-bot challenges by injecting randomized user agents, viewport settings, and stealth headers. Ensure the output is safely mapped to the normalized JSON schema containing profile metrics, bio, and verification badges.
```

4. **Add Credit Accounting & Billing Safeguards** — Implement the credit deduction logic with error exemption rules ensuring 503 failures do not debit user balances.

```
Implement the credit accounting engine in the API gateway. Configure transactional debiting rules such that successful lookups (including lookupStatus: not_found) deduct exactly 1 credit from the user's balance, while infrastructure failures (lookup_failed, 503 temporarily_unavailable) strictly bypass credit debiting. Include unique requestId generation on every response payload for audit tracing.
```

5. **Create Developer Documentation & TypeScript SDK** — Publish a lightweight TypeScript SDK mirroring the API routes and provision a developer playground UI.

```
Create a companion TypeScript SDK package (@socialfetch/sdk) featuring typed client methods that mirror the implemented REST routes, returning Result types and lookupStatus guards. Build an internal developer dashboard page with an interactive API playground, live request logs showing requestId and credit charges, and a balance top-up view.
```

### Cost vs paying

**Starting costs (one-time):**

- Proxy provider pool subscription: $50/mo
- Total: ~$70 initial

**Ongoing costs (monthly):**

- Proxy network bandwidth & rotation: $50/mo
- Fly.io + Turso hosting: $10/mo
- Total: ~$60/mo

- Paying for the SaaS instead: $14.00+ (PAYG credits)
- Build time: 80-120 hours
- AI tool credits: $20 (Claude Pro)
- Break-even: Never (maintenance overhead exceeds cost)

## Sources

- [Social Fetch Official Website](https://socialfetch.dev)
- [Social Fetch Pricing Documentation](https://socialfetch.dev/pricing)
- [Social Fetch Blog: Effect in Production](https://socialfetch.dev/blog/effect-typescript-production)