# How to Vibe Code Your Own Cleanvoice (and Stop Paying for It) > Automated post-production SaaS for audio and video - Site: https://cleanvoice.ai - Category: Audio & Video AI SaaS - Verdict: **Serious undertaking** (42/100 vibecodeable) - Estimated effort: 3-4 weeks of focused engineering ## Verdict Build a simple single-track script wrapper instead of paying, but skip cloning the full multitrack SaaS production pipeline. Replicating Cleanvoice involves stitching together complex audio ML models for source separation and forced alignment, building a robust asynchronous worker queue for heavy media files, and managing precise waveform splicing. While you can easily write a Python script that calls Whisper and a local Demucs model to strip out silence and filler words for personal use, building a reliable multitrack SaaS with webhooks and API keys requires serious backend infrastructure engineering. ### What you can't replicate - Proprietary fine-tuned acoustic models optimized specifically for low-latency multi-language filler word detection - Enterprise ISO 27001 compliance trust and established brand moat serving 15,000+ podcasters ## What it does An automated AI audio and video post-production platform that removes filler words, background noise, mouth sounds, stutters, and silences while generating transcriptions and summaries. ### Core features - Filler word detection and excision across 20+ languages - Background noise and room reverb removal - Mouth sound, saliva crackle, and heavy breath attenuation - Silence and dead air trimming - Studio sound normalization and vocal balancing - Automatic transcription and show notes generation - Multitrack synchronization and simultaneous editing - REST API and SDK for automated pipelines ## The business ### Pricing - Free Trial: Free — 30 minutes of free audio/video processing credit. - Pay-as-you-go 10 Hours: $20 — Flexible credits valid for 2 years. - Subscription 30h: $30/mo — Monthly recurring plan with credit rollover. Founded 2021. Team size: 1-100. ## The hard parts - Acoustic source separation and forced alignment models to pinpoint precise timestamps of filler words without clipping natural speech - Multitrack time-alignment and phase preservation while independently editing separate guest tracks - Asynchronous heavy media processing pipelines requiring GPU worker queues to prevent HTTP timeouts - Complex digital signal processing (DSP) logic to splice audio waveforms cleanly without audible clicks or phase artifacts ## How to vibe code Cleanvoice ### Prerequisites - Node.js (free): Runtime environment for the Next.js web application and dashboard frontend. - Python (free): Required for backend audio processing workers utilizing PyTorch, Demucs, and Whisper. - GitHub (free): Repository hosting and CI/CD deployment pipeline. ### Recommended AI tools - Claude Code: Agentic terminal coding tool best suited for scaffolding complex full-stack web applications and background worker scripts. - Cursor: AI code editor ideal for iterative UI development on the dashboard and API route handlers. ### Stack - Frontend: Next.js with Tailwind CSS and shadcn/ui - Backend: Next.js API routes paired with a Python FastAPI audio processing worker service - Database: Turso for SQLite metadata storage and user tracking - Auth: better-auth for self-hosted secure user management - Payments: Stripe for billing and credit purchases - Other: Cloudflare R2 for audio/video file storage, Upstash Redis for background processing queue ### Hosting - Vercel (Hosting the Next.js web dashboard and API frontend): $0-20/mo - Fly.io (Running persistent Python FastAPI worker containers with GPU/CPU support for heavy audio DSP and ML pipelines): $10-30/mo ### Build guide 1. **Scaffold Next.js Dashboard and Database** — Initialize the Next.js project with Tailwind CSS, shadcn/ui components, better-auth integration, and Turso database connection for user tracking and file metadata. ``` Create a new Next.js project configured with TypeScript, Tailwind CSS, and App Router. Set up better-auth with email/password authentication backed by a Turso SQLite database via Drizzle ORM. Build a clean dashboard layout featuring a sidebar navigation, file upload dropzone, processing history table, and user credit balance counter. Ensure all UI components follow a modern SaaS aesthetic with dark mode support. ``` 2. **Configure Cloudflare R2 File Storage & Presigned Uploads** — Implement direct-to-R2 presigned upload endpoints in Next.js to handle large audio and video files securely without hitting serverless payload limits. ``` Implement server actions and API routes in Next.js that generate secure presigned URLs for Cloudflare R2 bucket uploads. Build a frontend drag-and-drop upload component that uploads audio (.mp3, .wav) and video (.mp4) files directly to R2 with progress bars. Add validation for file types and maximum size limits, storing file metadata in the Turso database once uploads complete. ``` 3. **Build Python FastAPI Processing Worker Service** — Create a standalone Python FastAPI service that pulls audio tasks from an Upstash Redis queue, downloads files from R2, and prepares them for AI processing. ``` Create a Python FastAPI service structured for background audio processing. Connect it to an Upstash Redis queue to listen for incoming audio processing jobs containing file keys and user settings (e.g., filler word removal, noise reduction, silence trimming). Implement job status tracking (pending, processing, completed, failed) stored in Turso, and include robust error handling with automatic retries. ``` 4. **Implement AI Audio DSP and Filler Word Removal Pipeline** — Integrate Python libraries for voice separation, forced alignment transcription, and waveform trimming to execute the core audio cleaning logic. ``` Within the FastAPI worker, implement an audio processing pipeline using Whisper for forced alignment transcription and Demucs for background noise isolation. Write DSP logic using librosa or pydub to identify timestamps of filler words ('um', 'uh'), long silences (>2 seconds), and heavy breaths, then splice the audio waveform accordingly without introducing phase pops or clicks. Save the resulting processed audio file back to Cloudflare R2 and update the job record. ``` 5. **Build Result Review, Audio Player, and Export Dashboard** — Develop an interactive web audio player showing edited timestamps, allowing users to preview cleaned audio and download files or edit logs. ``` Build a dashboard results page in Next.js featuring an interactive audio waveform player (using wavesurfer.js) that highlights removed filler words and silences as markers. Add controls to toggle specific filters on/off, view the generated text transcription and summary, and download either the final cleaned audio file or edit timeline export files (.srt / JSON markers) for external digital audio workstations. ``` 6. **Implement Developer REST API and Webhooks** — Expose programmatic REST endpoints for file upload, job submission, and status retrieval to enable developer pipeline integrations. ``` Create developer API routes in Next.js secured by API key authentication. Implement POST /api/v1/jobs to accept audio URLs or direct uploads, queue the processing task, and return a job ID. Implement GET /api/v1/jobs/{id} to check status and retrieve download links for cleaned files and transcripts. Generate a developer settings page in the dashboard allowing users to create, view, and revoke their API keys. ``` ### Cost vs paying **Starting costs (one-time):** - AI Coding Assistant (Claude Pro): $20 - Domain Name: $12/yr - Total: ~$32 one-time **Ongoing costs (monthly):** - Fly.io Worker Hosting: $15/mo - Cloudflare R2 & Turso DB: $0-5/mo - Total: ~$15-20/mo - Paying for the SaaS instead: $30/mo - Build time: 60-80 hours - AI tool credits: $20 (Claude Pro) - Break-even: Not a financial saving for light users due to server and ML hosting costs, but viable for custom automated API workflows. ## Sources - [Cleanvoice AI Official Website](https://cleanvoice.ai) - [Cleanvoice API Documentation](https://docs.cleanvoice.ai/)