Build a Labeled Reddit Opinion Dataset

How It Works

This workflow builds a growing dataset of Reddit opinions. Once a week it scrapes posts and comments from a list of subreddits with the Apify Reddit Scraper, asks OpenAI to label the reaction type of every comment, and writes the results to Supabase. Posts and comments land in their own tables, and OpenAI embeddings of both go into a documents table so the dataset supports vector search as well as SQL analysis.

Setup Steps

1. Connect Apify: Add your Apify API token as a credential in n8n so the workflow can run the Reddit Scraper actor.
2. Prepare Supabase: Create a Supabase project, enable the pgvector extension, and create the posts, comments, and documents tables.
3. Add the Supabase Credential: Connect n8n to Supabase with your project URL and service_role key.
4. Connect OpenAI: Add an OpenAI API credential; it powers both the comment classification and the embedding steps.
5. Set the Dataset Parameters: Edit the input configuration with your subreddits and per-subreddit max results, sort order, time filter, minScore, and maxCommentsPerPost or maxItems.
6. Activate the Workflow: Turn the workflow on so the weekly schedule starts collecting data.

Automation Workflow

Weekly Schedule Trigger
↓
Load dataset inputs (subreddits, sort, time window, min score, limits)
↓
Split subreddit plan into individual scrape jobs
↓
Apify Reddit Scraper → posts and comments
↓
Posts branch:
  Filter and format posts → Supabase "posts" table
  ↓
  OpenAI embeddings of post context → Supabase "documents" table
↓
Comments branch:
  Filter and format comments → OpenAI gpt-4.1-mini reaction label
  ↓
  Supabase "comments" table → OpenAI embeddings → Supabase "documents" table

Benefits

• Hands-Off Data Collection: The weekly schedule keeps adding fresh posts and comments without manual exports.
• Labeled Out of the Box: Every comment carries a reaction type from gpt-4.1-mini, so the data is ready for opinion analysis without a separate annotation pass.
• SQL and Semantic Search in One Place: Structured tables sit next to pgvector embeddings in the same Supabase project.
• Tunable Scope: Per-subreddit limits, a minimum score, and comment caps keep scraping volume and OpenAI costs predictable.

Advanced Features

• Splits one subreddit plan into separate Apify scrape jobs, each with its own result limit
• Runs parallel branches for posts and comments with different formatting and storage logic
• Classifies each comment into a single reaction type with an LLM before insertion
• Embeds both post context and comment text into a shared documents table for retrieval
• Filters low-score content before it reaches the database or the OpenAI API

Use Cases

• Track how a product community reacts to launches or pricing changes week over week
• Build a labeled training or evaluation set for opinion and sentiment models
• Power a RAG chatbot that answers questions from real Reddit discussions
• Compare reaction patterns across competing subreddits or niches
• Feed market research reports with quotable, categorized user opinions