A high-performance, real-time trending topics and article aggregation dashboard designed for editorial teams. It aggregates trends from Google Search and X (formerly Twitter), then semantically matches them against latest news stories using optional LLM intent verification.
- 🌐 Multi-Source Aggegation: Real-time trending data from Google Trends (RSS) and Twitter (scraped via Trends24).
- 📝 High-Volume Article Fetching: Pulls and crawls thousands of articles from regional/national sitemaps.
- 🧠 Semantic Filtering: Uses keyword-based logic (Recall) followed by optional LLM-based re-ranking (Precision) to match articles to trends.
- ⚡ Performance First: Local caching and optimized aggregation logic for near-instant dashboard loads.
- 📊 Trend History: Supports persistent history and velocity tracking via Upstash Redis.
- 🎨 Premium UI: Modern, glassmorphism-inspired dark mode interface built with Next.js and Tailwind CSS.
- Framework: Next.js 15 (App Router)
- Language: TypeScript
- Styling: Tailwind CSS 4
- Database (Optional): Upstash Redis
- LLM Integration: Ollama or any OpenAI-compatible API (e.g., NVIDIA NIM)
- Scraping: Cheerio
- Node.js 20+
- npm / pnpm / yarn
- (Optional) Upstash Redis account for history tracking.
- (Optional) Ollama running locally for LLM-based article matching.
git clone https://github.com/your-username/trending-engine.git
cd trending-engine
npm installCopy the example environment file and fill in your values:
cp .env.example .env.localRun the development server:
npm run devVisit http://localhost:3000 to see the dashboard.
| Variable | Description | Default |
|---|---|---|
LLM_RERANK_ENABLED |
Enable LLM verification for article-trend matching | false |
LLM_BASE_URL |
Base URL for the LLM API (Ollama/NVIDIA) | http://localhost:11434/v1 |
GOOGLE_TRENDS_MAX |
Max number of trends to pull from Google | 20 |
TWITTER_TRENDS_MAX |
Max number of trends to pull from Twitter | 20 |
TOP_N |
Number of trends shown in the final merged list | 10 |
/src/app: Next.js pages and API routes./src/lib: Core logic including trend fetchers, scorers, and LLM utilities./data: Local scratch storage for transient historical data.