rss-automation

RSS Automation

• Author: Daniel Li
• Copyright © Daniel Li. All rights reserved.

Monitor and aggregate RSS/Atom feeds for automated content tracking and notification.

Capabilities

• Parse RSS 2.0 and Atom feeds
• Filter entries by keywords, date, author
• Track seen/new entries to avoid duplicates
• Push new items to Telegram, Slack, or other channels
• Schedule periodic feed checks via cron

Usage

Parse a Feed

import feedparser

feed = feedparser.parse("https://example.com/feed.xml")
for entry in feed.entries[:10]:
    print(f"{entry.title} - {entry.link}")
    print(f"  Published: {entry.get('published', 'N/A')}")

Monitor Multiple Feeds

import feedparser, json, hashlib
from pathlib import Path

SEEN_FILE = Path("~/.openclaw/rss-seen.json").expanduser()

def load_seen():
    if SEEN_FILE.exists():
        return json.loads(SEEN_FILE.read_text())
    return {}

def save_seen(seen):
    SEEN_FILE.write_text(json.dumps(seen))

def check_feed(url, seen):
    feed = feedparser.parse(url)
    new_entries = []
    for entry in feed.entries:
        entry_id = hashlib.md5(entry.link.encode()).hexdigest()
        if entry_id not in seen.get(url, []):
            new_entries.append(entry)
            seen.setdefault(url, []).append(entry_id)
    return new_entries

Cron Integration

Set up a cron job to check feeds periodically:

Schedule: every 2 hours
Payload: "Check RSS feeds and push new items"

Dependencies

pip install feedparser

Feed Sources Examples

Source	Feed URL
Hacker News	https://hnrss.org/frontpage
TechCrunch	https://techcrunch.com/feed/
ArXiv CS.AI	http://arxiv.org/rss/cs.AI
GitHub Trending	Use web scraping instead

Important Notes

• Some feeds require User-Agent headers
• Rate limit feed checks (don't poll more than every 15 minutes)
• Store seen entries persistently to survive restarts