cybrkyd

Sending a RSS feed to X (Twitter) with Python

 Mon, 03 Jun 2024 10:37 UTC
Sending a RSS feed to X (Twitter) with Python
Image: CC BY 4.0 by cybrkyd

I would like to connect my website's RSS feed to X (Twitter). How do I go about doing this in a self-hosted way without having to rely on third-party services?

On my website, as soon as a new blog post is published, the RSS feed is automatically updated. My aim is therefore to read the RSS feed and if a new post is found, automatically create a tweet and post it to X. I do not necessarily need this to happen in real time so I'll be using a cron job to check my RSS feed every couple of hours.

The formatting of the tweet is also important and will need to be taken care of in the script. Also, if there happens to be more than one new item on the RSS feed, the script will take care of them one-by-one and send them in with a 30-second delay between the posts. This is to ensure that we don't flood X's API with everything all at once.

Finally, we don't want to ever repeat a post. To ensure that this never happens, we need a log file which only contains one line: the time that the last post to X was made.

To enable my website's RSS feed to automatically post to X every time there is a new item published on my blog, I first need to setup a developer account and app on X. Once setup, obtain the API keys and tokens.

Ready? Let's write some Python.

Install the required Python libraries

The next step involves installing the required Python libraries. For the purpose of this script I will be using the requests and requests_oauthlib libraries as well as feedparser, time and html. Install the required libraries in your environment.

The RSS

What we will do in sequence is:

  1. Read the last run time from a log file.
  2. Fetch and parse the RSS feed.
  3. Compare feed entry publication times to the last run time.
  4. Format and post all new entries as tweets.
  5. Update the log to note the last post time.

The log file is called last_run.txt. Normally, I would code in that if the file is not found, create it but since this is a run-once-only operation, this file can be created manually.

The file should contain one line with a past date, for example:

Tue, 02 Jan 2024 15:32:05 UTC

This ensures that at first run, every item on the RSS feed which is timestamped later than 02-Jan-2024 15:32:05 UTC will be posted to X.

Formatting

After fetching the RSS feed, the script processes each entry to prepare it for posting. This involves iterating through the entries and extracting relevant information such as the title, description, publication date, and link.

  1. Publication Time Check: The script compares the publication time of each entry with the last run time (the last time the script posted to X, not the last time it ran). This ensures that only new entries, published after the last post, are considered.
  2. HTML "Unescaping": Titles and descriptions often contain HTML entities (e.g., & for &, " for quotes, etc.). The html.unescape function converts these entities into their corresponding characters, ensuring the tweet content is clean and readable.
  3. Content Formatting: The tweet content is constructed by combining the "unescaped" title, description, and link, separated by new lines.

The script

#!/usr/bin/env python3

import requests
import feedparser
import time
from datetime import datetime
from requests_oauthlib import OAuth1
import html  # For unescaping HTML entities

# Your Twitter API credentials
API_KEY = 'your_api_key'
API_SECRET_KEY = 'your_api_secret_key'
ACCESS_TOKEN = 'your_access_token'
ACCESS_TOKEN_SECRET = 'your_access_token_secret'

# Endpoint for posting a tweet
url = "https://api.twitter.com/2/tweets"

# Function to post a tweet
def post_tweet(tweet_content):
    auth = OAuth1(API_KEY, API_SECRET_KEY, ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
    payload = {"text": tweet_content}
    response = requests.post(url, auth=auth, json=payload)
    return response

# Function to read the last run time from the log file
def read_last_run_time(log_file):
    try:
        with open(log_file, 'r') as file:
            return file.readline().strip()
    except FileNotFoundError:
        return None

# Function to update the last run time in the log file
def update_last_run_time(log_file, timestamp):
    with open(log_file, 'w') as file:
        file.write(timestamp)

# Function to fetch and parse the RSS feed
def fetch_rss_feed(feed_url):
    return feedparser.parse(feed_url)

# Main logic
def main():
    feed_url = "https://cybrkyd.com/index.xml"
    log_file = "last_run.txt"
    last_run_time_str = read_last_run_time(log_file)

    if last_run_time_str:
        last_run_time = datetime.strptime(last_run_time_str, "%a, %d %b %Y %H:%M:%S %Z")
    else:
        last_run_time = datetime.min

    feed = fetch_rss_feed(feed_url)
    new_items = []

    for entry in feed.entries:
        published_time = datetime(*entry.published_parsed[:6])
        if published_time > last_run_time:
            title = html.unescape(entry.title)  # Unescape HTML entities
            description = html.unescape(entry.description)  # Unescape HTML entities
            link = entry.link
            tweet_content = f"Sending a RSS feed to X (Twitter) with Python\n{description}\n{link}"
            new_items.append(tweet_content)

    for tweet in new_items:
        response = post_tweet(tweet)
        if response.status_code == 201:
            print("Tweet posted successfully!")
        else:
            print(f"Failed to post tweet: {response.status_code}")
            print(response.json())
        time.sleep(30)  # 30-second delay between posts

    if new_items:
        now_utc = datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S UTC")
        update_last_run_time(log_file, now_utc)

if __name__ == "__main__":
    main()

This Python script provides an efficient and automated way to keep my X account updated with the latest content from my blog's RSS feed. This automation saves time and ensures my social media presence remains active and current with fresh content.

By using powerful libraries such as requests, feedparser, and requests_oauthlib, the script fetches and parses the RSS feed, formats the entries, and posts them to X with minimal manual intervention. It smartly keeps track of the last run time to ensure that only new entries are posted, and it includes rate-limiting mechanisms to avoid flooding the API.

»
Visitors: Loading...