Welcome, guest | Sign In | My Account | Store | Cart

A quick script that lets you download all of your tweets and write them to a text file.

Python, 22 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
#!/usr/bin/python

import time
from urllib2 import urlopen
from BeautifulSoup import BeautifulSoup

# Replace USERNAME with your twitter username
url = u'http://twitter.com/USERNAME?page=%s'
tweets_file = open('tweets', 'w')

for x in range(10*10000):
    f = urlopen(url % x)
    soup = BeautifulSoup(f.read())
    f.close()
    tweets = soup.findAll('span', {'class': 'entry-content'})
    if len(tweets) == 0:
        break
    [tweets_file.write(x.renderContents() + '\n') for x in tweets]
    # being nice to twitter's servers
    time.sleep(5)

tweets_file.close()

Loops through all of your public timeline pages, grabs the individual tweets, and throws them into a text file.

7 comments

Caleb Herbert 15 years, 3 months ago  # | flag

Cool! I'm gonna keep that in my library for when I just have BSD or a console-only system. This is great.

Alex 14 years, 7 months ago  # | flag

ImportError: No module named BeautifulSoup

I've never worked with BeautifulSoup before. I googled their site and downloaded the zip they have, but I don't know what I should be doing with it to be able to make this work.

Zach Seifts (author) 14 years, 7 months ago  # | flag

Alex: You should be able to install it with easy_install or pip. BeautifulSoup is in the cheese shop.

Sridhar Ratnakumar 14 years, 4 months ago  # | flag

Alex, you can also install it using PyPM that comes with ActivePython.

C:> pypm install beautifulsoup
alex 12 years, 7 months ago  # | flag
  1. page - read and parse Traceback (most recent call last): File "twitterbak.py", line 151, in <module> main(sys.argv[1:]) File "twitterbak.py", line 108, in main date_time = tweet.find('span', 'timestamp').get("data", None) AttributeError: 'NoneType' object has no attribute 'get
hossein 12 years, 5 months ago  # | flag

Hi guys I am new to python. How can I use this code to extract tweets. I copied the codes in python but recieved no result even no error message.

Julio 9 years, 11 months ago  # | flag

Hello! I dont understand limit to for "(10*10000)", you can explain me? Thanks!