4 replies to this topic

[roxygirl123]

I want to pull text strings off a website, and write them to a .txt file or even a spreadsheet.
Does anyone have an idea of how to got about this?
I will be very grateful :)




-----------
Damsel in distress

[Flying Dutchman]

You can do it super easy with urllib and urllib2 modules. To parse text you can do it by hand or use regexes (regular expressions).

import urllib
import re


URL = "http://www.xkcd.com"


for line in urllib.urlopen(URL).readlines():
	if "<title>" in line:
		print line[9:-9] # print without html tags
		break


regex = re.compile(r"\<title\>(.*?)\</title\>")


for line in regex.findall(urllib.urlopen(URL).read()):
	print line

Above code gets the latest comic title from xkcd.

[brokenbylaw]

I'd use urllib2 and you can use HTMLParser to parse the html.

[Flying Dutchman]

Forgot to mention earlier, there's also a BeautifulSoup module for parsing html.

[ReekenX]

There is also LXML which can help you parse HTML easilly if you know XPATH.