The support source code for Jäger is now available (licensed under the LGPL), http://jaeger.blogmatrix.com/source/. This encompasses three libraries, each in their own directory:

• generic: general helper classes and functions
• druecken: the Drücken HTML downloading and scraping libraries
• search: the Universal Search Parser

We will describe the files in "generic" and "druecken" at a later date. Some of this code is very useful, some less so.

The Universal Search Parser (USP) is an attempt to provide a consistent extensible method to utilize online search resources within a Python program (and in the future, as RSS or RDF results). It's quite easy to use; for example, here's how you can get all the listings for TRS-80s available on EBay:

import Search
for result in Search.search('search:ebay TRS-80 pages:all'):
 pprint.pprint(result)

Yielding:

{'Bidders': 0,
 'BuyItNow': u'$15.00',
 'Price': u'$10.00',
 '_link': 'http://cgi.ebay.com/ws/...1247&item=5122154379&rd=1',
 '_title': u'Sol 20, Exidy, TRS-80, NorthStar, BASIC, Assembler, etc'}
{'Bidders': 4,
 'Price': u'$15.50',
 '_link': 'http://cgi.ebay.com/ws/...=74947&item=5123221664&rd=1',
 '_title': u'RADIO SHACK TRS-80 POCKET COMPUTER MODEL PC-4 W/CASE'}
{'Bidders': 0,
 'Price': u'$0.99',
 '_link': 'http://cgi.ebay.com/ws/...=74947&item=5123225351&rd=1',
 '_title': u'Downland for TRS 80 Color Computer'}
...

What else can we search? Right now, we have the following modules implemented (there are example searches within each file):

• SearchAmazon.py (requires pyamazon)
• SearchBBC.py
• SearchCBC.py
• SearchCNN.py
• SearchCanada411.py
• SearchEbay.py
• SearchGoogle.py
• SearchIMDB.py
• SearchSourceforge.py
• SearchTechnorati.py (requires pytechnorati)
• SearchWhitepages.py
• SearchYahoo.py

To install the USP (and all the other libraries), do the following:

• download the latest copy of "jaeger-support-*.tar.gz" file
• unpack it
• "cd search"
• try "python Search.py 'some query'"

The main source code for Jäger will be along later this week or by Monday at the latest. The USP will be integrated into Jäger for both "immediate" and "persistent" searching in the very near future.

If you give this code a try and you like it (or hate it), please send along an e-mail (or even send a donation): I'd love to hear from you in any case. If there's any other Search modules you'd like implemented, send me a note and I'll see what I can do, or if you're so inclined try it yourself – it's not too difficult.