Skip to content
janderse edited this page Aug 30, 2014 · 2 revisions

You need four or five tools:

  1. An IDE such as eclipse/spe/intellij: for writing code
  2. lxml-xpath-tester: for testing xpath queries; See http://xpath.courtlistener.com
  3. ipython: run and test code (your IDE may do this)
  4. http://www.gskinner.com/RegExr/ : test regular expressions
  5. git and github.

Assuming you have git installed, have a github login and have forked juriscraper on github, and have locally cloned your fork of juriscraper (do: git clone https://github.com/YOURUSERNAME/juriscraper.git )

An ipython example, assuming you are editing the file "armfor.py":

cd /usr/local/juriscraper

ipython

import juriscraper

from juriscraper.opinions.united_states.federal_special import armfor

site = armfor.Site(); site.parse()

print site

(Make some changes to armfor.py in your IDE)

reload(armfor); site = armfor.Site(); site.parse();

print site

(continue the last three steps until it works)

But always test with:

cd /usr/local/juriscraper

python sample_caller.py -c opinions.united_states.federal_special.armfor -b -vv

and

python tests/tests.py

git add [path_to_your_file/file.py]

git status -do things look sane?

git commit

or if you just have one thing to commit:

git commit -m "commit message" filename

git push -use your github login

go to github and issue pull request.

Common mistake: if you create a new scraper, be sure to add its filename to the __init__.py file in its directory.

Clone this wiki locally