The functionality is written in Typescript.
The program uses Puppeteer (with Plugins) to generate a headless browser instance that traverses the Circuit court pages.
- Install yarn if you haven't already.
- Clone repository.
- run yarn install
- run yarn start
If you don't want to have to restart the scraper everytime you make a change, you can start the program by using yarn dev instead of yarn start.
What this does is enable nodemon to watch the /src directory for any changes in the *.ts files and restarts the index.ts module.
WARNING: If you are not careful about closing the automated browsers after making changes, you may end up with dozens of chromium app instances running.
Update! Nodemon now ends the current browser session on restart.
Before running the program in a cloud production environment, change the headless option to false in the initBrowser.ts file. Otherwise the cloud instance will try to run the browser gui and crash.
The following env variables need to be set in a .env file
# Court login credentials
[email protected]
LOGIN_PASSWORD=iamanfbiagent