Parses schedule page, gets links to .xlsx-files, parses them then, builds table models for each and every possible study group, updates DB schedule for each group of those ones.
If necessary, notifies via notifier.
- Install all dependencies
npm install - Run scrapper for first time
npm run scrapper
Cron job for running scrapper on it's own schedule: at minute 0 past every 3rd hour. Very simple, very reliable.
0 */3 * * * cd $MSS/scrapper && node scrapper.js
scapper.fixes.json – File for correcting typos in original .xlsx-files.
scapper.config.json – File for tokens, URLs, etc.