Skip to content
This repository has been archived by the owner on Apr 5, 2024. It is now read-only.

[Epic] Polish Core Datasets #268

Closed
7 of 14 tasks
sglavoie opened this issue May 11, 2020 · 1 comment
Closed
7 of 14 tasks

[Epic] Polish Core Datasets #268

sglavoie opened this issue May 11, 2020 · 1 comment

Comments

@sglavoie
Copy link

sglavoie commented May 11, 2020

As a PM, I want to review core datasets and make sure all of them are up to date so that I'm sure in their quality

As a PM, I want to re-run and make sure scrapers/scripts are working fine so that I can update data anytime

As a PM, I want to translate scrapers/scripts into dataflows, where possible, so that we use our tools to get the data, plus we can easily update them

As a PM, I want to review READMEs of the core dataset and update where necessary, so that I (and users) are sure that dataset descriptions are accurate enough

Acceptance Criteria

  • We have the latest data for all core datasets
  • We have all of the non-complex scripts working OK (unless the source is broken)
  • We have the missing sources and complex scripts fixed up
  • READMEs are up to date
  • We use dataflows to get the data
    • Automated by Travis

Tasks

  • List all datasets
  • Find and fix the non-complex scripts that are not currently working and review READMEs
  • Fix scripts that are complex and have non-common errors
  • translate scripts to dataflows
  • Fix/refactor more simple scripts addendum #267
  • Fix the broken source datasets #266
  • Fix scripts that require further analysis and debugging #265
  • Run on schedule by travis

Created by @zelima

@rufuspollock
Copy link
Member

FIXED or DUPLICATE of datasets/awesome-data#376

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants