Download data #72
Merged
Download data #72
+342
−127
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
these are intended to be run as pipenv scripts: pipenv run download_data pipenv run delete_data they can also be run from file
Updated Pipfile to specify versions. Updated Scripts definitions. Moved scripts to process_data repository. Edited header comment in download_data.
jhanley634
approved these changes
Jan 24, 2025
|
Hullo, James. Just checking in. This branch looks like it's nearly ready for a merge. Do my remarks mostly make sense? Is there any trouble with |
skyfenton
reviewed
Feb 4, 2025
Corrected nits and did some clean-up. Simplified the download_data function with pathlib.
generate new pipfile
Manually combined pipfile and regenerated pipfile.lock, reverted requests version to try and avoid problems
tried to fix errors in pipfile
jhanley634
reviewed
Feb 5, 2025
skyfenton
previously requested changes
Feb 10, 2025
There was a problem hiding this comment.
Heya, just a couple extra comments to help get this finished! Basically it seems like just five(ish) changes (take as many commits as you need):
- Modify the download_data operations to extract the tar.gz download to the same folder, delete the original file, then extract training_set.tar into training_set/.
- Simplify clean_up_data (honestly, probably could just remove it and use just rm).
- Reorder pipfile alphabetically
- Delete pipfile.lock and run pipenv install again to regenerate it.
- Pull from main and resolve any merge conflicts.
|
I've fixed the merge conflict with the following commands: The Let's merge this now and work on follow ups in #85. I've copied any remain comments into that issue and resolved them. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Converted my fork into a branch called Download-Data