Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure TERRA REF backed up to TACC #12

Open
17 of 90 tasks
dlebauer opened this issue Feb 13, 2020 · 0 comments
Open
17 of 90 tasks

Ensure TERRA REF backed up to TACC #12

dlebauer opened this issue Feb 13, 2020 · 0 comments
Assignees
@julianpistorius
Labels
Epic ua-gantry

Comments

@dlebauer
Copy link
Contributor

dlebauer commented Feb 13, 2020 •
edited by julianpistorius
Loading

  • first priority is season 3-9 raw_data/ plus Level_1/laser3Dply
    • in raw_data, you can skip 3DScannerRawDataTopTmp, UAV, ua-mac, TestIn, 3DTest, scanner3DLowerOnWestSide, FLIR_TG167_handheld, phenotractor
  • for seasons 1-2, there is no raw_data folder
    • back up raw_data/irrigation and Level_1 for this season
  • for data outside season date ranges, call it 'noseason'
  • other derived data are lower priority b/c they can be re-generated (albiet slowly)
  • create a table that lists locations and content of each tar file
    • tar filename + tarfile md5 and for each file: directory path, filename, file size
  • use same tar as Sean used for archiving (not the backup which was less structured)

Progress and notes: https://hackmd.io/@julianp-diag/SJRDa0vSw

Currently on Ranch@TACC:

  • Season 9

    • Level 0
      • VNIR
      • stereoTop
      • scanner3DTop
      • SWIR
      • sensor_group
    • Level 1
    • Level 2
    • Level 3

  • Season 8

    • Level 0
      • VNIR
      • stereoTop
      • scanner3DTop
      • SWIR
      • sensor_group
    • Level 1
    • Level 2
    • Level 3

  • Season 7

    • Level 0
      • VNIR
      • stereoTop
      • scanner3DTop
      • SWIR
      • sensor_group
    • Level 1
    • Level 2
    • Level 3

  • Season 6

    • Level 0
      • VNIR
      • stereoTop
      • scanner3DTop
      • SWIR
      • sensor_group (retrying tar step)
    • Level 1
    • Level 2
    • Level 3

  • Season 5

    • Level 0
      • VNIR
      • stereoTop
      • scanner3DTop
      • SWIR
      • sensor_group (FAILED)
    • Level 1
    • Level 2
    • Level 3

  • Season 4

    • Level 0
      • VNIR
      • stereoTop (in progress: transferring)
      • scanner3DTop (in progress: transferring)
      • SWIR
      • sensor_group
    • Level 1
    • Level 2
    • Level 3

  • Season 3

    • Level 0
      • VNIR
      • stereoTop
      • scanner3DTop
      • SWIR
      • sensor_group
    • Level 1
    • Level 2
    • Level 3

  • Season 2

    • Level 0
      • VNIR
      • stereoTop
      • scanner3DTop
      • SWIR
      • sensor_group
    • Level 1
    • Level 2
    • Level 3

  • Season 1

    • Level 0
      • VNIR
      • stereoTop
      • scanner3DTop
      • SWIR
      • sensor_group
    • Level 1
    • Level 2
    • Level 3
@dlebauer dlebauer added the Epic label Feb 13, 2020
@dlebauer dlebauer changed the title Ensure TERRA REF and Open data backed up to TACC and GDRIVE Ensure TERRA REF backed up to TACC and GDRIVE Mar 26, 2020
@julianpistorius julianpistorius mentioned this issue Jun 22, 2020
Closed
@julianpistorius julianpistorius mentioned this issue Jul 6, 2020
Closed
7 tasks
@dlebauer dlebauer changed the title Ensure TERRA REF backed up to TACC and GDRIVE Ensure TERRA REF backed up to TACC Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@julianpistorius julianpistorius

Labels
Epic ua-gantry
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dlebauer @julianpistorius