DM-48349: Initial draft #16
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: CI | |
| "on": [push, pull_request] | |
| jobs: | |
| build: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v4 | |
| with: | |
| fetch-depth: 0 # full history for metadata | |
| submodules: true | |
| - name: Set up Python | |
| uses: actions/setup-python@v5 | |
| with: | |
| python-version: "3.10" | |
| - name: Python install | |
| run: | | |
| python -m pip install --upgrade pip | |
| python -m pip install "lander<2.0.0" | |
| - name: Login to Docker Hub | |
| uses: docker/login-action@v1 | |
| with: | |
| username: sqrereadonly | |
| password: ${{ secrets.DOCKERHUB_SQREREADONLY_TOKEN }} | |
| - name: TeX build | |
| run: | | |
| docker run --rm -v `pwd`:/workspace -w /workspace lsstsqre/lsst-texmf:latest sh -c 'make' | |
| - name: Landing page upload | |
| if: ${{ github.event_name == 'push' }} | |
| env: | |
| LTD_PASSWORD: ${{ secrets.LTD_PASSWORD }} | |
| LTD_USERNAME: ${{ secrets.LTD_USERNAME }} | |
| run: | | |
| lander --upload --pdf DMTN-306.pdf --lsstdoc DMTN-306.tex --ltd-product dmtn-306 --author "Fabio Hernandez" --author "George Beckett" --author "Andrew Hanushevsky" --author "Tim Jenness" --author "Kian-Tat Lim" --author "Peter Love" --author "Timothy Noble" --author "Stephen R. Pietrowicz" --author "Wei Yang" --abstract "The set of sky images recorded nightly by the camera mounted on the telescope of the Vera C. Rubin Observatory will be processed in facilities located on three continents. Data acquisition will take place at Cerro Pachón in the Andes mountains in Chile, where the observatory is located. A first copy of the raw image data set is stored at the summit site of the observatory and immediately transferred through dedicated network links to the archive site and the US Data Facility hosted at SLAC National Laboratory in California, USA. After an embargo period of a few days, the full image set is copied to the UK and French Data Facilities, where a third copy is located. During its 10 years in operation starting late 2025, annual processing campaigns across all images taken to date will be jointly performed by the three facilities, involving sophisticated algorithms to extract the physical properties of the celestial objects and producing science-ready images and catalogs. Data products resulting from the processing campaigns at each facility will be sent to SLAC and combined to create a consistent Data Release, which is served to the scientific community for its science studies via Data Access Centers in the US and Chile and Independent Data Access Centers elsewhere. In this contribution we present an overall view of how we leverage the tools selected for managing the movement of data among the Rubin processing and serving facilities, including Rucio and FTS3. We will also present the tools we developed to integrate Rucio's data model and Rubin's Data Butler, the software abstraction layer that mediates all access to storage by pipeline tasks that implement science algorithms." |