Checking FTS3 Works

For the data challenge DC4 we are using exclusively fts3-public.cern.ch

Monitoring can be seen at

https://fts3-public.cern.ch:8449/fts3/ftsmon

Ingest daemon generates transfers that go from np04-srv-002.cern.ch and np04-srv-003.cern.ch to eospublic.cern.ch

Declaration daemon generates transfers that go from eospublic.cern.ch to fndca1.fnal.gov

You can watch both kinds of transfers and make sure that they keep happening and keep succeeding.

There is also a grafana page at

https://monit-grafana.cern.ch/d/veRQSWBGz/fts-servers-dashboard?orgId=25&var-group_by=vo&var-vo=dune&var-src_country=All&var-dst_country=All&var-src_site=All&var-source_se=All&var-dst_site=All&var-dest_se=All&var-fts_server=All&var-staging=All&var-bin=1h&var-dest_hostname=All&var-source_hostname=All

If you do not see any dune FTS3 transfers being generated, then there is something wrong. Either (1) the file creation daemons aren't making files or metadata, or (2) the ingest daemon isn't pulling them to EOS (np04-srv-002 and 003 transfers would be missing in this case) or (3) the declaration daemon isn't making rucio rules (cern->fnal transfers would be missing in this case, or (4) rucio is busted and can't submit any rules (cern->fnal transfers also missing in this case.

Errors in the declaration daemon logs would tell a lot about (3) or (4).