Arcctl job datastaging
Have a new action remote_dds
[almalinux@uio-atlas-ce1 control]$ arcctl datastaging -h
usage: arcctl datastaging [-h] ACTION ...
optional arguments:
-h, --help show this help message and exit
DataStaging Control Actions:
ACTION DESCRIPTION
job Job Datastaging Information for a preparing or running job.
summary Job Datastaging Summary Information for jobs preparing or running.
remote_dds
Remote Datastaging file information for preparing jobs.
dtr Data-delivery transfer (DTR) information
When executing arcctl datastaging remote_dds action you will get a list of all remote datastaging services where there are ongoing or done file transfers for jobs that currently are in the PREPARING state.
Sample output:
sudo arcctl datastaging remote_dds
[almalinux@uio-atlas-ce1 control]$ sudo arcctl datastaging remote_dds
Datadelivery service and file info for jobs in PREPARING state, which are currently been staged in or are in progress of being staged in at a remote datadelivery service.
COUNTER REMOTE-DELIVERY ARC-ID FILENAME SOURCE SIZE (MB) START END SECONDS
1 http://10.2.2.125:33555/datadeliveryservice 5439ff287f13 pilot3.tar.gz http://pandaserver.cern.ch:25085/cache/pilot 0.4 2024-08-23 11:19:54 2024-08-23T09:19:56Z 2
2 http://10.2.2.125:33555/datadeliveryservice 543955dbdea9 user.weitao.31350488._000047.CxAOD.root rucio://rucio-lb-prod.cern.ch/replicas/user.weitao 4174.5 2024-08-23 11:19:54 2024-08-23T09:20:55Z 60
3 http://10.2.0.137:33555/datadeliveryservice 5439ff287f13 user.weitao.31350488._000046.CxAOD.root rucio://rucio-lb-prod.cern.ch/replicas/user.weitao 4004.0 2024-08-23 11:19:54 2024-08-23T09:20:12Z 18
4 http://10.2.0.150:33555/datadeliveryservice 543961bcb6f2 user.weitao.31350488._000048.CxAOD.root rucio://rucio-lb-prod.cern.ch/replicas/user.weitao 4179.0 2024-08-23 11:19:55 2024-08-23T09:20:44Z 49
5 http://10.2.0.231:33555/datadeliveryservice 543a3b74797b user.weitao.31350488._000049.CxAOD.root rucio://rucio-lb-prod.cern.ch/replicas/user.weitao 2646.9 2024-08-23 11:19:55 2024-08-23T09:20:13Z 18
6 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 pandaJobData.out - -1.0 2024-08-23 11:24:19 - -1
7 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 runpilot2-wrapper.sh - -1.0 2024-08-23 11:24:19 - -1
8 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 pilot3.tar.gz - -1.0 2024-08-23 11:24:19 - -1
9 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 queuedata.json - -1.0 2024-08-23 11:24:19 - -1
10 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 panda.um.user.hpacey.601355.e8547_s4231_r13145_p6266.METPerf - -1.0 2024-08-23 11:24:19 - -1
11 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 panda.um.user.hpacey.601355.e8547_s4231_r13145_p6266.METPerf - -1.0 2024-08-23 11:24:19 - -1
12 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 panda.um.user.hpacey.601355.e8547_s4231_r13145_p6266.METPerf - -1.0 2024-08-23 11:24:19 - -1
13 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 panda.um.user.hpacey.601355.e8547_s4231_r13145_p6266.METPerf - -1.0 2024-08-23 11:24:19 - -1
14 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92 panda.um.user.hpacey.601355.e8547_s4231_r13145_p6266.METPerf - -1.0 2024-08-23 11:24:19 - -1
15 http://10.2.0.78:33555/datadeliveryservice 5538e30f8f92
In addition the
arcctl job datastaging get-details <jobid> now shows also what remote datadelivery service the files got downloaded at, if such exists
Example:
[root@uio-atlas-ce1 ~]# arcctl job datastaging get-details 4ab87bfb8307
Information about input-files for arcid 4ab87bfb8307
State of input-files:
COUNTER FILENAME STAGED-IN
1 pandaJobData.out no
2 runpilot2-wrapper.sh no
3 pilot3.tar.gz yes
4 queuedata.json no
5 data23_13p6TeV.00456729.physics_Main.daq.RAW._lb0411._SFO-12 yes
6 data23_13p6TeV.00456729.physics_Main.daq.RAW._lb0411._SFO-12 yes
7 data23_13p6TeV.00456729.physics_Main.daq.RAW._lb0411._SFO-12 yes
8 data23_13p6TeV.00456729.physics_Main.daq.RAW._lb0411._SFO-13 yes
9 UIO_CLOUD.all.json yes
Note: files uploaded by the client appear to not be staged-in, ignore these as AREX does not handle the stage-in of these files.
Details for files that have been staged in - both downloaded and cached:
COUNTER FILENAME SOURCE SIZE (MB) START END SECONDS CACHED
1 pilot3.tar.gz http://pandaserver.cern.ch:25085/cache/pilot 0.447 2024-08-23T08:39:21Z 2024-08-23T08:41:48Z 147 yes
2 data23_13p6TeV.00456729.physics_Main.daq.RAW._lb0411._SFO-12 rucio://rucio-lb-prod.cern.ch/replicas/data23_13p6TeV 5000.295 2024-08-23T08:39:21Z 2024-08-23T08:51:46Z 599 no
3 data23_13p6TeV.00456729.physics_Main.daq.RAW._lb0411._SFO-12 rucio://rucio-lb-prod.cern.ch/replicas/data23_13p6TeV 4999.507 2024-08-23T08:39:21Z 2024-08-23T08:51:46Z 599 no
4 data23_13p6TeV.00456729.physics_Main.daq.RAW._lb0411._SFO-12 rucio://rucio-lb-prod.cern.ch/replicas/data23_13p6TeV 1101.420 2024-08-23T08:39:21Z 2024-08-23T08:51:46Z 599 no
5 data23_13p6TeV.00456729.physics_Main.daq.RAW._lb0411._SFO-13 rucio://rucio-lb-prod.cern.ch/replicas/data23_13p6TeV 5000.219 2024-08-23T08:39:21Z 2024-08-23T08:51:46Z 599 no
6 UIO_CLOUD.all.json http://pandaserver.cern.ch:25085/cache/schedconfig 0.005 2024-08-23T08:39:21Z 2024-08-23T08:54:21Z 900 yes
Fine-grained details for files that have been staged-in by download (not cached):
COUNT FILENAME SIZE (MB) START END SCHEDULER-START DELIVERY-START TRANSFER-DONE ALL-DONE (s) (MB/s) DELIVERY-SERVICE
1 data23_13p6TeV. 5000.295 2024-08-23T08:39:21Z 2024-08-23T08:51:46Z 2024-08-23 10:39:21 2024-08-23 10:41:38 2024-08-23 10:51:37 2024-08-23 10:51:39 599 8.3 http://10.2.0.231:33555/datadeliveryservice
2 data23_13p6TeV. 4999.507 2024-08-23T08:39:21Z 2024-08-23T08:51:46Z 2024-08-23 10:39:21 2024-08-23 10:41:38 2024-08-23 10:51:37 2024-08-23 10:51:39 599 8.3 http://10.2.0.214:33555/datadeliveryservice
3 data23_13p6TeV. 1101.420 2024-08-23T08:39:21Z 2024-08-23T08:51:46Z 2024-08-23 10:39:21 2024-08-23 10:41:38 2024-08-23 10:51:37 2024-08-23 10:51:39 599 1.8 http://10.2.1.42:33555/datadeliveryservice
4 data23_13p6TeV. 5000.219 2024-08-23T08:39:21Z 2024-08-23T08:51:46Z 2024-08-23 10:39:21 2024-08-23 10:41:38 2024-08-23 10:51:37 2024-08-23 10:51:39 599 8.3 http://10.2.0.180:33555/datadeliveryservice
Edited by Maiken