Jump to content

ALICE/CCDB

From MukeWiki

ALICE CCDB URL

$ curl http://alice-ccdb.cern.ch
Usage of /*:
  GET:
    task name / detector name / start time [ / UUID] - return the content of that UUID, or
    task name / detector name / [ / time [ / key = value]* ]

  POST:
    task name / detector name / start time / end time [ / UUID ] [ / key = value ]*
    binary blob as multipart parameter called 'blob'
  PUT:
    task name / detector name / start time [ / new end time ] [ / UUID ] [? (key=newvalue&)* ]
  DELETE:
    task name / detector name / start time [ / UUID ]
    or any other selection string, the matching object will be deleted

Usage of /browse/* or /latest/*:
  GET:
    task name / detector name / [start time, default = now] [/key=value]*
    Use the Accept header to control the output format (one of text/plain, text/html, text/xml, application/json)

HTTP (RESTfull) method GET - retrieve the content (query by path / time). Usage of browse - GET listing of all matching objects and subfolders. Other HTTP methods - POST (upload new objects), PUT (update existing objects) and DELETE, are not interesting yet (and of course require CERN certificate).

http://alice-ccdb.cern.ch/browse/CTP   # 2025-08-21 (Central Trigger Processor)

CTP/Calib/Align               # one object, valid interval: 01 Jan 1970 — 20 Nov 2286, last modified: 05 May 2022
CTP/Calib/CTFDictionary       # two objects, replaced by CTP/Calib/CTFDictionaryTree, see also O2-3260
CTP/Calib/CTFDictionaryTree   # one object, valid interval: 01 Jan 2019 — 31 Dec 2025, last modified: 06 Mar 2025
CTP/Calib/FirstRunOrbit       #   6523 objects, last object: 21 Aug 2025, first object: 15 Aug 2023
CTP/Calib/OrbitReset          # 209798 objects, last object: 21 Aug 2025, first object: 15 Nov 2021
CTP/Calib/Scalers             #  12793 objects, last object: 21 Aug 2025, first object: 29 Jun 2022

CTP/ClassNameToIndexMap       #   5528 objects, last object: 11 Oct 2022, first object: 08 Sep 2021

CTP/Config/Config             #  14401 objects, last object: 21 Aug 2025, first object: 29 Jun 2022
CTP/Config/TriggerOffsets     # two objects, valid interval: 10 Oct 2023 — 20 Nov 2286, last modified: 22 Oct 2023
http://alice-ccdb.cern.ch/browse/RCT   # 2025-08-21 (Run Condition Table)

RCT/Flags/RunFlags        #   2385 objects, last object: 07 Aug 2025, first object: 04 Feb 2025, writing in "batches"

RCT/Info/RunInformation   # 243393 objects, last object: 21 Aug 2025, first object: 02 May 2022

RCT/RunInformation        # 224144 objects, before start Run 3, last modified: 29 Mar 2022

NOTE Run 3 - LHC became operational again on 22 April 2022, physics season from 5 July 2022. The run numbering for ALICE Run 3 starts at ~ 500 000. At this time, RCT/RunInformation was completely replaced by RCT/Info/RunInformation.

Prehliadat cez web browser CCDB zoznamy objektov, obzvlast pri vacsom objeme dat, nie je prilis vhodne (pomale, resp. moznost zahltenia az padnutia browser-a). Stiahnutie konkretneho CCDB zoznamu (napr. pomocou curl(1)rpm) na lokalny disk (radovo sekundy) je praktickejsie.

$ curl http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers > alice-ccdb.CTP.Calib.Scalers.$(date +%F)
$ curl http://alice-ccdb.cern.ch/browse/CTP/Config/Config > alice-ccdb.CTP.Config.Config.$(date +%F)
$ curl http://alice-ccdb.cern.ch/browse/RCT/Info/RunInformation > alice-ccdb.RCT.Info.RunInformation.$(date +%F)
# Output format controllable by the Accept header or URL parameter
$ curl -H "Accept: text/xml"         http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers > CTP.Calib.Scalers.xml
$ curl -H "Accept: application/json" http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers > CTP.Calib.Scalers.json
# $ curl -H "Accept: text/html"    # => for browser
# $ curl -H "Accept: text/plain"   # => by default

HTTP REST

Query format for any HTTP method is /Detector/Category/Param/tStart[/tEnd][/UUID][/key=value]. Folder structure and the reference time for most requests are mandatory. User defined metadata associated to each object, can also filter by it.

NOTE Each object has an Interval of Validity (IoV) that is defined as an interval that includes the Start of Validity (SoV), but excludes the End of Validity (EoV). Thus IoV = [SoV, EoV). ALICE CCDB uses PostgreSQL's timestamp range data type tsrange, that stores a time interval as a single column with dedicated operators, including the CONTAINS one (@>).

# HTTP method GET retrieve the content of latest (recent) object
$ curl    http://alice-ccdb.cern.ch/CTP/Calib/Scalers/1754317315067   # do nothing
$ curl -i http://alice-ccdb.cern.ch/CTP/Calib/Scalers/1754317315067   # 1 object
# /browse GET listing of all matching objects and subfolders
$ curl    http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers/1754317315067   # 3 objects
$ curl -i http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers/1754317315067   # 3 objects
# /latest GET like the above but only the most recent object
$ curl    http://alice-ccdb.cern.ch/latest/CTP/Calib/Scalers/1754317315067   # 1 object
$ curl -i http://alice-ccdb.cern.ch/latest/CTP/Calib/Scalers/1754317315067   # 1 object

# curl(1) option -i, --show-headers show response headers in the output
Example of timestamp range (path: CTP/Calib/Scalers, run: 565118)
Interval of Validity (IoV) < 1754317315067, 1754582883000 ) milliseconds
1754317315067 run 565118 1754582882999 another run (565190)
1754317315066 run 565097 1754582882999/runNumber=565118 run 565118
1754317315068 run 565118 1754582883000 another run (565190)
1754417315067 another run (565140) 1754582883000/runNumber=565118 not found
1754417315067/runNumber=565118 run 565118
$ curl -i http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers/1754317315067                  # 3 run objects
$ curl -i http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers/1754317315067/runNumber=565118 # 1 run object
$ curl -i http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers/runNumber=565118 # works without ? reference time ?
$ curl -i http://alice-ccdb.cern.ch/CTP/Calib/Scalers/runNumber=565118          # not working

$ curl -i http://alice-ccdb.cern.ch/browse/RCT/Info/RunInformation/565118
$ curl -i http://alice-ccdb.cern.ch/browse/RCT/Info/RunInformation/565118/SOR=1754317914890
$ curl -i http://alice-ccdb.cern.ch/browse/RCT/Info/RunInformation/SOR=1754317914890

Timestamp

# $ curl http://alice-ccdb.cern.ch/browse/CTP/Calib/Scalers/1754317315067/337db9d2-714d-11f0-87fb-0aa140641b9a
Listing content of CTP/Calib/Scalers

ID: 337db9d2-714d-11f0-87fb-0aa140641b9a
Path: CTP/Calib/Scalers
Validity: 1754317315067 - 1754582883000 (Mon Aug 04 16:21:55 CEST 2025 - Thu Aug 07 18:08:03 CEST 2025)
Initial validity limit: 1754582883000 (Thu Aug 07 18:08:03 CEST 2025)
Created: 1754323683878 (Mon Aug 04 18:08:03 CEST 2025)
Last modified: 1754323683880 (Mon Aug 04 18:08:03 CEST 2025)
Original file: o2-ctp-CTPRunScalers_1754323683866.root, size: 223232, md5: a9a3987283edadd7cc52200b3430bd5b, content type: application/root
Uploaded from: 10.161.64.100
Metadata:
  ObjectType = o2::ctp::CTPRunScalers
  runNumber = 565118
  partName = send

Timestamp 1754323683878 predstavuje Unix epoch time. ALICE CCDB vsak pouziva millisecond resolution, na rozdiel od klasickeho Unix epoch time, ktory pouziva "len" second resolution. Pri prevode zo/do second resolution sa jednoducho odsekne/prida 3-digit millisecond part. Analogicky, vsetky ostatne timestamps pouzivaju millisecond resolution.

$ epoch=1754323683878
$ date --date="@$epoch"
Sun Apr 22 08:24:38 AM CEST 57562   # not correct, date(1)rpm works with second resolution
$ epoch="${epoch::-3}"              # remove last 3 (milliseconds) chars
$ echo $epoch
1754323683
$ date --date="@$epoch" "+%a %b %d %T %Z %Y"   # or LC_TIME=C date --date="@$epoch"
Mon Aug 04 18:08:03 CEST 2025

$ date +%s; date +%s%3N; date +%s%N; date +%s%N | cut -b1-13   # get the current time in milliseconds

Timestamp 337db9d2-714d-11f0-87fb-0aa140641b9a predstavuje time based UUID, identicky timestamp ako Created, ale v time based UUID formate (version 1, date-time and MAC address). Dekodovanie UUID v Linux najjednoduchsie pomocou prikazu uuidparse(1)rpm.

$ uuid=337db9d2-714d-11f0-87fb-0aa140641b9a
$ uuidparse "$uuid"
UUID                                  VARIANT TYPE       TIME
337db9d2-714d-11f0-87fb-0aa140641b9a  DCE     time-based 2025-08-04 18:08:03,878344+02:00

$ uuid_datime="$(uuidparse --noheadings --output TIME $uuid)"
$ echo "$uuid_datime"
2025-08-04 18:08:03,878344+02:00
$ date --date="$uuid_datime" "+%a %b %d %T %Z %Y"
Mon Aug 04 18:08:03 CEST 2025
$ date --date="$uuid_datime" "+%s"
1754323683   # only second resolution (manually add the 3-digit millisecond part if necessary)
millis="${uuid_datime#*,}"   # keep all chars after first ','
millis="${millis:0:3}"       # keep first 3 chars
$ date --date="$uuid_datime" "+%s$millis"
1754323683878
$ date --date="$uuid_datime" "+%s%3N"   # simple solution
1754323683878

CTP and RCT

RCT/Info/RunInformation
ID: c5d553a0-713f-11f0-87fb-0aa145271b9a
Path: RCT/Info/RunInformation
Validity: 565118 - 565119 (Thu Jan 01 01:09:25 CET 1970 - Thu Jan 01 01:09:25 CET 1970)
Initial validity limit: 565119 (Thu Jan 01 01:09:25 CET 1970)
Created: 1754317916445 (Mon Aug 04 16:31:56 CEST 2025)
Last modified: 1754323749606 (Mon Aug 04 18:09:09 CEST 2025)
Original file: tmp.dat, size: 1, md5: 93b885adfe0da089cdf634904fd59f71, content type: application/x-ns-proxy-autoconfig
Uploaded from: 10.161.69.39
Metadata:
  ObjectType = char
  STF = 1754317965555
  ETF = 1754323682151
  SOR = 1754317914890
  UpdatedFrom = 10.161.69.39
  EOR = 1754323749155
  SOX = 1754317966072
  partName = send
  EOX = 1754323683903

# alien:///alice/data/CCDB/RCT/Info/RunInformation/15/35932/c5d553a0-713f-11f0-87fb-0aa145271b9a
# http://alice-ccdb.cern.ch/download/c5d553a0-713f-11f0-87fb-0aa145271b9a => tmp.dat
# 1754317914890 | Mon Aug 04 16:31:54,890 CEST 2025 | Metadata SOR (Start of Run)
# 1754317916445 | Mon Aug 04 16:31:56,445 CEST 2025 | Created (ID: c5d553a0-713f-11f0-87fb-0aa145271b9a)
# 1754317965555 | Mon Aug 04 16:32:45,555 CEST 2025 | Metadata STF (Start of Time Frame)
# 1754317966072 | Mon Aug 04 16:32:46,072 CEST 2025 | Metadata SOX (Start of eXchange)
# 1754323682151 | Mon Aug 04 18:08:02,151 CEST 2025 | Metadata ETF (End of Time Frame)
# 1754323683903 | Mon Aug 04 18:08:03,903 CEST 2025 | Metadata EOX (End of eXchange)
# 1754323749155 | Mon Aug 04 18:09:09,155 CEST 2025 | Metadata EOR (End of Run)
# 1754323749606 | Mon Aug 04 18:09:09,606 CEST 2025 | Last modified
CTP/Config/Config
ID: c5037599-713f-11f0-87fb-0aa140641b9a
Path: CTP/Config/Config
Validity: 1754317315067 - 1754577115067 (Mon Aug 04 16:21:55 CEST 2025 - Thu Aug 07 16:31:55 CEST 2025)
Initial validity limit: 1754577115067 (Thu Aug 07 16:31:55 CEST 2025)
Created: 1754317915070 (Mon Aug 04 16:31:55 CEST 2025)
Last modified: 1754317915072 (Mon Aug 04 16:31:55 CEST 2025)
Original file: o2-ctp-CTPConfiguration_1754317915068.root, size: 4096, md5: 5b6026d38ca715983392563b3cd26a9d, content type: application/root
Uploaded from: 10.161.64.100
Metadata:
  ObjectType = o2::ctp::CTPConfiguration
  runNumber = 565118
  partName = send

# alien:///alice/data/CCDB/CTP/Config/Config/08/61111/c5037599-713f-11f0-87fb-0aa140641b9a
# http://alice-ccdb.cern.ch/download/c5037599-713f-11f0-87fb-0aa140641b9a => o2-ctp-CTPConfiguration_1754317915068.root
# 1754317315067 | Mon Aug 04 16:21:55,067 CEST 2025 | Valid from
# 1754317915068 | Mon Aug 04 16:31:55,068 CEST 2025 | o2-ctp-CTPConfiguration_1754317915068.root
# 1754317915070 | Mon Aug 04 16:31:55,070 CEST 2025 | Created (ID: c5037599-713f-11f0-87fb-0aa140641b9a)
# 1754317915072 | Mon Aug 04 16:31:55,072 CEST 2025 | Last modified
# 1754577115067 | Thu Aug 07 16:31:55,067 CEST 2025 | Initial validity limit
# 1754577115067 | Thu Aug 07 16:31:55,067 CEST 2025 | Valid until
CTP/Calib/Scalers
ID: 337db9d2-714d-11f0-87fb-0aa140641b9a
Path: CTP/Calib/Scalers
Validity: 1754317315067 - 1754582883000 (Mon Aug 04 16:21:55 CEST 2025 - Thu Aug 07 18:08:03 CEST 2025)
Initial validity limit: 1754582883000 (Thu Aug 07 18:08:03 CEST 2025)
Created: 1754323683878 (Mon Aug 04 18:08:03 CEST 2025)
Last modified: 1754323683880 (Mon Aug 04 18:08:03 CEST 2025)
Original file: o2-ctp-CTPRunScalers_1754323683866.root, size: 223232, md5: a9a3987283edadd7cc52200b3430bd5b, content type: application/root
Uploaded from: 10.161.64.100
Metadata:
  ObjectType = o2::ctp::CTPRunScalers
  runNumber = 565118
  partName = send

# alien:///alice/data/CCDB/CTP/Calib/Scalers/02/64462/337db9d2-714d-11f0-87fb-0aa140641b9a
# http://alice-ccdb.cern.ch/download/337db9d2-714d-11f0-87fb-0aa140641b9a => o2-ctp-CTPRunScalers_1754323683866.root
# 1754317315067 | Mon Aug 04 16:21:55,067 CEST 2025 | Valid from
# 1754323683866 | Mon Aug 04 18:08:03,866 CEST 2025 | o2-ctp-CTPRunScalers_1754323683866.root
# 1754323683878 | Mon Aug 04 18:08:03,878 CEST 2025 | Created (ID: 337db9d2-714d-11f0-87fb-0aa140641b9a)
# 1754323683880 | Mon Aug 04 18:08:03,880 CEST 2025 | Last modified
# 1754582883000 | Thu Aug 07 18:08:03,000 CEST 2025 | Initial validity limit
# 1754582883000 | Thu Aug 07 18:08:03,000 CEST 2025 | Valid until

Priblizne kazdych 10 sekund sa citaju vsetky hodnoty CTP counters (counters = scalers), aktualne je to 1085 hodnot. Tieto hodnoty sa raz za 24 hodin dump-nu (Tono Jusko python script) do obycajneho textoveho suboru YYYYMMDD.cc pre vnutorne potreby CTP group. V pripade ak bezi run, tak sa niektore vybrane hodnoty CTP counters (suvisiace s luminosity) na konci run-u ulozia do CTP/Calib/Scalers CCDB.

Aktualna CTP counters konfiguracia (see also DataFormats/Detectors/CTP) sa na zaciatku run-u ulozi do CTP/Config/Config CCDB. Pre vnutorne potreby CTP group sa tato konfiguracia na zaciatku run-u dump-ne do textoveho suboru RunNumber.rcfg2.

Informacie, ktore suvisia s run-ami sa ukladaju do RCT/Info/RunInformation CCDB. Pre vyssie uvedeny priklad sa run 565118 (remember include <565118,565119) exclude in PostgreSQL timestamp ranges) spustil v case 16:31:54, resp. "SOR (Start of Run)". Tento cas minus 10 minut (16:21:55) je cas pre CTP/Calib/Scalers "Validity from". V case 18:08:03, resp. "EOX (End of data eXchange)", kedy sa asi pravdepodobne poslal nejaky vnutorny signal pre CTP, sa zapisali vybrane hodnoty CTP counters do CCDB. Tento cas plus 3 dni je cas pre pre CTP/Calib/Scalers "Validity until". Hodnoty -10 minut a +3 dni (a funkcie, ktore s nimi pracuju) su definovane v Detectors/CTP/workflowScalers/src/ctpCCDBManager.cxx. Interval +3 dni sa ukazal ako zbytocne velky.

Analogicky sa zapisuju hodnoty do CTP/Config/Config CCDB, s tym rozdielom, ze CTP/Config/Config sa zapise na zaciatku run-u a CTP/Calib/Scalers na jeho konci. See also AliceO2 Bookkeeping run 565118 with panel "CTP Trigger counters" (data from CTP/Calib/Scalers) and panel "CTP Trigger config" (data from CTP/Config/Config).

Pre prvy mozny CTP/Calib/Scalers timestamp 1754317315067 existuju 2 runs: 565118 a 565097 (run 80 je zjavne nejaky fake run). Run 565118 je v zozname ako prvy, kedze je to najnovsi run pre dany timestamp. Naopak, pre posledny mozny CTP/Calib/Scalers timestamp 1754582882999 (remember <1754317315067,1754582883000) in PostgreSQL timestamp ranges) existuje az 7 runs, a run 565118 je v zozname ako posledny, kedze je to najstarsi run.

Kedze pre konkretny timestamp vacsinou existuje viac ako jeden run, je mozne (a potrebne) specifikovat cislo run-u, ktore je ulozene ako metadata. Napr. pre 1754317315067/runNumber=565118 a 1754582882999/runNumber=565118 je CTP/Calib/Scalers ten isty CCDB object, ale pre 1754582883000/runNumber=565118 je to uz neexistujuci objekt. Alebo ine cislo run-u (iny CCDB object) pre ten isty timestamp 1754317315067/runNumber=565097.

Chronologia zapisovania v RCT/Info/RunInformation (priklad pre run 564661)

# 1752788702451 | Thu Jul 17 23:45:02 CEST 2025 | Metadata SOR (Start of Run)
# 1752788704000 | Thu Jul 17 23:45:04 CEST 2025 | Created (ID: 4c2ffce0-6357-11f0-8791-0aa145271b9a)
# 1752788754242 | Thu Jul 17 23:45:54 CEST 2025 | Metadata STF (Start of Time Frame)
# 1752788754274 | Thu Jul 17 23:45:54 CEST 2025 | Metadata SOX (Start of eXchange)
# 1752797021327 | Fri Jul 18 02:03:41 CEST 2025 | Metadata ETF (End of Time Frame)
# 1752797023066 | Fri Jul 18 02:03:43 CEST 2025 | Metadata EOX (End of eXchange)
# 1752797088144 | Fri Jul 18 02:04:48 CEST 2025 | Metadata EOR (End of Run)
# 1752797088584 | Fri Jul 18 02:04:48 CEST 2025 | Last modified
# curl v case 23:50:20 (cca 5 minut od spustenia run-u), diff s curl v case 02:06:11, t.j. potom ako run skoncil v 02:04:48
ID: 4c2ffce0-6357-11f0-8791-0aa145271b9a
Path: RCT/Info/RunInformation
Validity: 564661 - 564662 (Thu Jan 01 01:09:24 CET 1970 - Thu Jan 01 01:09:24 CET 1970)
Initial validity limit: 564662 (Thu Jan 01 01:09:24 CET 1970)
Created: 1752788704000 (Thu Jul 17 23:45:04 CEST 2025)
Last modified: 1752789019243 (Thu Jul 17 23:50:19 CEST 2025)
Original file: tmp.dat, size: 1, md5: 93b885adfe0da089cdf634904fd59f71, content type: application/x-ns-proxy-autoconfig
Uploaded from: 10.161.69.39
Metadata:
  ObjectType = char
  STF = 1752788754242
  ETF = 1752789016529
  SOR = 1752788702451
  UpdatedFrom = 10.162.0.51
  EOR = 0
  SOX = 0
  partName = send
  EOX = 0
# curl v case 01:00:16 (priblizne v polovici run-u), diff s curl v case 23:50:20
ID: 4c2ffce0-6357-11f0-8791-0aa145271b9a
Path: RCT/Info/RunInformation
Validity: 564661 - 564662 (Thu Jan 01 01:09:24 CET 1970 - Thu Jan 01 01:09:24 CET 1970)
Initial validity limit: 564662 (Thu Jan 01 01:09:24 CET 1970)
Created: 1752788704000 (Thu Jul 17 23:45:04 CEST 2025)
Last modified: 1752793215862 (Fri Jul 18 01:00:15 CEST 2025)
Original file: tmp.dat, size: 1, md5: 93b885adfe0da089cdf634904fd59f71, content type: application/x-ns-proxy-autoconfig
Uploaded from: 10.161.69.39
Metadata:
  ObjectType = char
  STF = 1752788754242
  ETF = 1752793213126
  SOR = 1752788702451
  UpdatedFrom = 10.162.0.51
  EOR = 0
  SOX = 0
  partName = send
  EOX = 0
# curl v case 02:03:46 (cca 1 minuta pred koncom run-u), diff s curl v case 23:50:20
ID: 4c2ffce0-6357-11f0-8791-0aa145271b9a
Path: RCT/Info/RunInformation
Validity: 564661 - 564662 (Thu Jan 01 01:09:24 CET 1970 - Thu Jan 01 01:09:24 CET 1970)
Initial validity limit: 564662 (Thu Jan 01 01:09:24 CET 1970)
Created: 1752788704000 (Thu Jul 17 23:45:04 CEST 2025)
Last modified: 1752797024436 (Fri Jul 18 02:03:44 CEST 2025)
Original file: tmp.dat, size: 1, md5: 93b885adfe0da089cdf634904fd59f71, content type: application/x-ns-proxy-autoconfig
Uploaded from: 10.161.69.39
Metadata:
  ObjectType = char
  STF = 1752788754242
  ETF = 1752797021256
  SOR = 1752788702451
  UpdatedFrom = 10.162.0.51
  EOR = 0
  SOX = 0
  partName = send
  EOX = 0
# curl v case 02:06:11, t.j. potom ako run skoncil v 02:04:48
ID: 4c2ffce0-6357-11f0-8791-0aa145271b9a
Path: RCT/Info/RunInformation
Validity: 564661 - 564662 (Thu Jan 01 01:09:24 CET 1970 - Thu Jan 01 01:09:24 CET 1970)
Initial validity limit: 564662 (Thu Jan 01 01:09:24 CET 1970)
Created: 1752788704000 (Thu Jul 17 23:45:04 CEST 2025)
Last modified: 1752797088584 (Fri Jul 18 02:04:48 CEST 2025)
Original file: tmp.dat, size: 1, md5: 93b885adfe0da089cdf634904fd59f71, content type: application/x-ns-proxy-autoconfig
Uploaded from: 10.161.69.39
Metadata:
  ObjectType = char
  STF = 1752788754242
  ETF = 1752797021327
  SOR = 1752788702451
  UpdatedFrom = 10.161.69.39
  EOR = 1752797088144
  SOX = 1752788754274
  partName = send
  EOX = 1752797023066