Data Acquisition Manual for Nuclear Physics Experiments

Version as of 10.06.2020

The data acquistion set up for nuclear physics experiments at ELBE consists of a mixed synchronized readout of various digitizers in different form factors (crates). The system is based on the Multi-Branch System developped at GSI. The help of the data acquisition group at GSI, esp. Dr. Nikolaus Kurz, is gratefully acknowledged.
A detailed description about the data acquisition scheme can be found here. You are asked to type in the bold face emphasized text. As an example we assume the frontend machine to be rio3-3 and the account nrfdaq.

Data acquisition has been upgraded for GiPS experiments to a linux-based rio4 hardware, high-res. TDC and VME digitizers.

Starting the data acquisition from an x-terminal window:

  • Start the NoMachine nxclient process on any of the x-terminals or from a Windows PC and log in as user eposdaq
  • Open several xterm windows
    Two xterm windows should be opened on the LYNX-OS system on RIO3-3. We start with the process first which delivers and takes data from the accelerator by an OPC-server. The communication to the readout software is done via a shared memory segment.
      • /home/nrfdaq> ssh -y rio3-3
        Connected to rio3-3.
        Escape character is '^]'.

        LynxOS (rio)

        user name:eposdaq

        @ W E L C O M E   to   LynxOS 3.1.0a   on   CES RIO2 8064   
        @ in case of problems call N.Kurz (2979) at GSI

        /etc/csh.login common login ...
        # MBS Version: 6.2
        Created message queue MBSMessages (1)
        Created command queue MBSCommands (2)
        RIO3-3 eposdaq >

        If this message wasn't displayed exactly as shown above, then you have to reboot the machine by toggling the little switch on the front panel labelled reset. Alternatively you may just kill all running data acquisition processes using the resa command twice. 
        RIO3-3 eposdaq > resa
        RIO3-3 eposdaq > resa
        Now, you can proceed with
        RIO3-3 eposdaq > cd mbsrun/pos16100557 (this may change; look at the whiteboard or in the elog)
        RIO3-3 eposdaq > opc2shm (starts the opc client which takes data, settings, beam position monitors, shutter, etc., from the accelerator control and delivers the overall count rate)
      • Open a second xterm window and again open a telnet session on RIO3-3.
        RIO3-3 eposdaq > cd mbsrun/pos16100577
        RIO3-3 eposdaq > mbs
        c_path: /mbs/v62/bin_RIO3
        c_task: m_msg_log
        mbs> -RIO3-3   :msg_log    :Message logger running
        mbs> @start
        -RIO3-3   :util       :task m_util started
        -RIO3-3   :util       :setup file setup.usf successfully loaded
        -RIO3-3   :util       :cleared daqstatus counter
        -RIO3-3   :util       :trigger module set up as MASTER, crate nr: 0
        -RIO3-3   :util       :enabled interrupt
        -RIO3-3   :read_meb   :Pipe type 0: smem_create for base 0x4000000
        -RIO3-3   :collector  :Pipe type 0: smem_create for base 0x4000000
        -RIO3-3   :read_meb   :task m_read_meb started
        -RIO3-3   :collector  :task m_collector started
        -RIO3-3   :transport  :task m_transport started
        -RIO3-3   :stream_serv:task m_stream_serv started
        mbs> -RIO3-3   :stream_serv:NO client (asynch), keep none, scale=1
        -RIO3-3   :collector  :old flushtime: 10
        -RIO3-3   :collector  :new flushtime: 5
        -RIO3-3 :esone_serv :Esone server started
        -RIO3-3 :esone_serv :Esone rpc server started

      • mbs> sta acq -cle
        mbs> -RIO3-3   :util       :start acquisition
        -RIO3-3   :read_meb   :found trig type 14 == start acquisition
        <I>: locate and attach shared memory segment for OPC data
        <I>: shared memory segment located
        <I>: shared memory segment attached
        user_readout: start acquisition triggered
        <I>: NFI ungated singles mode enabled for nfi in slot 1!
        <I>: NFI ungated singles mode enabled for nfi in slot 2!
        <I>: NFI ungated singles mode enabled for nfi in slot 3!
        <I>: NFI ungated singles mode enabled for nfi in slot 4!
        <I>: NFI ungated singles mode enabled for nfi in slot 5!
        <I>: NFI ungated singles mode enabled for nfi in slot 6!
        <I>: NFI ungated singles mode enabled for nfi in slot 7!
        <I>: NFI ungated singles mode enabled for nfi in slot 8!
        <I>: ZFK timer trig #4 (OPC) set to 10 s
        <I>: ZFK timer trig #8 (NFI) set to 600 s
        <I>: ZFK timer clock set to 2048 s
        <I>: acquisition started at: 1384447463 790043
        <I>: data taking started: Thu Nov 14 17:44:23 2013
        -RIO3-3   :collector  :acquisition running
        -RIO3-3   :stream_serv: connected

    Now, we start the file output using the remote file server.

    eposdaq@xbee78:>ssh -X eposdaq@fwklux5
    PassWord: xxxxxxxxxxx
    eposdaq@fwklux5:~$ cd bin
    eposdaq@fwklux5:~/bin$ rawDispRFIO64b
    -I- 64 bit RFIO server (fwklux5) listening on port 1974
    forks './rawServRFIO64b' for each client request

  • mbs> connect rfio fwklux5 -disk
    -RIO3-3    :transport  :Connected RFIO server!
    mbs> open file /data/fwkk/eposdaq/RPVS-.lmd size=950 -auto -disk -rfio
    -RIO3-3    :transport  :open   file at server fwklux5::
    -RIO3-3    :transport  :/data/fwkk/eposdaq/RPVS-0001

  • eposdaq@fwklux5:~/bin$:
  • -I- client no 1 accepted
    log file client no. 1: ./eposdaq.t1384492050.p22571.log

    It is necessary to fully qualify the file name including the full path. Take care that the account which started the rawDispRFIO64b server has write permission on the directory selected.
    Now, the file is open and you can start to take data.
    • mbs> sta acq
      -rio :util :start acquisition
      mbs> -rio :read_meb :found trig type 14 == start acquisition
      user_readout: start acquisition triggered
      -rio :collector:acquisition running
  • Now, the system is taking data and would serve them over the ethernet where data are taken by a remote event server.
  • The status of the acquisition can be inquired by:

  • mbs> show acq
    -RIO3-3   :util       :--------------------------------------------------------------
    -RIO3-3   :util       :ACQUISITION INITIALIZED 15-Nov-13 07:46:37: active tasks:
    -RIO3-3   :util       :m_dispatch m_msg_log m_util m_read_meb m_collector
    -RIO3-3   :util       :m_transport m_stream_serv
    -RIO3-3   :util       :SB setup LOADED from setup.usf.
    -RIO3-3   :util       :Readout table NOT LOADED, acquisition RUNNING.
    -RIO3-3   :util       :There are 3 streams with 128 buffers a 32768 [b] each.
    -RIO3-3   :util       :Current crate is 0, event builder is RIO3.
    -RIO3-3   :util       :--------------------------------------------------------------
    -RIO3-3   :util       :Crate   0: ID=    1, LOCAL   SYNC RIO2 , subevent slots= 5000.
    -RIO3-3   :util       :  trig  1: cvt= 300, fclrt= 100 [usec], max se length=  1024 b
    -RIO3-3   :util       :  trig  2: cvt= 300, fclrt= 100 [usec], max se length=  1024 b
    -RIO3-3   :util       :  trig  4: cvt= 300, fclrt= 100 [usec], max se length=  1024 b
    -RIO3-3   :util       :  trig  6: cvt= 300, fclrt= 100 [usec], max se length=  1024 b
    -RIO3-3   :util       :  trig  8: cvt= 300, fclrt= 100 [usec], max se length=4194304 b
    -RIO3-3   :util       :  trig 14: cvt= 300, fclrt= 100 [usec], max se length=  1024 b
    -RIO3-3   :util       :  trig 15: cvt= 300, fclrt= 100 [usec], max se length=4194304 b
    -RIO3-3   :util       :--------------------------------------------------------------
    -RIO3-3   :util       :Name of output device = RFIO
    -RIO3-3   :util       :RFIO server  fwklux5, file OPENED
    -RIO3-3   :util       :/data/fwkk/eposdaq/ZnO-0001.lmd
    -RIO3-3   :util       :       0.000 [MB] written to tape,      504.234 to file
    -RIO3-3   :util       :--------------------------------------------------------------
    -RIO3-3   :util       :Collected:   945.8483 MB,   28865 Buffers,   13288442 Events.
    -RIO3-3   :util       :Rate     :        229 KB/s,     7 Buffers/s,     2579 Events/s
    -RIO3-3   :util       :--------------------------------------------------------------
    Shutdown data acquisition:
In order to stop to data taking, one has to issue:

  • mbs> stop acq
    user_readout: stop acquisition triggered
    -rio :util :stop acquisition
    mbs> -rio :read_meb :found trig type 15 == stop acquisition
    -rio :collector:acquisition NOT running
    mbs> close file
    mbs> disconnect rfio

To do a full shutdown one has to issue:

  • mbs> @stop
    user_readout: stop acquisition triggered
    -rio :util :stop acquisition
    -rio :read_meb :found trig type 15 == stop acquisition
    -rio :collector:acquisition NOT running
    -rio :stream_se:m_stream_serv exiting...
    -rio :transport:/mbs/prod/bin_RIO2/m_transport exiting
    -rio :collector:/mbs/prod/bin_RIO2/m_collector exiting..
    -rio :read_meb :./m_read_meb exiting..
    -rio :util :/mbs/prod/bin_RIO2/m_util exiting..
    mbs> exit
    rio nrfdaq >

Refresh rate of online spectra:

  • The refresh rate between two readout cycles of the single spectra is fixed to 600 s and the OPC data are taken every 10 s.

Trouble shooting:

  • In case of hanging processes one can restart the system without a reboot by resetting the processes in a defined way:
    • RIO3-3 nrfdaq > resl
      ---- Reset node rio
      RIO3-3: kill 60 # m_msg_log
      -rio :msg_log :exiting.. SIGTERM (kill) signal
      RIO3-3: Clear DAQ status...
      RIO3-3: Create new message files...
      ---- Reset node RIO3-3 finished.
  • If the X-terminal freezes (white stripes on the screen, no response) then you have to restart the X-windows server by:
    • CTRL-ALT F1 and CTRL-ALT F7 (function keys)
  • If the RIO3-3 does not boot automatically after a cold start or power failure:

Writing data files:

  • The system can write data files to a RFIO server running on the host machine fwklux5 (or zirkonia) using automatic file name creation as desribed above. The file filenum.set contains the next file name to be written. To start with the first file with number nnnn you have to use:
    • mbs> connect rfio fwklux5 -disk

    • mbs> open file /data/fwkk/eposdaq/RPVS-.lmd first=nnnn size=950 -auto -rfio

    • mbs> start acquisition

This open file command takes the file specification from a file named filenum.set: This is the default procedure which avoids of having double file names, other conventions and so on. Nevertheless, you can specify the file name directly with the instructions given explicitly in man open file.

Please, try to follow the sequence of

  1. mbs> open file -rfio -auto

  2. mbs> start acq

  3. mbs> stop acq

  4. mbs> close file

all the time. This sequence avoids problems with improper file ends and arbitrarily filled single spectra.

Data analysis:

  • Online data can be analysed using LeA (GSI Lean Analysis). The software should run on zirkonia. You can invoke it in the following way:
  • Now, the graphical user interface should pop up showing three windows. Since the default input has been set to the stream server on rio you can start the analysis by simply pressing the analyse events button in the main window. After finishing a run you can save the online spectra into a data base using the pull-down menu database -> save (replace/new). The individual spectra may be saved to file, as well, using: histogram -> dump/restore -> dump . Remember, you have to press return after a parameter change in any input window in LeA.
  • DAQ Scheme (replace urania by zirkonia)

DAQ Schema der Kernphysik an ELBE

  • Error handling:
  • Message from MBS:
  •  RIO3-3 :read_meb :exiting.. SIGSEGV (segment. fault) signal
     RIO3-3 :read_meb :E-exit_daq_proc-SYSERR: System error: Out of free memory.
  • Action: Close file, exit MBS and restart MBS from scratch using RESL and mbs> @start