Data Acquisition Manual for GiPS (Gamma-induced Positron Spectroscopy) Experiments

Version as of 10.02.2021 (original one from 06.02.2019)

The data acquistion set up consists of a event-synchronized readout of various digitizers in VME standard. The system is based on the Multi-Branch System developped at GSI. The continuous support of the data acquisition group at GSI, esp. Dr. Nikolaus Kurz, is gratefully acknowledged.
You are asked to type in the bold face emphasized text. As an example we assume the frontend machine to be rio4l-3 and the account eposdaq.

Starting the data acquisition from an x-terminal window:

    • Start the NoMachine nxclient process on any of the x-terminals or from a Windows PC and log in as user eposdaq
    • Open several xterm windows
      Two xterm windows should be opened on the Sugarhat Linux-based operating system running on rio4l-3 (VME-card onboard computers or crate controllers from CES). The manual states: "Sugar Hat Linux 1.1 is an enhanced distribution of Open Source Linux based on the MontaVista Software’s Hard Hat Linux 2.0 distribution (Montavista Software, Inc) which provides the PowerPC Cross Development Kit (CDK) and target environment. Sugar Hat Linux 1.1 provides an interface to the CES hardware resource and a set of drivers and services that allow applications to exploit the CES RIO boards hardware features.".
      We start with the process first which delivers and takes data from the accelerator by an OPC-server. The communication to the readout software is done via a shared memory segment.
        • /home/eposdaq> ssh rio4l-3
          |------------------------------------------------------------------------------|
          | W E L C O M E to
          |
          | Sugarhat (3.3.10) Linux (2.6.33)
          | on CES RIO4 8070/72
          | at GSI
          |
          | in case of problems call N.Kurz (2979)
          |
          |------------------------------------------------------------------------------|
          /etc/csh.login common login ...
          # MBS Version: 6.2
          /mbs/v62/bin_RIO4_Linux_2.6.33_SuHa
          Message queue MBSMessages (1) already exists!
          Command queue MBSCommands (2) already exists!
          RIO4L-3 eposdaq >
        • If this message wasn't displayed exactly as shown above, then you have to reboot the machine by toggling the little switch on the front panel labelled reset. You may also go to the site 149.220.60.99 and use the button VME SYSRESET (credentials can be found on the hardware).
          Alternatively, if you still can access the machine you may just kill all running data acquisition processes using the resl command twice.
          rio4l-3 eposdaq > resl
          rio4l-3 eposdaq > resl
          Now, you can proceed with
          rio4l-3 eposdaq > cd mbsrun/GiPS (this may change; look at the whiteboard or in the elog)
          rio4l-3 eposdaq > mbs
          c_path: /mbs/v62/bin_RIO4_Linux_2.6.33_SuHa
          c_task: m_msg_log

          mbs> @start
          -RIO4L-3:util :task m_util started
          -RIO4L-3:util :setup file setup.usf successfully loaded
          -RIO4L-3:util :trigger module set up as MASTER, crate nr: 0
          -RIO4L-3:util :disabled interrupt
          -RIO4L-3:dispatch :-> 'sleep 1 '
          -RIO4L-3:read_meb :Pipe type 2: virtual mapping
          -RIO4L-3:read_meb :task m_read_meb started
          -RIO4L-3:dispatch :-> 'sleep 1 ' finished
          -RIO4L-3:collector :Pipe type 2: virtual mapping
          -RIO4L-3:transport :task m_transport started
          -RIO4L-3:stream_serv:task m_stream_serv started
          -RIO4L-3:stream_serv:NO client (asynch), keep none, scale=1
          -RIO4L-3:collector :task m_collector started
          -RIO4L-3:collector :old flushtime: 10
          -RIO4L-3:collector :new flushtime: 1
          -RIO4L-3:dispatch :-> 'sleep 1 '
          -RIO4L-3:dispatch :-> 'sleep 1 ' finished

          mbs> sta acq
          mbs> -RIO4L-3:util :start acquisition
          -RIO4L-3:read_meb :found trig type 14 == start acquisition
          -RIO4L-3:read_meb :TDC module: 0 register base address: 0x3005e000
          -RIO4L-3:read_meb :TDC modules done
          -RIO4L-3:read_meb :QDC module: 0 register base address: 0x3205e000
          -RIO4L-3:read_meb :QDC modules done
          -RIO4L-3:read_meb :ADC module: 0 register base address: 0x3305e000
          -RIO4L-3:read_meb :ADC modules done
          -RIO4L-3:read_meb :GDG module: base address: 0x3405e000
          -RIO4L-3:read_meb :Trigger input to LEMO selected
          -RIO4L-3:read_meb :Internal clock selected
          -RIO4L-3:read_meb :<I>: f_user_init: GSI VFTX init for module 0:
          -RIO4L-3:read_meb :
          -RIO4L-3:read_meb :module: 0 status before initialization: 0x63
          -RIO4L-3:read_meb :module: 0 status after setting trigger window: 0x63
          -RIO4L-3:read_meb :module: 0 trigger window: 0x788028
          -RIO4L-3:read_meb :module: 0 trg window begin: 600 ns before trigger
          -RIO4L-3:read_meb :module: 0 trg window end: 200 ns after trigger
          -RIO4L-3:read_meb :module: 0 trg window size: 800 ns
          -RIO4L-3:read_meb :module: 0 armed for coarse counter reset at first hardware trigger
          -RIO4L-3:read_meb :module: 0 status after initialization: 0x62
          -RIO4L-3:read_meb :
          -RIO4L-3:read_meb :<I>: f_user_init: CAEN V965 init for module 0:
          -RIO4L-3:read_meb :module: 0 status before initialization: 0x62
          -RIO4L-3:read_meb :module: 0 firmware: 0x00000904
          -RIO4L-3:read_meb :module: 0 version: 0x000000E3
          -RIO4L-3:read_meb :module: 0 SN: 00393374
          -RIO4L-3:read_meb :module: 0 OUI: 0x000040E6
          -RIO4L-3:read_meb :module: 0 BID: 0x000003C5
          -RIO4L-3:read_meb :module: 0 ctrl: 0x00000000
          -RIO4L-3:read_meb :module: 0 geo id: 00000006
          -RIO4L-3:read_meb :module: 0 evt counter: -0065281
          -RIO4L-3:read_meb :module: 0 stat reg 1: 0x00000050
          -RIO4L-3:read_meb :module: 0 stat reg 2: 0x0000002a
          -RIO4L-3:read_meb :module: 0 bit set 1: 0x00000000
          -RIO4L-3:read_meb :module: 0 bit set 2: 0x00004818
          -RIO4L-3:read_meb :module: 0 Iped (max. 255): 180
          -RIO4L-3:read_meb :module: 0 ch 0 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 1 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 2 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 3 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 4 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 5 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 6 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 7 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 8 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 9 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 10 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 11 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 12 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 13 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 14 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 15 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 16 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 17 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 18 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 19 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 20 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 21 thr reg >0 : 0x00000100
          -RIO4L-3:read_meb :module: 0 ch 22 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 23 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 24 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 25 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 26 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 27 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 28 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 29 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 30 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 31 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :<I>: f_user_init: CAEN V1785 init for module 0:
          -RIO4L-3:read_meb :module: 0 status before initialization: 0x62
          -RIO4L-3:read_meb :module: 0 firmware: 0x00000904
          -RIO4L-3:read_meb :module: 0 version: 0x000000E3
          -RIO4L-3:read_meb :module: 0 SN: 00000226
          -RIO4L-3:read_meb :module: 0 OUI: 0x000040E6
          -RIO4L-3:read_meb :module: 0 BID: 0x000006F9
          -RIO4L-3:read_meb :module: 0 ctrl: 0x00000000
          -RIO4L-3:read_meb :module: 0 geo id: 00000007
          <W>: TDC hit mult: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
          -RIO4L-3:read_meb :module: 0 evt counter: -0065281
          -RIO4L-3:read_meb :module: 0 stat reg 1: 0x00000050
          -RIO4L-3:read_meb :module: 0 stat reg 2: 0x0000009a
          -RIO4L-3:read_meb :module: 0 bit set 1: 0x00000000
          -RIO4L-3:read_meb :module: 0 bit set 2: 0x00004818
          -RIO4L-3:read_meb :module: 0 Iped (max. 255): 255
          -RIO4L-3:read_meb :module: 0 ch 0 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 1 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 2 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 3 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 4 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 5 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 6 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 7 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 8 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 9 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 10 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 11 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 12 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 13 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 14 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :module: 0 ch 15 thr reg >0 : 0x00000000
          -RIO4L-3:read_meb :<I>: f_user_init: CAEN V486 init
          -RIO4L-3:read_meb :module: user_readout: CAEN V486 identification:
          -RIO4L-3:read_meb : version: 0x000000
          -RIO4L-3:read_meb : SN: 0x00001e
          -RIO4L-3:read_meb : manufact: 0x000000
          -RIO4L-3:read_meb : module type: 0x00001b
          -RIO4L-3:read_meb : fixed code: 0x00faf5
          -RIO4L-3:read_meb : select: 0x00ffff
          -RIO4L-3:read_meb :<I>: f_user_init: done ...
          -RIO4L-3:read_meb :<I>: f_user_init: CAEN V965 set V965 0 online
          -RIO4L-3:read_meb :<I>: f_user_init: CAEN V1785 set V1785 0 online
          -RIO4L-3:collector :acquisition running

      Now, we may start the file output using the remote file server.

      eposdaq@xbee78:>ssh -X eposdaq@fwklux5
      PassWord: xxxxxxxxxxx
      eposdaq@fwklux5:~$ cd bin
      eposdaq@fwklux5:~/bin$ rawDispRFIO64b
      -I- 64 bit RFIO server (fwklux5) listening on port 1974
      forks './rawServRFIO64b' for each client request

    • mbs> connect rfio fwklux5 -disk
      -rio4l-3 :transport :Connected RFIO server!
      mbs> open file /data/fwkk/eposdaq/RPVS-.lmd size=950 -auto -disk -rfio
      -rio4l-3 :transport :open file at server fwklux5::-rio4l-3 :transport :/data/fwkk/eposdaq/RPVS-0001.lmd
    • eposdaq@fwklux5:~/bin$:

-I- client no 1 accepted
log file client no. 1: ./eposdaq.t1384492050.p22571.log

      It is necessary to fully qualify the file name including the full path. Take care that the account which started t

he rawDispRFIO64b server has write permission on the directory selected.

    Now, the file is open and you can start to take data. It is not necessary to stop the running DAQ and restart.
    • mbs> sta acq
      -rio :util :start acquisition
      mbs> -rio :read_meb :found trig type 14 == start acquisition
      user_readout: start acquisition triggered
      -rio :collector:acquisition running
  • Now, the system is taking data and would serve them over the ethernet where data are taken by a remote event server.
  • The status of the acquisition can be inquired by:

  • mbs> show acq
    -RIO4L-3:util :--------------------------------------------------------------
    -RIO4L-3:util :ACQUISITION INITIALIZED 24-Jan-17 10:48:00: active tasks:
    -RIO4L-3:util :m_dispatch m_msg_log m_util m_read_meb m_transport
    -RIO4L-3:util :m_collector m_daq_rate m_stream_serv
    -RIO4L-3:util :SB setup LOADED from setup.usf.
    -RIO4L-3:util :Readout table NOT LOADED, acquisition RUNNING.
    -RIO4L-3:util :There are 32 streams with 32 buffers a 32768 [b] each.
    -RIO4L-3:util :Current crate is 0, event builder is RIO4.
    -RIO4L-3:util :--------------------------------------------------------------
    -RIO4L-3:util :Crate 0: ID= 1, LOCAL SYNC SYNC , subevent slots=200000.
    -RIO4L-3:util : trig 1: cvt= 20, fclrt= 12 [usec], max se length=1000000 b
    -RIO4L-3:util : trig 2: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 3: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 4: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 5: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 6: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 7: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 8: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 9: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 10: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 11: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 12: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 13: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 14: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util : trig 15: cvt= 20, fclrt= 12 [usec], max se length=100000 b
    -RIO4L-3:util :--------------------------------------------------------------
    -RIO4L-3:util :Name of output device =
    -RIO4L-3:util :Tape label = , Dismounted, file CLOSED
    -RIO4L-3:util : 0.000 [MB] written to tape, 0.000 to file
    -RIO4L-3:util :--------------------------------------------------------------
    -RIO4L-3:util :Collected: 12.2552 MB, 374 Buffers, 66984 Events.
    -RIO4L-3:util :Rate : 98 KB/s, 3 Buffers/s, 429 Events/s
    -RIO4L-3:util :--------------------------------------------------------------
  • Shutdown data acquisition:
  •  
In order to stop to data taking, one has to issue:

  • mbs> stop acq
    user_readout: stop acquisition triggered
    -rio :util :stop acquisition
    mbs> -rio :read_meb :found trig type 15 == stop acquisition
    -rio :collector:acquisition NOT running

    To do a full shutdown one has to issue:

    • mbs> @stop

Trouble shooting:

  • In case of hanging processes one can restart the system without a reboot by resetting the processes in a defined way:
    • rio4l-3 nrfdaq > resl
      ---- Reset node rio
      rio4l-3: kill 60 # m_msg_log
      -rio :msg_log :exiting.. SIGTERM (kill) signal
      rio4l-3: Clear DAQ status...
      rio4l-3: Create new message files...
      ---- Reset node rio4l-3 finished.
  • If the X-terminal freezes (white stripes on the screen, no response) then you have to restart the X-windows server by:
    • CTRL-ALT F1 and CTRL-ALT F7 (function keys)
  • Writing data files:

    • The system can write data files to a RFIO server running on the host machine fwklux5 (or zirkonia) using automatic file name creation as desribed above. The file filenum.set contains the next file name to be written. To start with the first file with number nnnn you have to use:
      • mbs> connect rfio fwklux5 -disk

      • mbs> open file /data/fwkk/eposdaq/RPVS-.lmd first=nnnn size=950 -auto -rfio

      • mbs> start acquisition

    This open file command takes the file specification from a file named filenum.set: This is the default procedure which avoids of having double file names, other conventions and so on. Nevertheless, you can specify the file name directly with the instructions given explicitly in man open file.

    Please, try to follow the sequence of

    1. mbs> open file -rfio -auto

    2. mbs> start acq

    3. mbs> stop acq

    4. mbs> close file

    all the time. This sequence avoids problems with improper file ends and arbitrarily filled single spectra.

    Data analysis:

    • Online data can be analysed using Go4 (GSI Object Oriented On-line Off-line system). The software should run on fwklux5. You can invoke it in the following way:
      • eposdaq@fwklux5:~> cd go4/GiPS
        eposdaq@fwklux5:~/go4/GiPS$ . go4login
        eposdaq@fwklux5:~/go4/GiPS$ go4 or
        eposdaq@fwklux5:~/go4/GiPS$ go4 GiPS-online.hotstart           which will do all necessary startup commands and connect to the frontend data server.
    • Because the go4 system is very powerfull and also complex, a video will be used to show you the principal steps. Don't worry if you crash the analysis, the data will be recoorded on disk and they can be analysed later again and again.

    • Error handling:
    • Message from MBS:
    • rio4l-3 :read_meb :exiting.. SIGSEGV (segment. fault) signal
      rio4l-3 :read_meb :E-exit_daq_proc-SYSERR: System error: Out of free memory.
    • Action: Close file, exit MBS and restart MBS from scratch using RESL and
    • mbs> @start