Contact

Dr. Thomas Gruber

t.gruberAthzdr.de
Phone: +49 351 260 3846

The Working Group "Data Management Solutions" (DMS)

Datenlebenszyklus ©Copyright: Dr. Knodel, Oliver

Datenlebenszyklus

Foto: Oliver Knodel

Download

For a state-of-the-art research data management it is recommended to consider the data life cycle first. Our long-term goal is to document the entire life cycle in a Findable, Accessible, Interoperable and Reusable (FAIR) way. Especially the steps from the creation of the research data until the publication of data and results should be included in the final publication. We developed a concept for the entire experiment or project flow, which contains all essential components and their seamless interconnection. Descriptions about the individual components provided by our working group can be found in our publication in RODARE  (https://doi.org/10.14278/rodare.252). Since we are still in the development process only some of the components are available. These are referenced in the section Useful Links below. The remaining components will follow as soon as possible.

Mission

We are developing a technical implementation of an integrated digital research data ­manage­ment for researchers at HZDR. This implies as first steps the linkage of existing data sources,
documentation of experiments, ­manage­ment and integration of data analysis of primary data as well as establishing a complete data provenance.

Support

We provide support for existing or planned projects concerning the above presented topics. The electronic documentation is often the first point of contact with this subject. Automated data acquisition and interfaces with analysis programs are also common main topics. We will support the establishment and initialization of an executable version of data aquicition. The goal will always be that the involved scientists will be able to continue and optimize the projects by themselves afterwards. Our group will always be available for further questions.

Services

Get in touch with us for questions or assistance

  • all around the topic "Research Data Management",
  • for porting a project life cycle to our infrastructure,
  • on documentation of experiments (Lab Documentation System)
  • concerning the integration of the lab documentation system with different data sources
  • about establishment of workflows related to the FAIR principles,
  • how to archive of research data, workflows and the scientific publication itself.

The Team

Name Geb./Raum +49 351 260 Email Zuständigkeit
Gruber Dr., Thomas 312/6 3846 t.gruber@hzdr.de

Reseach Data Management, Lab Documentation

Knodel Dr., Oliver 312/6 3845 o.knodel@hzdr.de Data Aquisition, Data Bases, Workflows, Cloud Computing
Müller Dr., Stefan 312/6 3847 stefan.mueller@hzdr.de Dataprocessing and -analysis, Documentation (Sharelatex)
Pape, David 312/3 3808 d.pape@hzdr.de Cluster Middleware, Application Prototyping and Development

Useful Links

Datenmanagement Arbeitsablauf ©Copyright: Dr. Knodel, Oliver

Datenmanagement Arbeitsablauf

Foto: Oliver Knodel

Download

Current Projects

Post-processing for TELBE Experiment

Description: After each measurement the user would like to have the post-processed file available as soon as possible. The experiments are controlled with Labview and the post-processing should initiated automatically on the cluster.
Expected Results (Goals): After each measurement the post-processing "workflow" is initiated, runs on the cluster and the resulting file is available on bigdata
Owner: Thomas Gruber
Customer: TELBE (FWKP)
Project Page: https://gitlab.hzdr.de/fwcc/data-management/user-project-documentation/post-processing-for-telbe-experiment

Organize Documentation for FMR experiments

Description: Kilian Lenz would like to setup a documentation platform for all FMR experiments in his group.
Expected Results (Goals):
  • Copy of the current version (athene server) on the wiki server
  • Labview communication still works with the new server
Owner: Thomas Gruber
Customer: FMR experiments (FWIN-D)
Project Page: https://gitlab.hzdr.de/fwcc/data-management/user-project-documentation/documentation-of-fmr-experiments

Provide an ETL Workflow for Turbulence Fluid Dynamic Simulations

Description: Create an ETL Workflow based on Celery with PostgrSQL and optional Elasticsearch integration.
Expected Results (Goals):
  • Python-based ETL workflow for our FWCC PostgreSQL database
  • Setting up a celery workflow environment
  • Integration of the workflow into our celery infrastructure
  • Visualization and administration of the workflows using Flower or Airflow
  • Connect Jupyter Notebooks on hemera to the PostgreSQL database
  • Setting up an OpenDistro (Elastic Search + Kibana + LDAP)
  • Synchronize the PostgrSQL database with Elasticsearch using LogStash
  • Visualize the data with Kibana
Owner: Oliver Knodel
Customer: (FWDC, Thomas Ziegenhein)

Automated GitLab CI-Job for the bitstream cration on Hemera

Description: Create a CI-Job to automate the FPGA bitstream creation after every commit (with a special Tag) to provide a valid bitstream and to bring the GitLab project to the next level.
Expected Results (Goals):
  • Validated project sources to enable a bitstream build based on the data provided in the GitLab repository.
  • Creation of a reproduceable (command line based) FPGA development pipeline with necesasary tools/dependencys on Hemera.
  • Automated GitLab HPC Runner producing valid bitstreams as artefacts.
Owner: Oliver Knodel
Customer: FWDF (André Bieberle)
Project Page: https://gitlab.hzdr.de/fwdf/measurementscience/projects/ufxct/fpga-dev

Manage DRESDYN Simulations

Description: Document every simulation and store selected values/parameters from results.
Expected Results (Goals):
  • The input parameters and specific results should be stored in an LDS system (first suggestion SQL database).
  • Easy interface to query simulations and plot selected parameters (first try with Kibana like tool)
Owner: Thomas Gruber
Customer: DRESDYN Simulation (FWDH)
Project Page: https://gitlab.hzdr.de/fwcc/data-management/user-project-documentation/manage-dresdyn-simulations

Organize Documentation in Blitzlab

Description: Documentation of every experiment of the Helmholtz Innovation Lab Blitzlab in openBIS
Expected Results (Goals):
  • Organize projects and document experiments in openBIS
  • Provide access for industry partners
Owner: Thomas Gruber
Customer: Blitzlab (FWIM)
Project Page: https://gitlab.hzdr.de/fwcc/data-management/user-project-documentation/experiment-documentation-in-blitzlab

Provide a Toolflow for FPGA-DAQ Development using High-Level-Synthesis

Description: Create a service which generates FPGA designs from OpenCL code using the High-Level-Synthesis (HLS) Tools from Xilinx on Hemera and implement first data aquisition cores.
Expected Results (Goals):
  • setup the toolflow on Hemera
  • implement first cores in pure C or OpenCL
  • document the project in GitLab and use CI for code validation
  • validate the core using SW/HW Cosimulation
  • optimize the code using directives and create different solutions on the provided FPGA (FWKK)
  • create the hardware design and deploy it on the ELBE-FPGA
Owner: Oliver Knodel
Customer: ELBE Experimant (FWKK, Andreas Wagner)

Provide project IDs with and without proposals

Description: Create a service which validates proposal IDs or provides a "HZDR-ID" for non proposal projects
Expected Results (Goals):
  • setup DMS Guidance System (Webfrontend and API)
  • mirror GATE database using OAuth and cURL
  • provide validation function for user + proposal ID requests
  • provide new "HZDR-ID" and validation for non proposal projects
  • provide additional information for validated IDs
Owner: Oliver Knodel
Customer:
  • Laser group collecting laboratory environmental sensor data (FWKT)
  • TELBE group using the THz beam (FWKP)
  • HZDR-wide customers
Project Page:

https://gitlab.hzdr.de/fwcc/data-management/user-project-documentation/project-id

and

https://gitlab.hzdr.de/fwcc/data-management/dms-guidance-system

Provide access to environmental data of ELBE laboratories

Description: Find secure tool to get selected data from Gebäudeleittechnik (GLT) without comprising the controlling functionality.
Expected Results (Goals): Provide parameter data which influence the laser characteristic/quality
Owner: Stefan Müller
Customer: Experimental groups which use laser, e.g. FWKT, FWKP
Project Page: https://gitlab.hzdr.de/fwcc/data-management/user-project-documentation/glt-data

Test OPC-UA Client

Description: Interact via OPC-UA with ELBE data and make it available for user in different ways.
Expected Results (Goals):
  • write sensor data into ELBE database via OPC-UA client
  • access power of THz pulses with Labview using OCA-UA plugin
  • read data with free OPC-UA client and write into an SQL and noSQL database
Owner: Stefan Müller
Customer:
  • Laser group collecting laboratory environmental sensor data (FWKT)
  • TELBE group using the THz beam (FWKP)
Project Page: https://gitlab.hzdr.de/muelle94/opcua_test_elbe

Prepare data for upload to HEPData repository

Description: Make data sets available in a consistent and useful way.
Expected Results (Goals): Prepare data sets of KLOE05, KLOE08, KLOE10, KLOE12 and the updated sets of
KLOE17 for upload to the HEPData repository using the hepdata_lib python library
Owner: Stefan Müller
Customer: http://www.strong-2020.eu/
Project Page: https://redmine.hzdr.de/projects/fluka/repository/show/MISC/KLOE/HEPdata

Organize Documentation for flotation model experiments

Description: Document flotation model experiments and subsequent analysis steps and results.
Expected Results (Goals):
  • Organize projects and document experiments in openBIS
  • surf as platform for further analysis tools in Matlab
Owner: Thomas Gruber
Customer: Flotation model experiments (FWDT)
Project Page: https://gitlab.hzdr.de/fwcc/data-management/user-project-documentation/documentation-of-flotation-model-exp.

Contact

Dr. Thomas Gruber

t.gruberAthzdr.de
Phone: +49 351 260 3846