DocumentationDDRapp overview
DDRapp is a multi-tenant research data management platform from TUL that collects instrument data automatically with Apache Airflow and stores them in S3.
DDRapp (Application for data-driven research) is a multi-tenant research data management platform for scientific experiments, developed at the Institute for Nanomaterials, Advanced Technologies and Innovation (CXI) of the Technical University of Liberec (TUL). It manages instruments, datasets, experiments and metadata, and collects data from instrument PCs automatically, without installing anything on them. The data are stored in S3, such as CESNET S3.
DDRapp is the web application at the centre of the wider DDRplatform, together with DDRflow (Apache Airflow pipelines) and S3 storage.
What DDRapp does
- Instrument registry: every instrument is registered with its responsible users, a logbook, an activation state and metadata, including PIDInst-compatible instrument descriptions.
- Experiment tracking: operators start and stop measurements in the web app. DDRapp knows which data belong to which project, dataset and experiment.
- Automated collection: when a measurement starts, a folder is created on the instrument PC’s shared disk. When it ends, Airflow pulls the data over SMB (Windows) or SFTP (Linux) and backs them up to S3.
- Metadata on every file: information about the user, instrument, project, dataset and experiment is attached to each stored S3 object and indexed in Elasticsearch.
- Secure instruments: instrument PCs can stay in a private deny-all network. Only the Airflow server talks to them.
- Explorer: a built-in browser for the stored data, with sharing, time-limited download links and zip downloads.
- Publication: datasets can be published to the National Repository Platform (NRP) as drafts.
- Manual upload: instruments that cannot be connected can still be recorded, with data uploaded through the web UI.
How data are organised
Each resource maps to an S3 bucket, datasets and experiments to prefixes (“folders”) inside it:
s3://<internal number>-<activity>-<acronym>/<dataset>/<experiment>/Instances
| Instance | Operated by | URL |
|---|---|---|
| TUL production | TUL CXI | ddr.tul.cz |
| Public instance | J. Heyrovský Institute of Physical Chemistry, on e-INFRA CZ Kubernetes | ddrapp.nrp1.du.cesnet.cz |
Institutions can also run their own instance; see For administrators.
Who is DDRapp for?
- Laboratories with valuable instruments on old or unpatched PCs that must be isolated from the network.
- Institutes that want a uniform workflow and S3 backup across many instruments and departments, with each department’s data and users separated.
- Teams publishing to NRP that want metadata captured at the source.
Project and funding
DDRapp is developed within the EOSC CZ initiative, project National Repository Platform for Research Data (CZ.02.01.01/00/23_014/0008787), Programme Johannes Amos Comenius, Ministry of Education, Youth and Sports of the Czech Republic.
The authoritative, detailed documentation is maintained by the developers at heytul.gitlab.io/ddrapp.
