SciPy Proceedings
article
Jul 13, 2026
dagster-slurm: Dagster Assets on Slurm-Managed Scientific Workflows
An open-source integration that runs Dagster asset source code across local, CI, and Slurm-managed execution targets, demonstrated on a web-scale European firm-discovery pipeline.
Hernan Picatto, Maximilian Hess, Georg Heiler, Martin Pfister
doi
none
For Editors
submission
Build Status
COMPLETED
Submission Status
DRAFT
Timestamp
Jun 21, 2026 03:54:05
Submission
scipy-2026-dagster_slurm
Source
command line
github.com/scipy-conference/scipy_proceedings
HEAD
ee21d3a
papers/dagster_slurm
Storage
019ee850-5fe4-76b2-8d6d-18c1c4dc949f
https://tmp.curvenote.com/
temporary CDN, 30-day retention policy
Checks

45 checks passed

16 checks failed (1 required)

ORCID (optional)
Add an ORCID to each author
CRediT Roles (optional)
Add CRediT roles to each author
DOI Exists (optional)
No DOI available for dagster (skipped)
No DOI available for pixi (skipped)
No DOI available for dagsterpipes (skipped)
No DOI available for moritz2018ray (skipped)
No DOI available for docker (skipped)
No DOI available for commoncrawl (skipped)
No DOI available for resiliparse (skipped)
No DOI available for nuextract (skipped)
No DOI available for daskjobqueue (skipped)
Add a DOI for airflow if it exists
No DOI available for cwl_wiki (skipped)