Run DuMuX simulation inside container via Snakemake by Sarbani-Roy · Pull Request #49 · BAMresearch/NFDI4IngModelValidationPlatform

Sarbani-Roy · 2026-01-07T17:50:45Z

Related to #48

This pull request adds the complete DuMux rotating-cylinders benchmark.

Includes scripts, Snakemake workflows, and a GitHub Actions CI pipeline.
Provisions a Conda/Mamba environment and Singularity, generates the benchmark configuration, and runs the full Snakemake pipeline in a containerized DuMux runtime.
Automatically post-processes simulation results to produce standardized metrics and field data.
Aggregates all outputs into per-tool summary JSON files under snakemake_results.
Currently, only two input parameters are parameterized; the remaining inputs use fixed values.

Once this is validated, we will proceed to parameterize most of the remaining input parameters.

…n_done)

Sarbani-Roy · 2026-01-28T21:31:30Z

We believe this is ready from our side. @joergfunger or @jpthiele, could you please review and merge this? We are looking forward to your feedback.

joergfunger

Thanks, a couple of comments from my side.

.github/workflows/run-dumux-benchmark.yml

benchmarks/rotating-cylinders/dumux/Snakefile

joergfunger · 2026-01-29T05:36:05Z

benchmarks/rotating-cylinders/generate_rc_config.py

+
+        configurations.append(config_name)
+        config_to_param[config_name] = f"params_{config_name}.input"
+        config_to_vtu[config_name] = [


The parameter files are mapped to the config {Path("parameters_1.json"): "1", ...} and the inverse map has to be stored. But is that true here, just use the config string when creating the output.

The configuration layer is defined explicitly through two mappings, for example:

"configuration_to_parameter_file": { "10_80": "params_10_80.input" }, "configuration_to_solution_vtu_files": { "10_80": [ "dumux/test_rotatingcylinders_10_80-00000.vtu", "dumux/test_rotatingcylinders_10_80-00001.vtu" ] }

With this structure, the config string (10_80) serves as the single key that links the parameter file and the corresponding set of solution files. This avoids the need for an explicit inverse map while still preserving a clear and deterministic association between inputs and outputs.

true, but in the same way you could just write in you Snakefile something like below, where you would not need that additional variables. IMO, this is easier to read, because the definition of these additional variables is at another place than the actual usage.

rule postprocess_dumux_results: input: parameters = lambda wildcards: configuration_to_parameter_file[wildcards.configuration], result_vtk0 = f"{result_dir}/{tool}/{{configuration}}/test_rotatingcylinders_10_80-00000.vtu", result_vtk1 = f"{result_dir}/{tool}/{{configuration}}/test_rotatingcylinders_10_80-00001.vtu", script = f"{tool}/postprocess_results.py", output: zip = f"{result_dir}/{tool}/solution_field_data_{{configuration}}.zip", metrics = f"{result_dir}/{tool}/solution_metrics_{{configuration}}.json", conda: "environment_simulation.yml", shell: """ python3 {input.script} \ --input_parameter_file {input.parameters} \ --input_result_vtk0 {input.result_vtk0} \ --input_result_vtk1 {input.result_vtk1} \ --output_solution_file_zip {output.zip} \ --output_metrics_file {output.metrics} """

joergfunger · 2026-01-29T05:38:25Z

benchmarks/rotating-cylinders/Snakefile

+    input:
+        expand(f"{result_dir}/{{tool}}/summary.json", tool=tools)
+
+rule generate_dumux_inputs:


How general is that meshing? (see comment above, when being implemented by another CFD code

This will work for any DUNE-based solvers.

But it is in the general Snakefile (which would also be true for openFOAM, or any other CFD solver).

@berndflemisch this is the general snakefile (for all tools the same), so IMO generate_dumux_input feels wrong to me.

joergfunger · 2026-01-29T05:58:16Z

benchmarks/rotating-cylinders/generate_rc_config.py

+        config_name = f"{cells0}_{cells1}"
+
+        configurations.append(config_name)
+        config_to_param[config_name] = f"params_{config_name}.input"


these parameter files should be json. At a later stage, we would like to replace this generation of the parameter files by automatically extracting it from the benchmark knowledge graph

According to my best knowledge DuMuX does not natively use JSON as its primary input format. It uses DUNE’s parameter system, which expects a .input file (INI-like / key–value format). However, we can use JSON as a preprocessing layer and convert it into a DuMuX-compatible .input file before running the simulation. If this is okay I can certainly do this.

The goal is that these parameter files are tool-independent definitions. We replace the key/values in these json that are converted to dict in the tool-specific input_template. So you would have to create a native DuMuX input template, write the variable/parameters that are to be replaced by some $$my_variable$$ and then replace in a preprocessing step $$my_variable$$ by 42 in this template to generate the real input for dumux (or whatever is given in the parameter.json file).

I agree with Jörg. It seems a bit overkill for the first benchmark but we should make it extensible.
The idea with the template is good and the python package jinja2 works well for these things, so you can convert a currently generated DuMuX input file into a jinja2 template and write a relatively short preprocessing scripts that takes the generic JSON and inserts the values into the template.

But I would first check if Dune added the functionality of reading JSON. I know that in deal.II it is a relatively new addition, so maybe. And that would possibly be the easiest way

joergfunger · 2026-01-29T05:59:38Z

benchmarks/rotating-cylinders/Snakefile

+rule generate_dumux_inputs:
+    input:
+        grid_t = f"{dumux_dir}/grid_files/grid_template.json",
+        dumux_t = f"{dumux_dir}/input_files/dumux_config.json"


Why are those dumux config files (rather than tool independent configurations that define what is to be computed in the benchmark)?

Sorry but can you please explain a bit more.

This is the general Snakefile that is supposed to be valid for all tools, and then there is the tool-specific part, that is integrated right after this part that does the tool-specific stuff. I was just wondering, why the apparently tool-specific files are handled in the general snakefile.

srosenbu

Nice work. I have a few comments regarding the conda environments.

srosenbu · 2026-02-17T09:58:05Z

benchmarks/rotating-cylinders/environment_dumux.yaml

In the other benchmarks, we use the https://github.com/BAMresearch/NFDI4IngModelValidationPlatform/blob/main/environment_benchmarks.yml file to start snakemake. Ich each of the rules,we then define the environment or container used to run the rule. So I tthink this file should be removed

srosenbu · 2026-02-17T09:59:36Z

benchmarks/rotating-cylinders/dumux/Snakefile

+        """
+
+# Rule 2: Post-processing
+rule postprocess_dumux:


If you use anything in the postprocessing that is not in the environment_benchmarks.yml, this rule needs a different environment. Otherwise it's fine.

srosenbu · 2026-02-17T10:12:59Z

.github/workflows/run-dumux-benchmark.yml

+          apptainer-version: 1.4.5
+
+      - name: Update environment
+        run: mamba env update -n rc-dumux -f $GITHUB_WORKSPACE/benchmarks/rotating-cylinders/environment_dumux.yaml


here you should activate the environment_benchmarks.yml. If you think that we need anything else in this environment, we can think about adding it instead of each tool having their own environment for snakemake

Sarbani-Roy added 3 commits January 7, 2026 18:31

Run DuMuX simulation inside container via Snakemake

ab25875

Update .gitignore and remove unnecessary files (.DS_Store, .simulatio…

9d2e868

…n_done)

dumux simulation can run via snakemake rule

91fd400

Sarbani-Roy force-pushed the 48-add-rotating-cylinders-benchmark-using-dumux branch from dabd6b9 to 91fd400 Compare January 22, 2026 13:37

Sarbani-Roy added 3 commits January 23, 2026 12:04

executing dumux simulation with three different input files

ef52760

Merge main and restore run-benchmark.yml

edc0793

dumux benchmark implementation working

ad3298d

joergfunger requested changes Jan 29, 2026

View reviewed changes

joergfunger assigned srosenbu and jpthiele Feb 3, 2026

srosenbu reviewed Feb 17, 2026

View reviewed changes

Conversation

Sarbani-Roy commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Sarbani-Roy commented Jan 28, 2026

Uh oh!

joergfunger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srosenbu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Sarbani-Roy commented Jan 7, 2026 •

edited

Loading