8000 [ENH]Dynamic ram estimator by mauriliogenovese · Pull Request #3740 · nipy/nipype · GitHub
[go: up one dir, main page]

Skip to content

[ENH]Dynamic ram estimator - #3740

Merged
effigies merged 2 commits into
nipy:masterfrom
mauriliogenovese:enh/dynamic-ram-estimators
Mar 2, 2026
Merged

effigies merged 2 commits into
nipy:masterfrom
mauriliogenovese:enh/dynamic-ram-estimators

Conversation

@mauriliogenovese
Copy link
Copy Markdown
Contributor

This PR introduces a simple and extensible mechanism to estimate node RAM usage at runtime when executing workflows with MultiProcPlugin.

A new RamEstimator base class allows users (and interface authors) to attach a RAM estimator to a Node or MapNode via the node.ram_estimator attribute.
The estimator computes a per-node memory estimate before execution, based on selected input traits, and is fully user-configurable.

Estimators aggregate RAM contributions from:

  • file-like inputs (based on image voxel counts)
  • numeric inputs (scalars or lists)

A human-readable debug string describing the estimate is stored in _report/report.rst under the runtime section.

For MapNodes, the estimator is inherited by subnodes and evaluated on a representative iteration; the resulting value is interpreted as the per-task peak RAM requirement.

No changes to interface execution logic are required; this only affects scheduling/resource management.

This is intended as an opt-in, non-invasive improvement to resource-aware scheduling in multiprocess workflows.

The usage would be something like this:

class FlirtRamEstimator(RamEstimator):
            def __init__(self):
                super().__init__(
                    input_multipliers={
                        'in_file': 32,
                        'reference': 4,
                    },
                    overhead_gb=0.3,
                    min_gb=0.5,
                    max_gb=4.0
                )

from nipype.interfaces.fsl import FLIRT
from nipype.pipeline.engine import Node

flirt = Node(FLIRT(dof=6), name="flirt")
flirt.ram_estimator = FlirtRamEstimator()

@codecov
codecov Bot commented Mar 2, 2026 •
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 25.00000% with 66 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.85%. Comparing base (23e003b) to head (376ab30).
⚠️ Report is 45 commits behind head on master.

Files with missing lines Patch % Lines
nipype/utils/ram_estimator.py 15.06% 62 Missing ⚠️
nipype/pipeline/engine/nodes.py 75.00% 3 Missing ⚠️
nipype/pipeline/engine/utils.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3740      +/-   ##
==========================================
- Coverage   72.92%   72.85%   -0.08%     
==========================================
  Files        1278     1279       +1     
  Lines       59234    59321      +87     
==========================================
+ Hits        43198    43219      +21     
- Misses      16036    16102      +66     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@effigies effigies left a comment
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks!

@effigies
effigies merged commit 16ea5ec into nipy:master Mar 2, 2026
20 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0