Buffer access issue in high-resolution IBM grids

[SpeedyTurtle599]

Describe the bug
When running the examples/2D_forward_facing_step/case.py file, MFC crashes in the simulation step (after successfully preprocessing). The terminal shows this log:

+ mpirun -np 14 /Users/speedyturtle/Developer/mfc/build/install/51b5b52d6c/bin/simulation
 Simulating a regular 1499x499x0 case on 14 rank(s) on CPUs.
At line 514 of file /Users/speedyturtle/Developer/mfc/src/simulation/m_ibm.fpp
Fortran runtime error: Index '-11' of dimension 1 of array 's_cc' below lower bound of -10

Error termination. Backtrace:

Could not print backtrace: executable file is not an executable
#0  0x1052a2103
#1  0x1052a2cd7
#2  0x1052a3077
#3  0x10476ea63
#4  0x104778b2b
#5  0x10486e76f
#6  0x10494a377
--------------------------------------------------------------------------
prterun detected that one or more processes exited with non-zero status,
thus causing the job to be terminated. The first process to do so was:

   Process name: [prterun-BGC-MBP14-14574@1,2]
   Exit code:    2
--------------------------------------------------------------------------

mfc: ERROR > :( /Users/speedyturtle/Developer/mfc/build/install/51b5b52d6c/bin/simulation failed with exit code 2.

 

Error: Submitting batch file for Interactive failed. It can be found here: /Users/speedyturtle/Developer/mfc/examples/2D_forward_facing_step/MFC.sh. Please check the file for errors.

./mfc.sh: line 74: 98008 Terminated: 15          python3 "$(pwd)/toolchain/main.py" "$@"

mfc: ERROR > main.py finished with a 143 exit code.
mfc: (venv) Exiting the Python virtual environment.

To Reproduce

1. Clone latest mfc from source
2. Compile using ./mfc.sh build -j14
3. Run example using ./mfc.sh run examples/2D_forward_facing_step/case.py -n14
4. See error in simulation executable

Expected behavior
Simulation correctly accesses ghost points in high-resolution grids

Environment

• OS: macOS 26.2
• Compiler versions: Apple clang-1700.6.3.2, gfortran 15.2.0, Python 3.14.3

Additional context
Best guess, this happens because of a hardcoded buffer size limit for IBM simulations. I was able to sidestep the problem by increasing the buffer limit from 10 to 20 in /src/m_helper_basic.fpp. As of writing, that's lines 147 to 149.

[sbryngelson]

i didn't see this until now, @danieljvickers or @anandrdbz would know what's going on

[danieljvickers]

I will load up this particular case locally and see. I can tell you immediately that you can fix this problem for yourself quickly by simply using fewer nodes. The error that is printed is related to the fact that there is an issue with the ghost-point method for immersed boundaries where sometimes the ghost point ends up in a different computational domain than the image point. Using a different number of processors will prevent this from happening.

As for longer-term fixes, I have actually been working on this bug this week, so you are in luck. My new PR branch actually cleared a the rectangle patch with 10k random configurations of rectangles during testing. That doesn't mean it is perfect, just that any error in rectangle IBs should be low. But I will give it a look. If you want to try out my new branch and see if it has resolved the issue, you can pull it here: #1123

[anandrdbz]

I believe this is because the image point for a particular ghost point is outside the range of values that are accessible. It should print an error message saying buffer size should be increased beyond 10, but it also prints s_cc at this index outside it's range leading to a memory access fault

The only solution to this is to increase buffer size beyond 10 on line 148 in src/common/m_helper_basic.fpp. So keep increasing it until you no longer get this error (or use fewer procs or increase resolution)

[danieljvickers]

That is another solution, yes. In my opinion it is not an ideal one because it involves ignoring the underlying issue of non-rotated rectangle geometries should never point out of bounds. won't go through the details too much here, but if we actually compute the levelset and levelset norm correctly, the maximum grid spacing a cells an image point should be from a ghost point (without grid stretching) is 8 cells. So 10 is a safety buffer that we have for some of the odd shaped geometries where these values are only approximations, like airfoils and STL models. Rectangles don't fall into that camp. They should work perfectly fine. So the fact that this happens at all tells me there is some other bug in the middle here. And ideally we have well posed behavior to ensure that it gives the same answer on all ranks. Give me a bit to remote to a machine and let you know.

[danieljvickers]

I run the test and can report the issue. The problem is because of a bad set up in the case file. First the domain has a lower bounds of "y_domain%beg": 0.0, and the IB is placed at the bottom of the domain with "patch_ib(1)%y_centroid": 0 * h,. Finally the Ib has a height of "patch_ib(1)%length_y": 2 * h, where h=0.2.

You will notice that it means that only half of the vertical component is in the overall computational domain. What this means is that no ghost points get computed outside the domain. The values on the bottom of the rectangle get marked as a ghost point, but those values are actually right at the center of the domain. By swapping out "patch_ib(1)%length_y": h, and "patch_ib(1)%y_centroid": 0.5 * h, the error in the y direction goes away.

For exactly the same reasons, the values of "patch_icpp(1)%x_centroid": 9 * h, and "patch_icpp(1)%length_x": 12 * h, are needed in the x direction. If those 4 lines are modified, it passes:

"patch_ib(1)%x_centroid": 9 * h,
"patch_ib(1)%y_centroid": 0.5 * h,
"patch_ib(1)%length_x": 12 * h,
"patch_ib(1)%length_y": h,

I have mode these changes to the example in my latest PR.

[danieljvickers]

Issue is solved and plan to add protection against bad use cases has been slated for future development.