Stay focused
I’m teaching two classes at ESC Boston in September:
[ESC-442] Practical Migration of Sequential C/C++ Code to Multi-core Systems
[ESC-322] Coprocessing and Multiprocessing Techniques to Accelerate Software
and I have been busy working up examples and polishing slides.
Lots of people suggest that parallel programming is hard, so I thought I would share a few of my lessons learned during the run up to ESC Boston. Of course, some parallel programming lessons are subtle and hard won, but then there are others which remind you that no matter how complex things get, some mistakes are just plain dumb. Might as well start there…
I was reworking an image processing example to demonstrate a loop fusion technique which makes it more efficient to parallelize a processing pipeline. Part of the algorithm uses a Sobel filter, part of which includes:
x_sum = 1 * in_pixels[rn * ncols + cw];
x_sum += 2 * in_pixels[r * ncols + cw];
x_sum += 1 * in_pixels[rs * ncols + cw];
x_sum -= 1 * in_pixels[rn * ncols + ce];
x_sum -= 2 * in_pixels[r * ncols + ce];
x_sum -= 1 * in_pixels[rs * ncols + ce];
This works fine (I’ve used it before), but as I was refactoring the algorithm and adding threading, I decided that the computation might look a wee bit clearer if I stopped repeating the x_sum assignment. The quick code modification looks like this:
x_sum = 1 * in_pixels[rn * ncols + cw];
+ 2 * in_pixels[r * ncols + cw];
+ 1 * in_pixels[rs * ncols + cw];
- 1 * in_pixels[rn * ncols + ce];
- 2 * in_pixels[r * ncols + ce];
- 1 * in_pixels[rs * ncols + ce];
I just couldn’t leave well enough alone. I finished up all the refactoring, and pretty quickly I got a clean compile. Great!
But of course, when I ran it, I got nothing close to the test image I was expecting. Well that’s not too unexpected- I figured I’d need to do a little debugging- I was fusing loops and I had to lag some computations in buffers to make everything pipe out correctly- it’s pretty easy to make an error in there somewhere.
You probably know the rest. After adding a fair amount of debug scaffolding to examine what was happening, I concluded that I had done all the refactoring and fusing correctly. I finally narrowed the problem down to the Sobel, and it took a lot of head scratching to realize that for the x_sum computation, I’d removed ‘=‘ assignments, but I had failed to remove the corresponding ‘;‘ at all but the last line. No complaints from my compiler; it’s perfectly legal though not terribly useful C code.
So today’s important parallel programming lesson- stay focused. As you migrate from sequential to parallel code, change the minimum necessary to move from one stable program to another. Resist the urge to clean up a few incidentals along the way- some people think parallel programming is hard enough without making it more so.
- Skip
BTW: for GCC, anybody know which compiler switches to add (or remove) to warn about a statement with no lvalue and no side effects?