Skip to content
Permalink
d72cc01465
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

370CT_21-22_Open/lectures/12_combining_MPI_and_OpenMP.org

Go to file
 
 
Cannot retrieve contributors at this time
Combining MPI and OpenMP setup Combining MPI and OpenMP Obligatory XKCD
executable file 70 lines (62 sloc) 3.06 KB
Raw Blame

Combining MPI and OpenMP

setup

Combining MPI and OpenMP

  • While MPI could be used as a parallelism library, it isn’t a particularly efficient one.
  • OpenMP was designed intentional for exactly this purpose, whereas MPI was designed primarily for use on distributed computing systems. It just happened that a consequence of this was it could do paralelism as well.
  • Thus far with MPI we’ve looked at how to move data over a cluster and build our own data structures. However, the nodes are rarely, if ever single core machines, so we need to make use of a parallelism library to take advantage of the nodes properly.

  • On this module I’ve only covered OpenMP in any detail, but any parallelism library would work in combination with MPI.
  • In the commercial world OneTBB, Intels Threading Building Blocks (TBB), is gaining popularity. It is more advanced, feature wise than OpenMP.
  • It removes some of the things I consider to be unnecessary now, such as barrier control, and provides fine grain parallelism, thus reducing wasted computational time.
  • This in turn saves energy, as more can be done using less power.

  • There’s no inherent advantage, in terms of speed alone, between OpenMP and TBB, as they both falter at the same number of cores due to the inter core communication problem, starting to suffer at 14 to 16 cores, slowly worsening, then topping out at around 32 cores, at which point the computer is spending more time managing the code than running the problem it’s trying to solve.
  • There are various counts I’ve found for the exact number of cores, but the problem remains the same, they’re both limited, just as threaded programming is limited by mutex lock queues, and seriel programming is limited by its very nature.

  • To integrate any Parallelism library, or a Concurrency library if you wanted to doesn’t take anything like the detail you might think.
  • To continue, open the source code in code/distributed/demonstration_code/OpenMP_and_MPI and we’ll go through this example program together.

Obligatory XKCD

file:img/multi.png