diff options
Diffstat (limited to 'doc/html/mpi/python.html')
| -rw-r--r-- | doc/html/mpi/python.html | 298 |
1 files changed, 298 insertions, 0 deletions
diff --git a/doc/html/mpi/python.html b/doc/html/mpi/python.html new file mode 100644 index 0000000000..226f6d28c2 --- /dev/null +++ b/doc/html/mpi/python.html @@ -0,0 +1,298 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> +<html> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII"> +<title>Python Bindings</title> +<link rel="stylesheet" href="../../../doc/src/boostbook.css" type="text/css"> +<meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> +<link rel="home" href="../index.html" title="The Boost C++ Libraries BoostBook Documentation Subset"> +<link rel="up" href="../mpi.html" title="Chapter 26. Boost.MPI"> +<link rel="prev" href="../boost/mpi/timer.html" title="Class timer"> +<link rel="next" href="design.html" title="Design Philosophy"> +</head> +<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> +<table cellpadding="2" width="100%"><tr> +<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../boost.png"></td> +<td align="center"><a href="../../../index.html">Home</a></td> +<td align="center"><a href="../../../libs/libraries.htm">Libraries</a></td> +<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> +<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> +<td align="center"><a href="../../../more/index.htm">More</a></td> +</tr></table> +<hr> +<div class="spirit-nav"> +<a accesskey="p" href="../boost/mpi/timer.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="design.html"><img src="../../../doc/src/images/next.png" alt="Next"></a> +</div> +<div class="section"> +<div class="titlepage"><div><div><h2 class="title" style="clear: both"> +<a name="mpi.python"></a><a class="link" href="python.html" title="Python Bindings">Python Bindings</a> +</h2></div></div></div> +<div class="toc"><dl class="toc"> +<dt><span class="section"><a href="python.html#mpi.python.quickstart">Quickstart</a></span></dt> +<dt><span class="section"><a href="python.html#mpi.python.user_data">Transmitting User-Defined Data</a></span></dt> +<dt><span class="section"><a href="python.html#mpi.python.collectives">Collectives</a></span></dt> +<dt><span class="section"><a href="python.html#mpi.python.skeleton_content">Skeleton/Content Mechanism</a></span></dt> +<dt><span class="section"><a href="python.html#mpi.python.compatibility">C++/Python MPI Compatibility</a></span></dt> +<dt><span class="section"><a href="python.html#mpi.python.reference">Reference</a></span></dt> +</dl></div> +<p> + Boost.MPI provides an alternative MPI interface from the <a href="http://www.python.org" target="_top">Python</a> + programming language via the <code class="computeroutput"><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span></code> module. + The Boost.MPI Python bindings, built on top of the C++ Boost.MPI using the + <a href="http://www.boost.org/libs/python/doc" target="_top">Boost.Python</a> library, + provide nearly all of the functionality of Boost.MPI within a dynamic, object-oriented + language. + </p> +<p> + The Boost.MPI Python module can be built and installed from the <code class="computeroutput"><span class="identifier">libs</span><span class="special">/</span><span class="identifier">mpi</span><span class="special">/</span><span class="identifier">build</span></code> directory. + Just follow the <a class="link" href="getting_started.html#mpi.getting_started.config" title="Configure and Build">configuration</a> + and <a class="link" href="getting_started.html#mpi.getting_started.config.installation" title="Installation">installation</a> + instructions for the C++ Boost.MPI. Once you have installed the Python module, + be sure that the installation location is in your <code class="computeroutput"><span class="identifier">PYTHONPATH</span></code>. + </p> +<div class="section"> +<div class="titlepage"><div><div><h3 class="title"> +<a name="mpi.python.quickstart"></a><a class="link" href="python.html#mpi.python.quickstart" title="Quickstart">Quickstart</a> +</h3></div></div></div> +<p> + Getting started with the Boost.MPI Python module is as easy as importing + <code class="computeroutput"><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span></code>. Our first "Hello, World!" + program is just two lines long: + </p> +<pre class="programlisting"><span class="keyword">import</span> <span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span> <span class="keyword">as</span> <span class="identifier">mpi</span> +<span class="keyword">print</span> <span class="string">"I am process %d of %d."</span> <span class="special">%</span> <span class="special">(</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">rank</span><span class="special">,</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">size</span><span class="special">)</span> +</pre> +<p> + Go ahead and run this program with several processes. Be sure to invoke the + <code class="computeroutput"><span class="identifier">python</span></code> interpreter from + <code class="computeroutput"><span class="identifier">mpirun</span></code>, e.g., + </p> +<pre class="programlisting">mpirun -np 5 python hello_world.py +</pre> +<p> + This will return output such as: + </p> +<pre class="programlisting">I am process 1 of 5. +I am process 3 of 5. +I am process 2 of 5. +I am process 4 of 5. +I am process 0 of 5. +</pre> +<p> + Point-to-point operations in Boost.MPI have nearly the same syntax in Python + as in C++. We can write a simple two-process Python program that prints "Hello, + world!" by transmitting Python strings: + </p> +<pre class="programlisting"><span class="keyword">import</span> <span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span> <span class="keyword">as</span> <span class="identifier">mpi</span> + +<span class="keyword">if</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">rank</span> <span class="special">==</span> <span class="number">0</span><span class="special">:</span> + <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="string">'Hello'</span><span class="special">)</span> + <span class="identifier">msg</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">1</span><span class="special">)</span> + <span class="keyword">print</span> <span class="identifier">msg</span><span class="special">,</span><span class="string">'!'</span> +<span class="keyword">else</span><span class="special">:</span> + <span class="identifier">msg</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">)</span> + <span class="keyword">print</span> <span class="special">(</span><span class="identifier">msg</span> <span class="special">+</span> <span class="string">', '</span><span class="special">),</span> + <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">1</span><span class="special">,</span> <span class="string">'world'</span><span class="special">)</span> +</pre> +<p> + There are only a few notable differences between this Python code and the + example <a class="link" href="tutorial.html#mpi.tutorial.point_to_point" title="Point-to-Point communication">in the C++ tutorial</a>. + First of all, we don't need to write any initialization code in Python: just + loading the <code class="computeroutput"><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span></code> module makes the appropriate <code class="computeroutput"><span class="identifier">MPI_Init</span></code> and <code class="computeroutput"><span class="identifier">MPI_Finalize</span></code> + calls. Second, we're passing Python objects from one process to another through + MPI. Any Python object that can be pickled can be transmitted; the next section + will describe in more detail how the Boost.MPI Python layer transmits objects. + Finally, when we receive objects with <code class="computeroutput"><span class="identifier">recv</span></code>, + we don't need to specify the type because transmission of Python objects + is polymorphic. + </p> +<p> + When experimenting with Boost.MPI in Python, don't forget that help is always + available via <code class="computeroutput"><span class="identifier">pydoc</span></code>: just + pass the name of the module or module entity on the command line (e.g., + <code class="computeroutput"><span class="identifier">pydoc</span> <span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">communicator</span></code>) to receive complete reference + documentation. When in doubt, try it! + </p> +</div> +<div class="section"> +<div class="titlepage"><div><div><h3 class="title"> +<a name="mpi.python.user_data"></a><a class="link" href="python.html#mpi.python.user_data" title="Transmitting User-Defined Data">Transmitting User-Defined Data</a> +</h3></div></div></div> +<p> + Boost.MPI can transmit user-defined data in several different ways. Most + importantly, it can transmit arbitrary <a href="http://www.python.org" target="_top">Python</a> + objects by pickling them at the sender and unpickling them at the receiver, + allowing arbitrarily complex Python data structures to interoperate with + MPI. + </p> +<p> + Boost.MPI also supports efficient serialization and transmission of C++ objects + (that have been exposed to Python) through its C++ interface. Any C++ type + that provides (de-)serialization routines that meet the requirements of the + Boost.Serialization library is eligible for this optimization, but the type + must be registered in advance. To register a C++ type, invoke the C++ function + <code class="computeroutput"><a class="link" href="../boost/mpi/python/register_serialized.html" title="Function template register_serialized">register_serialized</a></code>. + If your C++ types come from other Python modules (they probably will!), those + modules will need to link against the <code class="computeroutput"><span class="identifier">boost_mpi</span></code> + and <code class="computeroutput"><span class="identifier">boost_mpi_python</span></code> libraries + as described in the <a class="link" href="getting_started.html#mpi.getting_started.config.installation" title="Installation">installation + section</a>. Note that you do <span class="bold"><strong>not</strong></span> need + to link against the Boost.MPI Python extension module. + </p> +<p> + Finally, Boost.MPI supports separation of the structure of an object from + the data it stores, allowing the two pieces to be transmitted separately. + This "skeleton/content" mechanism, described in more detail in + a later section, is a communication optimization suitable for problems with + fixed data structures whose internal data changes frequently. + </p> +</div> +<div class="section"> +<div class="titlepage"><div><div><h3 class="title"> +<a name="mpi.python.collectives"></a><a class="link" href="python.html#mpi.python.collectives" title="Collectives">Collectives</a> +</h3></div></div></div> +<p> + Boost.MPI supports all of the MPI collectives (<code class="computeroutput"><span class="identifier">scatter</span></code>, + <code class="computeroutput"><span class="identifier">reduce</span></code>, <code class="computeroutput"><span class="identifier">scan</span></code>, + <code class="computeroutput"><span class="identifier">broadcast</span></code>, etc.) for any + type of data that can be transmitted with the point-to-point communication + operations. For the MPI collectives that require a user-specified operation + (e.g., <code class="computeroutput"><span class="identifier">reduce</span></code> and <code class="computeroutput"><span class="identifier">scan</span></code>), the operation can be an arbitrary + Python function. For instance, one could concatenate strings with <code class="computeroutput"><span class="identifier">all_reduce</span></code>: + </p> +<pre class="programlisting"><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">all_reduce</span><span class="special">(</span><span class="identifier">my_string</span><span class="special">,</span> <span class="keyword">lambda</span> <span class="identifier">x</span><span class="special">,</span><span class="identifier">y</span><span class="special">:</span> <span class="identifier">x</span> <span class="special">+</span> <span class="identifier">y</span><span class="special">)</span> +</pre> +<p> + The following module-level functions implement MPI collectives: all_gather + Gather the values from all processes. all_reduce Combine the results from + all processes. all_to_all Every process sends data to every other process. + broadcast Broadcast data from one process to all other processes. gather + Gather the values from all processes to the root. reduce Combine the results + from all processes to the root. scan Prefix reduction of the values from + all processes. scatter Scatter the values stored at the root to all processes. + </p> +</div> +<div class="section"> +<div class="titlepage"><div><div><h3 class="title"> +<a name="mpi.python.skeleton_content"></a><a class="link" href="python.html#mpi.python.skeleton_content" title="Skeleton/Content Mechanism">Skeleton/Content Mechanism</a> +</h3></div></div></div> +<p> + Boost.MPI provides a skeleton/content mechanism that allows the transfer + of large data structures to be split into two separate stages, with the skeleton + (or, "shape") of the data structure sent first and the content + (or, "data") of the data structure sent later, potentially several + times, so long as the structure has not changed since the skeleton was transferred. + The skeleton/content mechanism can improve performance when the data structure + is large and its shape is fixed, because while the skeleton requires serialization + (it has an unknown size), the content transfer is fixed-size and can be done + without extra copies. + </p> +<p> + To use the skeleton/content mechanism from Python, you must first register + the type of your data structure with the skeleton/content mechanism <span class="bold"><strong>from C++</strong></span>. The registration function is <code class="computeroutput"><a class="link" href="../boost/mpi/python/regist_1_3_27_7_20_2_1_1_2.html" title="Function template register_skeleton_and_content">register_skeleton_and_content</a></code> + and resides in the <code class="computeroutput"><a class="link" href="reference.html#header.boost.mpi.python_hpp" title="Header <boost/mpi/python.hpp>"><boost/mpi/python.hpp></a></code> + header. + </p> +<p> + Once you have registered your C++ data structures, you can extract the skeleton + for an instance of that data structure with <code class="computeroutput"><span class="identifier">skeleton</span><span class="special">()</span></code>. The resulting <code class="computeroutput"><span class="identifier">skeleton_proxy</span></code> + can be transmitted via the normal send routine, e.g., + </p> +<pre class="programlisting"><span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">skeleton</span><span class="special">(</span><span class="identifier">my_data_structure</span><span class="special">))</span> +</pre> +<p> + <code class="computeroutput"><span class="identifier">skeleton_proxy</span></code> objects can + be received on the other end via <code class="computeroutput"><span class="identifier">recv</span><span class="special">()</span></code>, which stores a newly-created instance + of your data structure with the same "shape" as the sender in its + <code class="computeroutput"><span class="string">"object"</span></code> attribute: + </p> +<pre class="programlisting"><span class="identifier">shape</span> <span class="special">=</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">)</span> +<span class="identifier">my_data_structure</span> <span class="special">=</span> <span class="identifier">shape</span><span class="special">.</span><span class="identifier">object</span> +</pre> +<p> + Once the skeleton has been transmitted, the content (accessed via <code class="computeroutput"><span class="identifier">get_content</span></code>) can be transmitted in much + the same way. Note, however, that the receiver also specifies <code class="computeroutput"><span class="identifier">get_content</span><span class="special">(</span><span class="identifier">my_data_structure</span><span class="special">)</span></code> + in its call to receive: + </p> +<pre class="programlisting"><span class="keyword">if</span> <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">rank</span> <span class="special">==</span> <span class="number">0</span><span class="special">:</span> + <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">get_content</span><span class="special">(</span><span class="identifier">my_data_structure</span><span class="special">))</span> +<span class="keyword">else</span><span class="special">:</span> + <span class="identifier">mpi</span><span class="special">.</span><span class="identifier">world</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">get_content</span><span class="special">(</span><span class="identifier">my_data_structure</span><span class="special">))</span> +</pre> +<p> + Of course, this transmission of content can occur repeatedly, if the values + in the data structure--but not its shape--changes. + </p> +<p> + The skeleton/content mechanism is a structured way to exploit the interaction + between custom-built MPI datatypes and <code class="computeroutput"><span class="identifier">MPI_BOTTOM</span></code>, + to eliminate extra buffer copies. + </p> +</div> +<div class="section"> +<div class="titlepage"><div><div><h3 class="title"> +<a name="mpi.python.compatibility"></a><a class="link" href="python.html#mpi.python.compatibility" title="C++/Python MPI Compatibility">C++/Python MPI Compatibility</a> +</h3></div></div></div> +<p> + Boost.MPI is a C++ library whose facilities have been exposed to Python via + the Boost.Python library. Since the Boost.MPI Python bindings are build directly + on top of the C++ library, and nearly every feature of C++ library is available + in Python, hybrid C++/Python programs using Boost.MPI can interact, e.g., + sending a value from Python but receiving that value in C++ (or vice versa). + However, doing so requires some care. Because Python objects are dynamically + typed, Boost.MPI transfers type information along with the serialized form + of the object, so that the object can be received even when its type is not + known. This mechanism differs from its C++ counterpart, where the static + types of transmitted values are always known. + </p> +<p> + The only way to communicate between the C++ and Python views on Boost.MPI + is to traffic entirely in Python objects. For Python, this is the normal + state of affairs, so nothing will change. For C++, this means sending and + receiving values of type <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">python</span><span class="special">::</span><span class="identifier">object</span></code>, + from the <a href="http://www.boost.org/libs/python/doc" target="_top">Boost.Python</a> + library. For instance, say we want to transmit an integer value from Python: + </p> +<pre class="programlisting"><span class="identifier">comm</span><span class="special">.</span><span class="identifier">send</span><span class="special">(</span><span class="number">1</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="number">17</span><span class="special">)</span> +</pre> +<p> + In C++, we would receive that value into a Python object and then <code class="computeroutput"><span class="identifier">extract</span></code> an integer value: + </p> +<pre class="programlisting"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">python</span><span class="special">::</span><span class="identifier">object</span> <span class="identifier">value</span><span class="special">;</span> +<span class="identifier">comm</span><span class="special">.</span><span class="identifier">recv</span><span class="special">(</span><span class="number">0</span><span class="special">,</span> <span class="number">0</span><span class="special">,</span> <span class="identifier">value</span><span class="special">);</span> +<span class="keyword">int</span> <span class="identifier">int_value</span> <span class="special">=</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">python</span><span class="special">::</span><span class="identifier">extract</span><span class="special"><</span><span class="keyword">int</span><span class="special">>(</span><span class="identifier">value</span><span class="special">);</span> +</pre> +<p> + In the future, Boost.MPI will be extended to allow improved interoperability + with the C++ Boost.MPI and the C MPI bindings. + </p> +</div> +<div class="section"> +<div class="titlepage"><div><div><h3 class="title"> +<a name="mpi.python.reference"></a><a class="link" href="python.html#mpi.python.reference" title="Reference">Reference</a> +</h3></div></div></div> +<p> + The Boost.MPI Python module, <code class="computeroutput"><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span></code>, has + its own <a href="../boost.mpi.html" target="_top">reference documentation</a>, which + is also available using <code class="computeroutput"><span class="identifier">pydoc</span></code> + (from the command line) or <code class="computeroutput"><span class="identifier">help</span><span class="special">(</span><span class="identifier">boost</span><span class="special">.</span><span class="identifier">mpi</span><span class="special">)</span></code> + (from the Python interpreter). + </p> +</div> +</div> +<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> +<td align="left"></td> +<td align="right"><div class="copyright-footer">Copyright © 2005-2007 Douglas Gregor, + Matthias Troyer, Trustees of Indiana University<p> + Distributed under the Boost Software License, Version 1.0. (See accompanying + file LICENSE_1_0.txt or copy at <ulink url="http://www.boost.org/LICENSE_1_0.txt"> + http://www.boost.org/LICENSE_1_0.txt </ulink>) + </p> +</div></td> +</tr></table> +<hr> +<div class="spirit-nav"> +<a accesskey="p" href="../boost/mpi/timer.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../mpi.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="design.html"><img src="../../../doc/src/images/next.png" alt="Next"></a> +</div> +</body> +</html> |