diff options
Diffstat (limited to 'doc/source/reference')
49 files changed, 1091 insertions, 713 deletions
diff --git a/doc/source/reference/alignment.rst b/doc/source/reference/alignment.rst index ebc8f353c..5e4315b38 100644 --- a/doc/source/reference/alignment.rst +++ b/doc/source/reference/alignment.rst @@ -67,7 +67,7 @@ There are 4 relevant uses of the word ``align`` used in numpy: field alignment. In either case ``dtype.isalignedstruct`` is also set to True. * ``IsUintAligned`` is used to determine if an ndarray is "uint aligned" in - an analagous way to how ``IsAligned`` checks for true-alignment. + an analogous way to how ``IsAligned`` checks for true-alignment. Consequences of alignment ------------------------- diff --git a/doc/source/reference/arrays.classes.rst b/doc/source/reference/arrays.classes.rst index 0fafc593e..9dcbb6267 100644 --- a/doc/source/reference/arrays.classes.rst +++ b/doc/source/reference/arrays.classes.rst @@ -51,7 +51,7 @@ NumPy provides several hooks that classes can customize: .. versionadded:: 1.13 Any class, ndarray subclass or not, can define this method or set it to - :obj:`None` in order to override the behavior of NumPy's ufuncs. This works + None in order to override the behavior of NumPy's ufuncs. This works quite similarly to Python's ``__mul__`` and other binary operation routines. - *ufunc* is the ufunc object that was called. @@ -82,7 +82,7 @@ NumPy provides several hooks that classes can customize: :func:`~numpy.matmul`, which currently is not a Ufunc, but could be relatively easily be rewritten as a (set of) generalized Ufuncs. The same may happen with functions such as :func:`~numpy.median`, - :func:`~numpy.min`, and :func:`~numpy.argsort`. + :func:`~numpy.amin`, and :func:`~numpy.argsort`. Like with some other special methods in python, such as ``__hash__`` and ``__iter__``, it is possible to indicate that your class does *not* @@ -94,13 +94,13 @@ NumPy provides several hooks that classes can customize: :class:`ndarray` handles binary operations like ``arr + obj`` and ``arr < obj`` when ``arr`` is an :class:`ndarray` and ``obj`` is an instance of a custom class. There are two possibilities. If - ``obj.__array_ufunc__`` is present and not :obj:`None`, then + ``obj.__array_ufunc__`` is present and not None, then ``ndarray.__add__`` and friends will delegate to the ufunc machinery, meaning that ``arr + obj`` becomes ``np.add(arr, obj)``, and then :func:`~numpy.add` invokes ``obj.__array_ufunc__``. This is useful if you want to define an object that acts like an array. - Alternatively, if ``obj.__array_ufunc__`` is set to :obj:`None`, then as a + Alternatively, if ``obj.__array_ufunc__`` is set to None, then as a special case, special methods like ``ndarray.__add__`` will notice this and *unconditionally* raise :exc:`TypeError`. This is useful if you want to create objects that interact with arrays via binary operations, but @@ -135,7 +135,7 @@ NumPy provides several hooks that classes can customize: place rather than separately by the ufunc machinery and by the binary operation rules (which gives preference to special methods of subclasses; the alternative way to enforce a one-place only hierarchy, - of setting :func:`__array_ufunc__` to :obj:`None`, would seem very + of setting :func:`__array_ufunc__` to None, would seem very unexpected and thus confusing, as then the subclass would not work at all with ufuncs). - :class:`ndarray` defines its own :func:`__array_ufunc__`, which, @@ -280,7 +280,7 @@ NumPy provides several hooks that classes can customize: .. py:method:: class.__array_prepare__(array, context=None) - At the beginning of every :ref:`ufunc <ufuncs.output-type>`, this + At the beginning of every :ref:`ufunc <ufuncs-output-type>`, this method is called on the input object with the highest array priority, or the output object if one was specified. The output array is passed in and whatever is returned is passed to the ufunc. @@ -295,7 +295,7 @@ NumPy provides several hooks that classes can customize: .. py:method:: class.__array_wrap__(array, context=None) - At the end of every :ref:`ufunc <ufuncs.output-type>`, this method + At the end of every :ref:`ufunc <ufuncs-output-type>`, this method is called on the input object with the highest array priority, or the output object if one was specified. The ufunc-computed array is passed in and whatever is returned is passed to the user. @@ -322,7 +322,7 @@ NumPy provides several hooks that classes can customize: If a class (ndarray subclass or not) having the :func:`__array__` method is used as the output object of an :ref:`ufunc - <ufuncs.output-type>`, results will be written to the object + <ufuncs-output-type>`, results will be written to the object returned by :func:`__array__`. Similar conversion is done on input arrays. @@ -570,7 +570,7 @@ object, then the Python code:: some code involving val ... -calls ``val = myiter.next()`` repeatedly until :exc:`StopIteration` is +calls ``val = next(myiter)`` repeatedly until :exc:`StopIteration` is raised by the iterator. There are several ways to iterate over an array that may be useful: default iteration, flat iteration, and :math:`N`-dimensional enumeration. diff --git a/doc/source/reference/arrays.datetime.rst b/doc/source/reference/arrays.datetime.rst index 387515f59..9c45e04c7 100644 --- a/doc/source/reference/arrays.datetime.rst +++ b/doc/source/reference/arrays.datetime.rst @@ -26,7 +26,9 @@ be either a :ref:`date unit <arrays.dtypes.dateunits>` or a :ref:`time unit <arrays.dtypes.timeunits>`. The date units are years ('Y'), months ('M'), weeks ('W'), and days ('D'), while the time units are hours ('h'), minutes ('m'), seconds ('s'), milliseconds ('ms'), and -some additional SI-prefix seconds-based units. +some additional SI-prefix seconds-based units. The datetime64 data type +also accepts the string "NAT", in any combination of lowercase/uppercase +letters, for a "Not A Time" value. .. admonition:: Example @@ -50,6 +52,11 @@ some additional SI-prefix seconds-based units. >>> np.datetime64('2005-02-25T03:30') numpy.datetime64('2005-02-25T03:30') + NAT (not a time): + + >>> numpy.datetime64('nat') + numpy.datetime64('NaT') + When creating an array of datetimes from a string, it is still possible to automatically select the unit from the inputs, by using the datetime type with generic units. @@ -100,7 +107,21 @@ Datetime and Timedelta Arithmetic NumPy allows the subtraction of two Datetime values, an operation which produces a number with a time unit. Because NumPy doesn't have a physical quantities system in its core, the timedelta64 data type was created -to complement datetime64. +to complement datetime64. The arguments for timedelta64 are a number, +to represent the number of units, and a date/time unit, such as +(D)ay, (M)onth, (Y)ear, (h)ours, (m)inutes, or (s)econds. The timedelta64 +data type also accepts the string "NAT" in place of the number for a "Not A Time" value. + +.. admonition:: Example + + >>> numpy.timedelta64(1, 'D') + numpy.timedelta64(1,'D') + + >>> numpy.timedelta64(4, 'h') + numpy.timedelta64(4,'h') + + >>> numpy.timedelta64('nAt') + numpy.timedelta64('NaT') Datetimes and Timedeltas work together to provide ways for simple datetime calculations. @@ -122,6 +143,12 @@ simple datetime calculations. >>> np.timedelta64(1,'W') % np.timedelta64(10,'D') numpy.timedelta64(7,'D') + >>> numpy.datetime64('nat') - numpy.datetime64('2009-01-01') + numpy.timedelta64('NaT','D') + + >>> numpy.datetime64('2009-01-01') + numpy.timedelta64('nat') + numpy.datetime64('NaT') + There are two Timedelta units ('Y', years and 'M', months) which are treated specially, because how much time they represent changes depending on when they are used. While a timedelta day unit is equivalent to @@ -341,7 +368,7 @@ times in UTC. By default, creating a datetime64 object from a string or printing it would convert from or to local time:: # old behavior - >>>> np.datetime64('2000-01-01T00:00:00') + >>> np.datetime64('2000-01-01T00:00:00') numpy.datetime64('2000-01-01T00:00:00-0800') # note the timezone offset -08:00 A consensus of datetime64 users agreed that this behavior is undesirable @@ -351,7 +378,7 @@ most use cases, a timezone naive datetime type is preferred, similar to the datetime64 no longer assumes that input is in local time, nor does it print local times:: - >>>> np.datetime64('2000-01-01T00:00:00') + >>> np.datetime64('2000-01-01T00:00:00') numpy.datetime64('2000-01-01T00:00:00') For backwards compatibility, datetime64 still parses timezone offsets, which @@ -367,131 +394,3 @@ with date units and datetimes with timeunits. With timezone naive datetimes, the rule for casting from dates to times is no longer ambiguous. .. _pandas: http://pandas.pydata.org - - -Differences Between 1.6 and 1.7 Datetimes -========================================= - -The NumPy 1.6 release includes a more primitive datetime data type -than 1.7. This section documents many of the changes that have taken -place. - -String Parsing -`````````````` - -The datetime string parser in NumPy 1.6 is very liberal in what it accepts, -and silently allows invalid input without raising errors. The parser in -NumPy 1.7 is quite strict about only accepting ISO 8601 dates, with a few -convenience extensions. 1.6 always creates microsecond (us) units by -default, whereas 1.7 detects a unit based on the format of the string. -Here is a comparison.:: - - # NumPy 1.6.1 - >>> np.datetime64('1979-03-22') - 1979-03-22 00:00:00 - # NumPy 1.7.0 - >>> np.datetime64('1979-03-22') - numpy.datetime64('1979-03-22') - - # NumPy 1.6.1, unit default microseconds - >>> np.datetime64('1979-03-22').dtype - dtype('datetime64[us]') - # NumPy 1.7.0, unit of days detected from string - >>> np.datetime64('1979-03-22').dtype - dtype('<M8[D]') - - # NumPy 1.6.1, ignores invalid part of string - >>> np.datetime64('1979-03-2corruptedstring') - 1979-03-02 00:00:00 - # NumPy 1.7.0, raises error for invalid input - >>> np.datetime64('1979-03-2corruptedstring') - Traceback (most recent call last): - File "<stdin>", line 1, in <module> - ValueError: Error parsing datetime string "1979-03-2corruptedstring" at position 8 - - # NumPy 1.6.1, 'nat' produces today's date - >>> np.datetime64('nat') - 2012-04-30 00:00:00 - # NumPy 1.7.0, 'nat' produces not-a-time - >>> np.datetime64('nat') - numpy.datetime64('NaT') - - # NumPy 1.6.1, 'garbage' produces today's date - >>> np.datetime64('garbage') - 2012-04-30 00:00:00 - # NumPy 1.7.0, 'garbage' raises an exception - >>> np.datetime64('garbage') - Traceback (most recent call last): - File "<stdin>", line 1, in <module> - ValueError: Error parsing datetime string "garbage" at position 0 - - # NumPy 1.6.1, can't specify unit in scalar constructor - >>> np.datetime64('1979-03-22T19:00', 'h') - Traceback (most recent call last): - File "<stdin>", line 1, in <module> - TypeError: function takes at most 1 argument (2 given) - # NumPy 1.7.0, unit in scalar constructor - >>> np.datetime64('1979-03-22T19:00', 'h') - numpy.datetime64('1979-03-22T19:00-0500','h') - - # NumPy 1.6.1, reads ISO 8601 strings w/o TZ as UTC - >>> np.array(['1979-03-22T19:00'], dtype='M8[h]') - array([1979-03-22 19:00:00], dtype=datetime64[h]) - # NumPy 1.7.0, reads ISO 8601 strings w/o TZ as local (ISO specifies this) - >>> np.array(['1979-03-22T19:00'], dtype='M8[h]') - array(['1979-03-22T19-0500'], dtype='datetime64[h]') - - # NumPy 1.6.1, doesn't parse all ISO 8601 strings correctly - >>> np.array(['1979-03-22T12'], dtype='M8[h]') - array([1979-03-22 00:00:00], dtype=datetime64[h]) - >>> np.array(['1979-03-22T12:00'], dtype='M8[h]') - array([1979-03-22 12:00:00], dtype=datetime64[h]) - # NumPy 1.7.0, handles this case correctly - >>> np.array(['1979-03-22T12'], dtype='M8[h]') - array(['1979-03-22T12-0500'], dtype='datetime64[h]') - >>> np.array(['1979-03-22T12:00'], dtype='M8[h]') - array(['1979-03-22T12-0500'], dtype='datetime64[h]') - -Unit Conversion -``````````````` - -The 1.6 implementation of datetime does not convert between units correctly.:: - - # NumPy 1.6.1, the representation value is untouched - >>> np.array(['1979-03-22'], dtype='M8[D]') - array([1979-03-22 00:00:00], dtype=datetime64[D]) - >>> np.array(['1979-03-22'], dtype='M8[D]').astype('M8[M]') - array([2250-08-01 00:00:00], dtype=datetime64[M]) - # NumPy 1.7.0, the representation is scaled accordingly - >>> np.array(['1979-03-22'], dtype='M8[D]') - array(['1979-03-22'], dtype='datetime64[D]') - >>> np.array(['1979-03-22'], dtype='M8[D]').astype('M8[M]') - array(['1979-03'], dtype='datetime64[M]') - -Datetime Arithmetic -``````````````````` - -The 1.6 implementation of datetime only works correctly for a small subset of -arithmetic operations. Here we show some simple cases.:: - - # NumPy 1.6.1, produces invalid results if units are incompatible - >>> a = np.array(['1979-03-22T12'], dtype='M8[h]') - >>> b = np.array([3*60], dtype='m8[m]') - >>> a + b - array([1970-01-01 00:00:00.080988], dtype=datetime64[us]) - # NumPy 1.7.0, promotes to higher-resolution unit - >>> a = np.array(['1979-03-22T12'], dtype='M8[h]') - >>> b = np.array([3*60], dtype='m8[m]') - >>> a + b - array(['1979-03-22T15:00-0500'], dtype='datetime64[m]') - - # NumPy 1.6.1, arithmetic works if everything is microseconds - >>> a = np.array(['1979-03-22T12:00'], dtype='M8[us]') - >>> b = np.array([3*60*60*1000000], dtype='m8[us]') - >>> a + b - array([1979-03-22 15:00:00], dtype=datetime64[us]) - # NumPy 1.7.0 - >>> a = np.array(['1979-03-22T12:00'], dtype='M8[us]') - >>> b = np.array([3*60*60*1000000], dtype='m8[us]') - >>> a + b - array(['1979-03-22T15:00:00.000000-0500'], dtype='datetime64[us]') diff --git a/doc/source/reference/arrays.dtypes.rst b/doc/source/reference/arrays.dtypes.rst index ab743a8ee..231707b11 100644 --- a/doc/source/reference/arrays.dtypes.rst +++ b/doc/source/reference/arrays.dtypes.rst @@ -128,7 +128,7 @@ What can be converted to a data-type object is described below: Used as-is. -:const:`None` +None .. index:: triple: dtype; construction; from None @@ -392,7 +392,7 @@ Type strings their values must each be lists of the same length as the *names* and *formats* lists. The *offsets* value is a list of byte offsets (limited to `ctypes.c_int`) for each field, while the *titles* value is a - list of titles for each field (:const:`None` can be used if no title is + list of titles for each field (None can be used if no title is desired for that field). The *titles* can be any :class:`string` or :class:`unicode` object and will add another entry to the fields dictionary keyed by the title and referencing the same diff --git a/doc/source/reference/arrays.indexing.rst b/doc/source/reference/arrays.indexing.rst index 0c0c8dff6..8ec8d8330 100644 --- a/doc/source/reference/arrays.indexing.rst +++ b/doc/source/reference/arrays.indexing.rst @@ -3,6 +3,10 @@ Indexing ======== +.. seealso:: + + :ref:`Indexing basics <basics.indexing>` + .. sectionauthor:: adapted from "Guide to NumPy" by Travis E. Oliphant .. currentmodule:: numpy diff --git a/doc/source/reference/arrays.interface.rst b/doc/source/reference/arrays.interface.rst index f361ccb06..f36a083aa 100644 --- a/doc/source/reference/arrays.interface.rst +++ b/doc/source/reference/arrays.interface.rst @@ -138,18 +138,18 @@ This approach to the interface consists of the object having an This attribute can also be an object exposing the :c:func:`buffer interface <PyObject_AsCharBuffer>` which will be used to share the data. If this key is not present (or - returns :class:`None`), then memory sharing will be done + returns None), then memory sharing will be done through the buffer interface of the object itself. In this case, the offset key can be used to indicate the start of the buffer. A reference to the object exposing the array interface must be stored by the new object if the memory area is to be secured. - **Default**: :const:`None` + **Default**: None **strides** (optional) - Either :const:`None` to indicate a C-style contiguous array or + Either None to indicate a C-style contiguous array or a Tuple of strides which provides the number of bytes needed to jump to the next array element in the corresponding dimension. Each entry must be an integer (a Python @@ -157,29 +157,29 @@ This approach to the interface consists of the object having an be larger than can be represented by a C "int" or "long"; the calling code should handle this appropriately, either by raising an error, or by using :c:type:`Py_LONG_LONG` in C. The - default is :const:`None` which implies a C-style contiguous + default is None which implies a C-style contiguous memory buffer. In this model, the last dimension of the array varies the fastest. For example, the default strides tuple for an object whose array entries are 8 bytes long and whose shape is (10,20,30) would be (4800, 240, 8) - **Default**: :const:`None` (C-style contiguous) + **Default**: None (C-style contiguous) **mask** (optional) - :const:`None` or an object exposing the array interface. All + None or an object exposing the array interface. All elements of the mask array should be interpreted only as true or not true indicating which elements of this array are valid. The shape of this object should be `"broadcastable" <arrays.broadcasting.broadcastable>` to the shape of the original array. - **Default**: :const:`None` (All array values are valid) + **Default**: None (All array values are valid) **offset** (optional) An integer offset into the array data region. This can only be - used when data is :const:`None` or returns a :class:`buffer` + used when data is None or returns a :class:`buffer` object. **Default**: 0. diff --git a/doc/source/reference/arrays.ndarray.rst b/doc/source/reference/arrays.ndarray.rst index 8f431bc9c..831d211bc 100644 --- a/doc/source/reference/arrays.ndarray.rst +++ b/doc/source/reference/arrays.ndarray.rst @@ -329,7 +329,7 @@ Item selection and manipulation ------------------------------- For array methods that take an *axis* keyword, it defaults to -:const:`None`. If axis is *None*, then the array is treated as a 1-D +*None*. If axis is *None*, then the array is treated as a 1-D array. Any other value for *axis* represents the dimension along which the operation should proceed. diff --git a/doc/source/reference/arrays.nditer.rst b/doc/source/reference/arrays.nditer.rst index fa8183f75..7dab09a71 100644 --- a/doc/source/reference/arrays.nditer.rst +++ b/doc/source/reference/arrays.nditer.rst @@ -115,13 +115,18 @@ context is exited. array([[ 0, 2, 4], [ 6, 8, 10]]) +If you are writing code that needs to support older versions of numpy, +note that prior to 1.15, :class:`nditer` was not a context manager and +did not have a `close` method. Instead it relied on the destructor to +initiate the writeback of the buffer. + Using an External Loop ---------------------- In all the examples so far, the elements of `a` are provided by the iterator one at a time, because all the looping logic is internal to the -iterator. While this is simple and convenient, it is not very efficient. A -better approach is to move the one-dimensional innermost loop into your +iterator. While this is simple and convenient, it is not very efficient. +A better approach is to move the one-dimensional innermost loop into your code, external to the iterator. This way, NumPy's vectorized operations can be used on larger chunks of the elements being visited. @@ -156,41 +161,29 @@ element in a computation. For example, you may want to visit the elements of an array in memory order, but use a C-order, Fortran-order, or multidimensional index to look up values in a different array. -The Python iterator protocol doesn't have a natural way to query these -additional values from the iterator, so we introduce an alternate syntax -for iterating with an :class:`nditer`. This syntax explicitly works -with the iterator object itself, so its properties are readily accessible -during iteration. With this looping construct, the current value is -accessible by indexing into the iterator, and the index being tracked -is the property `index` or `multi_index` depending on what was requested. - -The Python interactive interpreter unfortunately prints out the -values of expressions inside the while loop during each iteration of the -loop. We have modified the output in the examples using this looping -construct in order to be more readable. +The index is tracked by the iterator object itself, and accessible +through the `index` or `multi_index` properties, depending on what was +requested. The examples below show printouts demonstrating the +progression of the index: .. admonition:: Example >>> a = np.arange(6).reshape(2,3) >>> it = np.nditer(a, flags=['f_index']) - >>> while not it.finished: - ... print("%d <%d>" % (it[0], it.index), end=' ') - ... it.iternext() + >>> for x in it: + ... print("%d <%d>" % (x, it.index), end=' ') ... 0 <0> 1 <2> 2 <4> 3 <1> 4 <3> 5 <5> >>> it = np.nditer(a, flags=['multi_index']) - >>> while not it.finished: - ... print("%d <%s>" % (it[0], it.multi_index), end=' ') - ... it.iternext() + >>> for x in it: + ... print("%d <%s>" % (x, it.multi_index), end=' ') ... 0 <(0, 0)> 1 <(0, 1)> 2 <(0, 2)> 3 <(1, 0)> 4 <(1, 1)> 5 <(1, 2)> - >>> it = np.nditer(a, flags=['multi_index'], op_flags=['writeonly']) - >>> with it: - .... while not it.finished: - ... it[0] = it.multi_index[1] - it.multi_index[0] - ... it.iternext() + >>> with np.nditer(a, flags=['multi_index'], op_flags=['writeonly']) as it: + ... for x in it: + ... x[...] = it.multi_index[1] - it.multi_index[0] ... >>> a array([[ 0, 1, 2], @@ -199,7 +192,7 @@ construct in order to be more readable. Tracking an index or multi-index is incompatible with using an external loop, because it requires a different index value per element. If you try to combine these flags, the :class:`nditer` object will -raise an exception +raise an exception. .. admonition:: Example @@ -209,6 +202,42 @@ raise an exception File "<stdin>", line 1, in <module> ValueError: Iterator flag EXTERNAL_LOOP cannot be used if an index or multi-index is being tracked +Alternative Looping and Element Access +-------------------------------------- + +To make its properties more readily accessible during iteration, +:class:`nditer` has an alternative syntax for iterating, which works +explicitly with the iterator object itself. With this looping construct, +the current value is accessible by indexing into the iterator. Other +properties, such as tracked indices remain as before. The examples below +produce identical results to the ones in the previous section. + +.. admonition:: Example + + >>> a = np.arange(6).reshape(2,3) + >>> it = np.nditer(a, flags=['f_index']) + >>> while not it.finished: + ... print("%d <%d>" % (it[0], it.index), end=' ') + ... it.iternext() + ... + 0 <0> 1 <2> 2 <4> 3 <1> 4 <3> 5 <5> + + >>> it = np.nditer(a, flags=['multi_index']) + >>> while not it.finished: + ... print("%d <%s>" % (it[0], it.multi_index), end=' ') + ... it.iternext() + ... + 0 <(0, 0)> 1 <(0, 1)> 2 <(0, 2)> 3 <(1, 0)> 4 <(1, 1)> 5 <(1, 2)> + + >>> with np.nditer(a, flags=['multi_index'], op_flags=['writeonly']) as it: + ... while not it.finished: + ... it[0] = it.multi_index[1] - it.multi_index[0] + ... it.iternext() + ... + >>> a + array([[ 0, 1, 2], + [-1, 0, 1]]) + Buffering the Array Elements ---------------------------- diff --git a/doc/source/reference/c-api.array.rst b/doc/source/reference/c-api/array.rst index bd6062b16..0530a5747 100644 --- a/doc/source/reference/c-api.array.rst +++ b/doc/source/reference/c-api/array.rst @@ -20,27 +20,44 @@ Array API Array structure and data access ------------------------------- -These macros all access the :c:type:`PyArrayObject` structure members. The input -argument, arr, can be any :c:type:`PyObject *<PyObject>` that is directly interpretable -as a :c:type:`PyArrayObject *` (any instance of the :c:data:`PyArray_Type` and its -sub-types). +These macros access the :c:type:`PyArrayObject` structure members and are +defined in ``ndarraytypes.h``. The input argument, *arr*, can be any +:c:type:`PyObject *<PyObject>` that is directly interpretable as a +:c:type:`PyArrayObject *` (any instance of the :c:data:`PyArray_Type` +and itssub-types). .. c:function:: int PyArray_NDIM(PyArrayObject *arr) The number of dimensions in the array. -.. c:function:: npy_intp *PyArray_DIMS(PyArrayObject *arr) +.. c:function:: int PyArray_FLAGS(PyArrayObject* arr) - Returns a pointer to the dimensions/shape of the array. The - number of elements matches the number of dimensions - of the array. Can return ``NULL`` for 0-dimensional arrays. + Returns an integer representing the :ref:`array-flags<array-flags>`. -.. c:function:: npy_intp *PyArray_SHAPE(PyArrayObject *arr) +.. c:function:: int PyArray_TYPE(PyArrayObject* arr) + + Return the (builtin) typenumber for the elements of this array. + +.. c:function:: int PyArray_SETITEM( \ + PyArrayObject* arr, void* itemptr, PyObject* obj) + + Convert obj and place it in the ndarray, *arr*, at the place + pointed to by itemptr. Return -1 if an error occurs or 0 on + success. + +.. c:function:: void PyArray_ENABLEFLAGS(PyArrayObject* arr, int flags) .. versionadded:: 1.7 - A synonym for PyArray_DIMS, named to be consistent with the - 'shape' usage within Python. + Enables the specified array flags. This function does no validation, + and assumes that you know what you're doing. + +.. c:function:: void PyArray_CLEARFLAGS(PyArrayObject* arr, int flags) + + .. versionadded:: 1.7 + + Clears the specified array flags. This function does no validation, + and assumes that you know what you're doing. .. c:function:: void *PyArray_DATA(PyArrayObject *arr) @@ -53,6 +70,19 @@ sub-types). array then be sure you understand how to access the data in the array to avoid memory and/or alignment problems. +.. c:function:: npy_intp *PyArray_DIMS(PyArrayObject *arr) + + Returns a pointer to the dimensions/shape of the array. The + number of elements matches the number of dimensions + of the array. Can return ``NULL`` for 0-dimensional arrays. + +.. c:function:: npy_intp *PyArray_SHAPE(PyArrayObject *arr) + + .. versionadded:: 1.7 + + A synonym for :c:func:`PyArray_DIMS`, named to be consistent with the + `shape <numpy.ndarray.shape>` usage within Python. + .. c:function:: npy_intp *PyArray_STRIDES(PyArrayObject* arr) Returns a pointer to the strides of the array. The @@ -67,6 +97,27 @@ sub-types). Return the stride in the *n* :math:`^{\textrm{th}}` dimension. +.. c:function:: npy_intp PyArray_ITEMSIZE(PyArrayObject* arr) + + Return the itemsize for the elements of this array. + + Note that, in the old API that was deprecated in version 1.7, this function + had the return type ``int``. + +.. c:function:: npy_intp PyArray_SIZE(PyArrayObject* arr) + + Returns the total size (in number of elements) of the array. + +.. c:function:: npy_intp PyArray_Size(PyArrayObject* obj) + + Returns 0 if *obj* is not a sub-class of ndarray. Otherwise, + returns the total number of elements in the array. Safer version + of :c:func:`PyArray_SIZE` (*obj*). + +.. c:function:: npy_intp PyArray_NBYTES(PyArrayObject* arr) + + Returns the total number of bytes consumed by the array. + .. c:function:: PyObject *PyArray_BASE(PyArrayObject* arr) This returns the base object of the array. In most cases, this @@ -93,60 +144,12 @@ sub-types). A synonym for PyArray_DESCR, named to be consistent with the 'dtype' usage within Python. -.. c:function:: void PyArray_ENABLEFLAGS(PyArrayObject* arr, int flags) - - .. versionadded:: 1.7 - - Enables the specified array flags. This function does no validation, - and assumes that you know what you're doing. - -.. c:function:: void PyArray_CLEARFLAGS(PyArrayObject* arr, int flags) - - .. versionadded:: 1.7 - - Clears the specified array flags. This function does no validation, - and assumes that you know what you're doing. - -.. c:function:: int PyArray_FLAGS(PyArrayObject* arr) - -.. c:function:: npy_intp PyArray_ITEMSIZE(PyArrayObject* arr) - - Return the itemsize for the elements of this array. - - Note that, in the old API that was deprecated in version 1.7, this function - had the return type ``int``. - -.. c:function:: int PyArray_TYPE(PyArrayObject* arr) - - Return the (builtin) typenumber for the elements of this array. - .. c:function:: PyObject *PyArray_GETITEM(PyArrayObject* arr, void* itemptr) - Get a Python object of a builtin type from the ndarray, *arr*, + Get a Python object of a builtin type from the ndarray, *arr*, at the location pointed to by itemptr. Return ``NULL`` on failure. - - `numpy.ndarray.item` is identical to PyArray_GETITEM. - -.. c:function:: int PyArray_SETITEM( \ - PyArrayObject* arr, void* itemptr, PyObject* obj) - - Convert obj and place it in the ndarray, *arr*, at the place - pointed to by itemptr. Return -1 if an error occurs or 0 on - success. - -.. c:function:: npy_intp PyArray_SIZE(PyArrayObject* arr) - - Returns the total size (in number of elements) of the array. -.. c:function:: npy_intp PyArray_Size(PyArrayObject* obj) - - Returns 0 if *obj* is not a sub-class of ndarray. Otherwise, - returns the total number of elements in the array. Safer version - of :c:func:`PyArray_SIZE` (*obj*). - -.. c:function:: npy_intp PyArray_NBYTES(PyArrayObject* arr) - - Returns the total number of bytes consumed by the array. + `numpy.ndarray.item` is identical to PyArray_GETITEM. Data access @@ -223,7 +226,7 @@ From scratch If *data* is not ``NULL``, then it is assumed to point to the memory to be used for the array and the *flags* argument is used as the - new flags for the array (except the state of :c:data:`NPY_OWNDATA`, + new flags for the array (except the state of :c:data:`NPY_ARRAY_OWNDATA`, :c:data:`NPY_ARRAY_WRITEBACKIFCOPY` and :c:data:`NPY_ARRAY_UPDATEIFCOPY` flags of the new array will be reset). @@ -913,122 +916,126 @@ enumerated array data type. For the array type checking macros the argument must be a :c:type:`PyObject *<PyObject>` that can be directly interpreted as a :c:type:`PyArrayObject *`. -.. c:function:: PyTypeNum_ISUNSIGNED(num) +.. c:function:: PyTypeNum_ISUNSIGNED(int num) -.. c:function:: PyDataType_ISUNSIGNED(descr) +.. c:function:: PyDataType_ISUNSIGNED(PyArray_Descr *descr) -.. c:function:: PyArray_ISUNSIGNED(obj) +.. c:function:: PyArray_ISUNSIGNED(PyArrayObject *obj) Type represents an unsigned integer. -.. c:function:: PyTypeNum_ISSIGNED(num) +.. c:function:: PyTypeNum_ISSIGNED(int num) -.. c:function:: PyDataType_ISSIGNED(descr) +.. c:function:: PyDataType_ISSIGNED(PyArray_Descr *descr) -.. c:function:: PyArray_ISSIGNED(obj) +.. c:function:: PyArray_ISSIGNED(PyArrayObject *obj) Type represents a signed integer. -.. c:function:: PyTypeNum_ISINTEGER(num) +.. c:function:: PyTypeNum_ISINTEGER(int num) -.. c:function:: PyDataType_ISINTEGER(descr) +.. c:function:: PyDataType_ISINTEGER(PyArray_Descr* descr) -.. c:function:: PyArray_ISINTEGER(obj) +.. c:function:: PyArray_ISINTEGER(PyArrayObject *obj) Type represents any integer. -.. c:function:: PyTypeNum_ISFLOAT(num) +.. c:function:: PyTypeNum_ISFLOAT(int num) -.. c:function:: PyDataType_ISFLOAT(descr) +.. c:function:: PyDataType_ISFLOAT(PyArray_Descr* descr) -.. c:function:: PyArray_ISFLOAT(obj) +.. c:function:: PyArray_ISFLOAT(PyArrayObject *obj) Type represents any floating point number. -.. c:function:: PyTypeNum_ISCOMPLEX(num) +.. c:function:: PyTypeNum_ISCOMPLEX(int num) -.. c:function:: PyDataType_ISCOMPLEX(descr) +.. c:function:: PyDataType_ISCOMPLEX(PyArray_Descr* descr) -.. c:function:: PyArray_ISCOMPLEX(obj) +.. c:function:: PyArray_ISCOMPLEX(PyArrayObject *obj) Type represents any complex floating point number. -.. c:function:: PyTypeNum_ISNUMBER(num) +.. c:function:: PyTypeNum_ISNUMBER(int num) -.. c:function:: PyDataType_ISNUMBER(descr) +.. c:function:: PyDataType_ISNUMBER(PyArray_Descr* descr) -.. c:function:: PyArray_ISNUMBER(obj) +.. c:function:: PyArray_ISNUMBER(PyArrayObject *obj) Type represents any integer, floating point, or complex floating point number. -.. c:function:: PyTypeNum_ISSTRING(num) +.. c:function:: PyTypeNum_ISSTRING(int num) -.. c:function:: PyDataType_ISSTRING(descr) +.. c:function:: PyDataType_ISSTRING(PyArray_Descr* descr) -.. c:function:: PyArray_ISSTRING(obj) +.. c:function:: PyArray_ISSTRING(PyArrayObject *obj) Type represents a string data type. -.. c:function:: PyTypeNum_ISPYTHON(num) +.. c:function:: PyTypeNum_ISPYTHON(int num) -.. c:function:: PyDataType_ISPYTHON(descr) +.. c:function:: PyDataType_ISPYTHON(PyArray_Descr* descr) -.. c:function:: PyArray_ISPYTHON(obj) +.. c:function:: PyArray_ISPYTHON(PyArrayObject *obj) Type represents an enumerated type corresponding to one of the standard Python scalar (bool, int, float, or complex). -.. c:function:: PyTypeNum_ISFLEXIBLE(num) +.. c:function:: PyTypeNum_ISFLEXIBLE(int num) -.. c:function:: PyDataType_ISFLEXIBLE(descr) +.. c:function:: PyDataType_ISFLEXIBLE(PyArray_Descr* descr) -.. c:function:: PyArray_ISFLEXIBLE(obj) +.. c:function:: PyArray_ISFLEXIBLE(PyArrayObject *obj) Type represents one of the flexible array types ( :c:data:`NPY_STRING`, :c:data:`NPY_UNICODE`, or :c:data:`NPY_VOID` ). -.. c:function:: PyDataType_ISUNSIZED(descr): +.. c:function:: PyDataType_ISUNSIZED(PyArray_Descr* descr): Type has no size information attached, and can be resized. Should only be called on flexible dtypes. Types that are attached to an array will always be sized, hence the array form of this macro not existing. -.. c:function:: PyTypeNum_ISUSERDEF(num) + .. versionchanged:: 1.18 -.. c:function:: PyDataType_ISUSERDEF(descr) + For structured datatypes with no fields this function now returns False. -.. c:function:: PyArray_ISUSERDEF(obj) +.. c:function:: PyTypeNum_ISUSERDEF(int num) + +.. c:function:: PyDataType_ISUSERDEF(PyArray_Descr* descr) + +.. c:function:: PyArray_ISUSERDEF(PyArrayObject *obj) Type represents a user-defined type. -.. c:function:: PyTypeNum_ISEXTENDED(num) +.. c:function:: PyTypeNum_ISEXTENDED(int num) -.. c:function:: PyDataType_ISEXTENDED(descr) +.. c:function:: PyDataType_ISEXTENDED(PyArray_Descr* descr) -.. c:function:: PyArray_ISEXTENDED(obj) +.. c:function:: PyArray_ISEXTENDED(PyArrayObject *obj) Type is either flexible or user-defined. -.. c:function:: PyTypeNum_ISOBJECT(num) +.. c:function:: PyTypeNum_ISOBJECT(int num) -.. c:function:: PyDataType_ISOBJECT(descr) +.. c:function:: PyDataType_ISOBJECT(PyArray_Descr* descr) -.. c:function:: PyArray_ISOBJECT(obj) +.. c:function:: PyArray_ISOBJECT(PyArrayObject *obj) Type represents object data type. -.. c:function:: PyTypeNum_ISBOOL(num) +.. c:function:: PyTypeNum_ISBOOL(int num) -.. c:function:: PyDataType_ISBOOL(descr) +.. c:function:: PyDataType_ISBOOL(PyArray_Descr* descr) -.. c:function:: PyArray_ISBOOL(obj) +.. c:function:: PyArray_ISBOOL(PyArrayObject *obj) Type represents Boolean data type. -.. c:function:: PyDataType_HASFIELDS(descr) +.. c:function:: PyDataType_HASFIELDS(PyArray_Descr* descr) -.. c:function:: PyArray_HASFIELDS(obj) +.. c:function:: PyArray_HASFIELDS(PyArrayObject *obj) Type has fields associated with it. @@ -1397,6 +1404,7 @@ Special functions for NPY_OBJECT Returns 0 for success, -1 for failure. +.. _array-flags: Array flags ----------- @@ -1574,9 +1582,9 @@ Flag checking ^^^^^^^^^^^^^ For all of these macros *arr* must be an instance of a (subclass of) -:c:data:`PyArray_Type`, but no checking is done. +:c:data:`PyArray_Type`. -.. c:function:: PyArray_CHKFLAGS(arr, flags) +.. c:function:: PyArray_CHKFLAGS(PyObject *arr, flags) The first parameter, arr, must be an ndarray or subclass. The parameter, *flags*, should be an integer consisting of bitwise @@ -1586,60 +1594,60 @@ For all of these macros *arr* must be an instance of a (subclass of) :c:data:`NPY_ARRAY_WRITEABLE`, :c:data:`NPY_ARRAY_WRITEBACKIFCOPY`, :c:data:`NPY_ARRAY_UPDATEIFCOPY`. -.. c:function:: PyArray_IS_C_CONTIGUOUS(arr) +.. c:function:: PyArray_IS_C_CONTIGUOUS(PyObject *arr) Evaluates true if *arr* is C-style contiguous. -.. c:function:: PyArray_IS_F_CONTIGUOUS(arr) +.. c:function:: PyArray_IS_F_CONTIGUOUS(PyObject *arr) Evaluates true if *arr* is Fortran-style contiguous. -.. c:function:: PyArray_ISFORTRAN(arr) +.. c:function:: PyArray_ISFORTRAN(PyObject *arr) Evaluates true if *arr* is Fortran-style contiguous and *not* C-style contiguous. :c:func:`PyArray_IS_F_CONTIGUOUS` is the correct way to test for Fortran-style contiguity. -.. c:function:: PyArray_ISWRITEABLE(arr) +.. c:function:: PyArray_ISWRITEABLE(PyObject *arr) Evaluates true if the data area of *arr* can be written to -.. c:function:: PyArray_ISALIGNED(arr) +.. c:function:: PyArray_ISALIGNED(PyObject *arr) Evaluates true if the data area of *arr* is properly aligned on the machine. -.. c:function:: PyArray_ISBEHAVED(arr) +.. c:function:: PyArray_ISBEHAVED(PyObject *arr) Evaluates true if the data area of *arr* is aligned and writeable and in machine byte-order according to its descriptor. -.. c:function:: PyArray_ISBEHAVED_RO(arr) +.. c:function:: PyArray_ISBEHAVED_RO(PyObject *arr) Evaluates true if the data area of *arr* is aligned and in machine byte-order. -.. c:function:: PyArray_ISCARRAY(arr) +.. c:function:: PyArray_ISCARRAY(PyObject *arr) Evaluates true if the data area of *arr* is C-style contiguous, and :c:func:`PyArray_ISBEHAVED` (*arr*) is true. -.. c:function:: PyArray_ISFARRAY(arr) +.. c:function:: PyArray_ISFARRAY(PyObject *arr) Evaluates true if the data area of *arr* is Fortran-style contiguous and :c:func:`PyArray_ISBEHAVED` (*arr*) is true. -.. c:function:: PyArray_ISCARRAY_RO(arr) +.. c:function:: PyArray_ISCARRAY_RO(PyObject *arr) Evaluates true if the data area of *arr* is C-style contiguous, aligned, and in machine byte-order. -.. c:function:: PyArray_ISFARRAY_RO(arr) +.. c:function:: PyArray_ISFARRAY_RO(PyObject *arr) Evaluates true if the data area of *arr* is Fortran-style contiguous, aligned, and in machine byte-order **.** -.. c:function:: PyArray_ISONESEGMENT(arr) +.. c:function:: PyArray_ISONESEGMENT(PyObject *arr) Evaluates true if the data area of *arr* consists of a single (C-style or Fortran-style) contiguous segment. @@ -2045,9 +2053,20 @@ Calculation .. tip:: Pass in :c:data:`NPY_MAXDIMS` for axis in order to achieve the same - effect that is obtained by passing in *axis* = :const:`None` in Python + effect that is obtained by passing in ``axis=None`` in Python (treating the array as a 1-d array). + +.. note:: + + The out argument specifies where to place the result. If out is + NULL, then the output array is created, otherwise the output is + placed in out which must be the correct size and type. A new + reference to the output array is always returned even when out + is not NULL. The caller of the routine has the responsibility + to ``Py_DECREF`` out if not NULL or a memory-leak will occur. + + .. c:function:: PyObject* PyArray_ArgMax( \ PyArrayObject* self, int axis, PyArrayObject* out) @@ -2060,18 +2079,6 @@ Calculation Equivalent to :meth:`ndarray.argmin<numpy.ndarray.argmin>` (*self*, *axis*). Return the index of the smallest element of *self* along *axis*. - - - -.. note:: - - The out argument specifies where to place the result. If out is - NULL, then the output array is created, otherwise the output is - placed in out which must be the correct size and type. A new - reference to the output array is always returned even when out - is not NULL. The caller of the routine has the responsibility - to ``DECREF`` out if not NULL or a memory-leak will occur. - .. c:function:: PyObject* PyArray_Max( \ PyArrayObject* self, int axis, PyArrayObject* out) @@ -2651,22 +2658,33 @@ cost of a slight overhead. The mode should be one of: - * NPY_NEIGHBORHOOD_ITER_ZERO_PADDING: zero padding. Outside bounds values - will be 0. - * NPY_NEIGHBORHOOD_ITER_ONE_PADDING: one padding, Outside bounds values - will be 1. - * NPY_NEIGHBORHOOD_ITER_CONSTANT_PADDING: constant padding. Outside bounds - values will be the same as the first item in fill_value. - * NPY_NEIGHBORHOOD_ITER_MIRROR_PADDING: mirror padding. Outside bounds - values will be as if the array items were mirrored. For example, for the - array [1, 2, 3, 4], x[-2] will be 2, x[-2] will be 1, x[4] will be 4, - x[5] will be 1, etc... - * NPY_NEIGHBORHOOD_ITER_CIRCULAR_PADDING: circular padding. Outside bounds - values will be as if the array was repeated. For example, for the - array [1, 2, 3, 4], x[-2] will be 3, x[-2] will be 4, x[4] will be 1, - x[5] will be 2, etc... - - If the mode is constant filling (NPY_NEIGHBORHOOD_ITER_CONSTANT_PADDING), + .. c:macro:: NPY_NEIGHBORHOOD_ITER_ZERO_PADDING + + Zero padding. Outside bounds values will be 0. + + .. c:macro:: NPY_NEIGHBORHOOD_ITER_ONE_PADDING + + One padding, Outside bounds values will be 1. + + .. c:macro:: NPY_NEIGHBORHOOD_ITER_CONSTANT_PADDING + + Constant padding. Outside bounds values will be the + same as the first item in fill_value. + + .. c:macro:: NPY_NEIGHBORHOOD_ITER_MIRROR_PADDING + + Mirror padding. Outside bounds values will be as if the + array items were mirrored. For example, for the array [1, 2, 3, 4], + x[-2] will be 2, x[-2] will be 1, x[4] will be 4, x[5] will be 1, + etc... + + .. c:macro:: NPY_NEIGHBORHOOD_ITER_CIRCULAR_PADDING + + Circular padding. Outside bounds values will be as if the array + was repeated. For example, for the array [1, 2, 3, 4], x[-2] will + be 3, x[-2] will be 4, x[4] will be 1, x[5] will be 2, etc... + + If the mode is constant filling (`NPY_NEIGHBORHOOD_ITER_CONSTANT_PADDING`), fill_value should point to an array object which holds the filling value (the first item will be the filling value if the array contains more than one item). For other cases, fill_value may be NULL. @@ -2788,10 +2806,7 @@ Array Scalars *arr* is not ``NULL`` and the first element is negative then :c:data:`NPY_INTNEG_SCALAR` is returned, otherwise :c:data:`NPY_INTPOS_SCALAR` is returned. The possible return values - are :c:data:`NPY_{kind}_SCALAR` where ``{kind}`` can be **INTPOS**, - **INTNEG**, **FLOAT**, **COMPLEX**, **BOOL**, or **OBJECT**. - :c:data:`NPY_NOSCALAR` is also an enumerated value - :c:type:`NPY_SCALARKIND` variables can take on. + are the enumerated values in :c:type:`NPY_SCALARKIND`. .. c:function:: int PyArray_CanCoerceScalar( \ char thistype, char neededtype, NPY_SCALARKIND scalar) @@ -3372,7 +3387,7 @@ Group 1 Useful to release the GIL only if *dtype* does not contain arbitrary Python objects which may need the Python interpreter - during execution of the loop. Equivalent to + during execution of the loop. .. c:function:: NPY_END_THREADS_DESCR(PyArray_Descr *dtype) @@ -3466,6 +3481,10 @@ Other constants The maximum number of dimensions allowed in arrays. +.. c:var:: NPY_MAXARGS + + The maximum number of array arguments that can be used in functions. + .. c:var:: NPY_VERSION The current version of the ndarray object (check to see if this @@ -3498,6 +3517,10 @@ Miscellaneous Macros Evaluates as True if arrays *a1* and *a2* have the same shape. +.. c:var:: a + +.. c:var:: b + .. c:macro:: PyArray_MAX(a,b) Returns the maximum of *a* and *b*. If (*a*) or (*b*) are @@ -3558,10 +3581,18 @@ Enumerated Types .. c:type:: NPY_SORTKIND - A special variable-type which can take on the values :c:data:`NPY_{KIND}` - where ``{KIND}`` is + A special variable-type which can take on different values to indicate + the sorting algorithm being used. + + .. c:var:: NPY_QUICKSORT - **QUICKSORT**, **HEAPSORT**, **MERGESORT**, **STABLESORT** + .. c:var:: NPY_HEAPSORT + + .. c:var:: NPY_MERGESORT + + .. c:var:: NPY_STABLESORT + + Used as an alias of :c:data:`NPY_MERGESORT` and vica versa. .. c:var:: NPY_NSORTS @@ -3575,11 +3606,21 @@ Enumerated Types A special variable type indicating the number of "kinds" of scalars distinguished in determining scalar-coercion rules. This - variable can take on the values :c:data:`NPY_{KIND}` where ``{KIND}`` can be + variable can take on the values: + + .. c:var:: NPY_NOSCALAR + + .. c:var:: NPY_BOOL_SCALAR + + .. c:var:: NPY_INTPOS_SCALAR + + .. c:var:: NPY_INTNEG_SCALAR + + .. c:var:: NPY_FLOAT_SCALAR + + .. c:var:: NPY_COMPLEX_SCALAR - **NOSCALAR**, **BOOL_SCALAR**, **INTPOS_SCALAR**, - **INTNEG_SCALAR**, **FLOAT_SCALAR**, **COMPLEX_SCALAR**, - **OBJECT_SCALAR** + .. c:var:: NPY_OBJECT_SCALAR .. c:var:: NPY_NSCALARKINDS diff --git a/doc/source/reference/c-api.config.rst b/doc/source/reference/c-api/config.rst index 05e6fe44d..05e6fe44d 100644 --- a/doc/source/reference/c-api.config.rst +++ b/doc/source/reference/c-api/config.rst diff --git a/doc/source/reference/c-api.coremath.rst b/doc/source/reference/c-api/coremath.rst index 7e00322f9..7e00322f9 100644 --- a/doc/source/reference/c-api.coremath.rst +++ b/doc/source/reference/c-api/coremath.rst diff --git a/doc/source/reference/c-api.deprecations.rst b/doc/source/reference/c-api/deprecations.rst index a382017a2..a382017a2 100644 --- a/doc/source/reference/c-api.deprecations.rst +++ b/doc/source/reference/c-api/deprecations.rst diff --git a/doc/source/reference/c-api.dtype.rst b/doc/source/reference/c-api/dtype.rst index 72e908861..72e908861 100644 --- a/doc/source/reference/c-api.dtype.rst +++ b/doc/source/reference/c-api/dtype.rst diff --git a/doc/source/reference/c-api.generalized-ufuncs.rst b/doc/source/reference/c-api/generalized-ufuncs.rst index b59f077ad..b59f077ad 100644 --- a/doc/source/reference/c-api.generalized-ufuncs.rst +++ b/doc/source/reference/c-api/generalized-ufuncs.rst diff --git a/doc/source/reference/c-api.rst b/doc/source/reference/c-api/index.rst index b8cbe97b2..56fe8e473 100644 --- a/doc/source/reference/c-api.rst +++ b/doc/source/reference/c-api/index.rst @@ -40,12 +40,12 @@ code. .. toctree:: :maxdepth: 2 - c-api.types-and-structures - c-api.config - c-api.dtype - c-api.array - c-api.iterator - c-api.ufunc - c-api.generalized-ufuncs - c-api.coremath - c-api.deprecations + types-and-structures + config + dtype + array + iterator + ufunc + generalized-ufuncs + coremath + deprecations diff --git a/doc/source/reference/c-api.iterator.rst b/doc/source/reference/c-api/iterator.rst index b77d029cc..b77d029cc 100644 --- a/doc/source/reference/c-api.iterator.rst +++ b/doc/source/reference/c-api/iterator.rst diff --git a/doc/source/reference/c-api.types-and-structures.rst b/doc/source/reference/c-api/types-and-structures.rst index a716b5a06..336dff211 100644 --- a/doc/source/reference/c-api.types-and-structures.rst +++ b/doc/source/reference/c-api/types-and-structures.rst @@ -1,3 +1,4 @@ + ***************************** Python Types and C-Structures ***************************** @@ -75,7 +76,8 @@ PyArray_Type and PyArrayObject these structure members should normally be accessed using the provided macros. If you need a shorter name, then you can make use of :c:type:`NPY_AO` (deprecated) which is defined to be equivalent to - :c:type:`PyArrayObject`. + :c:type:`PyArrayObject`. Direct access to the struct fields are + deprecated. Use the `PyArray_*(arr)` form instead. .. code-block:: c @@ -103,7 +105,8 @@ PyArray_Type and PyArrayObject .. c:member:: char *PyArrayObject.data - A pointer to the first element of the array. This pointer can + Accessible via :c:data:`PyArray_DATA`, this data member is a + pointer to the first element of the array. This pointer can (and normally should) be recast to the data type of the array. .. c:member:: int PyArrayObject.nd @@ -111,26 +114,29 @@ PyArray_Type and PyArrayObject An integer providing the number of dimensions for this array. When nd is 0, the array is sometimes called a rank-0 array. Such arrays have undefined dimensions and strides and - cannot be accessed. :c:data:`NPY_MAXDIMS` is the largest number of - dimensions for any array. + cannot be accessed. Macro :c:data:`PyArray_NDIM` defined in + ``ndarraytypes.h`` points to this data member. :c:data:`NPY_MAXDIMS` + is the largest number of dimensions for any array. .. c:member:: npy_intp PyArrayObject.dimensions An array of integers providing the shape in each dimension as long as nd :math:`\geq` 1. The integer is always large enough to hold a pointer on the platform, so the dimension size is - only limited by memory. + only limited by memory. :c:data:`PyArray_DIMS` is the macro + associated with this data member. .. c:member:: npy_intp *PyArrayObject.strides An array of integers providing for each dimension the number of bytes that must be skipped to get to the next element in that - dimension. + dimension. Associated with macro :c:data:`PyArray_STRIDES`. .. c:member:: PyObject *PyArrayObject.base - This member is used to hold a pointer to another Python object that - is related to this array. There are two use cases: + Pointed to by :c:data:`PyArray_BASE`, this member is used to hold a + pointer to another Python object that is related to this array. + There are two use cases: - If this array does not own its own memory, then base points to the Python object that owns it (perhaps another array object) @@ -149,11 +155,13 @@ PyArray_Type and PyArrayObject descriptor structure for each data type supported. This descriptor structure contains useful information about the type as well as a pointer to a table of function pointers to - implement specific functionality. + implement specific functionality. As the name suggests, it is + associated with the macro :c:data:`PyArray_DESCR`. .. c:member:: int PyArrayObject.flags - Flags indicating how the memory pointed to by data is to be + Pointed to by the macro :c:data:`PyArray_FLAGS`, this data member represents + the flags indicating how the memory pointed to by data is to be interpreted. Possible flags are :c:data:`NPY_ARRAY_C_CONTIGUOUS`, :c:data:`NPY_ARRAY_F_CONTIGUOUS`, :c:data:`NPY_ARRAY_OWNDATA`, :c:data:`NPY_ARRAY_ALIGNED`, :c:data:`NPY_ARRAY_WRITEABLE`, diff --git a/doc/source/reference/c-api.ufunc.rst b/doc/source/reference/c-api/ufunc.rst index 0499ccf5b..c9cc60141 100644 --- a/doc/source/reference/c-api.ufunc.rst +++ b/doc/source/reference/c-api/ufunc.rst @@ -21,7 +21,17 @@ Constants .. c:var:: PyUFunc_{VALUE} - ``{VALUE}`` can be **One** (1), **Zero** (0), or **None** (-1) + .. c:var:: PyUFunc_One + + .. c:var:: PyUFunc_Zero + + .. c:var:: PyUFunc_MinusOne + + .. c:var:: PyUFunc_ReorderableNone + + .. c:var:: PyUFunc_None + + .. c:var:: PyUFunc_IdentityValue Macros @@ -39,28 +49,6 @@ Macros Used in universal function code to re-acquire the Python GIL if it was released (because loop->obj was not true). -.. c:function:: UFUNC_CHECK_ERROR(loop) - - A macro used internally to check for errors and goto fail if - found. This macro requires a fail label in the current code - block. The *loop* variable must have at least members (obj, - errormask, and errorobj). If *loop* ->obj is nonzero, then - :c:func:`PyErr_Occurred` () is called (meaning the GIL must be held). If - *loop* ->obj is zero, then if *loop* ->errormask is nonzero, - :c:func:`PyUFunc_checkfperr` is called with arguments *loop* ->errormask - and *loop* ->errobj. If the result of this check of the IEEE - floating point registers is true then the code redirects to the - fail label which must be defined. - -.. c:function:: UFUNC_CHECK_STATUS(ret) - - Deprecated: use npy_clear_floatstatus from npy_math.h instead. - - A macro that expands to platform-dependent code. The *ret* - variable can be any integer. The :c:data:`UFUNC_FPE_{ERR}` bits are - set in *ret* according to the status of the corresponding error - flags of the floating point processor. - Functions --------- @@ -210,10 +198,10 @@ Functions to calling PyUFunc_FromFuncAndData. A copy of the string is made, so the passed in buffer can be freed. -.. c:function:: PyObject* PyUFunc_FromFuncAndDataAndSignatureAndIdentity( +.. c:function:: PyObject* PyUFunc_FromFuncAndDataAndSignatureAndIdentity( \ PyUFuncGenericFunction *func, void **data, char *types, int ntypes, \ - int nin, int nout, int identity, char *name, char *doc, int unused, char *signature, - PyObject *identity_value) + int nin, int nout, int identity, char *name, char *doc, int unused, \ + char *signature, PyObject *identity_value) This function is very similar to `PyUFunc_FromFuncAndDataAndSignature` above, but has an extra *identity_value* argument, to define an arbitrary identity diff --git a/doc/source/reference/distutils.rst b/doc/source/reference/distutils.rst index 46e5ec25e..a22db3e8e 100644 --- a/doc/source/reference/distutils.rst +++ b/doc/source/reference/distutils.rst @@ -22,38 +22,30 @@ information is available in the :ref:`distutils-user-guide`. Modules in :mod:`numpy.distutils` ================================= +.. toctree:: + :maxdepth: 2 -misc_util ---------- + distutils/misc_util -.. module:: numpy.distutils.misc_util + +.. currentmodule:: numpy.distutils .. autosummary:: :toctree: generated/ - get_numpy_include_dirs - dict_append - appendpath - allpath - dot_join - generate_config_py - get_cmd - terminal_has_colors - red_text - green_text - yellow_text - blue_text - cyan_text - cyg2win32 - all_strings - has_f_sources - has_cxx_sources - filter_sources - get_dependencies - is_local_src_dir - get_ext_source_files - get_script_files + ccompiler + cpuinfo.cpu + core.Extension + exec_command + log.set_verbosity + system_info.get_info + system_info.get_standard_file + + +Configuration class +=================== +.. currentmodule:: numpy.distutils.misc_util .. class:: Configuration(package_name=None, parent_name=None, top_path=None, package_path=None, **attrs) @@ -109,20 +101,6 @@ misc_util .. automethod:: get_info -Other modules -------------- - -.. currentmodule:: numpy.distutils - -.. autosummary:: - :toctree: generated/ - - system_info.get_info - system_info.get_standard_file - cpuinfo.cpu - log.set_verbosity - exec_command - Building Installable C libraries ================================ diff --git a/doc/source/reference/distutils/misc_util.rst b/doc/source/reference/distutils/misc_util.rst new file mode 100644 index 000000000..bbb83a5ab --- /dev/null +++ b/doc/source/reference/distutils/misc_util.rst @@ -0,0 +1,7 @@ +distutils.misc_util +=================== + +.. automodule:: numpy.distutils.misc_util + :members: + :undoc-members: + :exclude-members: Configuration diff --git a/doc/source/reference/index.rst b/doc/source/reference/index.rst index 6accb8535..6742d605a 100644 --- a/doc/source/reference/index.rst +++ b/doc/source/reference/index.rst @@ -24,7 +24,7 @@ For learning how to use NumPy, see also :ref:`user`. routines distutils distutils_guide - c-api + c-api/index internals swig diff --git a/doc/source/reference/maskedarray.baseclass.rst b/doc/source/reference/maskedarray.baseclass.rst index 204ebfe08..5bbdd0299 100644 --- a/doc/source/reference/maskedarray.baseclass.rst +++ b/doc/source/reference/maskedarray.baseclass.rst @@ -160,9 +160,9 @@ replaced with ``n`` integers which will be interpreted as an n-tuple. Item selection and manipulation ------------------------------- -For array methods that take an *axis* keyword, it defaults to `None`. -If axis is *None*, then the array is treated as a 1-D array. -Any other value for *axis* represents the dimension along which +For array methods that take an ``axis`` keyword, it defaults to None. +If axis is None, then the array is treated as a 1-D array. +Any other value for ``axis`` represents the dimension along which the operation should proceed. .. autosummary:: diff --git a/doc/source/reference/maskedarray.generic.rst b/doc/source/reference/maskedarray.generic.rst index 7375d60fb..41c3ee564 100644 --- a/doc/source/reference/maskedarray.generic.rst +++ b/doc/source/reference/maskedarray.generic.rst @@ -74,7 +74,7 @@ To create an array with the second element invalid, we would do:: To create a masked array where all values close to 1.e20 are invalid, we would do:: - >>> z = masked_values([1.0, 1.e20, 3.0, 4.0], 1.e20) + >>> z = ma.masked_values([1.0, 1.e20, 3.0, 4.0], 1.e20) For a complete discussion of creation methods for masked arrays please see section :ref:`Constructing masked arrays <maskedarray.generic.constructing>`. @@ -110,15 +110,15 @@ There are several ways to construct a masked array. >>> x = np.array([1, 2, 3]) >>> x.view(ma.MaskedArray) - masked_array(data = [1 2 3], - mask = False, - fill_value = 999999) + masked_array(data=[1, 2, 3], + mask=False, + fill_value=999999) >>> x = np.array([(1, 1.), (2, 2.)], dtype=[('a',int), ('b', float)]) >>> x.view(ma.MaskedArray) - masked_array(data = [(1, 1.0) (2, 2.0)], - mask = [(False, False) (False, False)], - fill_value = (999999, 1e+20), - dtype = [('a', '<i4'), ('b', '<f8')]) + masked_array(data=[(1, 1.0), (2, 2.0)], + mask=[(False, False), (False, False)], + fill_value=(999999, 1.e+20), + dtype=[('a', '<i8'), ('b', '<f8')]) * Yet another possibility is to use any of the following functions: @@ -195,9 +195,9 @@ index. The inverse of the mask can be calculated with the >>> x = ma.array([[1, 2], [3, 4]], mask=[[0, 1], [1, 0]]) >>> x[~x.mask] - masked_array(data = [1 4], - mask = [False False], - fill_value = 999999) + masked_array(data=[1, 4], + mask=[False, False], + fill_value=999999) Another way to retrieve the valid data is to use the :meth:`compressed` method, which returns a one-dimensional :class:`~numpy.ndarray` (or one of its @@ -223,27 +223,26 @@ as invalid is to assign the special value :attr:`masked` to them:: >>> x = ma.array([1, 2, 3]) >>> x[0] = ma.masked >>> x - masked_array(data = [-- 2 3], - mask = [ True False False], - fill_value = 999999) + masked_array(data=[--, 2, 3], + mask=[ True, False, False], + fill_value=999999) >>> y = ma.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) >>> y[(0, 1, 2), (1, 2, 0)] = ma.masked >>> y - masked_array(data = - [[1 -- 3] - [4 5 --] - [-- 8 9]], - mask = - [[False True False] - [False False True] - [ True False False]], - fill_value = 999999) + masked_array( + data=[[1, --, 3], + [4, 5, --], + [--, 8, 9]], + mask=[[False, True, False], + [False, False, True], + [ True, False, False]], + fill_value=999999) >>> z = ma.array([1, 2, 3, 4]) >>> z[:-2] = ma.masked >>> z - masked_array(data = [-- -- 3 4], - mask = [ True True False False], - fill_value = 999999) + masked_array(data=[--, --, 3, 4], + mask=[ True, True, False, False], + fill_value=999999) A second possibility is to modify the :attr:`~MaskedArray.mask` directly, @@ -263,9 +262,10 @@ mask:: >>> x = ma.array([1, 2, 3], mask=[0, 0, 1]) >>> x.mask = True >>> x - masked_array(data = [-- -- --], - mask = [ True True True], - fill_value = 999999) + masked_array(data=[--, --, --], + mask=[ True, True, True], + fill_value=999999, + dtype=int64) Finally, specific entries can be masked and/or unmasked by assigning to the mask a sequence of booleans:: @@ -273,9 +273,9 @@ mask a sequence of booleans:: >>> x = ma.array([1, 2, 3]) >>> x.mask = [0, 1, 0] >>> x - masked_array(data = [1 -- 3], - mask = [False True False], - fill_value = 999999) + masked_array(data=[1, --, 3], + mask=[False, True, False], + fill_value=999999) Unmasking an entry ~~~~~~~~~~~~~~~~~~ @@ -285,14 +285,14 @@ new valid values to them:: >>> x = ma.array([1, 2, 3], mask=[0, 0, 1]) >>> x - masked_array(data = [1 2 --], - mask = [False False True], - fill_value = 999999) + masked_array(data=[1, 2, --], + mask=[False, False, True], + fill_value=999999) >>> x[-1] = 5 >>> x - masked_array(data = [1 2 5], - mask = [False False False], - fill_value = 999999) + masked_array(data=[1, 2, 5], + mask=[False, False, False], + fill_value=999999) .. note:: Unmasking an entry by direct assignment will silently fail if the masked @@ -304,21 +304,27 @@ new valid values to them:: >>> x = ma.array([1, 2, 3], mask=[0, 0, 1], hard_mask=True) >>> x - masked_array(data = [1 2 --], - mask = [False False True], - fill_value = 999999) + masked_array(data=[1, 2, --], + mask=[False, False, True], + fill_value=999999) >>> x[-1] = 5 >>> x - masked_array(data = [1 2 --], - mask = [False False True], - fill_value = 999999) + masked_array(data=[1, 2, --], + mask=[False, False, True], + fill_value=999999) >>> x.soften_mask() + masked_array(data=[1, 2, --], + mask=[False, False, True], + fill_value=999999) >>> x[-1] = 5 >>> x - masked_array(data = [1 2 5], - mask = [False False False], - fill_value = 999999) + masked_array(data=[1, 2, 5], + mask=[False, False, False], + fill_value=999999) >>> x.harden_mask() + masked_array(data=[1, 2, 5], + mask=[False, False, False], + fill_value=999999) To unmask all masked entries of a masked array (provided the mask isn't a hard @@ -327,15 +333,14 @@ mask:: >>> x = ma.array([1, 2, 3], mask=[0, 0, 1]) >>> x - masked_array(data = [1 2 --], - mask = [False False True], - fill_value = 999999) + masked_array(data=[1, 2, --], + mask=[False, False, True], + fill_value=999999) >>> x.mask = ma.nomask >>> x - masked_array(data = [1 2 3], - mask = [False False False], - fill_value = 999999) - + masked_array(data=[1, 2, 3], + mask=[False, False, False], + fill_value=999999) Indexing and slicing @@ -353,9 +358,7 @@ the mask is ``True``):: >>> x[0] 1 >>> x[-1] - masked_array(data = --, - mask = True, - fill_value = 1e+20) + masked >>> x[-1] is ma.masked True @@ -370,10 +373,7 @@ is masked. >>> y[0] (1, 2) >>> y[-1] - masked_array(data = (3, --), - mask = (False, True), - fill_value = (999999, 999999), - dtype = [('a', '<i4'), ('b', '<i4')]) + (3, --) When accessing a slice, the output is a masked array whose @@ -385,20 +385,19 @@ required to ensure propagation of any modification of the mask to the original. >>> x = ma.array([1, 2, 3, 4, 5], mask=[0, 1, 0, 0, 1]) >>> mx = x[:3] >>> mx - masked_array(data = [1 -- 3], - mask = [False True False], - fill_value = 999999) + masked_array(data=[1, --, 3], + mask=[False, True, False], + fill_value=999999) >>> mx[1] = -1 >>> mx - masked_array(data = [1 -1 3], - mask = [False False False], - fill_value = 999999) + masked_array(data=[1, -1, 3], + mask=[False, False, False], + fill_value=999999) >>> x.mask - array([False, True, False, False, True]) + array([False, False, False, False, True]) >>> x.data array([ 1, -1, 3, 4, 5]) - Accessing a field of a masked array with structured datatype returns a :class:`MaskedArray`. @@ -421,9 +420,9 @@ ufuncs. Unary and binary functions that have a validity domain (such as constant whenever the input is masked or falls outside the validity domain:: >>> ma.log([-1, 0, 1, 2]) - masked_array(data = [-- -- 0.0 0.69314718056], - mask = [ True True False False], - fill_value = 1e+20) + masked_array(data=[--, --, 0.0, 0.6931471805599453], + mask=[ True, True, False, False], + fill_value=1e+20) Masked arrays also support standard numpy ufuncs. The output is then a masked array. The result of a unary ufunc is masked wherever the input is masked. The @@ -435,10 +434,9 @@ input fall outside the validity domain:: >>> x = ma.array([-1, 1, 0, 2, 3], mask=[0, 0, 0, 0, 1]) >>> np.log(x) - masked_array(data = [-- -- 0.0 0.69314718056 --], - mask = [ True True False False True], - fill_value = 1e+20) - + masked_array(data=[--, 0.0, --, 0.6931471805599453, --], + mask=[ True, False, True, False, True], + fill_value=1e+20) Examples @@ -454,11 +452,11 @@ of anomalies (deviations from the average):: >>> import numpy.ma as ma >>> x = [0.,1.,-9999.,3.,4.] >>> mx = ma.masked_values (x, -9999.) - >>> print mx.mean() + >>> print(mx.mean()) 2.0 - >>> print mx - mx.mean() + >>> print(mx - mx.mean()) [-2.0 -1.0 -- 1.0 2.0] - >>> print mx.anom() + >>> print(mx.anom()) [-2.0 -1.0 -- 1.0 2.0] @@ -468,7 +466,7 @@ Filling in the missing data Suppose now that we wish to print that same data, but with the missing values replaced by the average value. - >>> print mx.filled(mx.mean()) + >>> print(mx.filled(mx.mean())) [ 0. 1. 2. 3. 4.] @@ -478,10 +476,10 @@ Numerical operations Numerical operations can be easily performed without worrying about missing values, dividing by zero, square roots of negative numbers, etc.:: - >>> import numpy as np, numpy.ma as ma + >>> import numpy.ma as ma >>> x = ma.array([1., -1., 3., 4., 5., 6.], mask=[0,0,0,0,1,0]) >>> y = ma.array([1., 2., 0., 4., 5., 6.], mask=[0,0,0,0,0,1]) - >>> print np.sqrt(x/y) + >>> print(ma.sqrt(x/y)) [1.0 -- -- 1.0 -- --] Four values of the output are invalid: the first one comes from taking the @@ -492,8 +490,10 @@ the last two where the inputs were masked. Ignoring extreme values ----------------------- -Let's consider an array ``d`` of random floats between 0 and 1. We wish to +Let's consider an array ``d`` of floats between 0 and 1. We wish to compute the average of the values of ``d`` while ignoring any data outside -the range ``[0.1, 0.9]``:: +the range ``[0.2, 0.9]``:: - >>> print ma.masked_outside(d, 0.1, 0.9).mean() + >>> d = np.linspace(0, 1, 20) + >>> print(d.mean() - ma.masked_outside(d, 0.2, 0.9).mean()) + -0.05263157894736836 diff --git a/doc/source/reference/random/bit_generators/bitgenerators.rst b/doc/source/reference/random/bit_generators/bitgenerators.rst deleted file mode 100644 index 1474f7dac..000000000 --- a/doc/source/reference/random/bit_generators/bitgenerators.rst +++ /dev/null @@ -1,11 +0,0 @@ -:orphan: - -BitGenerator ------------- - -.. currentmodule:: numpy.random.bit_generator - -.. autosummary:: - :toctree: generated/ - - BitGenerator diff --git a/doc/source/reference/random/bit_generators/index.rst b/doc/source/reference/random/bit_generators/index.rst index 35d9e5d09..315657172 100644 --- a/doc/source/reference/random/bit_generators/index.rst +++ b/doc/source/reference/random/bit_generators/index.rst @@ -1,5 +1,3 @@ -.. _bit_generator: - .. currentmodule:: numpy.random Bit Generators @@ -21,7 +19,7 @@ The included BitGenerators are: and can be advanced by an arbitrary amount. See the documentation for :meth:`~.PCG64.advance`. PCG-64 has a period of :math:`2^{128}`. See the `PCG author's page`_ for more details about this class of PRNG. -* MT19937 - The standard Python BitGenerator. Adds a `~mt19937.MT19937.jumped` +* MT19937 - The standard Python BitGenerator. Adds a `MT19937.jumped` function that returns a new generator with state as-if :math:`2^{128}` draws have been made. * Philox - A counter-based generator capable of being advanced an @@ -35,14 +33,18 @@ The included BitGenerators are: .. _`Random123`: https://www.deshawresearch.com/resources_random123.html .. _`SFC author's page`: http://pracrand.sourceforge.net/RNG_engines.txt +.. autosummary:: + :toctree: generated/ + + BitGenerator + .. toctree:: - :maxdepth: 1 + :maxdepth: 1 - BitGenerator <bitgenerators> - MT19937 <mt19937> - PCG64 <pcg64> - Philox <philox> - SFC64 <sfc64> + MT19937 <mt19937> + PCG64 <pcg64> + Philox <philox> + SFC64 <sfc64> Seeding and Entropy ------------------- @@ -53,14 +55,14 @@ seed. All of the provided BitGenerators will take an arbitrary-sized non-negative integer, or a list of such integers, as a seed. BitGenerators need to take those inputs and process them into a high-quality internal state for the BitGenerator. All of the BitGenerators in numpy delegate that task to -`~SeedSequence`, which uses hashing techniques to ensure that even low-quality +`SeedSequence`, which uses hashing techniques to ensure that even low-quality seeds generate high-quality initial states. .. code-block:: python - from numpy.random import PCG64 + from numpy.random import PCG64 - bg = PCG64(12345678903141592653589793) + bg = PCG64(12345678903141592653589793) .. end_block @@ -75,14 +77,14 @@ user, which is up to you. .. code-block:: python - from numpy.random import PCG64, SeedSequence + from numpy.random import PCG64, SeedSequence - # Get the user's seed somehow, maybe through `argparse`. - # If the user did not provide a seed, it should return `None`. - seed = get_user_seed() - ss = SeedSequence(seed) - print('seed = {}'.format(ss.entropy)) - bg = PCG64(ss) + # Get the user's seed somehow, maybe through `argparse`. + # If the user did not provide a seed, it should return `None`. + seed = get_user_seed() + ss = SeedSequence(seed) + print('seed = {}'.format(ss.entropy)) + bg = PCG64(ss) .. end_block @@ -104,9 +106,6 @@ or using ``secrets.randbits(128)`` from the standard library are both convenient ways. .. autosummary:: - :toctree: generated/ + :toctree: generated/ SeedSequence - bit_generator.ISeedSequence - bit_generator.ISpawnableSeedSequence - bit_generator.SeedlessSeedSequence diff --git a/doc/source/reference/random/bit_generators/pcg64.rst b/doc/source/reference/random/bit_generators/pcg64.rst index 5881b7008..edac4620b 100644 --- a/doc/source/reference/random/bit_generators/pcg64.rst +++ b/doc/source/reference/random/bit_generators/pcg64.rst @@ -1,5 +1,5 @@ -Parallel Congruent Generator (64-bit, PCG64) --------------------------------------------- +Permuted Congruential Generator (64-bit, PCG64) +----------------------------------------------- .. currentmodule:: numpy.random diff --git a/doc/source/reference/random/c-api.rst b/doc/source/reference/random/c-api.rst new file mode 100644 index 000000000..0d60f4d9e --- /dev/null +++ b/doc/source/reference/random/c-api.rst @@ -0,0 +1,192 @@ +Cython API for random +--------------------- + +.. currentmodule:: numpy.random + +Typed versions of many of the `Generator` and `BitGenerator` methods as well as +the classes themselves can be accessed directly from Cython via + +.. code-block:: cython + + cimport numpy.random + +C API for random +---------------- + +Access to various distributions is available via Cython or C-wrapper libraries +like CFFI. All the functions accept a :c:type:`bitgen_t` as their first argument. + +.. c:type:: bitgen_t + + The :c:type:`bitgen_t` holds the current state of the BitGenerator and + pointers to functions that return standard C types while advancing the + state. + + .. code-block:: c + + struct bitgen: + void *state + npy_uint64 (*next_uint64)(void *st) nogil + uint32_t (*next_uint32)(void *st) nogil + double (*next_double)(void *st) nogil + npy_uint64 (*next_raw)(void *st) nogil + + ctypedef bitgen bitgen_t + +See :doc:`extending` for examples of using these functions. + +The functions are named with the following conventions: + +- "standard" refers to the reference values for any parameters. For instance + "standard_uniform" means a uniform distribution on the interval ``0.0`` to + ``1.0`` + +- "fill" functions will fill the provided ``out`` with ``cnt`` values. + +- The functions without "standard" in their name require additional parameters + to describe the distributions. + +- ``zig`` in the name are based on a ziggurat lookup algorithm is used instead + of calculating the ``log``, which is significantly faster. The non-ziggurat + variants are used in corner cases and for legacy compatibility. + + +.. c:function:: double random_standard_uniform(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_uniform_fill(bitgen_t* bitgen_state, npy_intp cnt, double *out) + +.. c:function:: double random_standard_exponential(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_exponential_fill(bitgen_t *bitgen_state, npy_intp cnt, double *out) + +.. c:function:: double random_standard_normal(bitgen_t* bitgen_state) + +.. c:function:: void random_standard_normal_fill(bitgen_t *bitgen_state, npy_intp count, double *out) + +.. c:function:: void random_standard_normal_fill_f(bitgen_t *bitgen_state, npy_intp count, float *out) + +.. c:function:: double random_standard_gamma(bitgen_t *bitgen_state, double shape) + +.. c:function:: float random_standard_uniform_f(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_uniform_fill_f(bitgen_t* bitgen_state, npy_intp cnt, float *out) + +.. c:function:: float random_standard_exponential_f(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_exponential_fill_f(bitgen_t *bitgen_state, npy_intp cnt, float *out) + +.. c:function:: float random_standard_normal_f(bitgen_t* bitgen_state) + +.. c:function:: float random_standard_gamma_f(bitgen_t *bitgen_state, float shape) + +.. c:function:: double random_normal(bitgen_t *bitgen_state, double loc, double scale) + +.. c:function:: double random_gamma(bitgen_t *bitgen_state, double shape, double scale) + +.. c:function:: float random_gamma_f(bitgen_t *bitgen_state, float shape, float scale) + +.. c:function:: double random_exponential(bitgen_t *bitgen_state, double scale) + +.. c:function:: double random_uniform(bitgen_t *bitgen_state, double lower, double range) + +.. c:function:: double random_beta(bitgen_t *bitgen_state, double a, double b) + +.. c:function:: double random_chisquare(bitgen_t *bitgen_state, double df) + +.. c:function:: double random_f(bitgen_t *bitgen_state, double dfnum, double dfden) + +.. c:function:: double random_standard_cauchy(bitgen_t *bitgen_state) + +.. c:function:: double random_pareto(bitgen_t *bitgen_state, double a) + +.. c:function:: double random_weibull(bitgen_t *bitgen_state, double a) + +.. c:function:: double random_power(bitgen_t *bitgen_state, double a) + +.. c:function:: double random_laplace(bitgen_t *bitgen_state, double loc, double scale) + +.. c:function:: double random_gumbel(bitgen_t *bitgen_state, double loc, double scale) + +.. c:function:: double random_logistic(bitgen_t *bitgen_state, double loc, double scale) + +.. c:function:: double random_lognormal(bitgen_t *bitgen_state, double mean, double sigma) + +.. c:function:: double random_rayleigh(bitgen_t *bitgen_state, double mode) + +.. c:function:: double random_standard_t(bitgen_t *bitgen_state, double df) + +.. c:function:: double random_noncentral_chisquare(bitgen_t *bitgen_state, double df, double nonc) +.. c:function:: double random_noncentral_f(bitgen_t *bitgen_state, double dfnum, double dfden, double nonc) +.. c:function:: double random_wald(bitgen_t *bitgen_state, double mean, double scale) + +.. c:function:: double random_vonmises(bitgen_t *bitgen_state, double mu, double kappa) + +.. c:function:: double random_triangular(bitgen_t *bitgen_state, double left, double mode, double right) + +.. c:function:: npy_int64 random_poisson(bitgen_t *bitgen_state, double lam) + +.. c:function:: npy_int64 random_negative_binomial(bitgen_t *bitgen_state, double n, double p) + +.. c:type:: binomial_t + + .. code-block:: c + + typedef struct s_binomial_t { + int has_binomial; /* !=0: following parameters initialized for binomial */ + double psave; + RAND_INT_TYPE nsave; + double r; + double q; + double fm; + RAND_INT_TYPE m; + double p1; + double xm; + double xl; + double xr; + double c; + double laml; + double lamr; + double p2; + double p3; + double p4; + } binomial_t; + + +.. c:function:: npy_int64 random_binomial(bitgen_t *bitgen_state, double p, npy_int64 n, binomial_t *binomial) + +.. c:function:: npy_int64 random_logseries(bitgen_t *bitgen_state, double p) + +.. c:function:: npy_int64 random_geometric_search(bitgen_t *bitgen_state, double p) + +.. c:function:: npy_int64 random_geometric_inversion(bitgen_t *bitgen_state, double p) + +.. c:function:: npy_int64 random_geometric(bitgen_t *bitgen_state, double p) + +.. c:function:: npy_int64 random_zipf(bitgen_t *bitgen_state, double a) + +.. c:function:: npy_int64 random_hypergeometric(bitgen_t *bitgen_state, npy_int64 good, npy_int64 bad, npy_int64 sample) + +.. c:function:: npy_uint64 random_interval(bitgen_t *bitgen_state, npy_uint64 max) + +.. c:function:: void random_multinomial(bitgen_t *bitgen_state, npy_int64 n, npy_int64 *mnix, double *pix, npy_intp d, binomial_t *binomial) + +.. c:function:: int random_multivariate_hypergeometric_count(bitgen_t *bitgen_state, npy_int64 total, size_t num_colors, npy_int64 *colors, npy_int64 nsample, size_t num_variates, npy_int64 *variates) + +.. c:function:: void random_multivariate_hypergeometric_marginals(bitgen_t *bitgen_state, npy_int64 total, size_t num_colors, npy_int64 *colors, npy_int64 nsample, size_t num_variates, npy_int64 *variates) + +Generate a single integer + +.. c:function:: npy_int64 random_positive_int64(bitgen_t *bitgen_state) + +.. c:function:: npy_int32 random_positive_int32(bitgen_t *bitgen_state) + +.. c:function:: npy_int64 random_positive_int(bitgen_t *bitgen_state) + +.. c:function:: npy_uint64 random_uint(bitgen_t *bitgen_state) + + +Generate random uint64 numbers in closed interval [off, off + rng]. + +.. c:function:: npy_uint64 random_bounded_uint64(bitgen_t *bitgen_state, npy_uint64 off, npy_uint64 rng, npy_uint64 mask, bint use_masked) + + diff --git a/doc/source/reference/random/examples/cffi.rst b/doc/source/reference/random/examples/cffi.rst new file mode 100644 index 000000000..04d52203b --- /dev/null +++ b/doc/source/reference/random/examples/cffi.rst @@ -0,0 +1,5 @@ +Extending via CFFI +------------------ + +.. literalinclude:: ../../../../../numpy/random/_examples/cffi/extending.py + :language: python diff --git a/doc/source/reference/random/examples/cython/extending.pyx b/doc/source/reference/random/examples/cython/extending.pyx new file mode 100644 index 000000000..0cfbc146f --- /dev/null +++ b/doc/source/reference/random/examples/cython/extending.pyx @@ -0,0 +1,4 @@ +extending.pyx +------------- + +.. include:: ../../../../../../numpy/random/examples/extending.pyx diff --git a/doc/source/reference/random/examples/cython/extending.pyx.rst b/doc/source/reference/random/examples/cython/extending.pyx.rst new file mode 100644 index 000000000..e2bba5aa4 --- /dev/null +++ b/doc/source/reference/random/examples/cython/extending.pyx.rst @@ -0,0 +1,5 @@ +extending.pyx +------------- + +.. literalinclude:: ../../../../../../numpy/random/_examples/cython/extending.pyx + :language: cython diff --git a/doc/source/reference/random/examples/cython/extending_distributions.pyx.rst b/doc/source/reference/random/examples/cython/extending_distributions.pyx.rst new file mode 100644 index 000000000..f64921c67 --- /dev/null +++ b/doc/source/reference/random/examples/cython/extending_distributions.pyx.rst @@ -0,0 +1,5 @@ +extending_distributions.pyx +--------------------------- + +.. literalinclude:: ../../../../../../numpy/random/_examples/cython/extending_distributions.pyx + :language: cython diff --git a/doc/source/reference/random/examples/cython/index.rst b/doc/source/reference/random/examples/cython/index.rst new file mode 100644 index 000000000..368f5fcd5 --- /dev/null +++ b/doc/source/reference/random/examples/cython/index.rst @@ -0,0 +1,11 @@ + +.. _extending_cython_example: + +Extending `numpy.random` via Cython +----------------------------------- + + +.. toctree:: + setup.py.rst + extending.pyx + extending_distributions.pyx diff --git a/doc/source/reference/random/examples/cython/setup.py.rst b/doc/source/reference/random/examples/cython/setup.py.rst new file mode 100644 index 000000000..bc7a74c59 --- /dev/null +++ b/doc/source/reference/random/examples/cython/setup.py.rst @@ -0,0 +1,5 @@ +setup.py +-------- + +.. literalinclude:: ../../../../../../numpy/random/_examples/cython/setup.py + :language: python diff --git a/doc/source/reference/random/examples/numba.rst b/doc/source/reference/random/examples/numba.rst new file mode 100644 index 000000000..b41a02568 --- /dev/null +++ b/doc/source/reference/random/examples/numba.rst @@ -0,0 +1,5 @@ +Extending via Numba +------------------- + +.. literalinclude:: ../../../../../numpy/random/_examples/numba/extending.py + :language: python diff --git a/doc/source/reference/random/examples/numba_cffi.rst b/doc/source/reference/random/examples/numba_cffi.rst new file mode 100644 index 000000000..fb2f85cce --- /dev/null +++ b/doc/source/reference/random/examples/numba_cffi.rst @@ -0,0 +1,5 @@ +Extending via Numba and CFFI +---------------------------- + +.. literalinclude:: ../../../../../numpy/random/_examples/numba/extending_distributions.py + :language: python diff --git a/doc/source/reference/random/extending.rst b/doc/source/reference/random/extending.rst new file mode 100644 index 000000000..4adb90c06 --- /dev/null +++ b/doc/source/reference/random/extending.rst @@ -0,0 +1,109 @@ +.. currentmodule:: numpy.random + +.. _extending: + +Extending +--------- +The BitGenerators have been designed to be extendable using standard tools for +high-performance Python -- numba and Cython. The `~Generator` object can also +be used with user-provided BitGenerators as long as these export a small set of +required functions. + +Numba +===== +Numba can be used with either CTypes or CFFI. The current iteration of the +BitGenerators all export a small set of functions through both interfaces. + +This example shows how numba can be used to produce gaussian samples using +a pure Python implementation which is then compiled. The random numbers are +provided by ``ctypes.next_double``. + +.. literalinclude:: ../../../../numpy/random/_examples/numba/extending.py + :language: python + :end-before: example 2 + +Both CTypes and CFFI allow the more complicated distributions to be used +directly in Numba after compiling the file distributions.c into a ``DLL`` or +``so``. An example showing the use of a more complicated distribution is in +the `examples` section below. + +.. _random_cython: + +Cython +====== + +Cython can be used to unpack the ``PyCapsule`` provided by a BitGenerator. +This example uses `PCG64` and the example from above. The usual caveats +for writing high-performance code using Cython -- removing bounds checks and +wrap around, providing array alignment information -- still apply. + +.. literalinclude:: ../../../../numpy/random/_examples/cython/extending_distributions.pyx + :language: cython + :end-before: example 2 + +The BitGenerator can also be directly accessed using the members of the basic +RNG structure. + +.. literalinclude:: ../../../../numpy/random/_examples/cython/extending_distributions.pyx + :language: cython + :start-after: example 2 + +See :ref:`extending_cython_example` for a complete working example including a +minimal setup and cython files. + +CFFI +==== + +CFFI can be used to directly access the functions in +``include/numpy/random/distributions.h``. Some "massaging" of the header +file is required: + +.. literalinclude:: ../../../../numpy/random/_examples/cffi/extending.py + :language: python + :end-before: dlopen + +Once the header is parsed by ``ffi.cdef``, the functions can be accessed +directly from the ``_generator`` shared object, using the `BitGenerator.cffi` interface. + +.. literalinclude:: ../../../../numpy/random/_examples/cffi/extending.py + :language: python + :start-after: dlopen + + +New Basic RNGs +============== +`~Generator` can be used with other user-provided BitGenerators. The simplest +way to write a new BitGenerator is to examine the pyx file of one of the +existing BitGenerators. The key structure that must be provided is the +``capsule`` which contains a ``PyCapsule`` to a struct pointer of type +``bitgen_t``, + +.. code-block:: c + + typedef struct bitgen { + void *state; + uint64_t (*next_uint64)(void *st); + uint32_t (*next_uint32)(void *st); + double (*next_double)(void *st); + uint64_t (*next_raw)(void *st); + } bitgen_t; + +which provides 5 pointers. The first is an opaque pointer to the data structure +used by the BitGenerators. The next three are function pointers which return +the next 64- and 32-bit unsigned integers, the next random double and the next +raw value. This final function is used for testing and so can be set to +the next 64-bit unsigned integer function if not needed. Functions inside +``Generator`` use this structure as in + +.. code-block:: c + + bitgen_state->next_uint64(bitgen_state->state) + +Examples +======== + +.. toctree:: + Numba <examples/numba> + CFFI + Numba <examples/numba_cffi> + Cython <examples/cython/index> + CFFI <examples/cffi> diff --git a/doc/source/reference/random/generator.rst b/doc/source/reference/random/generator.rst index c3803bcab..a2cbb493a 100644 --- a/doc/source/reference/random/generator.rst +++ b/doc/source/reference/random/generator.rst @@ -22,63 +22,64 @@ Accessing the BitGenerator .. autosummary:: :toctree: generated/ - ~Generator.bit_generator + ~numpy.random.Generator.bit_generator Simple random data ================== .. autosummary:: :toctree: generated/ - ~Generator.integers - ~Generator.random - ~Generator.choice - ~Generator.bytes + ~numpy.random.Generator.integers + ~numpy.random.Generator.random + ~numpy.random.Generator.choice + ~numpy.random.Generator.bytes Permutations ============ .. autosummary:: :toctree: generated/ - ~Generator.shuffle - ~Generator.permutation + ~numpy.random.Generator.shuffle + ~numpy.random.Generator.permutation Distributions ============= .. autosummary:: :toctree: generated/ - ~Generator.beta - ~Generator.binomial - ~Generator.chisquare - ~Generator.dirichlet - ~Generator.exponential - ~Generator.f - ~Generator.gamma - ~Generator.geometric - ~Generator.gumbel - ~Generator.hypergeometric - ~Generator.laplace - ~Generator.logistic - ~Generator.lognormal - ~Generator.logseries - ~Generator.multinomial - ~Generator.multivariate_normal - ~Generator.negative_binomial - ~Generator.noncentral_chisquare - ~Generator.noncentral_f - ~Generator.normal - ~Generator.pareto - ~Generator.poisson - ~Generator.power - ~Generator.rayleigh - ~Generator.standard_cauchy - ~Generator.standard_exponential - ~Generator.standard_gamma - ~Generator.standard_normal - ~Generator.standard_t - ~Generator.triangular - ~Generator.uniform - ~Generator.vonmises - ~Generator.wald - ~Generator.weibull - ~Generator.zipf + ~numpy.random.Generator.beta + ~numpy.random.Generator.binomial + ~numpy.random.Generator.chisquare + ~numpy.random.Generator.dirichlet + ~numpy.random.Generator.exponential + ~numpy.random.Generator.f + ~numpy.random.Generator.gamma + ~numpy.random.Generator.geometric + ~numpy.random.Generator.gumbel + ~numpy.random.Generator.hypergeometric + ~numpy.random.Generator.laplace + ~numpy.random.Generator.logistic + ~numpy.random.Generator.lognormal + ~numpy.random.Generator.logseries + ~numpy.random.Generator.multinomial + ~numpy.random.Generator.multivariate_hypergeometric + ~numpy.random.Generator.multivariate_normal + ~numpy.random.Generator.negative_binomial + ~numpy.random.Generator.noncentral_chisquare + ~numpy.random.Generator.noncentral_f + ~numpy.random.Generator.normal + ~numpy.random.Generator.pareto + ~numpy.random.Generator.poisson + ~numpy.random.Generator.power + ~numpy.random.Generator.rayleigh + ~numpy.random.Generator.standard_cauchy + ~numpy.random.Generator.standard_exponential + ~numpy.random.Generator.standard_gamma + ~numpy.random.Generator.standard_normal + ~numpy.random.Generator.standard_t + ~numpy.random.Generator.triangular + ~numpy.random.Generator.uniform + ~numpy.random.Generator.vonmises + ~numpy.random.Generator.wald + ~numpy.random.Generator.weibull + ~numpy.random.Generator.zipf diff --git a/doc/source/reference/random/index.rst b/doc/source/reference/random/index.rst index 641c2164a..bda9c4d96 100644 --- a/doc/source/reference/random/index.rst +++ b/doc/source/reference/random/index.rst @@ -22,34 +22,44 @@ Since Numpy version 1.17.0 the Generator can be initialized with a number of different BitGenerators. It exposes many different probability distributions. See `NEP 19 <https://www.numpy.org/neps/ nep-0019-rng-policy.html>`_ for context on the updated random Numpy number -routines. The legacy `.RandomState` random number routines are still +routines. The legacy `RandomState` random number routines are still available, but limited to a single BitGenerator. -For convenience and backward compatibility, a single `~.RandomState` +For convenience and backward compatibility, a single `RandomState` instance's methods are imported into the numpy.random namespace, see :ref:`legacy` for the complete list. +.. _random-quick-start: + Quick Start ----------- -By default, `~Generator` uses bits provided by `~pcg64.PCG64` which -has better statistical properties than the legacy mt19937 random -number generator in `~.RandomState`. +Call `default_rng` to get a new instance of a `Generator`, then call its +methods to obtain samples from different distributions. By default, +`Generator` uses bits provided by `PCG64` which has better statistical +properties than the legacy `MT19937` used in `RandomState`. .. code-block:: python - # Uses the old numpy.random.RandomState + # Do this + from numpy.random import default_rng + rng = default_rng() + vals = rng.standard_normal(10) + more_vals = rng.standard_normal(10) + + # instead of this from numpy import random - random.standard_normal() + vals = random.standard_normal(10) + more_vals = random.standard_normal(10) -`~Generator` can be used as a replacement for `~.RandomState`. Both class -instances now hold a internal `BitGenerator` instance to provide the bit +`Generator` can be used as a replacement for `RandomState`. Both class +instances hold a internal `BitGenerator` instance to provide the bit stream, it is accessible as ``gen.bit_generator``. Some long-overdue API cleanup means that legacy and compatibility methods have been removed from -`~.Generator` +`Generator` =================== ============== ============ -`~.RandomState` `~.Generator` Notes +`RandomState` `Generator` Notes ------------------- -------------- ------------ ``random_sample``, ``random`` Compatible with `random.random` ``rand`` @@ -57,22 +67,13 @@ cleanup means that legacy and compatibility methods have been removed from ``randint``, ``integers`` Add an ``endpoint`` kwarg ``random_integers`` ------------------- -------------- ------------ -``tomaxint`` removed Use ``integers(0, np.iinfo(np.int).max,`` - ``endpoint=False)`` +``tomaxint`` removed Use ``integers(0, np.iinfo(np.int_).max,`` + ``endpoint=False)`` ------------------- -------------- ------------ -``seed`` removed Use `~.SeedSequence.spawn` +``seed`` removed Use `SeedSequence.spawn` =================== ============== ============ -See `new-or-different` for more information - -.. code-block:: python - - # As replacement for RandomState(); default_rng() instantiates Generator with - # the default PCG64 BitGenerator. - from numpy.random import default_rng - rg = default_rng() - rg.standard_normal() - rg.bit_generator +See :ref:`new-or-different` for more information Something like the following code can be used to support both ``RandomState`` and ``Generator``, with the understanding that the interfaces are slightly @@ -87,9 +88,9 @@ different a = rg_integers(1000) Seeds can be passed to any of the BitGenerators. The provided value is mixed -via `~.SeedSequence` to spread a possible sequence of seeds across a wider -range of initialization states for the BitGenerator. Here `~.PCG64` is used and -is wrapped with a `~.Generator`. +via `SeedSequence` to spread a possible sequence of seeds across a wider +range of initialization states for the BitGenerator. Here `PCG64` is used and +is wrapped with a `Generator`. .. code-block:: python @@ -100,7 +101,7 @@ is wrapped with a `~.Generator`. Introduction ------------ The new infrastructure takes a different approach to producing random numbers -from the `~.RandomState` object. Random number generation is separated into +from the `RandomState` object. Random number generation is separated into two components, a bit generator and a random generator. The `BitGenerator` has a limited set of responsibilities. It manages state @@ -113,8 +114,8 @@ distributions, e.g., simulated normal random values. This structure allows alternative bit generators to be used with little code duplication. The `Generator` is the user-facing object that is nearly identical to -`.RandomState`. The canonical method to initialize a generator passes a -`~.PCG64` bit generator as the sole argument. +`RandomState`. The canonical method to initialize a generator passes a +`PCG64` bit generator as the sole argument. .. code-block:: python @@ -123,7 +124,7 @@ The `Generator` is the user-facing object that is nearly identical to rg.random() One can also instantiate `Generator` directly with a `BitGenerator` instance. -To use the older `~mt19937.MT19937` algorithm, one can instantiate it directly +To use the older `MT19937` algorithm, one can instantiate it directly and pass it to `Generator`. .. code-block:: python @@ -139,9 +140,9 @@ What's New or Different The Box-Muller method used to produce NumPy's normals is no longer available in `Generator`. It is not possible to reproduce the exact random values using Generator for the normal distribution or any other - distribution that relies on the normal such as the `.RandomState.gamma` or - `.RandomState.standard_t`. If you require bitwise backward compatible - streams, use `.RandomState`. + distribution that relies on the normal such as the `RandomState.gamma` or + `RandomState.standard_t`. If you require bitwise backward compatible + streams, use `RandomState`. * The Generator's normal, exponential and gamma functions use 256-step Ziggurat methods which are 2-10 times faster than NumPy's Box-Muller or inverse CDF @@ -152,16 +153,20 @@ What's New or Different * Optional ``out`` argument that allows existing arrays to be filled for select distributions * All BitGenerators can produce doubles, uint64s and uint32s via CTypes -* `~.Generator.integers` is now the canonical way to generate integer + (`PCG64.ctypes`) and CFFI (`PCG64.cffi`). This allows the bit generators + to be used in numba. +* The bit generators can be used in downstream projects via + :ref:`Cython <random_cython>`. +* `Generator.integers` is now the canonical way to generate integer random numbers from a discrete uniform distribution. The ``rand`` and - ``randn`` methods are only available through the legacy `~.RandomState`. + ``randn`` methods are only available through the legacy `RandomState`. The ``endpoint`` keyword can be used to specify open or closed intervals. This replaces both ``randint`` and the deprecated ``random_integers``. -* `~.Generator.random` is now the canonical way to generate floating-point - random numbers, which replaces `.RandomState.random_sample`, - `.RandomState.sample`, and `.RandomState.ranf`. This is consistent with +* `Generator.random` is now the canonical way to generate floating-point + random numbers, which replaces `RandomState.random_sample`, + `RandomState.sample`, and `RandomState.ranf`. This is consistent with Python's `random.random`. -* All BitGenerators in numpy use `~SeedSequence` to convert seeds into +* All BitGenerators in numpy use `SeedSequence` to convert seeds into initialized states. See :ref:`new-or-different` for a complete list of improvements and @@ -195,9 +200,12 @@ Features Multithreaded Generation <multithreading> new-or-different Comparing Performance <performance> + c-api + Examples of using Numba, Cython, CFFI <extending> -Original Source -~~~~~~~~~~~~~~~ +Original Source of the Generator and BitGenerators +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This package was developed independently of NumPy and was integrated in version 1.17.0. The original repo is at https://github.com/bashtage/randomgen. + diff --git a/doc/source/reference/random/legacy.rst b/doc/source/reference/random/legacy.rst index 413a42727..922d76a9a 100644 --- a/doc/source/reference/random/legacy.rst +++ b/doc/source/reference/random/legacy.rst @@ -121,3 +121,71 @@ Distributions ~RandomState.wald ~RandomState.weibull ~RandomState.zipf + +Functions in `numpy.random` +=========================== +Many of the RandomState methods above are exported as functions in +`numpy.random` This usage is discouraged, as it is implemented via a gloabl +`RandomState` instance which is not advised on two counts: + +- It uses global state, which means results will change as the code changes + +- It uses a `RandomState` rather than the more modern `Generator`. + +For backward compatible legacy reasons, we cannot change this. See +`random-quick-start`. + +.. autosummary:: + :toctree: generated/ + + beta + binomial + bytes + chisquare + choice + dirichlet + exponential + f + gamma + geometric + get_state + gumbel + hypergeometric + laplace + logistic + lognormal + logseries + multinomial + multivariate_normal + negative_binomial + noncentral_chisquare + noncentral_f + normal + pareto + permutation + poisson + power + rand + randint + randn + random + random_integers + random_sample + ranf + rayleigh + sample + seed + set_state + shuffle + standard_cauchy + standard_exponential + standard_gamma + standard_normal + standard_t + triangular + uniform + vonmises + wald + weibull + zipf + diff --git a/doc/source/reference/random/multithreading.rst b/doc/source/reference/random/multithreading.rst index 6883d3672..a0a31d0ea 100644 --- a/doc/source/reference/random/multithreading.rst +++ b/doc/source/reference/random/multithreading.rst @@ -41,7 +41,7 @@ seed will produce the same outputs. self.n = n self.executor = concurrent.futures.ThreadPoolExecutor(threads) self.values = np.empty(n) - self.step = np.ceil(n / threads).astype(np.int) + self.step = np.ceil(n / threads).astype(np.int_) def fill(self): def _fill(random_state, out, first, last): diff --git a/doc/source/reference/random/new-or-different.rst b/doc/source/reference/random/new-or-different.rst index 402f24ab0..b3bddb443 100644 --- a/doc/source/reference/random/new-or-different.rst +++ b/doc/source/reference/random/new-or-different.rst @@ -10,9 +10,10 @@ What's New or Different The Box-Muller method used to produce NumPy's normals is no longer available in `Generator`. It is not possible to reproduce the exact random values using ``Generator`` for the normal distribution or any other - distribution that relies on the normal such as the `gamma` or - `standard_t`. If you require bitwise backward compatible - streams, use `RandomState`. + distribution that relies on the normal such as the `Generator.gamma` or + `Generator.standard_t`. If you require bitwise backward compatible + streams, use `RandomState`, i.e., `RandomState.gamma` or + `RandomState.standard_t`. Quick comparison of legacy `mtrand <legacy>`_ to the new `Generator` @@ -20,9 +21,9 @@ Quick comparison of legacy `mtrand <legacy>`_ to the new `Generator` Feature Older Equivalent Notes ------------------ -------------------- ------------- `~.Generator` `~.RandomState` ``Generator`` requires a stream - source, called a `BitGenerator - <bit_generators>` A number of these - are provided. ``RandomState`` uses + source, called a `BitGenerator` + A number of these are provided. + ``RandomState`` uses the Mersenne Twister `~.MT19937` by default, but can also be instantiated with any BitGenerator. @@ -56,12 +57,13 @@ And in more detail: ``randn`` methods are only available through the legacy `~.RandomState`. This replaces both ``randint`` and the deprecated ``random_integers``. * The Box-Muller method used to produce NumPy's normals is no longer available. -* All bit generators can produce doubles, uint64s and uint32s via CTypes - (`~PCG64.ctypes`) and CFFI (`~PCG64.cffi`). This allows these bit generators - to be used in numba. +* All bit generators can produce doubles, uint64s and + uint32s via CTypes (`~PCG64.ctypes`) and CFFI (`~PCG64.cffi`). + This allows these bit generators to be used in numba. * The bit generators can be used in downstream projects via Cython. + .. ipython:: python from numpy.random import Generator, PCG64 diff --git a/doc/source/reference/random/parallel.rst b/doc/source/reference/random/parallel.rst index 2f79f22d8..721584014 100644 --- a/doc/source/reference/random/parallel.rst +++ b/doc/source/reference/random/parallel.rst @@ -18,10 +18,10 @@ a `~BitGenerator`. It uses hashing techniques to ensure that low-quality seeds are turned into high quality initial states (at least, with very high probability). -For example, `~mt19937.MT19937` has a state consisting of 624 +For example, `MT19937` has a state consisting of 624 `uint32` integers. A naive way to take a 32-bit integer seed would be to just set the last element of the state to the 32-bit seed and leave the rest 0s. This is -a valid state for `~mt19937.MT19937`, but not a good one. The Mersenne Twister +a valid state for `MT19937`, but not a good one. The Mersenne Twister algorithm `suffers if there are too many 0s`_. Similarly, two adjacent 32-bit integer seeds (i.e. ``12345`` and ``12346``) would produce very similar streams. @@ -91,15 +91,15 @@ territory ([2]_). .. [2] In this calculation, we can ignore the amount of numbers drawn from each stream. Each of the PRNGs we provide has some extra protection built in that avoids overlaps if the `~SeedSequence` pools differ in the - slightest bit. `~pcg64.PCG64` has :math:`2^{127}` separate cycles + slightest bit. `PCG64` has :math:`2^{127}` separate cycles determined by the seed in addition to the position in the :math:`2^{128}` long period for each cycle, so one has to both get on or near the same cycle *and* seed a nearby position in the cycle. - `~philox.Philox` has completely independent cycles determined by the seed. - `~sfc64.SFC64` incorporates a 64-bit counter so every unique seed is at + `Philox` has completely independent cycles determined by the seed. + `SFC64` incorporates a 64-bit counter so every unique seed is at least :math:`2^{64}` iterations away from any other seed. And - finally, `~mt19937.MT19937` has just an unimaginably huge period. Getting - a collision internal to `~SeedSequence` is the way a failure would be + finally, `MT19937` has just an unimaginably huge period. Getting + a collision internal to `SeedSequence` is the way a failure would be observed. .. _`implements an algorithm`: http://www.pcg-random.org/posts/developing-a-seed_seq-alternative.html @@ -113,10 +113,10 @@ territory ([2]_). Independent Streams ------------------- -:class:`~philox.Philox` is a counter-based RNG based which generates values by +`Philox` is a counter-based RNG based which generates values by encrypting an incrementing counter using weak cryptographic primitives. The seed determines the key that is used for the encryption. Unique keys create -unique, independent streams. :class:`~philox.Philox` lets you bypass the +unique, independent streams. `Philox` lets you bypass the seeding algorithm to directly set the 128-bit key. Similar, but different, keys will still create independent streams. diff --git a/doc/source/reference/random/performance.rst b/doc/source/reference/random/performance.rst index 2d5fca496..d70dd064a 100644 --- a/doc/source/reference/random/performance.rst +++ b/doc/source/reference/random/performance.rst @@ -5,21 +5,21 @@ Performance Recommendation ************** -The recommended generator for general use is :class:`~pcg64.PCG64`. It is +The recommended generator for general use is `PCG64`. It is statistically high quality, full-featured, and fast on most platforms, but somewhat slow when compiled for 32-bit processes. -:class:`~philox.Philox` is fairly slow, but its statistical properties have +`Philox` is fairly slow, but its statistical properties have very high quality, and it is easy to get assuredly-independent stream by using unique keys. If that is the style you wish to use for parallel streams, or you are porting from another system that uses that style, then -:class:`~philox.Philox` is your choice. +`Philox` is your choice. -:class:`~sfc64.SFC64` is statistically high quality and very fast. However, it +`SFC64` is statistically high quality and very fast. However, it lacks jumpability. If you are not using that capability and want lots of speed, even on 32-bit processes, this is your choice. -:class:`~mt19937.MT19937` `fails some statistical tests`_ and is not especially +`MT19937` `fails some statistical tests`_ and is not especially fast compared to modern PRNGs. For these reasons, we mostly do not recommend using it on its own, only through the legacy `~.RandomState` for reproducing old results. That said, it has a very long history as a default in @@ -31,20 +31,20 @@ Timings ******* The timings below are the time in ns to produce 1 random value from a -specific distribution. The original :class:`~mt19937.MT19937` generator is +specific distribution. The original `MT19937` generator is much slower since it requires 2 32-bit values to equal the output of the faster generators. Integer performance has a similar ordering. The pattern is similar for other, more complex generators. The normal -performance of the legacy :class:`~.RandomState` generator is much +performance of the legacy `RandomState` generator is much lower than the other since it uses the Box-Muller transformation rather than the Ziggurat generator. The performance gap for Exponentials is also large due to the cost of computing the log function to invert the CDF. The column labeled MT19973 is used the same 32-bit generator as -:class:`~.RandomState` but produces random values using -:class:`~Generator`. +`RandomState` but produces random values using +`Generator`. .. csv-table:: :header: ,MT19937,PCG64,Philox,SFC64,RandomState @@ -61,7 +61,7 @@ The column labeled MT19973 is used the same 32-bit generator as Poissons,67.6,52.4,69.2,46.4,78.1 The next table presents the performance in percentage relative to values -generated by the legacy generator, `RandomState(MT19937())`. The overall +generated by the legacy generator, ``RandomState(MT19937())``. The overall performance was computed using a geometric mean. .. csv-table:: diff --git a/doc/source/reference/routines.array-manipulation.rst b/doc/source/reference/routines.array-manipulation.rst index cc93d1029..bf43232ef 100644 --- a/doc/source/reference/routines.array-manipulation.rst +++ b/doc/source/reference/routines.array-manipulation.rst @@ -9,6 +9,7 @@ Basic operations :toctree: generated/ copyto + shape Changing array shape ==================== diff --git a/doc/source/reference/routines.char.rst b/doc/source/reference/routines.char.rst index 513f975e7..ed8393855 100644 --- a/doc/source/reference/routines.char.rst +++ b/doc/source/reference/routines.char.rst @@ -58,6 +58,7 @@ comparison. less_equal greater less + compare_chararrays String information ------------------ diff --git a/doc/source/reference/routines.ma.rst b/doc/source/reference/routines.ma.rst index 491bb6bff..5b2098c7a 100644 --- a/doc/source/reference/routines.ma.rst +++ b/doc/source/reference/routines.ma.rst @@ -264,17 +264,6 @@ Conversion operations ma.MaskedArray.tobytes -Pickling and unpickling -~~~~~~~~~~~~~~~~~~~~~~~ -.. autosummary:: - :toctree: generated/ - - ma.dump - ma.dumps - ma.load - ma.loads - - Filling a masked array ~~~~~~~~~~~~~~~~~~~~~~ .. autosummary:: diff --git a/doc/source/reference/routines.other.rst b/doc/source/reference/routines.other.rst index 0a3677904..28c9a1ad1 100644 --- a/doc/source/reference/routines.other.rst +++ b/doc/source/reference/routines.other.rst @@ -21,6 +21,7 @@ Memory ranges shares_memory may_share_memory + byte_bounds Array mixins ------------ @@ -35,3 +36,21 @@ NumPy version comparison :toctree: generated/ lib.NumpyVersion + +Utility +------- + +.. autosummary:: + :toctree: generated/ + + get_include + deprecate + deprecate_with_doc + +Matlab-like Functions +--------------------- +.. autosummary:: + :toctree: generated/ + + who + disp
\ No newline at end of file diff --git a/doc/source/reference/routines.testing.rst b/doc/source/reference/routines.testing.rst index c676dec07..98ce3f377 100644 --- a/doc/source/reference/routines.testing.rst +++ b/doc/source/reference/routines.testing.rst @@ -37,11 +37,11 @@ Decorators .. autosummary:: :toctree: generated/ - decorators.deprecated - decorators.knownfailureif - decorators.setastest - decorators.skipif - decorators.slow + dec.deprecated + dec.knownfailureif + dec.setastest + dec.skipif + dec.slow decorate_methods Test Running diff --git a/doc/source/reference/ufuncs.rst b/doc/source/reference/ufuncs.rst index d00e88b34..361cf11b9 100644 --- a/doc/source/reference/ufuncs.rst +++ b/doc/source/reference/ufuncs.rst @@ -100,7 +100,7 @@ is true: - *d* acts like a (5,6) array where the single value is repeated. -.. _ufuncs.output-type: +.. _ufuncs-output-type: Output type determination ========================= @@ -228,46 +228,47 @@ can generate this table for your system with the code given in the Figure. .. admonition:: Figure - Code segment showing the "can cast safely" table for a 32-bit system. + Code segment showing the "can cast safely" table for a 64-bit system. + Generally the output depends on the system; your system might result in + a different table. + >>> mark = {False: ' -', True: ' Y'} >>> def print_table(ntypes): - ... print 'X', - ... for char in ntypes: print char, - ... print + ... print('X ' + ' '.join(ntypes)) ... for row in ntypes: - ... print row, + ... print(row, end='') ... for col in ntypes: - ... print int(np.can_cast(row, col)), - ... print + ... print(mark[np.can_cast(row, col)], end='') + ... print() + ... >>> print_table(np.typecodes['All']) X ? b h i l q p B H I L Q P e f d g F D G S U V O M m - ? 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 - b 0 1 1 1 1 1 1 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 - h 0 0 1 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 - i 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 0 0 - l 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 0 0 - q 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 0 0 - p 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 0 0 - B 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 - H 0 0 0 1 1 1 1 0 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 0 0 - I 0 0 0 0 1 1 1 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 1 1 0 0 - L 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 1 1 0 1 1 1 1 1 1 0 0 - Q 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 1 1 0 1 1 1 1 1 1 0 0 - P 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 1 1 0 1 1 1 1 1 1 0 0 - e 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 - f 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 - d 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 0 0 - g 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 1 1 1 1 0 0 - F 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 - D 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 - G 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 - S 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 - U 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 - V 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 - O 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 - M 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 - m 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 - + ? Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y - Y + b - Y Y Y Y Y Y - - - - - - Y Y Y Y Y Y Y Y Y Y Y - Y + h - - Y Y Y Y Y - - - - - - - Y Y Y Y Y Y Y Y Y Y - Y + i - - - Y Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y + l - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y + q - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y + p - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y + B - - Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y - Y + H - - - Y Y Y Y - Y Y Y Y Y - Y Y Y Y Y Y Y Y Y Y - Y + I - - - - Y Y Y - - Y Y Y Y - - Y Y - Y Y Y Y Y Y - Y + L - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - + Q - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - + P - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - + e - - - - - - - - - - - - - Y Y Y Y Y Y Y Y Y Y Y - - + f - - - - - - - - - - - - - - Y Y Y Y Y Y Y Y Y Y - - + d - - - - - - - - - - - - - - - Y Y - Y Y Y Y Y Y - - + g - - - - - - - - - - - - - - - - Y - - Y Y Y Y Y - - + F - - - - - - - - - - - - - - - - - Y Y Y Y Y Y Y - - + D - - - - - - - - - - - - - - - - - - Y Y Y Y Y Y - - + G - - - - - - - - - - - - - - - - - - - Y Y Y Y Y - - + S - - - - - - - - - - - - - - - - - - - - Y Y Y Y - - + U - - - - - - - - - - - - - - - - - - - - - Y Y Y - - + V - - - - - - - - - - - - - - - - - - - - - - Y Y - - + O - - - - - - - - - - - - - - - - - - - - - - Y Y - - + M - - - - - - - - - - - - - - - - - - - - - - Y Y Y - + m - - - - - - - - - - - - - - - - - - - - - - Y Y - Y You should note that, while included in the table for completeness, the 'S', 'U', and 'V' types cannot be operated on by ufuncs. Also, @@ -319,7 +320,7 @@ advanced usage and will not typically be used. .. versionadded:: 1.10 The 'out' keyword argument is expected to be a tuple with one entry per - output (which can be `None` for arrays to be allocated by the ufunc). + output (which can be None for arrays to be allocated by the ufunc). For ufuncs with a single output, passing a single array (instead of a tuple holding a single array) is also valid. @@ -493,7 +494,7 @@ keyword, and an *out* keyword, and the arrays must all have dimension >= 1. The *axis* keyword specifies the axis of the array over which the reduction will take place (with negative values counting backwards). Generally, it is an integer, though for :meth:`ufunc.reduce`, it can also be a tuple of `int` to -reduce over several axes at once, or `None`, to reduce over all axes. +reduce over several axes at once, or None, to reduce over all axes. The *dtype* keyword allows you to manage a very common problem that arises when naively using :meth:`ufunc.reduce`. Sometimes you may have an array of a certain data type and wish to add up all of its |