summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorDongHun Kwak <dh0128.kwak@samsung.com>2020-12-31 09:39:50 +0900
committerDongHun Kwak <dh0128.kwak@samsung.com>2020-12-31 09:39:50 +0900
commit0ec8d92146702c23a3edd4113a0c37d746fc0c26 (patch)
tree3ae0f51bf05b2ccb995af7690a86df20f9e8bc7f
parent92fba4b9b454bc82b27770074a248dd685053832 (diff)
downloadpython-numpy-0ec8d92146702c23a3edd4113a0c37d746fc0c26.tar.gz
python-numpy-0ec8d92146702c23a3edd4113a0c37d746fc0c26.tar.bz2
python-numpy-0ec8d92146702c23a3edd4113a0c37d746fc0c26.zip
Imported Upstream version 1.18.1upstream/1.18.1
-rw-r--r--.travis.yml13
-rw-r--r--INSTALL.rst.txt55
-rw-r--r--azure-pipelines.yml22
-rw-r--r--azure-steps-windows.yml2
-rw-r--r--doc/HOWTO_RELEASE.rst.txt14
-rw-r--r--doc/changelog/1.16.6-changelog.rst36
-rw-r--r--doc/changelog/1.17.5-changelog.rst26
-rw-r--r--doc/changelog/1.18.1-changelog.rst33
-rw-r--r--doc/source/release.rst3
-rw-r--r--doc/source/release/1.16.6-notes.rst87
-rw-r--r--doc/source/release/1.17.5-notes.rst45
-rw-r--r--doc/source/release/1.18.0-notes.rst17
-rw-r--r--doc/source/release/1.18.1-notes.rst52
-rw-r--r--doc/source/user/building.rst68
-rw-r--r--numpy/core/tests/test_multiarray.py33
-rw-r--r--numpy/distutils/ccompiler.py5
-rw-r--r--numpy/distutils/system_info.py9
-rw-r--r--numpy/lib/function_base.py17
-rw-r--r--numpy/lib/tests/test_function_base.py34
-rw-r--r--numpy/linalg/tests/test_linalg.py28
-rw-r--r--numpy/random/_examples/cython/setup.py10
-rw-r--r--numpy/random/tests/test_extending.py38
-rw-r--r--numpy/testing/tests/test_utils.py15
-rw-r--r--pavement.py62
-rw-r--r--pyproject.toml2
-rwxr-xr-xsetup.py44
-rw-r--r--shippable.yml15
-rwxr-xr-xtools/cythonize.py4
-rw-r--r--tools/openblas_support.py111
-rwxr-xr-xtools/pypy-test.sh15
-rwxr-xr-xtools/travis-before-install.sh6
31 files changed, 708 insertions, 213 deletions
diff --git a/.travis.yml b/.travis.yml
index 64b7be051..65e55eb6d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -76,7 +76,17 @@ matrix:
- USE_ASV=1
- python: 3.7
- env: NPY_RELAXED_STRIDES_CHECKING=0
+ env:
+ - NPY_RELAXED_STRIDES_CHECKING=0
+ # use custom symbol-suffixed openblas build, not system ATLAS
+ - DOWNLOAD_OPENBLAS=1
+ - CHECK_BLAS=1
+ - NPY_USE_BLAS_ILP64=1
+ addons:
+ apt:
+ packages:
+ - gfortran
+ - eatmydata
- python: 3.7
env: USE_WHEEL=1 NPY_RELAXED_STRIDES_DEBUG=1
@@ -95,6 +105,7 @@ matrix:
arch: ppc64le
env:
# use ppc64le OpenBLAS build, not system ATLAS
+ - DOWNLOAD_OPENBLAS=1
- ATLAS=None
- python: 3.7
diff --git a/INSTALL.rst.txt b/INSTALL.rst.txt
index d6c42bad7..50cd7ab1f 100644
--- a/INSTALL.rst.txt
+++ b/INSTALL.rst.txt
@@ -14,24 +14,17 @@ Prerequisites
Building NumPy requires the following installed software:
-1) For Python 3, Python__ 3.5.x or newer.
+1) Python__ 3.5.x or newer.
- On Debian and derivative (Ubuntu): python python-dev
+ Please note that the Python development headers also need to be installed,
+ e.g., on Debian/Ubuntu one needs to install both `python3` and
+ `python3-dev`. On Windows and macOS this is normally not an issue.
- On Windows: the official python installer on Python__ is enough
-
- Make sure that the Python package distutils is installed before
- continuing. For example, in Debian GNU/Linux, distutils is included
- in the python-dev package.
-
- Python must also be compiled with the zlib module enabled.
-
-2) Cython >= 0.29.2 (for development versions of numpy, not for released
- versions)
+2) Cython >= 0.29.14
3) pytest__ (optional) 1.15 or later
- This is required for testing numpy, but not for using it.
+ This is required for testing NumPy, but not for using it.
Python__ http://www.python.org
pytest__ http://pytest.readthedocs.io
@@ -45,14 +38,14 @@ pytest__ http://pytest.readthedocs.io
.. note::
- More extensive information on building NumPy (and Scipy) is maintained at
+ More extensive information on building NumPy (and SciPy) is maintained at
https://scipy.github.io/devdocs/building/
Basic Installation
==================
-To install numpy run::
+To install NumPy, run::
python setup.py build -j 4 install --prefix $HOME/.local
@@ -88,18 +81,18 @@ installed then ``g77`` will be detected and used first. To explicitly select
Windows
-------
-On Windows, building from source can be difficult. Currently, the most robust
-option is to use the Intel compilers, or alternatively MSVC (the same version
-as used to build Python itself) with Intel ifort. Intel itself maintains a
-good `application note <https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl>`_
+On Windows, building from source can be difficult (in particular if you need to
+build SciPy as well, because that requires a Fortran compiler). Currently, the
+most robust option is to use MSVC (for NumPy only). If you also need SciPy,
+you can either use MSVC + Intel Fortran or the Intel compiler suite.
+Intel itself maintains a good `application note
+<https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl>`_
on this.
-If you want to use a free compiler toolchain, the recommended compiler is MingwPy__.
-The older MinGW32 compiler set used to produce older .exe installers for NumPy
-itself is still available at https://github.com/numpy/numpy-vendor, but not
-recommended for use anymore.
-
-MingwPy__ https://mingwpy.github.io
+If you want to use a free compiler toolchain, our current recommendation is to
+use Docker or Windows subsystem for Linux (WSL). See
+https://scipy.github.io/devdocs/dev/contributor/contributor_toc.html#development-environment
+for more details.
Building with optimized BLAS support
@@ -114,16 +107,16 @@ Windows
-------
The Intel compilers work with Intel MKL, see the application note linked above.
-MingwPy__ works with OpenBLAS.
+
For an overview of the state of BLAS/LAPACK libraries on Windows, see
`here <https://mingwpy.github.io/blas_lapack.html>`_.
-OS X
-----
+macOS
+-----
-OS X ships the Accelerate framework, which NumPy can build against without any
-manual configuration. Other BLAS/LAPACK implementations (OpenBLAS, Intel MKL,
-ATLAS) will also work.
+You will need to install a BLAS/LAPACK library. We recommend using OpenBLAS or
+Intel MKL. Apple's Accelerate also still works, however it has bugs and we are
+likely to drop support for it in the near future.
Ubuntu/Debian
-------------
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 9e1e52952..0ac3b6471 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -40,8 +40,9 @@ stages:
locale-gen fr_FR && update-locale && \
apt-get -y install gfortran-5 wget && \
target=\$(python3 tools/openblas_support.py) && \
- cp -r \$target/usr/local/lib/* /usr/lib && \
- cp \$target/usr/local/include/* /usr/include && \
+ ls -lR \$target && \
+ cp -r \$target/lib/* /usr/lib && \
+ cp \$target/include/* /usr/include && \
python3 -m pip install --user --upgrade pip setuptools && \
python3 -m pip install --user -r test_requirements.txt && \
python3 -m pip install . && \
@@ -65,13 +66,21 @@ stages:
# azure config for mac os -- Microsoft has indicated
# they will patch this issue
vmImage: macOS-10.14
+ strategy:
+ maxParallel: 2
+ matrix:
+ Python36:
+ PYTHON_VERSION: '3.6'
+ Python36-ILP64:
+ PYTHON_VERSION: '3.6'
+ NPY_USE_BLAS_ILP64: '1'
steps:
# the @0 refers to the (major) version of the *task* on Microsoft's
# end, not the order in the build matrix nor anything to do
# with version of Python selected
- task: UsePythonVersion@0
inputs:
- versionSpec: '3.6'
+ versionSpec: $(PYTHON_VERSION)
addToPath: true
architecture: 'x64'
# NOTE: do we have a compelling reason to use older / newer
@@ -97,9 +106,10 @@ stages:
# primarily on file size / name details
- script: |
target=$(python tools/openblas_support.py)
+ ls -lR $target
# manually link to appropriate system paths
- cp $target/usr/local/lib/* /usr/local/lib/
- cp $target/usr/local/include/* /usr/local/include/
+ cp $target/lib/* /usr/local/lib/
+ cp $target/include/* /usr/local/include/
displayName: 'install pre-built openblas'
- script: python -m pip install --upgrade pip setuptools wheel
displayName: 'Install tools'
@@ -176,6 +186,8 @@ stages:
PYTHON_ARCH: 'x64'
TEST_MODE: full
BITS: 64
+ NPY_USE_BLAS_ILP64: '1'
+ OPENBLAS_SUFFIX: '64_'
steps:
- template: azure-steps-windows.yml
- job: Linux_PyPy3
diff --git a/azure-steps-windows.yml b/azure-steps-windows.yml
index 26d7a667d..f17039455 100644
--- a/azure-steps-windows.yml
+++ b/azure-steps-windows.yml
@@ -11,7 +11,7 @@ steps:
- powershell: |
$pyversion = python -c "from __future__ import print_function; import sys; print(sys.version.split()[0])"
Write-Host "Python Version: $pyversion"
- $target = "C:\\hostedtoolcache\\windows\\Python\\$pyversion\\$(PYTHON_ARCH)\\lib\\openblas.a"
+ $target = "C:\\hostedtoolcache\\windows\\Python\\$pyversion\\$(PYTHON_ARCH)\\lib\\openblas$env:OPENBLAS_SUFFIX.a"
Write-Host "target path: $target"
$openblas = python tools/openblas_support.py
cp $openblas $target
diff --git a/doc/HOWTO_RELEASE.rst.txt b/doc/HOWTO_RELEASE.rst.txt
index 53359f0a2..ff56c6d78 100644
--- a/doc/HOWTO_RELEASE.rst.txt
+++ b/doc/HOWTO_RELEASE.rst.txt
@@ -101,11 +101,13 @@ Building source archives and wheels
You will need write permission for numpy-wheels in order to trigger wheel
builds.
-* Python(s) from `python.org <https://python.org>`_ or linux distro.
-* cython
-* virtualenv (pip)
-* Paver (pip)
-* numpy-wheels `<https://github.com/MacPython/numpy-wheels>`_ (clone)
+- Python(s) from `python.org <https://python.org>`_ or linux distro.
+- cython (pip)
+- virtualenv (pip)
+- Paver (pip)
+- pandoc `pandoc.org <https://www.pandoc.org>`_ or linux distro.
+- numpy-wheels `<https://github.com/MacPython/numpy-wheels>`_ (clone)
+
Building docs
-------------
@@ -376,7 +378,7 @@ Make the release
Build the changelog and notes for upload with::
- paver write_release_and_log
+ paver write_release
Build and archive documentation
-------------------------------
diff --git a/doc/changelog/1.16.6-changelog.rst b/doc/changelog/1.16.6-changelog.rst
new file mode 100644
index 000000000..62ff46c34
--- /dev/null
+++ b/doc/changelog/1.16.6-changelog.rst
@@ -0,0 +1,36 @@
+
+Contributors
+============
+
+A total of 10 people contributed to this release.
+
+* CakeWithSteak
+* Charles Harris
+* Chris Burr
+* Eric Wieser
+* Fernando Saravia
+* Lars Grueter
+* Matti Picus
+* Maxwell Aladago
+* Qiming Sun
+* Warren Weckesser
+
+Pull requests merged
+====================
+
+A total of 14 pull requests were merged for this release.
+
+* `#14211 <https://github.com/numpy/numpy/pull/14211>`__: BUG: Fix uint-overflow if padding with linear_ramp and negative...
+* `#14275 <https://github.com/numpy/numpy/pull/14275>`__: BUG: fixing to allow unpickling of PY3 pickles from PY2
+* `#14340 <https://github.com/numpy/numpy/pull/14340>`__: BUG: Fix misuse of .names and .fields in various places (backport...
+* `#14423 <https://github.com/numpy/numpy/pull/14423>`__: BUG: test, fix regression in converting to ctypes.
+* `#14434 <https://github.com/numpy/numpy/pull/14434>`__: BUG: Fixed maximum relative error reporting in assert_allclose
+* `#14509 <https://github.com/numpy/numpy/pull/14509>`__: BUG: Fix regression in boolean matmul.
+* `#14686 <https://github.com/numpy/numpy/pull/14686>`__: BUG: properly define PyArray_DescrCheck
+* `#14853 <https://github.com/numpy/numpy/pull/14853>`__: BLD: add 'apt update' to shippable
+* `#14854 <https://github.com/numpy/numpy/pull/14854>`__: BUG: Fix _ctypes class circular reference. (#13808)
+* `#14856 <https://github.com/numpy/numpy/pull/14856>`__: BUG: Fix `np.einsum` errors on Power9 Linux and z/Linux
+* `#14863 <https://github.com/numpy/numpy/pull/14863>`__: BLD: Prevent -flto from optimising long double representation...
+* `#14864 <https://github.com/numpy/numpy/pull/14864>`__: BUG: lib: Fix histogram problem with signed integer arrays.
+* `#15172 <https://github.com/numpy/numpy/pull/15172>`__: ENH: Backport improvements to testing functions.
+* `#15191 <https://github.com/numpy/numpy/pull/15191>`__: REL: Prepare for 1.16.6 release.
diff --git a/doc/changelog/1.17.5-changelog.rst b/doc/changelog/1.17.5-changelog.rst
new file mode 100644
index 000000000..7ac758075
--- /dev/null
+++ b/doc/changelog/1.17.5-changelog.rst
@@ -0,0 +1,26 @@
+
+Contributors
+============
+
+A total of 6 people contributed to this release. People with a "+" by their
+names contributed a patch for the first time.
+
+* Charles Harris
+* Eric Wieser
+* Ilhan Polat
+* Matti Picus
+* Michael Hudson-Doyle
+* Ralf Gommers
+
+Pull requests merged
+====================
+
+A total of 7 pull requests were merged for this release.
+
+* `#14593 <https://github.com/numpy/numpy/pull/14593>`__: MAINT: backport Cython API cleanup to 1.17.x, remove docs
+* `#14937 <https://github.com/numpy/numpy/pull/14937>`__: BUG: fix integer size confusion in handling array's ndmin argument
+* `#14939 <https://github.com/numpy/numpy/pull/14939>`__: BUILD: remove SSE2 flag from numpy.random builds
+* `#14993 <https://github.com/numpy/numpy/pull/14993>`__: MAINT: Added Python3.8 branch to dll lib discovery
+* `#15038 <https://github.com/numpy/numpy/pull/15038>`__: BUG: Fix refcounting in ufunc object loops
+* `#15067 <https://github.com/numpy/numpy/pull/15067>`__: BUG: Exceptions tracebacks are dropped
+* `#15175 <https://github.com/numpy/numpy/pull/15175>`__: ENH: Backport improvements to testing functions.
diff --git a/doc/changelog/1.18.1-changelog.rst b/doc/changelog/1.18.1-changelog.rst
new file mode 100644
index 000000000..d3df29198
--- /dev/null
+++ b/doc/changelog/1.18.1-changelog.rst
@@ -0,0 +1,33 @@
+
+Contributors
+============
+
+A total of 7 people contributed to this release. People with a "+" by their
+names contributed a patch for the first time.
+
+* Charles Harris
+* Matti Picus
+* Maxwell Aladago
+* Pauli Virtanen
+* Ralf Gommers
+* Tyler Reddy
+* Warren Weckesser
+
+Pull requests merged
+====================
+
+A total of 13 pull requests were merged for this release.
+
+* `#15158 <https://github.com/numpy/numpy/pull/15158>`__: MAINT: Update pavement.py for towncrier.
+* `#15159 <https://github.com/numpy/numpy/pull/15159>`__: DOC: add moved modules to 1.18 release note
+* `#15161 <https://github.com/numpy/numpy/pull/15161>`__: MAINT, DOC: Minor backports and updates for 1.18.x
+* `#15176 <https://github.com/numpy/numpy/pull/15176>`__: TST: Add assert_array_equal test for big integer arrays
+* `#15184 <https://github.com/numpy/numpy/pull/15184>`__: BUG: use tmp dir and check version for cython test (#15170)
+* `#15220 <https://github.com/numpy/numpy/pull/15220>`__: BUG: distutils: fix msvc+gfortran openblas handling corner case
+* `#15221 <https://github.com/numpy/numpy/pull/15221>`__: BUG: remove -std=c99 for c++ compilation (#15194)
+* `#15222 <https://github.com/numpy/numpy/pull/15222>`__: MAINT: unskip test on win32
+* `#15223 <https://github.com/numpy/numpy/pull/15223>`__: TST: add BLAS ILP64 run in Travis & Azure
+* `#15245 <https://github.com/numpy/numpy/pull/15245>`__: MAINT: only add --std=c99 where needed
+* `#15246 <https://github.com/numpy/numpy/pull/15246>`__: BUG: lib: Fix handling of integer arrays by gradient.
+* `#15247 <https://github.com/numpy/numpy/pull/15247>`__: MAINT: Do not use private Python function in testing
+* `#15250 <https://github.com/numpy/numpy/pull/15250>`__: REL: Prepare for the NumPy 1.18.1 release.
diff --git a/doc/source/release.rst b/doc/source/release.rst
index 0343275a5..cbeca3660 100644
--- a/doc/source/release.rst
+++ b/doc/source/release.rst
@@ -5,12 +5,15 @@ Release Notes
.. toctree::
:maxdepth: 3
+ 1.18.1 <release/1.18.1-notes>
1.18.0 <release/1.18.0-notes>
+ 1.17.5 <release/1.17.5-notes>
1.17.4 <release/1.17.4-notes>
1.17.3 <release/1.17.3-notes>
1.17.2 <release/1.17.2-notes>
1.17.1 <release/1.17.1-notes>
1.17.0 <release/1.17.0-notes>
+ 1.16.6 <release/1.16.6-notes>
1.16.5 <release/1.16.5-notes>
1.16.4 <release/1.16.4-notes>
1.16.3 <release/1.16.3-notes>
diff --git a/doc/source/release/1.16.6-notes.rst b/doc/source/release/1.16.6-notes.rst
new file mode 100644
index 000000000..0aeba3cd3
--- /dev/null
+++ b/doc/source/release/1.16.6-notes.rst
@@ -0,0 +1,87 @@
+.. currentmodule:: numpy
+
+==========================
+NumPy 1.16.6 Release Notes
+==========================
+
+The NumPy 1.16.6 release fixes bugs reported against the 1.16.5 release, and
+also backports several enhancements from master that seem appropriate for a
+release series that is the last to support Python 2.7. The wheels on PyPI are
+linked with OpenBLAS v0.3.7, which should fix errors on Skylake series
+cpus.
+
+Downstream developers building this release should use Cython >= 0.29.2 and, if
+using OpenBLAS, OpenBLAS >= v0.3.7. The supported Python versions are 2.7 and
+3.5-3.7.
+
+Highlights
+==========
+
+- The ``np.testing.utils`` functions have been updated from 1.19.0-dev0.
+ This improves the function documentation and error messages as well
+ extending the ``assert_array_compare`` function to additional types.
+
+
+New functions
+=============
+
+Allow matmul (`@` operator) to work with object arrays.
+-------------------------------------------------------
+This is an enhancement that was added in NumPy 1.17 and seems reasonable to
+include in the LTS 1.16 release series.
+
+
+Compatibility notes
+===================
+
+Fix regression in matmul (`@` operator) for boolean types
+---------------------------------------------------------
+Booleans were being treated as integers rather than booleans,
+which was a regression from previous behavior.
+
+
+Improvements
+============
+
+Array comparison assertions include maximum differences
+-------------------------------------------------------
+Error messages from array comparison tests such as ``testing.assert_allclose``
+now include "max absolute difference" and "max relative difference," in
+addition to the previous "mismatch" percentage. This information makes it
+easier to update absolute and relative error tolerances.
+
+Contributors
+============
+
+A total of 10 people contributed to this release.
+
+* CakeWithSteak
+* Charles Harris
+* Chris Burr
+* Eric Wieser
+* Fernando Saravia
+* Lars Grueter
+* Matti Picus
+* Maxwell Aladago
+* Qiming Sun
+* Warren Weckesser
+
+Pull requests merged
+====================
+
+A total of 14 pull requests were merged for this release.
+
+* `#14211 <https://github.com/numpy/numpy/pull/14211>`__: BUG: Fix uint-overflow if padding with linear_ramp and negative...
+* `#14275 <https://github.com/numpy/numpy/pull/14275>`__: BUG: fixing to allow unpickling of PY3 pickles from PY2
+* `#14340 <https://github.com/numpy/numpy/pull/14340>`__: BUG: Fix misuse of .names and .fields in various places (backport...
+* `#14423 <https://github.com/numpy/numpy/pull/14423>`__: BUG: test, fix regression in converting to ctypes.
+* `#14434 <https://github.com/numpy/numpy/pull/14434>`__: BUG: Fixed maximum relative error reporting in assert_allclose
+* `#14509 <https://github.com/numpy/numpy/pull/14509>`__: BUG: Fix regression in boolean matmul.
+* `#14686 <https://github.com/numpy/numpy/pull/14686>`__: BUG: properly define PyArray_DescrCheck
+* `#14853 <https://github.com/numpy/numpy/pull/14853>`__: BLD: add 'apt update' to shippable
+* `#14854 <https://github.com/numpy/numpy/pull/14854>`__: BUG: Fix _ctypes class circular reference. (#13808)
+* `#14856 <https://github.com/numpy/numpy/pull/14856>`__: BUG: Fix `np.einsum` errors on Power9 Linux and z/Linux
+* `#14863 <https://github.com/numpy/numpy/pull/14863>`__: BLD: Prevent -flto from optimising long double representation...
+* `#14864 <https://github.com/numpy/numpy/pull/14864>`__: BUG: lib: Fix histogram problem with signed integer arrays.
+* `#15172 <https://github.com/numpy/numpy/pull/15172>`__: ENH: Backport improvements to testing functions.
+* `#15191 <https://github.com/numpy/numpy/pull/15191>`__: REL: Prepare for 1.16.6 release.
diff --git a/doc/source/release/1.17.5-notes.rst b/doc/source/release/1.17.5-notes.rst
new file mode 100644
index 000000000..0f1d3e1a5
--- /dev/null
+++ b/doc/source/release/1.17.5-notes.rst
@@ -0,0 +1,45 @@
+.. currentmodule:: numpy
+
+==========================
+NumPy 1.17.5 Release Notes
+==========================
+
+This release contains fixes for bugs reported against NumPy 1.17.4 along with
+some build improvements. The Python versions supported in this release
+are 3.5-3.8.
+
+Downstream developers should use Cython >= 0.29.14 for Python 3.8 support and
+OpenBLAS >= 3.7 to avoid errors on the Skylake architecture.
+
+It is recommended that developers interested in the new random bit generators
+upgrade to the NumPy 1.18.x series, as it has updated documentation and
+many small improvements.
+
+
+Contributors
+============
+
+A total of 6 people contributed to this release. People with a "+" by their
+names contributed a patch for the first time.
+
+* Charles Harris
+* Eric Wieser
+* Ilhan Polat
+* Matti Picus
+* Michael Hudson-Doyle
+* Ralf Gommers
+
+
+Pull requests merged
+====================
+
+A total of 8 pull requests were merged for this release.
+
+* `#14593 <https://github.com/numpy/numpy/pull/14593>`__: MAINT: backport Cython API cleanup to 1.17.x, remove docs
+* `#14937 <https://github.com/numpy/numpy/pull/14937>`__: BUG: fix integer size confusion in handling array's ndmin argument
+* `#14939 <https://github.com/numpy/numpy/pull/14939>`__: BUILD: remove SSE2 flag from numpy.random builds
+* `#14993 <https://github.com/numpy/numpy/pull/14993>`__: MAINT: Added Python3.8 branch to dll lib discovery
+* `#15038 <https://github.com/numpy/numpy/pull/15038>`__: BUG: Fix refcounting in ufunc object loops
+* `#15067 <https://github.com/numpy/numpy/pull/15067>`__: BUG: Exceptions tracebacks are dropped
+* `#15175 <https://github.com/numpy/numpy/pull/15175>`__: ENH: Backport improvements to testing functions.
+* `#15213 <https://github.com/numpy/numpy/pull/15213>`__: REL: Prepare for the NumPy 1.17.5 release.
diff --git a/doc/source/release/1.18.0-notes.rst b/doc/source/release/1.18.0-notes.rst
index ad02d74d3..24aa94445 100644
--- a/doc/source/release/1.18.0-notes.rst
+++ b/doc/source/release/1.18.0-notes.rst
@@ -9,7 +9,7 @@ the new random C-API, expires a large number of old deprecations, and improves
the appearance of the documentation. The Python versions supported are 3.5-3.8.
This is the last NumPy release series that will support Python 3.5.
-Downstream developers should use Cython >= 0.29.13 for Python 3.8 support and
+Downstream developers should use Cython >= 0.29.14 for Python 3.8 support and
OpenBLAS >= 3.7 to avoid problems on the Skylake
architecture.
@@ -199,6 +199,15 @@ exception will require adaptation, and code that mistakenly called
``numpy.nanmax`` and ``numpy.nanmin``.
(`gh-14841 <https://github.com/numpy/numpy/pull/14841>`__)
+Moved modules in ``numpy.random``
+---------------------------------
+As part of the API cleanup, the submodules in ``numpy.random``
+``bit_generator``, ``philox``, ``pcg64``, ``sfc64, ``common``, ``generator``,
+and ``bounded_integers`` were moved to ``_bit_generator``, ``_philox``,
+``_pcg64``, ``_sfc64, ``_common``, ``_generator``, and ``_bounded_integers``
+respectively to indicate that they are not part of the public interface.
+(`gh-14608 <https://github.com/numpy/numpy/pull/14608>`__)
+
C API changes
=============
@@ -366,9 +375,9 @@ Added two new configuration options. During the ``build_src`` subcommand, as
part of configuring NumPy, the files ``_numpyconfig.h`` and ``config.h`` are
created by probing support for various runtime functions and routines.
Previously, the very verbose compiler output during this stage clouded more
-important information. By default the output is silenced. Running ``runtests.py
---debug-info`` will add ``--verbose-cfg`` to the ``build_src`` subcommand,
-which will restore the previous behaviour.
+important information. By default the output is silenced. Running
+``runtests.py --debug-info`` will add ``--verbose-cfg`` to the ``build_src``
+subcommand,which will restore the previous behaviour.
Adding ``CFLAGS=-Werror`` to turn warnings into errors would trigger errors
during the configuration. Now ``runtests.py --warn-error`` will add
diff --git a/doc/source/release/1.18.1-notes.rst b/doc/source/release/1.18.1-notes.rst
new file mode 100644
index 000000000..8bc502ecb
--- /dev/null
+++ b/doc/source/release/1.18.1-notes.rst
@@ -0,0 +1,52 @@
+.. currentmodule:: numpy
+
+==========================
+NumPy 1.18.1 Release Notes
+==========================
+
+This release contains fixes for bugs reported against NumPy 1.18.0. Two bugs
+in particular that caused widespread problems downstream were:
+
+- The cython random extension test was not using a temporary directory for
+ building, resulting in a permission violation. Fixed.
+
+- Numpy distutils was appending `-std=c99` to all C compiler runs, leading to
+ changed behavior and compile problems downstream. That flag is now only
+ applied when building numpy C code.
+
+The Python versions supported in this release are 3.5-3.8. Downstream
+developers should use Cython >= 0.29.14 for Python 3.8 support and OpenBLAS >=
+3.7 to avoid errors on the Skylake architecture.
+
+Contributors
+============
+
+A total of 7 people contributed to this release. People with a "+" by their
+names contributed a patch for the first time.
+
+* Charles Harris
+* Matti Picus
+* Maxwell Aladago
+* Pauli Virtanen
+* Ralf Gommers
+* Tyler Reddy
+* Warren Weckesser
+
+Pull requests merged
+====================
+
+A total of 13 pull requests were merged for this release.
+
+* `#15158 <https://github.com/numpy/numpy/pull/15158>`__: MAINT: Update pavement.py for towncrier.
+* `#15159 <https://github.com/numpy/numpy/pull/15159>`__: DOC: add moved modules to 1.18 release note
+* `#15161 <https://github.com/numpy/numpy/pull/15161>`__: MAINT, DOC: Minor backports and updates for 1.18.x
+* `#15176 <https://github.com/numpy/numpy/pull/15176>`__: TST: Add assert_array_equal test for big integer arrays
+* `#15184 <https://github.com/numpy/numpy/pull/15184>`__: BUG: use tmp dir and check version for cython test (#15170)
+* `#15220 <https://github.com/numpy/numpy/pull/15220>`__: BUG: distutils: fix msvc+gfortran openblas handling corner case
+* `#15221 <https://github.com/numpy/numpy/pull/15221>`__: BUG: remove -std=c99 for c++ compilation (#15194)
+* `#15222 <https://github.com/numpy/numpy/pull/15222>`__: MAINT: unskip test on win32
+* `#15223 <https://github.com/numpy/numpy/pull/15223>`__: TST: add BLAS ILP64 run in Travis & Azure
+* `#15245 <https://github.com/numpy/numpy/pull/15245>`__: MAINT: only add --std=c99 where needed
+* `#15246 <https://github.com/numpy/numpy/pull/15246>`__: BUG: lib: Fix handling of integer arrays by gradient.
+* `#15247 <https://github.com/numpy/numpy/pull/15247>`__: MAINT: Do not use private Python function in testing
+* `#15250 <https://github.com/numpy/numpy/pull/15250>`__: REL: Prepare for the NumPy 1.18.1 release.
diff --git a/doc/source/user/building.rst b/doc/source/user/building.rst
index 1588de964..e70391ed0 100644
--- a/doc/source/user/building.rst
+++ b/doc/source/user/building.rst
@@ -11,19 +11,11 @@ Prerequisites
Building NumPy requires the following software installed:
-1) Python 2.7.x, 3.4.x or newer
+1) Python 3.5.x or newer
- On Debian and derivatives (Ubuntu): python, python-dev (or python3-dev)
-
- On Windows: the official python installer at
- `www.python.org <https://www.python.org>`_ is enough
-
- Make sure that the Python package distutils is installed before
- continuing. For example, in Debian GNU/Linux, installing python-dev
- also installs distutils.
-
- Python must also be compiled with the zlib module enabled. This is
- practically always the case with pre-packaged Pythons.
+ Please note that the Python development headers also need to be installed,
+ e.g., on Debian/Ubuntu one needs to install both `python3` and
+ `python3-dev`. On Windows and macOS this is normally not an issue.
2) Compilers
@@ -42,19 +34,16 @@ Building NumPy requires the following software installed:
NumPy does not require any external linear algebra libraries to be
installed. However, if these are available, NumPy's setup script can detect
them and use them for building. A number of different LAPACK library setups
- can be used, including optimized LAPACK libraries such as ATLAS, MKL or the
- Accelerate/vecLib framework on OS X.
+ can be used, including optimized LAPACK libraries such as OpenBLAS or MKL.
4) Cython
- To build development versions of NumPy, you'll need a recent version of
- Cython. Released NumPy sources on PyPi include the C files generated from
- Cython code, so for released versions having Cython installed isn't needed.
+ For building NumPy, you'll need a recent version of Cython.
Basic Installation
------------------
-To install NumPy run::
+To install NumPy, run::
pip install .
@@ -62,10 +51,6 @@ To perform an in-place build that can be run from the source folder run::
python setup.py build_ext --inplace
-The NumPy build system uses ``setuptools`` (from numpy 1.11.0, before that it
-was plain ``distutils``) and ``numpy.distutils``.
-Using ``virtualenv`` should work as expected.
-
*Note: for build instructions to do development work on NumPy itself, see*
:ref:`development-environment`.
@@ -83,7 +68,7 @@ For detailed info on testing, see :ref:`testing-builds`.
Parallel builds
~~~~~~~~~~~~~~~
-From NumPy 1.10.0 on it's also possible to do a parallel build with::
+It's possible to do a parallel build with::
python setup.py build -j 4 install --prefix $HOME/.local
@@ -95,22 +80,11 @@ to perform a parallel in-place build, run::
The number of build jobs can also be specified via the environment variable
``NPY_NUM_BUILD_JOBS``.
-
-FORTRAN ABI mismatch
---------------------
-
-The two most popular open source fortran compilers are g77 and gfortran.
-Unfortunately, they are not ABI compatible, which means that concretely you
-should avoid mixing libraries built with one with another. In particular, if
-your blas/lapack/atlas is built with g77, you *must* use g77 when building
-numpy and scipy; on the contrary, if your atlas is built with gfortran, you
-*must* build numpy/scipy with gfortran. This applies for most other cases
-where different FORTRAN compilers might have been used.
-
Choosing the fortran compiler
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-To build with gfortran::
+Compilers are auto-detected; building with a particular compiler can be done
+with ``--fcompiler``. E.g. to select gfortran::
python setup.py build --fcompiler=gnu95
@@ -118,14 +92,14 @@ For more information see::
python setup.py build --help-fcompiler
-How to check the ABI of blas/lapack/atlas
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+How to check the ABI of BLAS/LAPACK libraries
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
One relatively simple and reliable way to check for the compiler used to build
a library is to use ldd on the library. If libg2c.so is a dependency, this
-means that g77 has been used. If libgfortran.so is a dependency, gfortran
-has been used. If both are dependencies, this means both have been used, which
-is almost always a very bad idea.
+means that g77 has been used (note: g77 is no longer supported for building NumPy).
+If libgfortran.so is a dependency, gfortran has been used. If both are dependencies,
+this means both have been used, which is almost always a very bad idea.
Accelerated BLAS/LAPACK libraries
---------------------------------
@@ -145,7 +119,6 @@ The default order for the libraries are:
5. Accelerate (MacOS)
6. BLAS (NetLIB)
-
If you wish to build against OpenBLAS but you also have BLIS available one
may predefine the order of searching via the environment variable
``NPY_BLAS_ORDER`` which is a comma-separated list of the above names which
@@ -235,14 +208,3 @@ Additional compiler flags can be supplied by setting the ``OPT``,
``FOPT`` (for Fortran), and ``CC`` environment variables.
When providing options that should improve the performance of the code ensure
that you also set ``-DNDEBUG`` so that debugging code is not executed.
-
-
-Building with ATLAS support
----------------------------
-
-Ubuntu
-~~~~~~
-
-You can install the necessary package for optimized ATLAS with this command::
-
- sudo apt-get install libatlas-base-dev
diff --git a/numpy/core/tests/test_multiarray.py b/numpy/core/tests/test_multiarray.py
index 22d550ecc..2464dee8b 100644
--- a/numpy/core/tests/test_multiarray.py
+++ b/numpy/core/tests/test_multiarray.py
@@ -20,7 +20,6 @@ import gc
import weakref
import pytest
from contextlib import contextmanager
-from test.support import no_tracing
from numpy.compat import pickle
@@ -97,6 +96,26 @@ def _aligned_zeros(shape, dtype=float, order="C", align=None):
data.fill(0)
return data
+def _no_tracing(func):
+ """
+ Decorator to temporarily turn off tracing for the duration of a test.
+ Needed in tests that check refcounting, otherwise the tracing itself
+ influences the refcounts
+ """
+ if not hasattr(sys, 'gettrace'):
+ return func
+ else:
+ @functools.wraps(func)
+ def wrapper(*args, **kwargs):
+ original_trace = sys.gettrace()
+ try:
+ sys.settrace(None)
+ return func(*args, **kwargs)
+ finally:
+ sys.settrace(original_trace)
+ return wrapper
+
+
class TestFlags(object):
def setup(self):
@@ -5098,7 +5117,7 @@ class TestFlat(object):
class TestResize(object):
- @no_tracing
+ @_no_tracing
def test_basic(self):
x = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]])
if IS_PYPY:
@@ -5115,7 +5134,7 @@ class TestResize(object):
assert_raises(ValueError, x.resize, (5, 1))
del y # avoid pyflakes unused variable warning.
- @no_tracing
+ @_no_tracing
def test_int_shape(self):
x = np.eye(3)
if IS_PYPY:
@@ -5149,7 +5168,7 @@ class TestResize(object):
assert_raises(TypeError, np.eye(3).resize, order=1)
assert_raises(TypeError, np.eye(3).resize, refcheck='hi')
- @no_tracing
+ @_no_tracing
def test_freeform_shape(self):
x = np.eye(3)
if IS_PYPY:
@@ -5158,7 +5177,7 @@ class TestResize(object):
x.resize(3, 2, 1)
assert_(x.shape == (3, 2, 1))
- @no_tracing
+ @_no_tracing
def test_zeros_appended(self):
x = np.eye(3)
if IS_PYPY:
@@ -5168,7 +5187,7 @@ class TestResize(object):
assert_array_equal(x[0], np.eye(3))
assert_array_equal(x[1], np.zeros((3, 3)))
- @no_tracing
+ @_no_tracing
def test_obj_obj(self):
# check memory is initialized on resize, gh-4857
a = np.ones(10, dtype=[('k', object, 2)])
@@ -7780,7 +7799,7 @@ if not IS_PYPY:
d = np.ones(100)
assert_(sys.getsizeof(d) < sys.getsizeof(d.reshape(100, 1, 1).copy()))
- @no_tracing
+ @_no_tracing
def test_resize(self):
d = np.ones(100)
old = sys.getsizeof(d)
diff --git a/numpy/distutils/ccompiler.py b/numpy/distutils/ccompiler.py
index 684c7535b..643879023 100644
--- a/numpy/distutils/ccompiler.py
+++ b/numpy/distutils/ccompiler.py
@@ -532,11 +532,6 @@ def CCompiler_customize(self, dist, need_cxx=0):
'g++' in self.compiler[0] or
'clang' in self.compiler[0]):
self._auto_depends = True
- if 'gcc' in self.compiler[0]:
- # add std=c99 flag for gcc
- # TODO: does this need to be more specific?
- self.compiler.append('-std=c99')
- self.compiler_so.append('-std=c99')
elif os.name == 'posix':
import tempfile
import shutil
diff --git a/numpy/distutils/system_info.py b/numpy/distutils/system_info.py
index 4786b3a0c..fc7018af3 100644
--- a/numpy/distutils/system_info.py
+++ b/numpy/distutils/system_info.py
@@ -2102,16 +2102,17 @@ class openblas_info(blas_info):
return None
# Generate numpy.distutils virtual static library file
- tmpdir = os.path.join(os.getcwd(), 'build', 'openblas')
+ basename = self.__class__.__name__
+ tmpdir = os.path.join(os.getcwd(), 'build', basename)
if not os.path.isdir(tmpdir):
os.makedirs(tmpdir)
info = {'library_dirs': [tmpdir],
- 'libraries': ['openblas'],
+ 'libraries': [basename],
'language': 'f77'}
- fake_lib_file = os.path.join(tmpdir, 'openblas.fobjects')
- fake_clib_file = os.path.join(tmpdir, 'openblas.cobjects')
+ fake_lib_file = os.path.join(tmpdir, basename + '.fobjects')
+ fake_clib_file = os.path.join(tmpdir, basename + '.cobjects')
with open(fake_lib_file, 'w') as f:
f.write("\n".join(library_paths))
with open(fake_clib_file, 'w') as f:
diff --git a/numpy/lib/function_base.py b/numpy/lib/function_base.py
index 499120630..df06d10cc 100644
--- a/numpy/lib/function_base.py
+++ b/numpy/lib/function_base.py
@@ -979,13 +979,18 @@ def gradient(f, *varargs, **kwargs):
# scalar or 1d array for each axis
dx = list(varargs)
for i, distances in enumerate(dx):
- if np.ndim(distances) == 0:
+ distances = np.asanyarray(distances)
+ if distances.ndim == 0:
continue
- elif np.ndim(distances) != 1:
+ elif distances.ndim != 1:
raise ValueError("distances must be either scalars or 1d")
if len(distances) != f.shape[axes[i]]:
raise ValueError("when 1d, distances must match "
"the length of the corresponding dimension")
+ if np.issubdtype(distances.dtype, np.integer):
+ # Convert numpy integer types to float64 to avoid modular
+ # arithmetic in np.diff(distances).
+ distances = distances.astype(np.float64)
diffx = np.diff(distances)
# if distances are constant reduce to the scalar case
# since it brings a consistent speedup
@@ -1024,8 +1029,12 @@ def gradient(f, *varargs, **kwargs):
elif np.issubdtype(otype, np.inexact):
pass
else:
- # all other types convert to floating point
- otype = np.double
+ # All other types convert to floating point.
+ # First check if f is a numpy integer type; if so, convert f to float64
+ # to avoid modular arithmetic when computing the changes in f.
+ if np.issubdtype(otype, np.integer):
+ f = f.astype(np.float64)
+ otype = np.float64
for axis, ax_dx in zip(axes, dx):
if f.shape[axis] < edge_order + 1:
diff --git a/numpy/lib/tests/test_function_base.py b/numpy/lib/tests/test_function_base.py
index 9075ff538..d5d62a8d1 100644
--- a/numpy/lib/tests/test_function_base.py
+++ b/numpy/lib/tests/test_function_base.py
@@ -1084,6 +1084,40 @@ class TestGradient(object):
assert_raises(ValueError, gradient, np.arange(1), edge_order=2)
assert_raises(ValueError, gradient, np.arange(2), edge_order=2)
+ @pytest.mark.parametrize('f_dtype', [np.uint8, np.uint16,
+ np.uint32, np.uint64])
+ def test_f_decreasing_unsigned_int(self, f_dtype):
+ f = np.array([5, 4, 3, 2, 1], dtype=f_dtype)
+ g = gradient(f)
+ assert_array_equal(g, [-1]*len(f))
+
+ @pytest.mark.parametrize('f_dtype', [np.int8, np.int16,
+ np.int32, np.int64])
+ def test_f_signed_int_big_jump(self, f_dtype):
+ maxint = np.iinfo(f_dtype).max
+ x = np.array([1, 3])
+ f = np.array([-1, maxint], dtype=f_dtype)
+ dfdx = gradient(f, x)
+ assert_array_equal(dfdx, [(maxint + 1) // 2]*2)
+
+ @pytest.mark.parametrize('x_dtype', [np.uint8, np.uint16,
+ np.uint32, np.uint64])
+ def test_x_decreasing_unsigned(self, x_dtype):
+ x = np.array([3, 2, 1], dtype=x_dtype)
+ f = np.array([0, 2, 4])
+ dfdx = gradient(f, x)
+ assert_array_equal(dfdx, [-2]*len(x))
+
+ @pytest.mark.parametrize('x_dtype', [np.int8, np.int16,
+ np.int32, np.int64])
+ def test_x_signed_int_big_jump(self, x_dtype):
+ minint = np.iinfo(x_dtype).min
+ maxint = np.iinfo(x_dtype).max
+ x = np.array([-1, maxint], dtype=x_dtype)
+ f = np.array([minint // 2, 0])
+ dfdx = gradient(f, x)
+ assert_array_equal(dfdx, [0.5, 0.5])
+
class TestAngle(object):
diff --git a/numpy/linalg/tests/test_linalg.py b/numpy/linalg/tests/test_linalg.py
index bd3df1ca4..ef05b595e 100644
--- a/numpy/linalg/tests/test_linalg.py
+++ b/numpy/linalg/tests/test_linalg.py
@@ -2016,3 +2016,31 @@ def test_blas64_dot():
a[0,-1] = 1
c = np.dot(b, a)
assert_equal(c[0,-1], 1)
+
+
+@pytest.mark.xfail(not HAS_LAPACK64,
+ reason="Numpy not compiled with 64-bit BLAS/LAPACK")
+def test_blas64_geqrf_lwork_smoketest():
+ # Smoke test LAPACK geqrf lwork call with 64-bit integers
+ dtype = np.float64
+ lapack_routine = np.linalg.lapack_lite.dgeqrf
+
+ m = 2**32 + 1
+ n = 2**32 + 1
+ lda = m
+
+ # Dummy arrays, not referenced by the lapack routine, so don't
+ # need to be of the right size
+ a = np.zeros([1, 1], dtype=dtype)
+ work = np.zeros([1], dtype=dtype)
+ tau = np.zeros([1], dtype=dtype)
+
+ # Size query
+ results = lapack_routine(m, n, a, lda, tau, work, -1, 0)
+ assert_equal(results['info'], 0)
+ assert_equal(results['m'], m)
+ assert_equal(results['n'], m)
+
+ # Should result to an integer of a reasonable size
+ lwork = int(work.item())
+ assert_(2**32 < lwork < 2**42)
diff --git a/numpy/random/_examples/cython/setup.py b/numpy/random/_examples/cython/setup.py
index 19f045fc0..20cedc4e3 100644
--- a/numpy/random/_examples/cython/setup.py
+++ b/numpy/random/_examples/cython/setup.py
@@ -9,9 +9,10 @@ import numpy as np
from distutils.core import setup
from Cython.Build import cythonize
from setuptools.extension import Extension
-from os.path import join, abspath, dirname
+from os.path import join, dirname
-path = abspath(dirname(__file__))
+path = dirname(__file__)
+defs = [('NPY_NO_DEPRECATED_API', 0)]
extending = Extension("extending",
sources=[join(path, 'extending.pyx')],
@@ -19,10 +20,13 @@ extending = Extension("extending",
np.get_include(),
join(path, '..', '..')
],
+ define_macros=defs,
)
distributions = Extension("extending_distributions",
sources=[join(path, 'extending_distributions.pyx')],
- include_dirs=[np.get_include()])
+ include_dirs=[np.get_include()],
+ define_macros=defs,
+ )
extensions = [extending, distributions]
diff --git a/numpy/random/tests/test_extending.py b/numpy/random/tests/test_extending.py
index 807de1a25..ce34c1b87 100644
--- a/numpy/random/tests/test_extending.py
+++ b/numpy/random/tests/test_extending.py
@@ -1,6 +1,8 @@
import os, sys
import pytest
import warnings
+import shutil
+import subprocess
try:
import cffi
@@ -22,30 +24,34 @@ except ImportError:
try:
import cython
+ from Cython.Compiler.Version import version as cython_version
except ImportError:
cython = None
+else:
+ from distutils.version import LooseVersion
+ # Cython 0.29.14 is required for Python 3.8 and there are
+ # other fixes in the 0.29 series that are needed even for earlier
+ # Python versions.
+ # Note: keep in sync with the one in pyproject.toml
+ required_version = LooseVersion('0.29.14')
+ if LooseVersion(cython_version) < required_version:
+ # too old or wrong cython, skip the test
+ cython = None
@pytest.mark.skipif(cython is None, reason="requires cython")
-def test_cython():
- curdir = os.getcwd()
- argv = sys.argv
- examples = (os.path.dirname(__file__), '..', '_examples')
- try:
- os.chdir(os.path.join(*examples))
- sys.argv = argv[:1] + ['build']
- with warnings.catch_warnings(record=True) as w:
- # setuptools issue gh-1885
- warnings.filterwarnings('always', '', DeprecationWarning)
- from numpy.random._examples.cython import setup
- finally:
- sys.argv = argv
- os.chdir(curdir)
+@pytest.mark.slow
+def test_cython(tmp_path):
+ examples = os.path.join(os.path.dirname(__file__), '..', '_examples')
+ # CPython 3.5 and below does not handle __fspath__ well: see bpo-26027
+ shutil.copytree(examples, str(tmp_path / '_examples'))
+ subprocess.check_call([sys.executable, 'setup.py', 'build'],
+ cwd=str(tmp_path / '_examples' / 'cython'))
@pytest.mark.skipif(numba is None or cffi is None,
reason="requires numba and cffi")
def test_numba():
- from numpy.random._examples.numba import extending
+ from numpy.random._examples.numba import extending
@pytest.mark.skipif(cffi is None, reason="requires cffi")
def test_cffi():
- from numpy.random._examples.cffi import extending
+ from numpy.random._examples.cffi import extending
diff --git a/numpy/testing/tests/test_utils.py b/numpy/testing/tests/test_utils.py
index 44f93a693..1e3f52715 100644
--- a/numpy/testing/tests/test_utils.py
+++ b/numpy/testing/tests/test_utils.py
@@ -90,6 +90,21 @@ class TestArrayEqual(_GenericTest):
for t in ['S1', 'U1']:
foo(t)
+ def test_0_ndim_array(self):
+ x = np.array(473963742225900817127911193656584771)
+ y = np.array(18535119325151578301457182298393896)
+ assert_raises(AssertionError, self._assert_func, x, y)
+
+ y = x
+ self._assert_func(x, y)
+
+ x = np.array(43)
+ y = np.array(10)
+ assert_raises(AssertionError, self._assert_func, x, y)
+
+ y = x
+ self._assert_func(x, y)
+
def test_generic_rank3(self):
"""Test rank 3 array for all dtypes."""
def foo(t):
diff --git a/pavement.py b/pavement.py
index 889a552f6..8a958fff3 100644
--- a/pavement.py
+++ b/pavement.py
@@ -41,7 +41,7 @@ from paver.easy import Bunch, options, task, sh
#-----------------------------------
# Path to the release notes
-RELEASE_NOTES = 'doc/source/release/1.18.0-notes.rst'
+RELEASE_NOTES = 'doc/source/release/1.18.1-notes.rst'
#-------------------------------------------------------
@@ -182,6 +182,18 @@ def compute_sha256(idirs):
def write_release_task(options, filename='README'):
"""Append hashes of release files to release notes.
+ This appends file hashes to the release notes ane creates
+ four README files of the result in various formats:
+
+ - README.rst
+ - README.rst.gpg
+ - README.md
+ - README.md.gpg
+
+ The md file are created using `pandoc` so that the links are
+ properly updated. The gpg files are kept separate, so that
+ the unsigned files may be edited before signing if needed.
+
Parameters
----------
options :
@@ -192,46 +204,44 @@ def write_release_task(options, filename='README'):
"""
idirs = options.installers.installersdir
- source = paver.path.path(RELEASE_NOTES)
- target = paver.path.path(filename + '.rst')
- if target.exists():
- target.remove()
+ notes = paver.path.path(RELEASE_NOTES)
+ rst_readme = paver.path.path(filename + '.rst')
+ md_readme = paver.path.path(filename + '.md')
- tmp_target = paver.path.path(filename + '.md')
- source.copy(tmp_target)
+ # append hashes
+ with open(rst_readme, 'w') as freadme:
+ with open(notes) as fnotes:
+ freadme.write(fnotes.read())
- with open(str(tmp_target), 'a') as ftarget:
- ftarget.writelines("""
+ freadme.writelines("""
Checksums
=========
MD5
---
+::
""")
- ftarget.writelines([' %s\n' % c for c in compute_md5(idirs)])
- ftarget.writelines("""
+ freadme.writelines([f' {c}\n' for c in compute_md5(idirs)])
+ freadme.writelines("""
SHA256
------
+::
""")
- ftarget.writelines([' %s\n' % c for c in compute_sha256(idirs)])
+ freadme.writelines([f' {c}\n' for c in compute_sha256(idirs)])
- # Sign release
- cmd = ['gpg', '--clearsign', '--armor']
+ # generate md file using pandoc before signing
+ sh(f"pandoc -s -o {md_readme} {rst_readme}")
+
+ # Sign files
if hasattr(options, 'gpg_key'):
- cmd += ['--default-key', options.gpg_key]
- cmd += ['--output', str(target), str(tmp_target)]
- subprocess.check_call(cmd)
- print("signed %s" % (target,))
-
- # Change PR links for github posting, don't sign this
- # as the signing isn't markdown compatible.
- with open(str(tmp_target), 'r') as ftarget:
- mdtext = ftarget.read()
- mdtext = re.sub(r'^\* `(\#[0-9]*).*?`__', r'* \1', mdtext, flags=re.M)
- with open(str(tmp_target), 'w') as ftarget:
- ftarget.write(mdtext)
+ cmd = f'gpg --clearsign --armor --default_key {options.gpg_key}'
+ else:
+ cmd = 'gpg --clearsign --armor'
+
+ sh(cmd + f' --output {rst_readme}.gpg {rst_readme}')
+ sh(cmd + f' --output {md_readme}.gpg {md_readme}')
@task
diff --git a/pyproject.toml b/pyproject.toml
index 918cbb278..d81b731d3 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -3,7 +3,7 @@
requires = [
"setuptools",
"wheel",
- "Cython>=0.29.13", # Note: keep in sync with tools/cythonize.py
+ "Cython>=0.29.14", # Note: keep in sync with tools/cythonize.py
]
diff --git a/setup.py b/setup.py
index cbd183afe..e6e696c11 100755
--- a/setup.py
+++ b/setup.py
@@ -25,6 +25,7 @@ import os
import sys
import subprocess
import textwrap
+import sysconfig
if sys.version_info[:2] < (3, 5):
@@ -57,7 +58,7 @@ Operating System :: MacOS
MAJOR = 1
MINOR = 18
-MICRO = 0
+MICRO = 1
ISRELEASED = True
VERSION = '%d.%d.%d' % (MAJOR, MINOR, MICRO)
@@ -228,6 +229,40 @@ class sdist_checked(sdist):
sdist.run(self)
+def get_build_overrides():
+ """
+ Custom build commands to add `-std=c99` to compilation
+ """
+ from numpy.distutils.command.build_clib import build_clib
+ from numpy.distutils.command.build_ext import build_ext
+
+ def _is_using_gcc(obj):
+ is_gcc = False
+ if obj.compiler.compiler_type == 'unix':
+ cc = sysconfig.get_config_var("CC")
+ if not cc:
+ cc = ""
+ compiler_name = os.path.basename(cc)
+ is_gcc = "gcc" in compiler_name
+ return is_gcc
+
+ class new_build_clib(build_clib):
+ def build_a_library(self, build_info, lib_name, libraries):
+ if _is_using_gcc(self):
+ args = build_info.get('extra_compiler_args') or []
+ args.append('-std=c99')
+ build_info['extra_compiler_args'] = args
+ build_clib.build_a_library(self, build_info, lib_name, libraries)
+
+ class new_build_ext(build_ext):
+ def build_extension(self, ext):
+ if _is_using_gcc(self):
+ if '-std=c99' not in ext.extra_compile_args:
+ ext.extra_compile_args.append('-std=c99')
+ build_ext.build_extension(self, ext)
+ return new_build_clib, new_build_ext
+
+
def generate_cython():
cwd = os.path.abspath(os.path.dirname(__file__))
print("Cythonizing sources")
@@ -390,6 +425,8 @@ def setup_package():
'f2py%s.%s = numpy.f2py.f2py2e:main' % sys.version_info[:2],
]
+ cmdclass={"sdist": sdist_checked,
+ }
metadata = dict(
name = 'numpy',
maintainer = "NumPy Developers",
@@ -408,8 +445,7 @@ def setup_package():
classifiers=[_f for _f in CLASSIFIERS.split('\n') if _f],
platforms = ["Windows", "Linux", "Solaris", "Mac OS-X", "Unix"],
test_suite='nose.collector',
- cmdclass={"sdist": sdist_checked,
- },
+ cmdclass=cmdclass,
python_requires='>=3.5',
zip_safe=False,
entry_points={
@@ -433,6 +469,8 @@ def setup_package():
generate_cython()
metadata['configuration'] = configuration
+ # Customize extension building
+ cmdclass['build_clib'], cmdclass['build_ext'] = get_build_overrides()
else:
# Version number is added to metadata inside configuration() if build
# is run.
diff --git a/shippable.yml b/shippable.yml
index b5ce17751..4313a6de2 100644
--- a/shippable.yml
+++ b/shippable.yml
@@ -25,31 +25,32 @@ build:
- sudo apt-get update
- sudo apt-get install gcc gfortran
- target=$(python tools/openblas_support.py)
- - sudo cp -r "${target}"/64/lib/* /usr/lib
- - sudo cp "${target}"/64/include/* /usr/include
- - pip install --upgrade pip
+ - ls -lR "${target}"
+ - sudo cp -r "${target}"/lib/* /usr/lib
+ - sudo cp "${target}"/include/* /usr/include
+ - python -m pip install --upgrade pip
# we will pay the ~13 minute cost of compiling Cython only when a new
# version is scraped in by pip; otherwise, use the cached
# wheel shippable places on Amazon S3 after we build it once
- - pip install -r test_requirements.txt --cache-dir=/root/.cache/pip/wheels/$SHIPPABLE_PYTHON_VERSION
+ - python -m pip install -r test_requirements.txt --cache-dir=/root/.cache/pip/wheels/$SHIPPABLE_PYTHON_VERSION
# install pytest-xdist to leverage a second core
# for unit tests
- - pip install pytest-xdist
+ - python -m pip install pytest-xdist
# build and test numpy
- export PATH=$PATH:$SHIPPABLE_REPO_DIR
# build first and adjust PATH so f2py is found in scripts dir
# use > 1 core for build sometimes slows down a fair bit,
# other times modestly speeds up, so avoid for now
- - pip install .
+ - python -m pip install .
- extra_directories=($SHIPPABLE_REPO_DIR/build/*scripts*)
- extra_path=$(printf "%s:" "${extra_directories[@]}")
- export PATH="${extra_path}${PATH}"
# check OpenBLAS version
- python tools/openblas_support.py --check_version 0.3.7
# run the test suite
- - python runtests.py --debug-info --show-build-log -- -rsx --junit-xml=$SHIPPABLE_REPO_DIR/shippable/testresults/tests.xml -n 2 --durations=10
+ - python runtests.py -n --debug-info --show-build-log -- -rsx --junit-xml=$SHIPPABLE_REPO_DIR/shippable/testresults/tests.xml -n 2 --durations=10
cache: true
cache_dir_list:
diff --git a/tools/cythonize.py b/tools/cythonize.py
index 5bea2d4ec..e5352a954 100755
--- a/tools/cythonize.py
+++ b/tools/cythonize.py
@@ -68,11 +68,11 @@ def process_pyx(fromfile, tofile):
# check the version, and invoke through python
from distutils.version import LooseVersion
- # Cython 0.29.13 is required for Python 3.8 and there are
+ # Cython 0.29.14 is required for Python 3.8 and there are
# other fixes in the 0.29 series that are needed even for earlier
# Python versions.
# Note: keep in sync with that in pyproject.toml
- required_version = LooseVersion('0.29.13')
+ required_version = LooseVersion('0.29.14')
if LooseVersion(cython_version) < required_version:
raise RuntimeError('Building {} requires Cython >= {}'.format(
diff --git a/tools/openblas_support.py b/tools/openblas_support.py
index 964adce6e..4a210cfe1 100644
--- a/tools/openblas_support.py
+++ b/tools/openblas_support.py
@@ -1,6 +1,8 @@
from __future__ import division, absolute_import, print_function
import os
import sys
+import glob
+import shutil
import textwrap
import platform
try:
@@ -40,7 +42,15 @@ def get_arch():
assert ret in ARCHITECTURES
return ret
-def download_openblas(target, arch):
+def get_ilp64():
+ if os.environ.get("NPY_USE_BLAS_ILP64", "0") == "0":
+ return None
+ if IS_32BIT:
+ raise RuntimeError("NPY_USE_BLAS_ILP64 set on 32-bit arch")
+ return "64_"
+
+def download_openblas(target, arch, ilp64):
+ fnsuffix = {None: "", "64_": "64_"}[ilp64]
filename = ''
if arch == 'arm':
# ARMv8 OpenBLAS built using script available here:
@@ -48,7 +58,7 @@ def download_openblas(target, arch):
# build done on GCC compile farm machine named gcc115
# tarball uploaded manually to an unshared Dropbox location
filename = ('https://www.dropbox.com/s/vdeckao4omss187/'
- 'openblas-{}-armv8.tar.gz?dl=1'.format(OPENBLAS_V))
+ 'openblas{}-{}-armv8.tar.gz?dl=1'.format(fnsuffix, OPENBLAS_V))
typ = 'tar.gz'
elif arch == 'ppc64':
# build script for POWER8 OpenBLAS available here:
@@ -56,28 +66,29 @@ def download_openblas(target, arch):
# built on GCC compile farm machine named gcc112
# manually uploaded tarball to an unshared Dropbox location
filename = ('https://www.dropbox.com/s/yt0d2j86x1j8nh1/'
- 'openblas-{}-ppc64le-power8.tar.gz?dl=1'.format(OPENBLAS_V))
+ 'openblas{}-{}-ppc64le-power8.tar.gz?dl=1'.format(fnsuffix, OPENBLAS_V))
typ = 'tar.gz'
elif arch == 'darwin':
- filename = '{0}/openblas-{1}-macosx_10_9_x86_64-gf_1becaaa.tar.gz'.format(
- RACKSPACE, OPENBLAS_LONG)
+ filename = '{0}/openblas{1}-{2}-macosx_10_9_x86_64-gf_1becaaa.tar.gz'.format(
+ RACKSPACE, fnsuffix, OPENBLAS_LONG)
typ = 'tar.gz'
elif arch == 'windows':
if IS_32BIT:
suffix = 'win32-gcc_7_1_0.zip'
else:
suffix = 'win_amd64-gcc_7_1_0.zip'
- filename = '{0}/openblas-{1}-{2}'.format(RACKSPACE, OPENBLAS_LONG, suffix)
+ filename = '{0}/openblas{1}-{2}-{3}'.format(RACKSPACE, fnsuffix, OPENBLAS_LONG, suffix)
typ = 'zip'
elif arch == 'x86':
if IS_32BIT:
suffix = 'manylinux1_i686.tar.gz'
else:
suffix = 'manylinux1_x86_64.tar.gz'
- filename = '{0}/openblas-{1}-{2}'.format(RACKSPACE, OPENBLAS_LONG, suffix)
+ filename = '{0}/openblas{1}-{2}-{3}'.format(RACKSPACE, fnsuffix, OPENBLAS_LONG, suffix)
typ = 'tar.gz'
if not filename:
return None
+ print("Downloading:", filename, file=sys.stderr)
try:
with open(target, 'wb') as fid:
fid.write(urlopen(filename).read())
@@ -86,7 +97,7 @@ def download_openblas(target, arch):
return None
return typ
-def setup_openblas(arch=get_arch()):
+def setup_openblas(arch=get_arch(), ilp64=get_ilp64()):
'''
Download and setup an openblas library for building. If successful,
the configuration script will find it automatically.
@@ -100,7 +111,7 @@ def setup_openblas(arch=get_arch()):
_, tmp = mkstemp()
if not arch:
raise ValueError('unknown architecture')
- typ = download_openblas(tmp, arch)
+ typ = download_openblas(tmp, arch, ilp64)
if not typ:
return ''
if arch == 'windows':
@@ -132,10 +143,33 @@ def unpack_targz(fname):
if not os.path.exists(target):
os.mkdir(target)
with tarfile.open(fname, 'r') as zf:
- # TODO: check that all the zf.getnames() files do not escape the
- # extract directory (no leading '../', '/')
- zf.extractall(target)
- return target
+ # Strip common prefix from paths when unpacking
+ prefix = os.path.commonpath(zf.getnames())
+ extract_tarfile_to(zf, target, prefix)
+ return target
+
+def extract_tarfile_to(tarfileobj, target_path, archive_path):
+ """Extract TarFile contents under archive_path/ to target_path/"""
+
+ target_path = os.path.abspath(target_path)
+
+ def get_members():
+ for member in tarfileobj.getmembers():
+ if archive_path:
+ norm_path = os.path.normpath(member.name)
+ if norm_path.startswith(archive_path + os.path.sep):
+ member.name = norm_path[len(archive_path)+1:]
+ else:
+ continue
+
+ dst_path = os.path.abspath(os.path.join(target_path, member.name))
+ if os.path.commonpath([target_path, dst_path]) != target_path:
+ # Path not under target_path, probably contains ../
+ continue
+
+ yield member
+
+ tarfileobj.extractall(target_path, members=get_members())
def make_init(dirname):
'''
@@ -180,19 +214,41 @@ def test_setup(arches):
'''
Make sure all the downloadable files exist and can be opened
'''
- for arch in arches:
+ def items():
+ for arch in arches:
+ yield arch, None
+ if arch in ('x86', 'darwin', 'windows'):
+ yield arch, '64_'
+
+ for arch, ilp64 in items():
if arch == '':
continue
+
+ target = None
try:
- target = setup_openblas(arch)
- except:
- print('Could not setup %s' % arch)
- raise
- if not target:
- raise RuntimeError('Could not setup %s' % arch)
- print(target)
-
-def test_version(expected_version):
+ try:
+ target = setup_openblas(arch, ilp64)
+ except:
+ print('Could not setup %s' % arch)
+ raise
+ if not target:
+ raise RuntimeError('Could not setup %s' % arch)
+ print(target)
+ if arch == 'windows':
+ if not target.endswith('.a'):
+ raise RuntimeError("Not .a extracted!")
+ else:
+ files = glob.glob(os.path.join(target, "lib", "*.a"))
+ if not files:
+ raise RuntimeError("No lib/*.a unpacked!")
+ finally:
+ if target is not None:
+ if os.path.isfile(target):
+ os.unlink(target)
+ else:
+ shutil.rmtree(target)
+
+def test_version(expected_version, ilp64=get_ilp64()):
"""
Assert that expected OpenBLAS version is
actually available via NumPy
@@ -201,12 +257,19 @@ def test_version(expected_version):
import ctypes
dll = ctypes.CDLL(numpy.core._multiarray_umath.__file__)
- get_config = dll.openblas_get_config
+ if ilp64 == "64_":
+ get_config = dll.openblas_get_config64_
+ else:
+ get_config = dll.openblas_get_config
get_config.restype=ctypes.c_char_p
res = get_config()
print('OpenBLAS get_config returned', str(res))
check_str = b'OpenBLAS %s' % expected_version[0].encode()
assert check_str in res
+ if ilp64:
+ assert b"USE64BITINT" in res
+ else:
+ assert b"USE64BITINT" not in res
if __name__ == '__main__':
import argparse
diff --git a/tools/pypy-test.sh b/tools/pypy-test.sh
index f4d56ba1a..33a97ad17 100755
--- a/tools/pypy-test.sh
+++ b/tools/pypy-test.sh
@@ -11,19 +11,20 @@ sudo apt-get -yq install libatlas-base-dev liblapack-dev gfortran-5
F77=gfortran-5 F90=gfortran-5 \
# Download the proper OpenBLAS x64 precompiled library
-target=$(python tools/openblas_support.py)
+target=$(python3 tools/openblas_support.py)
+ls -lR "$target"
echo getting OpenBLAS into $target
-export LD_LIBRARY_PATH=$target/usr/local/lib
-export LIB=$target/usr/local/lib
-export INCLUDE=$target/usr/local/include
+export LD_LIBRARY_PATH=$target/lib
+export LIB=$target/lib
+export INCLUDE=$target/include
# Use a site.cfg to build with local openblas
cat << EOF > site.cfg
[openblas]
libraries = openblas
-library_dirs = $target/usr/local/lib:$LIB
-include_dirs = $target/usr/local/lib:$LIB
-runtime_library_dirs = $target/usr/local/lib
+library_dirs = $target/lib:$LIB
+include_dirs = $target/lib:$LIB
+runtime_library_dirs = $target/lib
EOF
echo getting PyPy 3.6 nightly
diff --git a/tools/travis-before-install.sh b/tools/travis-before-install.sh
index 072ad3bf6..9f8b66a47 100755
--- a/tools/travis-before-install.sh
+++ b/tools/travis-before-install.sh
@@ -5,12 +5,12 @@ free -m
df -h
ulimit -a
-if [ -n "$PPC64_LE" ]; then
+if [ -n "$DOWNLOAD_OPENBLAS" ]; then
pwd
ls -ltrh
target=$(python tools/openblas_support.py)
- sudo cp -r $target/64/lib/* /usr/lib
- sudo cp $target/64/include/* /usr/include
+ sudo cp -r $target/lib/* /usr/lib
+ sudo cp $target/include/* /usr/include
fi
mkdir builds