diff options
| author | Chanho Park <chanho61.park@samsung.com> | 2014-08-19 19:35:08 +0900 |
|---|---|---|
| committer | Chanho Park <chanho61.park@samsung.com> | 2014-08-19 19:35:08 +0900 |
| commit | f153198bebe97a3d7576aefcec4841bc04b45ddc (patch) | |
| tree | fdc7119511266a613744d336ee58a2564f008d28 /Doc | |
| parent | bed873b4f7b3c21d12bf296eb3a3dbea1bc1dda0 (diff) | |
| download | python-f153198bebe97a3d7576aefcec4841bc04b45ddc.tar.gz python-f153198bebe97a3d7576aefcec4841bc04b45ddc.tar.bz2 python-f153198bebe97a3d7576aefcec4841bc04b45ddc.zip | |
Imported Upstream version 2.7.8upstream/2.7.8
Diffstat (limited to 'Doc')
336 files changed, 9815 insertions, 6354 deletions
diff --git a/Doc/ACKS.txt b/Doc/ACKS.txt deleted file mode 100644 index 7507c6c..0000000 --- a/Doc/ACKS.txt +++ /dev/null @@ -1,230 +0,0 @@ -Contributors to the Python Documentation ----------------------------------------- - -This section lists people who have contributed in some way to the Python -documentation. It is probably not complete -- if you feel that you or -anyone else should be on this list, please let us know (send email to -docs@python.org), and we'll be glad to correct the problem. - -.. acks:: - - * Aahz - * Michael Abbott - * Steve Alexander - * Jim Ahlstrom - * Fred Allen - * A. Amoroso - * Pehr Anderson - * Oliver Andrich - * Heidi Annexstad - * Jesús Cea Avión - * Manuel Balsera - * Daniel Barclay - * Chris Barker - * Don Bashford - * Anthony Baxter - * Alexander Belopolsky - * Bennett Benson - * Jonathan Black - * Robin Boerdijk - * Michal Bozon - * Aaron Brancotti - * Georg Brandl - * Keith Briggs - * Ian Bruntlett - * Lee Busby - * Arnaud Calmettes - * Lorenzo M. Catucci - * Carl Cerecke - * Mauro Cicognini - * Gilles Civario - * Mike Clarkson - * Steve Clift - * Dave Cole - * Matthew Cowles - * Jeremy Craven - * Andrew Dalke - * Ben Darnell - * L. Peter Deutsch - * Robert Donohue - * Fred L. Drake, Jr. - * Josip Dzolonga - * Jeff Epler - * Michael Ernst - * Blame Andy Eskilsson - * Carey Evans - * Martijn Faassen - * Carl Feynman - * Dan Finnie - * Hernán MartÃnez Foffani - * Stefan Franke - * Jim Fulton - * Peter Funk - * Lele Gaifax - * Matthew Gallagher - * Gabriel Genellina - * Ben Gertzfield - * Nadim Ghaznavi - * Jonathan Giddy - * Shelley Gooch - * Nathaniel Gray - * Grant Griffin - * Thomas Guettler - * Anders Hammarquist - * Mark Hammond - * Harald Hanche-Olsen - * Manus Hand - * Gerhard Häring - * Travis B. Hartwell - * Tim Hatch - * Janko Hauser - * Ben Hayden - * Thomas Heller - * Bernhard Herzog - * Magnus L. Hetland - * Konrad Hinsen - * Stefan Hoffmeister - * Albert Hofkamp - * Gregor Hoffleit - * Steve Holden - * Thomas Holenstein - * Gerrit Holl - * Rob Hooft - * Brian Hooper - * Randall Hopper - * Michael Hudson - * Eric Huss - * Jeremy Hylton - * Roger Irwin - * Jack Jansen - * Philip H. Jensen - * Pedro Diaz Jimenez - * Kent Johnson - * Lucas de Jonge - * Andreas Jung - * Robert Kern - * Jim Kerr - * Jan Kim - * Kamil Kisiel - * Greg Kochanski - * Guido Kollerie - * Peter A. Koren - * Daniel Kozan - * Andrew M. Kuchling - * Dave Kuhlman - * Erno Kuusela - * Ross Lagerwall - * Thomas Lamb - * Detlef Lannert - * Piers Lauder - * Glyph Lefkowitz - * Robert Lehmann - * Marc-André Lemburg - * Ross Light - * Ulf A. Lindgren - * Everett Lipman - * Mirko Liss - * Martin von Löwis - * Fredrik Lundh - * Jeff MacDonald - * John Machin - * Andrew MacIntyre - * Vladimir Marangozov - * Vincent Marchetti - * Westley MartÃnez - * Laura Matson - * Daniel May - * Rebecca McCreary - * Doug Mennella - * Paolo Milani - * Skip Montanaro - * Paul Moore - * Ross Moore - * Sjoerd Mullender - * Dale Nagata - * Michal Nowikowski - * Steffen Daode Nurpmeso - * Ng Pheng Siong - * Koray Oner - * Tomas Oppelstrup - * Denis S. Otkidach - * Zooko O'Whielacronx - * Shriphani Palakodety - * William Park - * Joonas Paalasmaa - * Harri Pasanen - * Bo Peng - * Tim Peters - * Benjamin Peterson - * Christopher Petrilli - * Justin D. Pettit - * Chris Phoenix - * François Pinard - * Paul Prescod - * Eric S. Raymond - * Edward K. Ream - * Terry J. Reedy - * Sean Reifschneider - * Bernhard Reiter - * Armin Rigo - * Wes Rishel - * Armin Ronacher - * Jim Roskind - * Guido van Rossum - * Donald Wallace Rouse II - * Mark Russell - * Nick Russo - * Chris Ryland - * Constantina S. - * Hugh Sasse - * Bob Savage - * Scott Schram - * Neil Schemenauer - * Barry Scott - * Joakim Sernbrant - * Justin Sheehy - * Charlie Shepherd - * Yue Shuaijie - * Michael Simcich - * Ionel Simionescu - * Michael Sloan - * Gregory P. Smith - * Roy Smith - * Clay Spence - * Nicholas Spies - * Tage Stabell-Kulo - * Frank Stajano - * Anthony Starks - * Greg Stein - * Peter Stoehr - * Mark Summerfield - * Reuben Sumner - * Kalle Svensson - * Jim Tittsler - * David Turner - * Sandro Tosi - * Ville Vainio - * Martijn Vries - * Charles G. Waldman - * Greg Ward - * Barry Warsaw - * Corran Webster - * Glyn Webster - * Bob Weiner - * Eddy Welbourne - * Jeff Wheeler - * Mats Wichmann - * Gerry Wiener - * Timothy Wild - * Paul Winkler - * Collin Winter - * Blake Winton - * Dan Wolfe - * Adam Woodbeck - * Steven Work - * Thomas Wouters - * Ka-Ping Yee - * Rory Yorke - * Moshe Zadka - * Milan Zamazal - * Cheng Zhang diff --git a/Doc/Makefile b/Doc/Makefile index 6e694c6..afdf35f 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -169,12 +169,19 @@ serve: # for development releases: always build autobuild-dev: make update - make dist SPHINXOPTS='-A daily=1' + make dist SPHINXOPTS='-A daily=1 -A versionswitcher=1' + -make suspicious -# for stable releases: only build if not in pre-release stage (alpha, beta, rc) +# for quick rebuilds (HTML only) +autobuild-html: + make html SPHINXOPTS='-A daily=1 -A versionswitcher=1' + +# for stable releases: only build if not in pre-release stage (alpha, beta) +# release candidate downloads are okay, since the stable tree can be in that stage autobuild-stable: - @case $(DISTVERSION) in *[abc]*) \ + @case $(DISTVERSION) in *[ab]*) \ echo "Not building; $(DISTVERSION) is not a release version."; \ exit 1;; \ esac @make autobuild-dev + diff --git a/Doc/README.txt b/Doc/README.txt index f03da76..b8ff47f 100644 --- a/Doc/README.txt +++ b/Doc/README.txt @@ -3,18 +3,17 @@ Python Documentation README This directory contains the reStructuredText (reST) sources to the Python documentation. You don't need to build them yourself, prebuilt versions are -available at http://docs.python.org/download/. +available at https://docs.python.org/2/download.html Documentation on the authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the -documentation. There's also a chapter intended to point out differences to -those familiar with the previous docs written in LaTeX. +documentation. Building the docs ================= -You need to have Python 2.4 or higher installed; the toolset used to build the +You need to have Python 2 installed; the toolset used to build the docs is written in Python. It is called *Sphinx*, it is not included in this tree, but maintained separately. Also needed are the docutils, supplying the base markup that Sphinx uses, Jinja, a templating engine, and optionally @@ -33,6 +32,9 @@ to check out the necessary toolset in the `tools/` subdirectory and build the HTML output files. To view the generated HTML, point your favorite browser at the top-level index `build/html/index.html` after running "make". +On Windows, we try to emulate the Makefile as closely as possible with a +``make.bat`` file. + Available make targets are: * "html", which builds standalone HTML files for offline viewing. @@ -65,43 +67,23 @@ Available make targets are: `tools/sphinxext/pyspecific.py` -- pydoc needs these to show topic and keyword help. + * "suspicious", which checks the parsed markup for text that looks like + malformed and thus unconverted reST. + A "make update" updates the Subversion checkouts in `tools/`. Without make ------------ -You'll need to install the Sphinx package, either by checking it out via :: - - svn co http://svn.python.org/projects/external/Sphinx-0.6.7/sphinx tools/sphinx - -or by installing it from PyPI. - -Then, you need to install Docutils, either by checking it out via :: - - svn co http://svn.python.org/projects/external/docutils-0.6/docutils tools/docutils - -or by installing it from http://docutils.sf.net/. - -You also need Jinja2, either by checking it out via :: - - svn co http://svn.python.org/projects/external/Jinja-2.3.1/jinja2 tools/jinja2 - -or by installing it from PyPI. - -You can optionally also install Pygments, either as a checkout via :: - - svn co http://svn.python.org/projects/external/Pygments-1.3.1/pygments tools/pygments - -or from PyPI at http://pypi.python.org/pypi/Pygments. - +Install the Sphinx package and its dependencies from PyPI. -Then, make an output directory, e.g. under `build/`, and run :: +Then, from the ``Docs`` directory, run :: - python tools/sphinx-build.py -b<builder> . build/<outputdirectory> + sphinx-build -b<builder> . build/<builder> -where `<builder>` is one of html, text, latex, or htmlhelp (for explanations see -the make targets above). +where ``<builder>`` is one of html, text, latex, or htmlhelp (for explanations +see the make targets above). Contributing @@ -127,7 +109,7 @@ The Python source is copyrighted, but you can freely use and copy it as long as you don't change or remove the copyright notice: ---------------------------------------------------------------------- -Copyright (c) 2000-2012 Python Software Foundation. +Copyright (c) 2000-2014 Python Software Foundation. All rights reserved. Copyright (c) 2000 BeOpen.com. diff --git a/Doc/about.rst b/Doc/about.rst index 2c229e6..678168b 100644 --- a/Doc/about.rst +++ b/Doc/about.rst @@ -7,14 +7,15 @@ These documents are generated from `reStructuredText`_ sources by `Sphinx`_, a document processor specifically written for the Python documentation. .. _reStructuredText: http://docutils.sf.net/rst.html -.. _Sphinx: http://sphinx.pocoo.org/ +.. _Sphinx: http://sphinx-doc.org/ .. In the online version of these documents, you can submit comments and suggest changes directly on the documentation pages. -Development of the documentation and its toolchain takes place on the -docs@python.org mailing list. We're always looking for volunteers wanting -to help with the docs, so feel free to send a mail there! +Development of the documentation and its toolchain is an entirely volunteer +effort, just like Python itself. If you want to contribute, please take a +look at the :ref:`reporting-bugs` page for information on how to do so. New +volunteers are always welcome! Many thanks go to: @@ -26,11 +27,13 @@ Many thanks go to: <http://effbot.org/zone/pyref.htm>`_ project from which Sphinx got many good ideas. -See :ref:`reporting-bugs` for information how to report bugs in this -documentation, or Python itself. -.. including the ACKS file here so that it can be maintained separately -.. include:: ACKS.txt +Contributors to the Python Documentation +---------------------------------------- + +Many people have contributed to the Python language, the Python standard +library, and the Python documentation. See :source:`Misc/ACKS` in the Python +source distribution for a partial list of contributors. It is only with the input and contributions of the Python community that Python has such wonderful documentation -- Thank You! diff --git a/Doc/bugs.rst b/Doc/bugs.rst index 3785ccb..847c010 100644 --- a/Doc/bugs.rst +++ b/Doc/bugs.rst @@ -13,15 +13,17 @@ Documentation bugs ================== If you find a bug in this documentation or would like to propose an improvement, -please send an e-mail to docs@python.org describing the bug and where you found -it. If you have a suggestion how to fix it, include that as well. +please submit a bug report on the :ref:`tracker <using-the-tracker>`. If you +have a suggestion how to fix it, include that as well. -docs@python.org is a mailing list run by volunteers; your request will be -noticed, even if it takes a while to be processed. +If you're short on time, you can also email your bug report to docs@python.org. +'docs@' is a mailing list run by volunteers; your request will be noticed, +though it may take a while to be processed. -Of course, if you want a more persistent record of your issue, you can use the -issue tracker for documentation bugs as well. +.. seealso:: + `Documentation bugs`_ on the Python issue tracker +.. _using-the-tracker: Using the Python issue tracker ============================== @@ -62,9 +64,6 @@ taken on the bug. .. seealso:: - `Python Developer's Guide <http://docs.python.org/devguide/>`_ - Detailed description of the issue workflow and developers tools. - `How to Report Bugs Effectively <http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>`_ Article which goes into some detail about how to create a useful bug report. This describes what kind of information is useful and why it is useful. @@ -73,3 +72,16 @@ taken on the bug. Information about writing a good bug report. Some of this is specific to the Mozilla project, but describes general good practices. + +Getting started contributing to Python yourself +=============================================== + +Beyond just reporting bugs that you find, you are also welcome to submit +patches to fix them. You can find more information on how to get started +patching Python in the `Python Developer's Guide`_. If you have questions, +the `core-mentorship mailing list`_ is a friendly place to get answers to +any and all questions pertaining to the process of fixing issues in Python. + +.. _Documentation bugs: http://bugs.python.org/issue?@filter=status&@filter=components&components=4&status=1&@columns=id,activity,title,status&@sort=-activity +.. _Python Developer's Guide: http://docs.python.org/devguide/ +.. _core-mentorship mailing list: https://mail.python.org/mailman/listinfo/core-mentorship/ diff --git a/Doc/c-api/allocation.rst b/Doc/c-api/allocation.rst index cb43cbf..32a414b 100644 --- a/Doc/c-api/allocation.rst +++ b/Doc/c-api/allocation.rst @@ -43,7 +43,7 @@ Allocating Objects on the Heap Allocate a new Python object using the C structure type *TYPE* and the Python type object *type*. Fields not defined by the Python object header are not initialized; the object's reference count will be one. The size of - the memory allocation is determined from the :attr:`tp_basicsize` field of + the memory allocation is determined from the :c:member:`~PyTypeObject.tp_basicsize` field of the type object. @@ -52,7 +52,7 @@ Allocating Objects on the Heap Allocate a new Python object using the C structure type *TYPE* and the Python type object *type*. Fields not defined by the Python object header are not initialized. The allocated memory allows for the *TYPE* structure - plus *size* fields of the size given by the :attr:`tp_itemsize` field of + plus *size* fields of the size given by the :c:member:`~PyTypeObject.tp_itemsize` field of *type*. This is useful for implementing objects like tuples, which are able to determine their size at construction time. Embedding the array of fields into the same allocation decreases the number of allocations, @@ -67,7 +67,7 @@ Allocating Objects on the Heap Releases memory allocated to an object using :c:func:`PyObject_New` or :c:func:`PyObject_NewVar`. This is normally called from the - :attr:`tp_dealloc` handler specified in the object's type. The fields of + :c:member:`~PyTypeObject.tp_dealloc` handler specified in the object's type. The fields of the object should not be accessed after this call as the memory is no longer a valid Python object. diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 7b6d1ae..74693ac 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -21,8 +21,10 @@ first. Two examples of objects that support the buffer interface are strings and arrays. The string object exposes the character contents in the buffer -interface's byte-oriented form. An array can also expose its contents, but it -should be noted that array elements may be multi-byte values. +interface's byte-oriented form. An array can only expose its contents via the +old-style buffer interface. This limitation does not apply to Python 3, +where :class:`memoryview` objects can be constructed from arrays, too. +Array elements may be multi-byte values. An example user of the buffer interface is the file object's :meth:`write` method. Any object that can export a series of bytes through the buffer @@ -33,7 +35,7 @@ returning data from the target object. Starting from version 1.6, Python has been providing Python-level buffer objects and a C-level buffer API so that any built-in or used-defined type can expose its characteristics. Both, however, have been deprecated because of -various shortcomings, and have been officially removed in Python 3.0 in favour +various shortcomings, and have been officially removed in Python 3 in favour of a new C-level buffer API and a new Python-level object named :class:`memoryview`. diff --git a/Doc/c-api/codec.rst b/Doc/c-api/codec.rst index 8207ae0..83252af 100644 --- a/Doc/c-api/codec.rst +++ b/Doc/c-api/codec.rst @@ -52,19 +52,19 @@ and *NULL* returned. .. c:function:: PyObject* PyCodec_IncrementalEncoder(const char *encoding, const char *errors) - Get an :class:`IncrementalEncoder` object for the given *encoding*. + Get an :class:`~codecs.IncrementalEncoder` object for the given *encoding*. .. c:function:: PyObject* PyCodec_IncrementalDecoder(const char *encoding, const char *errors) - Get an :class:`IncrementalDecoder` object for the given *encoding*. + Get an :class:`~codecs.IncrementalDecoder` object for the given *encoding*. .. c:function:: PyObject* PyCodec_StreamReader(const char *encoding, PyObject *stream, const char *errors) - Get a :class:`StreamReader` factory function for the given *encoding*. + Get a :class:`~codecs.StreamReader` factory function for the given *encoding*. .. c:function:: PyObject* PyCodec_StreamWriter(const char *encoding, PyObject *stream, const char *errors) - Get a :class:`StreamWriter` factory function for the given *encoding*. + Get a :class:`~codecs.StreamWriter` factory function for the given *encoding*. Registry API for Unicode encoding error handlers diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst index 3e967bd..3006b6c 100644 --- a/Doc/c-api/dict.rst +++ b/Doc/c-api/dict.rst @@ -210,8 +210,11 @@ Dictionary Objects .. c:function:: int PyDict_Update(PyObject *a, PyObject *b) - This is the same as ``PyDict_Merge(a, b, 1)`` in C, or ``a.update(b)`` in - Python. Return ``0`` on success or ``-1`` if an exception was raised. + This is the same as ``PyDict_Merge(a, b, 1)`` in C, and is similar to + ``a.update(b)`` in Python except that :c:func:`PyDict_Update` doesn't fall + back to the iterating over a sequence of key value pairs if the second + argument has no "keys" attribute. Return ``0`` on success or ``-1`` if an + exception was raised. .. versionadded:: 2.2 diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 025b75a..91964d0 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -192,12 +192,19 @@ is a separate error indicator for each thread. when the system call returns an error. -.. c:function:: PyObject* PyErr_SetFromErrnoWithFilename(PyObject *type, const char *filename) +.. c:function:: PyObject* PyErr_SetFromErrnoWithFilenameObject(PyObject *type, PyObject *filenameObject) Similar to :c:func:`PyErr_SetFromErrno`, with the additional behavior that if - *filename* is not *NULL*, it is passed to the constructor of *type* as a third - parameter. In the case of exceptions such as :exc:`IOError` and :exc:`OSError`, - this is used to define the :attr:`filename` attribute of the exception instance. + *filenameObject* is not *NULL*, it is passed to the constructor of *type* as + a third parameter. In the case of exceptions such as :exc:`IOError` and + :exc:`OSError`, this is used to define the :attr:`filename` attribute of the + exception instance. + + +.. c:function:: PyObject* PyErr_SetFromErrnoWithFilename(PyObject *type, const char *filename) + + Similar to :c:func:`PyErr_SetFromErrnoWithFilenameObject`, but the filename + is given as a C string. .. c:function:: PyObject* PyErr_SetFromWindowsErr(int ierr) @@ -220,14 +227,29 @@ is a separate error indicator for each thread. .. versionadded:: 2.3 -.. c:function:: PyObject* PyErr_SetFromWindowsErrWithFilename(int ierr, const char *filename) +.. c:function:: PyObject* PyErr_SetFromWindowsErrWithFilenameObject(int ierr, PyObject *filenameObject) Similar to :c:func:`PyErr_SetFromWindowsErr`, with the additional behavior that - if *filename* is not *NULL*, it is passed to the constructor of + if *filenameObject* is not *NULL*, it is passed to the constructor of :exc:`WindowsError` as a third parameter. Availability: Windows. -.. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilename(PyObject *type, int ierr, char *filename) +.. c:function:: PyObject* PyErr_SetFromWindowsErrWithFilename(int ierr, const char *filename) + + Similar to :c:func:`PyErr_SetFromWindowsErrWithFilenameObject`, but the + filename is given as a C string. Availability: Windows. + + +.. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilenameObject(PyObject *type, int ierr, PyObject *filename) + + Similar to :c:func:`PyErr_SetFromWindowsErrWithFilenameObject`, with an + additional parameter specifying the exception type to be raised. + Availability: Windows. + + .. versionadded:: 2.3 + + +.. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilename(PyObject *type, int ierr, const char *filename) Similar to :c:func:`PyErr_SetFromWindowsErrWithFilename`, with an additional parameter specifying the exception type to be raised. Availability: Windows. diff --git a/Doc/c-api/file.rst b/Doc/c-api/file.rst index 20a7a60..bbd4223 100644 --- a/Doc/c-api/file.rst +++ b/Doc/c-api/file.rst @@ -111,7 +111,8 @@ change in future releases of Python. .. index:: single: EOFError (built-in exception) Equivalent to ``p.readline([n])``, this function reads one line from the - object *p*. *p* may be a file object or any object with a :meth:`readline` + object *p*. *p* may be a file object or any object with a + :meth:`~io.IOBase.readline` method. If *n* is ``0``, exactly one line is read, regardless of the length of the line. If *n* is greater than ``0``, no more than *n* bytes will be read from the file; a partial line can be returned. In both cases, an empty string diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index 2a4fda4..b0a2d5c 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -15,10 +15,10 @@ collection. .. An example showing the use of these interfaces can be found in "Supporting the .. Cycle Collector (XXX not found: ../ext/example-cycle-support.html)". -To create a container type, the :attr:`tp_flags` field of the type object must +To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must include the :const:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the -:attr:`tp_traverse` handler. If instances of the type are mutable, a -:attr:`tp_clear` implementation must also be provided. +:c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a +:c:member:`~PyTypeObject.tp_clear` implementation must also be provided. .. data:: Py_TPFLAGS_HAVE_GC @@ -68,7 +68,7 @@ Constructors for container types must conform to two rules: Adds the object *op* to the set of container objects tracked by the collector. The collector can run at unexpected times so objects must be valid while being tracked. This should be called once all the fields - followed by the :attr:`tp_traverse` handler become valid, usually near the + followed by the :c:member:`~PyTypeObject.tp_traverse` handler become valid, usually near the end of the constructor. @@ -97,8 +97,8 @@ rules: Remove the object *op* from the set of container objects tracked by the collector. Note that :c:func:`PyObject_GC_Track` can be called again on this object to add it back to the set of tracked objects. The deallocator - (:attr:`tp_dealloc` handler) should call this for the object before any of - the fields used by the :attr:`tp_traverse` handler become invalid. + (:c:member:`~PyTypeObject.tp_dealloc` handler) should call this for the object before any of + the fields used by the :c:member:`~PyTypeObject.tp_traverse` handler become invalid. .. c:function:: void _PyObject_GC_UNTRACK(PyObject *op) @@ -106,19 +106,19 @@ rules: A macro version of :c:func:`PyObject_GC_UnTrack`. It should not be used for extension modules. -The :attr:`tp_traverse` handler accepts a function parameter of this type: +The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: .. c:type:: int (*visitproc)(PyObject *object, void *arg) - Type of the visitor function passed to the :attr:`tp_traverse` handler. + Type of the visitor function passed to the :c:member:`~PyTypeObject.tp_traverse` handler. The function should be called with an object to traverse as *object* and - the third parameter to the :attr:`tp_traverse` handler as *arg*. The + the third parameter to the :c:member:`~PyTypeObject.tp_traverse` handler as *arg*. The Python core uses several visitor functions to implement cyclic garbage detection; it's not expected that users will need to write their own visitor functions. -The :attr:`tp_traverse` handler must have the following type: +The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type: .. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg) @@ -130,15 +130,15 @@ The :attr:`tp_traverse` handler must have the following type: object argument. If *visit* returns a non-zero value that value should be returned immediately. -To simplify writing :attr:`tp_traverse` handlers, a :c:func:`Py_VISIT` macro is -provided. In order to use this macro, the :attr:`tp_traverse` implementation +To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers, a :c:func:`Py_VISIT` macro is +provided. In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse` implementation must name its arguments exactly *visit* and *arg*: .. c:function:: void Py_VISIT(PyObject *o) Call the *visit* callback, with arguments *o* and *arg*. If *visit* returns - a non-zero value, then return it. Using this macro, :attr:`tp_traverse` + a non-zero value, then return it. Using this macro, :c:member:`~PyTypeObject.tp_traverse` handlers look like:: static int @@ -151,7 +151,7 @@ must name its arguments exactly *visit* and *arg*: .. versionadded:: 2.4 -The :attr:`tp_clear` handler must be of the :c:type:`inquiry` type, or *NULL* +The :c:member:`~PyTypeObject.tp_clear` handler must be of the :c:type:`inquiry` type, or *NULL* if the object is immutable. diff --git a/Doc/c-api/index.rst b/Doc/c-api/index.rst index 12a1ec7..2ce7b98 100644 --- a/Doc/c-api/index.rst +++ b/Doc/c-api/index.rst @@ -4,9 +4,6 @@ Python/C API Reference Manual ################################## -:Release: |version| -:Date: |today| - This manual documents the API used by C and C++ programmers who want to write extension modules or embed Python. It is a companion to :ref:`extending-index`, which describes the general principles of extension writing but does not diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 6c58c5d..46fc93f 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -427,6 +427,9 @@ pointer. standard :mod:`zlib` and :mod:`hashlib` modules release the GIL when compressing or hashing data. + +.. _gilstate: + Non-Python created threads -------------------------- @@ -531,6 +534,7 @@ code, or when embedding the Python interpreter: .. index:: module: thread .. note:: + When only the main thread exists, no GIL operations are needed. This is a common situation (most Python programs do not use threads), and the lock operations slow the interpreter down a bit. Therefore, the lock is not @@ -905,41 +909,43 @@ Asynchronous Notifications A mechanism is provided to make asynchronous notifications to the main interpreter thread. These notifications take the form of a function -pointer and a void argument. - -.. index:: single: setcheckinterval() (in module sys) +pointer and a void pointer argument. -Every check interval, when the global interpreter lock is released and -reacquired, Python will also call any such provided functions. This can be used -for example by asynchronous IO handlers. The notification can be scheduled from -a worker thread and the actual call than made at the earliest convenience by the -main thread where it has possession of the global interpreter lock and can -perform any Python API calls. .. c:function:: int Py_AddPendingCall(int (*func)(void *), void *arg) .. index:: single: Py_AddPendingCall() - Post a notification to the Python main thread. If successful, *func* will be - called with the argument *arg* at the earliest convenience. *func* will be - called having the global interpreter lock held and can thus use the full - Python API and can take any action such as setting object attributes to - signal IO completion. It must return 0 on success, or -1 signalling an - exception. The notification function won't be interrupted to perform another - asynchronous notification recursively, but it can still be interrupted to - switch threads if the global interpreter lock is released, for example, if it - calls back into Python code. + Schedule a function to be called from the main interpreter thread. On + success, 0 is returned and *func* is queued for being called in the + main thread. On failure, -1 is returned without setting any exception. - This function returns 0 on success in which case the notification has been - scheduled. Otherwise, for example if the notification buffer is full, it - returns -1 without setting any exception. + When successfully queued, *func* will be *eventually* called from the + main interpreter thread with the argument *arg*. It will be called + asynchronously with respect to normally running Python code, but with + both these conditions met: - This function can be called on any thread, be it a Python thread or some - other system thread. If it is a Python thread, it doesn't matter if it holds - the global interpreter lock or not. + * on a :term:`bytecode` boundary; + * with the main thread holding the :term:`global interpreter lock` + (*func* can therefore use the full C API). - .. versionadded:: 2.7 + *func* must return 0 on success, or -1 on failure with an exception + set. *func* won't be interrupted to perform another asynchronous + notification recursively, but it can still be interrupted to switch + threads if the global interpreter lock is released. + + This function doesn't need a current thread state to run, and it doesn't + need the global interpreter lock. + .. warning:: + This is a low-level function, only useful for very special cases. + There is no guarantee that *func* will be called as quick as + possible. If the main thread is busy executing a system call, + *func* won't be called before the system call returns. This + function is generally **not** suitable for calling Python code from + arbitrary C threads. Instead, use the :ref:`PyGILState API<gilstate>`. + + .. versionadded:: 2.7 .. _profiling: diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst index 4216881..6414277 100644 --- a/Doc/c-api/intro.rst +++ b/Doc/c-api/intro.rst @@ -255,8 +255,10 @@ sets all items of a list (actually, any mutable sequence) to a given item:: PyObject *index = PyInt_FromLong(i); if (!index) return -1; - if (PyObject_SetItem(target, index, item) < 0) + if (PyObject_SetItem(target, index, item) < 0) { + Py_DECREF(index); return -1; + } Py_DECREF(index); } return 0; @@ -424,7 +426,7 @@ and lose important information about the exact cause of the error. .. index:: single: sum_sequence() A simple example of detecting exceptions and passing them on is shown in the -:c:func:`sum_sequence` example above. It so happens that that example doesn't +:c:func:`sum_sequence` example above. It so happens that this example doesn't need to clean up any owned references when it detects an error. The following example function shows some error cleanup. First, to remind you why you like Python, we show the equivalent Python code:: diff --git a/Doc/c-api/iter.rst b/Doc/c-api/iter.rst index 88ac0c1..8d1567c 100644 --- a/Doc/c-api/iter.rst +++ b/Doc/c-api/iter.rst @@ -7,7 +7,7 @@ Iterator Protocol .. versionadded:: 2.2 -There are only a couple of functions specifically for working with iterators. +There are two functions specifically for working with iterators. .. c:function:: int PyIter_Check(PyObject *o) @@ -17,11 +17,10 @@ There are only a couple of functions specifically for working with iterators. .. c:function:: PyObject* PyIter_Next(PyObject *o) - Return the next value from the iteration *o*. If the object is an iterator, - this retrieves the next value from the iteration, and returns *NULL* with no - exception set if there are no remaining items. If the object is not an - iterator, :exc:`TypeError` is raised, or if there is an error in retrieving the - item, returns *NULL* and passes along the exception. + Return the next value from the iteration *o*. The object must be an iterator + (it is up to the caller to check this). If there are no remaining values, + returns *NULL* with no exception set. If an error occurs while retrieving + the item, returns *NULL* and passes along the exception. To write a loop which iterates over an iterator, the C code should look something like this:: diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index b80b3d5..5465571 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -98,7 +98,7 @@ memory from the Python heap: Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the allocated memory, or *NULL* if the request fails. Requesting zero bytes returns - a distinct non-*NULL* pointer if possible, as if :c:func:`PyMem_Malloc(1)` had + a distinct non-*NULL* pointer if possible, as if ``PyMem_Malloc(1)`` had been called instead. The memory will not have been initialized in any way. @@ -106,7 +106,7 @@ memory from the Python heap: Resizes the memory block pointed to by *p* to *n* bytes. The contents will be unchanged to the minimum of the old and the new sizes. If *p* is *NULL*, the - call is equivalent to :c:func:`PyMem_Malloc(n)`; else if *n* is equal to zero, + call is equivalent to ``PyMem_Malloc(n)``; else if *n* is equal to zero, the memory block is resized but is not freed, and the returned pointer is non-*NULL*. Unless *p* is *NULL*, it must have been returned by a previous call to :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. If the request fails, @@ -118,7 +118,7 @@ memory from the Python heap: Frees the memory block pointed to by *p*, which must have been returned by a previous call to :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. Otherwise, or - if :c:func:`PyMem_Free(p)` has been called before, undefined behavior occurs. If + if ``PyMem_Free(p)`` has been called before, undefined behavior occurs. If *p* is *NULL*, no operation is performed. The following type-oriented macros are provided for convenience. Note that diff --git a/Doc/c-api/objbuffer.rst b/Doc/c-api/objbuffer.rst index 90dce62..c5228c6 100644 --- a/Doc/c-api/objbuffer.rst +++ b/Doc/c-api/objbuffer.rst @@ -8,7 +8,7 @@ Old Buffer Protocol This section describes the legacy buffer protocol, which has been introduced in Python 1.6. It is still supported but deprecated in the Python 2.x series. -Python 3.0 introduces a new buffer protocol which fixes weaknesses and +Python 3 introduces a new buffer protocol which fixes weaknesses and shortcomings of the protocol, and has been backported to Python 2.6. See :ref:`bufferobjects` for more information. diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index a02326f..50b83e9 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -47,8 +47,8 @@ Object Protocol Generic attribute getter function that is meant to be put into a type object's ``tp_getattro`` slot. It looks for a descriptor in the dictionary of classes in the object's MRO as well as an attribute in the object's - :attr:`__dict__` (if present). As outlined in :ref:`descriptors`, data - descriptors take preference over instance attributes, while non-data + :attr:`~object.__dict__` (if present). As outlined in :ref:`descriptors`, + data descriptors take preference over instance attributes, while non-data descriptors don't. Otherwise, an :exc:`AttributeError` is raised. @@ -72,8 +72,8 @@ Object Protocol object's ``tp_setattro`` slot. It looks for a data descriptor in the dictionary of classes in the object's MRO, and if found it takes preference over setting the attribute in the instance dictionary. Otherwise, the - attribute is set in the object's :attr:`__dict__` (if present). Otherwise, - an :exc:`AttributeError` is raised and ``-1`` is returned. + attribute is set in the object's :attr:`~object.__dict__` (if present). + Otherwise, an :exc:`AttributeError` is raised and ``-1`` is returned. .. c:function:: int PyObject_DelAttr(PyObject *o, PyObject *attr_name) @@ -180,9 +180,9 @@ Object Protocol be done against every entry in *cls*. The result will be ``1`` when at least one of the checks returns ``1``, otherwise it will be ``0``. If *inst* is not a class instance and *cls* is neither a type object, nor a class object, nor a - tuple, *inst* must have a :attr:`__class__` attribute --- the class relationship - of the value of that attribute with *cls* will be used to determine the result - of this function. + tuple, *inst* must have a :attr:`~instance.__class__` attribute --- the + class relationship of the value of that attribute with *cls* will be used + to determine the result of this function. .. versionadded:: 2.1 @@ -196,9 +196,9 @@ of. If :class:`A` and :class:`B` are class objects, :class:`B` is a subclass of either is not a class object, a more general mechanism is used to determine the class relationship of the two objects. When testing if *B* is a subclass of *A*, if *A* is *B*, :c:func:`PyObject_IsSubclass` returns true. If *A* and *B* -are different objects, *B*'s :attr:`__bases__` attribute is searched in a -depth-first fashion for *A* --- the presence of the :attr:`__bases__` attribute -is considered sufficient for this determination. +are different objects, *B*'s :attr:`~class.__bases__` attribute is searched in +a depth-first fashion for *A* --- the presence of the :attr:`~class.__bases__` +attribute is considered sufficient for this determination. .. c:function:: int PyObject_IsSubclass(PyObject *derived, PyObject *cls) diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst index 2b668a5..653f9ad 100644 --- a/Doc/c-api/sequence.rst +++ b/Doc/c-api/sequence.rst @@ -167,10 +167,10 @@ Sequence Protocol .. c:function:: PyObject* PySequence_Fast(PyObject *o, const char *m) - Returns the sequence *o* as a tuple, unless it is already a tuple or list, in - which case *o* is returned. Use :c:func:`PySequence_Fast_GET_ITEM` to access the - members of the result. Returns *NULL* on failure. If the object is not a - sequence, raises :exc:`TypeError` with *m* as the message text. + Return the sequence *o* as a list, unless it is already a tuple or list, in + which case *o* is returned. Use :c:func:`PySequence_Fast_GET_ITEM` to access + the members of the result. Returns *NULL* on failure. If the object is not + a sequence, raises :exc:`TypeError` with *m* as the message text. .. c:function:: PyObject* PySequence_Fast_GET_ITEM(PyObject *o, Py_ssize_t i) diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst index 41c4af4..258530b 100644 --- a/Doc/c-api/set.rst +++ b/Doc/c-api/set.rst @@ -156,7 +156,7 @@ subtypes but not for instances of :class:`frozenset` or its subtypes. Return 1 if found and removed, 0 if not found (no action taken), and -1 if an error is encountered. Does not raise :exc:`KeyError` for missing keys. Raise a - :exc:`TypeError` if the *key* is unhashable. Unlike the Python :meth:`discard` + :exc:`TypeError` if the *key* is unhashable. Unlike the Python :meth:`~set.discard` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise :exc:`PyExc_SystemError` if *set* is an not an instance of :class:`set` or its subtype. diff --git a/Doc/c-api/string.rst b/Doc/c-api/string.rst index ecf7050..32dc274 100644 --- a/Doc/c-api/string.rst +++ b/Doc/c-api/string.rst @@ -241,7 +241,7 @@ called with a non-string parameter. .. c:function:: PyObject* PyString_Format(PyObject *format, PyObject *args) Return a new string object from *format* and *args*. Analogous to ``format % - args``. The *args* argument must be a tuple. + args``. The *args* argument must be a tuple or dict. .. c:function:: void PyString_InternInPlace(PyObject **string) diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index f5007ac..e31687f 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -293,6 +293,6 @@ definition with the same method name. .. c:function:: PyObject* Py_FindMethod(PyMethodDef table[], PyObject *ob, char *name) Return a bound method object for an extension type implemented in C. This - can be useful in the implementation of a :attr:`tp_getattro` or - :attr:`tp_getattr` handler that does not use the + can be useful in the implementation of a :c:member:`~PyTypeObject.tp_getattro` or + :c:member:`~PyTypeObject.tp_getattr` handler that does not use the :c:func:`PyObject_GenericGetAttr` function. diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 5bda1ff..b545b06 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -35,7 +35,7 @@ definition found there: The type object structure extends the :c:type:`PyVarObject` structure. The :attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the -metatype) initializes :attr:`tp_itemsize`, which means that its instances (i.e. +metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. type objects) *must* have the :attr:`ob_size` field. @@ -108,7 +108,7 @@ type objects) *must* have the :attr:`ob_size` field. should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named :class:`T` defined in module :mod:`M` in subpackage :mod:`Q` in package :mod:`P` - should have the :attr:`tp_name` initializer ``"P.Q.M.T"``. + should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For dynamically allocated type objects, this should just be the type name, and the module name explicitly stored in the type dict as the value for key @@ -119,7 +119,7 @@ type objects) *must* have the :attr:`ob_size` field. attribute, and everything after the last dot is made accessible as the :attr:`__name__` attribute. - If no dot is present, the entire :attr:`tp_name` field is made accessible as the + If no dot is present, the entire :c:member:`~PyTypeObject.tp_name` field is made accessible as the :attr:`__name__` attribute, and the :attr:`__module__` attribute is undefined (unless explicitly set in the dictionary, as explained above). This means your type will be impossible to pickle. @@ -133,13 +133,13 @@ type objects) *must* have the :attr:`ob_size` field. These fields allow calculating the size in bytes of instances of the type. There are two kinds of types: types with fixed-length instances have a zero - :attr:`tp_itemsize` field, types with variable-length instances have a non-zero - :attr:`tp_itemsize` field. For a type with fixed-length instances, all - instances have the same size, given in :attr:`tp_basicsize`. + :c:member:`~PyTypeObject.tp_itemsize` field, types with variable-length instances have a non-zero + :c:member:`~PyTypeObject.tp_itemsize` field. For a type with fixed-length instances, all + instances have the same size, given in :c:member:`~PyTypeObject.tp_basicsize`. For a type with variable-length instances, the instances must have an - :attr:`ob_size` field, and the instance size is :attr:`tp_basicsize` plus N - times :attr:`tp_itemsize`, where N is the "length" of the object. The value of + :attr:`ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N + times :c:member:`~PyTypeObject.tp_itemsize`, where N is the "length" of the object. The value of N is typically stored in the instance's :attr:`ob_size` field. There are exceptions: for example, long ints use a negative :attr:`ob_size` to indicate a negative number, and N is ``abs(ob_size)`` there. Also, the presence of an @@ -152,21 +152,21 @@ type objects) *must* have the :attr:`ob_size` field. :c:macro:`PyObject_HEAD` or :c:macro:`PyObject_VAR_HEAD` (whichever is used to declare the instance struct) and this in turn includes the :attr:`_ob_prev` and :attr:`_ob_next` fields if they are present. This means that the only correct - way to get an initializer for the :attr:`tp_basicsize` is to use the + way to get an initializer for the :c:member:`~PyTypeObject.tp_basicsize` is to use the ``sizeof`` operator on the struct used to declare the instance layout. The basic size does not include the GC header size (this is new in Python 2.2; - in 2.1 and 2.0, the GC header size was included in :attr:`tp_basicsize`). + in 2.1 and 2.0, the GC header size was included in :c:member:`~PyTypeObject.tp_basicsize`). These fields are inherited separately by subtypes. If the base type has a - non-zero :attr:`tp_itemsize`, it is generally not safe to set - :attr:`tp_itemsize` to a different non-zero value in a subtype (though this + non-zero :c:member:`~PyTypeObject.tp_itemsize`, it is generally not safe to set + :c:member:`~PyTypeObject.tp_itemsize` to a different non-zero value in a subtype (though this depends on the implementation of the base type). A note about alignment: if the variable items require a particular alignment, - this should be taken care of by the value of :attr:`tp_basicsize`. Example: - suppose a type implements an array of ``double``. :attr:`tp_itemsize` is + this should be taken care of by the value of :c:member:`~PyTypeObject.tp_basicsize`. Example: + suppose a type implements an array of ``double``. :c:member:`~PyTypeObject.tp_itemsize` is ``sizeof(double)``. It is the programmer's responsibility that - :attr:`tp_basicsize` is a multiple of ``sizeof(double)`` (assuming this is the + :c:member:`~PyTypeObject.tp_basicsize` is a multiple of ``sizeof(double)`` (assuming this is the alignment requirement for ``double``). @@ -182,10 +182,10 @@ type objects) *must* have the :attr:`ob_size` field. destructor function should free all references which the instance owns, free all memory buffers owned by the instance (using the freeing function corresponding to the allocation function used to allocate the buffer), and finally (as its - last action) call the type's :attr:`tp_free` function. If the type is not + last action) call the type's :c:member:`~PyTypeObject.tp_free` function. If the type is not subtypable (doesn't have the :const:`Py_TPFLAGS_BASETYPE` flag bit set), it is permissible to call the object deallocator directly instead of via - :attr:`tp_free`. The object deallocator should be the one used to allocate the + :c:member:`~PyTypeObject.tp_free`. The object deallocator should be the one used to allocate the instance; this is normally :c:func:`PyObject_Del` if the instance was allocated using :c:func:`PyObject_New` or :c:func:`PyObject_VarNew`, or :c:func:`PyObject_GC_Del` if the instance was allocated using @@ -199,26 +199,26 @@ type objects) *must* have the :attr:`ob_size` field. An optional pointer to the instance print function. The print function is only called when the instance is printed to a *real* file; - when it is printed to a pseudo-file (like a :class:`StringIO` instance), the - instance's :attr:`tp_repr` or :attr:`tp_str` function is called to convert it to - a string. These are also called when the type's :attr:`tp_print` field is - *NULL*. A type should never implement :attr:`tp_print` in a way that produces - different output than :attr:`tp_repr` or :attr:`tp_str` would. + when it is printed to a pseudo-file (like a :class:`~StringIO.StringIO` instance), the + instance's :c:member:`~PyTypeObject.tp_repr` or :c:member:`~PyTypeObject.tp_str` function is called to convert it to + a string. These are also called when the type's :c:member:`~PyTypeObject.tp_print` field is + *NULL*. A type should never implement :c:member:`~PyTypeObject.tp_print` in a way that produces + different output than :c:member:`~PyTypeObject.tp_repr` or :c:member:`~PyTypeObject.tp_str` would. The print function is called with the same signature as :c:func:`PyObject_Print`: ``int tp_print(PyObject *self, FILE *file, int flags)``. The *self* argument is the instance to be printed. The *file* argument is the stdio file to which it is to be printed. The *flags* argument is composed of flag bits. The only flag bit currently defined is :const:`Py_PRINT_RAW`. When the :const:`Py_PRINT_RAW` - flag bit is set, the instance should be printed the same way as :attr:`tp_str` + flag bit is set, the instance should be printed the same way as :c:member:`~PyTypeObject.tp_str` would format it; when the :const:`Py_PRINT_RAW` flag bit is clear, the instance - should be printed the same was as :attr:`tp_repr` would format it. It should + should be printed the same was as :c:member:`~PyTypeObject.tp_repr` would format it. It should return ``-1`` and set an exception condition when an error occurred during the comparison. - It is possible that the :attr:`tp_print` field will be deprecated. In any case, - it is recommended not to define :attr:`tp_print`, but instead to rely on - :attr:`tp_repr` and :attr:`tp_str` for printing. + It is possible that the :c:member:`~PyTypeObject.tp_print` field will be deprecated. In any case, + it is recommended not to define :c:member:`~PyTypeObject.tp_print`, but instead to rely on + :c:member:`~PyTypeObject.tp_repr` and :c:member:`~PyTypeObject.tp_str` for printing. This field is inherited by subtypes. @@ -228,13 +228,13 @@ type objects) *must* have the :attr:`ob_size` field. An optional pointer to the get-attribute-string function. This field is deprecated. When it is defined, it should point to a function - that acts the same as the :attr:`tp_getattro` function, but taking a C string + that acts the same as the :c:member:`~PyTypeObject.tp_getattro` function, but taking a C string instead of a Python string object to give the attribute name. The signature is the same as for :c:func:`PyObject_GetAttrString`. - This field is inherited by subtypes together with :attr:`tp_getattro`: a subtype - inherits both :attr:`tp_getattr` and :attr:`tp_getattro` from its base type when - the subtype's :attr:`tp_getattr` and :attr:`tp_getattro` are both *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattro`: a subtype + inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when + the subtype's :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` are both *NULL*. .. c:member:: setattrfunc PyTypeObject.tp_setattr @@ -242,13 +242,13 @@ type objects) *must* have the :attr:`ob_size` field. An optional pointer to the set-attribute-string function. This field is deprecated. When it is defined, it should point to a function - that acts the same as the :attr:`tp_setattro` function, but taking a C string + that acts the same as the :c:member:`~PyTypeObject.tp_setattro` function, but taking a C string instead of a Python string object to give the attribute name. The signature is the same as for :c:func:`PyObject_SetAttrString`. - This field is inherited by subtypes together with :attr:`tp_setattro`: a subtype - inherits both :attr:`tp_setattr` and :attr:`tp_setattro` from its base type when - the subtype's :attr:`tp_setattr` and :attr:`tp_setattro` are both *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype + inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when + the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both *NULL*. .. c:member:: cmpfunc PyTypeObject.tp_compare @@ -260,10 +260,10 @@ type objects) *must* have the :attr:`ob_size` field. *other*, and ``-1`` if *self* less than *other*. It should return ``-1`` and set an exception condition when an error occurred during the comparison. - This field is inherited by subtypes together with :attr:`tp_richcompare` and - :attr:`tp_hash`: a subtypes inherits all three of :attr:`tp_compare`, - :attr:`tp_richcompare`, and :attr:`tp_hash` when the subtype's - :attr:`tp_compare`, :attr:`tp_richcompare`, and :attr:`tp_hash` are all *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_richcompare` and + :c:member:`~PyTypeObject.tp_hash`: a subtypes inherits all three of :c:member:`~PyTypeObject.tp_compare`, + :c:member:`~PyTypeObject.tp_richcompare`, and :c:member:`~PyTypeObject.tp_hash` when the subtype's + :c:member:`~PyTypeObject.tp_compare`, :c:member:`~PyTypeObject.tp_richcompare`, and :c:member:`~PyTypeObject.tp_hash` are all *NULL*. .. c:member:: reprfunc PyTypeObject.tp_repr @@ -292,7 +292,7 @@ type objects) *must* have the :attr:`ob_size` field. objects which implement the number protocol. These fields are documented in :ref:`number-structs`. - The :attr:`tp_as_number` field is not inherited, but the contained fields are + The :c:member:`~PyTypeObject.tp_as_number` field is not inherited, but the contained fields are inherited individually. @@ -302,7 +302,7 @@ type objects) *must* have the :attr:`ob_size` field. objects which implement the sequence protocol. These fields are documented in :ref:`sequence-structs`. - The :attr:`tp_as_sequence` field is not inherited, but the contained fields + The :c:member:`~PyTypeObject.tp_as_sequence` field is not inherited, but the contained fields are inherited individually. @@ -312,7 +312,7 @@ type objects) *must* have the :attr:`ob_size` field. objects which implement the mapping protocol. These fields are documented in :ref:`mapping-structs`. - The :attr:`tp_as_mapping` field is not inherited, but the contained fields + The :c:member:`~PyTypeObject.tp_as_mapping` field is not inherited, but the contained fields are inherited individually. @@ -336,14 +336,14 @@ type objects) *must* have the :attr:`ob_size` field. the Python level will result in the ``tp_hash`` slot being set to :c:func:`PyObject_HashNotImplemented`. - When this field is not set, two possibilities exist: if the :attr:`tp_compare` - and :attr:`tp_richcompare` fields are both *NULL*, a default hash value based on + When this field is not set, two possibilities exist: if the :c:member:`~PyTypeObject.tp_compare` + and :c:member:`~PyTypeObject.tp_richcompare` fields are both *NULL*, a default hash value based on the object's address is returned; otherwise, a :exc:`TypeError` is raised. - This field is inherited by subtypes together with :attr:`tp_richcompare` and - :attr:`tp_compare`: a subtypes inherits all three of :attr:`tp_compare`, - :attr:`tp_richcompare`, and :attr:`tp_hash`, when the subtype's - :attr:`tp_compare`, :attr:`tp_richcompare` and :attr:`tp_hash` are all *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_richcompare` and + :c:member:`~PyTypeObject.tp_compare`: a subtypes inherits all three of :c:member:`~PyTypeObject.tp_compare`, + :c:member:`~PyTypeObject.tp_richcompare`, and :c:member:`~PyTypeObject.tp_hash`, when the subtype's + :c:member:`~PyTypeObject.tp_compare`, :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` are all *NULL*. .. c:member:: ternaryfunc PyTypeObject.tp_call @@ -381,9 +381,9 @@ type objects) *must* have the :attr:`ob_size` field. convenient to set this field to :c:func:`PyObject_GenericGetAttr`, which implements the normal way of looking for object attributes. - This field is inherited by subtypes together with :attr:`tp_getattr`: a subtype - inherits both :attr:`tp_getattr` and :attr:`tp_getattro` from its base type when - the subtype's :attr:`tp_getattr` and :attr:`tp_getattro` are both *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattr`: a subtype + inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when + the subtype's :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` are both *NULL*. .. c:member:: setattrofunc PyTypeObject.tp_setattro @@ -394,9 +394,9 @@ type objects) *must* have the :attr:`ob_size` field. convenient to set this field to :c:func:`PyObject_GenericSetAttr`, which implements the normal way of setting object attributes. - This field is inherited by subtypes together with :attr:`tp_setattr`: a subtype - inherits both :attr:`tp_setattr` and :attr:`tp_setattro` from its base type when - the subtype's :attr:`tp_setattr` and :attr:`tp_setattro` are both *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattr`: a subtype + inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when + the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both *NULL*. .. c:member:: PyBufferProcs* PyTypeObject.tp_as_buffer @@ -405,7 +405,7 @@ type objects) *must* have the :attr:`ob_size` field. which implement the buffer interface. These fields are documented in :ref:`buffer-structs`. - The :attr:`tp_as_buffer` field is not inherited, but the contained fields are + The :c:member:`~PyTypeObject.tp_as_buffer` field is not inherited, but the contained fields are inherited individually. @@ -414,8 +414,8 @@ type objects) *must* have the :attr:`ob_size` field. This field is a bit mask of various flags. Some flags indicate variant semantics for certain situations; others are used to indicate that certain fields in the type object (or in the extension structures referenced via - :attr:`tp_as_number`, :attr:`tp_as_sequence`, :attr:`tp_as_mapping`, and - :attr:`tp_as_buffer`) that were historically not always present are valid; if + :c:member:`~PyTypeObject.tp_as_number`, :c:member:`~PyTypeObject.tp_as_sequence`, :c:member:`~PyTypeObject.tp_as_mapping`, and + :c:member:`~PyTypeObject.tp_as_buffer`) that were historically not always present are valid; if such a flag bit is clear, the type fields it guards must not be accessed and must be considered to have a zero or *NULL* value instead. @@ -425,14 +425,14 @@ type objects) *must* have the :attr:`ob_size` field. inherited if the extension structure is inherited, i.e. the base type's value of the flag bit is copied into the subtype together with a pointer to the extension structure. The :const:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with - the :attr:`tp_traverse` and :attr:`tp_clear` fields, i.e. if the + the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the - :attr:`tp_traverse` and :attr:`tp_clear` fields in the subtype exist (as + :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist (as indicated by the :const:`Py_TPFLAGS_HAVE_RICHCOMPARE` flag bit) and have *NULL* values. The following bit masks are currently defined; these can be ORed together using - the ``|`` operator to form the value of the :attr:`tp_flags` field. The macro + the ``|`` operator to form the value of the :c:member:`~PyTypeObject.tp_flags` field. The macro :c:func:`PyType_HasFeature` takes a type and a flags value, *tp* and *f*, and checks whether ``tp->tp_flags & f`` is non-zero. @@ -440,13 +440,13 @@ type objects) *must* have the :attr:`ob_size` field. .. data:: Py_TPFLAGS_HAVE_GETCHARBUFFER If this bit is set, the :c:type:`PyBufferProcs` struct referenced by - :attr:`tp_as_buffer` has the :attr:`bf_getcharbuffer` field. + :c:member:`~PyTypeObject.tp_as_buffer` has the :attr:`bf_getcharbuffer` field. .. data:: Py_TPFLAGS_HAVE_SEQUENCE_IN If this bit is set, the :c:type:`PySequenceMethods` struct referenced by - :attr:`tp_as_sequence` has the :attr:`sq_contains` field. + :c:member:`~PyTypeObject.tp_as_sequence` has the :attr:`sq_contains` field. .. data:: Py_TPFLAGS_GC @@ -458,8 +458,8 @@ type objects) *must* have the :attr:`ob_size` field. .. data:: Py_TPFLAGS_HAVE_INPLACEOPS If this bit is set, the :c:type:`PySequenceMethods` struct referenced by - :attr:`tp_as_sequence` and the :c:type:`PyNumberMethods` structure referenced by - :attr:`tp_as_number` contain the fields for in-place operators. In particular, + :c:member:`~PyTypeObject.tp_as_sequence` and the :c:type:`PyNumberMethods` structure referenced by + :c:member:`~PyTypeObject.tp_as_number` contain the fields for in-place operators. In particular, this means that the :c:type:`PyNumberMethods` structure has the fields :attr:`nb_inplace_add`, :attr:`nb_inplace_subtract`, :attr:`nb_inplace_multiply`, :attr:`nb_inplace_divide`, @@ -473,7 +473,7 @@ type objects) *must* have the :attr:`ob_size` field. .. data:: Py_TPFLAGS_CHECKTYPES If this bit is set, the binary and ternary operations in the - :c:type:`PyNumberMethods` structure referenced by :attr:`tp_as_number` accept + :c:type:`PyNumberMethods` structure referenced by :c:member:`~PyTypeObject.tp_as_number` accept arguments of arbitrary object types, and do their own type conversions if needed. If this bit is clear, those operations require that all arguments have the current type as their type, and the caller is supposed to perform a coercion @@ -485,31 +485,31 @@ type objects) *must* have the :attr:`ob_size` field. .. data:: Py_TPFLAGS_HAVE_RICHCOMPARE - If this bit is set, the type object has the :attr:`tp_richcompare` field, as - well as the :attr:`tp_traverse` and the :attr:`tp_clear` fields. + If this bit is set, the type object has the :c:member:`~PyTypeObject.tp_richcompare` field, as + well as the :c:member:`~PyTypeObject.tp_traverse` and the :c:member:`~PyTypeObject.tp_clear` fields. .. data:: Py_TPFLAGS_HAVE_WEAKREFS - If this bit is set, the :attr:`tp_weaklistoffset` field is defined. Instances - of a type are weakly referenceable if the type's :attr:`tp_weaklistoffset` field + If this bit is set, the :c:member:`~PyTypeObject.tp_weaklistoffset` field is defined. Instances + of a type are weakly referenceable if the type's :c:member:`~PyTypeObject.tp_weaklistoffset` field has a value greater than zero. .. data:: Py_TPFLAGS_HAVE_ITER - If this bit is set, the type object has the :attr:`tp_iter` and - :attr:`tp_iternext` fields. + If this bit is set, the type object has the :c:member:`~PyTypeObject.tp_iter` and + :c:member:`~PyTypeObject.tp_iternext` fields. .. data:: Py_TPFLAGS_HAVE_CLASS If this bit is set, the type object has several new fields defined starting in - Python 2.2: :attr:`tp_methods`, :attr:`tp_members`, :attr:`tp_getset`, - :attr:`tp_base`, :attr:`tp_dict`, :attr:`tp_descr_get`, :attr:`tp_descr_set`, - :attr:`tp_dictoffset`, :attr:`tp_init`, :attr:`tp_alloc`, :attr:`tp_new`, - :attr:`tp_free`, :attr:`tp_is_gc`, :attr:`tp_bases`, :attr:`tp_mro`, - :attr:`tp_cache`, :attr:`tp_subclasses`, and :attr:`tp_weaklist`. + Python 2.2: :c:member:`~PyTypeObject.tp_methods`, :c:member:`~PyTypeObject.tp_members`, :c:member:`~PyTypeObject.tp_getset`, + :c:member:`~PyTypeObject.tp_base`, :c:member:`~PyTypeObject.tp_dict`, :c:member:`~PyTypeObject.tp_descr_get`, :c:member:`~PyTypeObject.tp_descr_set`, + :c:member:`~PyTypeObject.tp_dictoffset`, :c:member:`~PyTypeObject.tp_init`, :c:member:`~PyTypeObject.tp_alloc`, :c:member:`~PyTypeObject.tp_new`, + :c:member:`~PyTypeObject.tp_free`, :c:member:`~PyTypeObject.tp_is_gc`, :c:member:`~PyTypeObject.tp_bases`, :c:member:`~PyTypeObject.tp_mro`, + :c:member:`~PyTypeObject.tp_cache`, :c:member:`~PyTypeObject.tp_subclasses`, and :c:member:`~PyTypeObject.tp_weaklist`. .. data:: Py_TPFLAGS_HEAPTYPE @@ -547,7 +547,7 @@ type objects) *must* have the :attr:`ob_size` field. is set, instances must be created using :c:func:`PyObject_GC_New` and destroyed using :c:func:`PyObject_GC_Del`. More information in section :ref:`supporting-cycle-detection`. This bit also implies that the - GC-related fields :attr:`tp_traverse` and :attr:`tp_clear` are present in + GC-related fields :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` are present in the type object; but those fields also exist when :const:`Py_TPFLAGS_HAVE_GC` is clear but :const:`Py_TPFLAGS_HAVE_RICHCOMPARE` is set. @@ -582,8 +582,8 @@ The following three fields only exist if the about Python's garbage collection scheme can be found in section :ref:`supporting-cycle-detection`. - The :attr:`tp_traverse` pointer is used by the garbage collector to detect - reference cycles. A typical implementation of a :attr:`tp_traverse` function + The :c:member:`~PyTypeObject.tp_traverse` pointer is used by the garbage collector to detect + reference cycles. A typical implementation of a :c:member:`~PyTypeObject.tp_traverse` function simply calls :c:func:`Py_VISIT` on each of the instance's members that are Python objects. For example, this is function :c:func:`local_traverse` from the :mod:`thread` extension module:: @@ -603,15 +603,15 @@ The following three fields only exist if the On the other hand, even if you know a member can never be part of a cycle, as a debugging aid you may want to visit it anyway just so the :mod:`gc` module's - :func:`get_referents` function will include it. + :func:`~gc.get_referents` function will include it. Note that :c:func:`Py_VISIT` requires the *visit* and *arg* parameters to :c:func:`local_traverse` to have these specific names; don't name them just anything. - This field is inherited by subtypes together with :attr:`tp_clear` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :attr:`tp_traverse`, and - :attr:`tp_clear` are all inherited from the base type if they are all zero in + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the + :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype *and* the subtype has the :const:`Py_TPFLAGS_HAVE_RICHCOMPARE` flag bit set. @@ -621,17 +621,17 @@ The following three fields only exist if the An optional pointer to a clear function for the garbage collector. This is only used if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is set. - The :attr:`tp_clear` member function is used to break reference cycles in cyclic - garbage detected by the garbage collector. Taken together, all :attr:`tp_clear` + The :c:member:`~PyTypeObject.tp_clear` member function is used to break reference cycles in cyclic + garbage detected by the garbage collector. Taken together, all :c:member:`~PyTypeObject.tp_clear` functions in the system must combine to break all reference cycles. This is - subtle, and if in any doubt supply a :attr:`tp_clear` function. For example, - the tuple type does not implement a :attr:`tp_clear` function, because it's + subtle, and if in any doubt supply a :c:member:`~PyTypeObject.tp_clear` function. For example, + the tuple type does not implement a :c:member:`~PyTypeObject.tp_clear` function, because it's possible to prove that no reference cycle can be composed entirely of tuples. - Therefore the :attr:`tp_clear` functions of other types must be sufficient to + Therefore the :c:member:`~PyTypeObject.tp_clear` functions of other types must be sufficient to break any cycle containing a tuple. This isn't immediately obvious, and there's - rarely a good reason to avoid implementing :attr:`tp_clear`. + rarely a good reason to avoid implementing :c:member:`~PyTypeObject.tp_clear`. - Implementations of :attr:`tp_clear` should drop the instance's references to + Implementations of :c:member:`~PyTypeObject.tp_clear` should drop the instance's references to those of its members that may be Python objects, and set its pointers to those members to *NULL*, as in the following example:: @@ -656,18 +656,18 @@ The following three fields only exist if the so that *self* knows the contained object can no longer be used. The :c:func:`Py_CLEAR` macro performs the operations in a safe order. - Because the goal of :attr:`tp_clear` functions is to break reference cycles, + Because the goal of :c:member:`~PyTypeObject.tp_clear` functions is to break reference cycles, it's not necessary to clear contained objects like Python strings or Python integers, which can't participate in reference cycles. On the other hand, it may be convenient to clear all contained Python objects, and write the type's - :attr:`tp_dealloc` function to invoke :attr:`tp_clear`. + :c:member:`~PyTypeObject.tp_dealloc` function to invoke :c:member:`~PyTypeObject.tp_clear`. More information about Python's garbage collection scheme can be found in section :ref:`supporting-cycle-detection`. - This field is inherited by subtypes together with :attr:`tp_traverse` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :attr:`tp_traverse`, and - :attr:`tp_clear` are all inherited from the base type if they are all zero in + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the + :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype *and* the subtype has the :const:`Py_TPFLAGS_HAVE_RICHCOMPARE` flag bit set. @@ -688,13 +688,13 @@ The following three fields only exist if the comparisons makes sense (e.g. ``==`` and ``!=``, but not ``<`` and friends), directly raise :exc:`TypeError` in the rich comparison function. - This field is inherited by subtypes together with :attr:`tp_compare` and - :attr:`tp_hash`: a subtype inherits all three of :attr:`tp_compare`, - :attr:`tp_richcompare`, and :attr:`tp_hash`, when the subtype's - :attr:`tp_compare`, :attr:`tp_richcompare`, and :attr:`tp_hash` are all *NULL*. + This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_compare` and + :c:member:`~PyTypeObject.tp_hash`: a subtype inherits all three of :c:member:`~PyTypeObject.tp_compare`, + :c:member:`~PyTypeObject.tp_richcompare`, and :c:member:`~PyTypeObject.tp_hash`, when the subtype's + :c:member:`~PyTypeObject.tp_compare`, :c:member:`~PyTypeObject.tp_richcompare`, and :c:member:`~PyTypeObject.tp_hash` are all *NULL*. The following constants are defined to be used as the third argument for - :attr:`tp_richcompare` and for :c:func:`PyObject_RichCompare`: + :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: +----------------+------------+ | Constant | Comparison | @@ -725,26 +725,26 @@ set. instance structure needs to include a field of type :c:type:`PyObject\*` which is initialized to *NULL*. - Do not confuse this field with :attr:`tp_weaklist`; that is the list head for + Do not confuse this field with :c:member:`~PyTypeObject.tp_weaklist`; that is the list head for weak references to the type object itself. This field is inherited by subtypes, but see the rules listed below. A subtype may override this offset; this means that the subtype uses a different weak reference list head than the base type. Since the list head is always found via - :attr:`tp_weaklistoffset`, this should not be a problem. + :c:member:`~PyTypeObject.tp_weaklistoffset`, this should not be a problem. - When a type defined by a class statement has no :attr:`__slots__` declaration, + When a type defined by a class statement has no :attr:`~object.__slots__` declaration, and none of its base types are weakly referenceable, the type is made weakly referenceable by adding a weak reference list head slot to the instance layout - and setting the :attr:`tp_weaklistoffset` of that slot's offset. + and setting the :c:member:`~PyTypeObject.tp_weaklistoffset` of that slot's offset. When a type's :attr:`__slots__` declaration contains a slot named :attr:`__weakref__`, that slot becomes the weak reference list head for instances of the type, and the slot's offset is stored in the type's - :attr:`tp_weaklistoffset`. + :c:member:`~PyTypeObject.tp_weaklistoffset`. When a type's :attr:`__slots__` declaration does not contain a slot named - :attr:`__weakref__`, the type inherits its :attr:`tp_weaklistoffset` from its + :attr:`__weakref__`, the type inherits its :c:member:`~PyTypeObject.tp_weaklistoffset` from its base type. The next two fields only exist if the :const:`Py_TPFLAGS_HAVE_ITER` flag bit is @@ -772,7 +772,7 @@ set. are iterators (although classic instances always have this function, even if they don't define a :meth:`next` method). - Iterator types should also define the :attr:`tp_iter` function, and that + Iterator types should also define the :c:member:`~PyTypeObject.tp_iter` function, and that function should return the iterator instance itself (not a new iterator instance). @@ -780,7 +780,7 @@ set. This field is inherited by subtypes. -The next fields, up to and including :attr:`tp_weaklist`, only exist if the +The next fields, up to and including :c:member:`~PyTypeObject.tp_weaklist`, only exist if the :const:`Py_TPFLAGS_HAVE_CLASS` flag bit is set. @@ -790,7 +790,7 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the structures, declaring regular methods of this type. For each entry in the array, an entry is added to the type's dictionary (see - :attr:`tp_dict` below) containing a method descriptor. + :c:member:`~PyTypeObject.tp_dict` below) containing a method descriptor. This field is not inherited by subtypes (methods are inherited through a different mechanism). @@ -803,7 +803,7 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the this type. For each entry in the array, an entry is added to the type's dictionary (see - :attr:`tp_dict` below) containing a member descriptor. + :c:member:`~PyTypeObject.tp_dict` below) containing a member descriptor. This field is not inherited by subtypes (members are inherited through a different mechanism). @@ -815,7 +815,7 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the structures, declaring computed attributes of instances of this type. For each entry in the array, an entry is added to the type's dictionary (see - :attr:`tp_dict` below) containing a getset descriptor. + :c:member:`~PyTypeObject.tp_dict` below) containing a getset descriptor. This field is not inherited by subtypes (computed attributes are inherited through a different mechanism). @@ -894,7 +894,7 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the the instance variable dictionary; this offset is used by :c:func:`PyObject_GenericGetAttr`. - Do not confuse this field with :attr:`tp_dict`; that is the dictionary for + Do not confuse this field with :c:member:`~PyTypeObject.tp_dict`; that is the dictionary for attributes of the type object itself. If the value of this field is greater than zero, it specifies the offset from @@ -903,20 +903,20 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the offset is more expensive to use, and should only be used when the instance structure contains a variable-length part. This is used for example to add an instance variable dictionary to subtypes of :class:`str` or :class:`tuple`. Note - that the :attr:`tp_basicsize` field should account for the dictionary added to + that the :c:member:`~PyTypeObject.tp_basicsize` field should account for the dictionary added to the end in that case, even though the dictionary is not included in the basic object layout. On a system with a pointer size of 4 bytes, - :attr:`tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is + :c:member:`~PyTypeObject.tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is at the very end of the structure. The real dictionary offset in an instance can be computed from a negative - :attr:`tp_dictoffset` as follows:: + :c:member:`~PyTypeObject.tp_dictoffset` as follows:: dictoffset = tp_basicsize + abs(ob_size)*tp_itemsize + tp_dictoffset if dictoffset is not aligned on sizeof(void*): round up to sizeof(void*) - where :attr:`tp_basicsize`, :attr:`tp_itemsize` and :attr:`tp_dictoffset` are + where :c:member:`~PyTypeObject.tp_basicsize`, :c:member:`~PyTypeObject.tp_itemsize` and :c:member:`~PyTypeObject.tp_dictoffset` are taken from the type object, and :attr:`ob_size` is taken from the instance. The absolute value is taken because long ints use the sign of :attr:`ob_size` to store the sign of the number. (There's never a need to do this calculation @@ -925,17 +925,17 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the This field is inherited by subtypes, but see the rules listed below. A subtype may override this offset; this means that the subtype instances store the dictionary at a difference offset than the base type. Since the dictionary is - always found via :attr:`tp_dictoffset`, this should not be a problem. + always found via :c:member:`~PyTypeObject.tp_dictoffset`, this should not be a problem. - When a type defined by a class statement has no :attr:`__slots__` declaration, + When a type defined by a class statement has no :attr:`~object.__slots__` declaration, and none of its base types has an instance variable dictionary, a dictionary - slot is added to the instance layout and the :attr:`tp_dictoffset` is set to + slot is added to the instance layout and the :c:member:`~PyTypeObject.tp_dictoffset` is set to that slot's offset. When a type defined by a class statement has a :attr:`__slots__` declaration, - the type inherits its :attr:`tp_dictoffset` from its base type. + the type inherits its :c:member:`~PyTypeObject.tp_dictoffset` from its base type. - (Adding a slot named :attr:`__dict__` to the :attr:`__slots__` declaration does + (Adding a slot named :attr:`~object.__dict__` to the :attr:`__slots__` declaration does not have the expected effect, it just causes confusion. Maybe this should be added as a feature just like :attr:`__weakref__` though.) @@ -957,15 +957,15 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the arguments represent positional and keyword arguments of the call to :meth:`__init__`. - The :attr:`tp_init` function, if not *NULL*, is called when an instance is - created normally by calling its type, after the type's :attr:`tp_new` function - has returned an instance of the type. If the :attr:`tp_new` function returns an + The :c:member:`~PyTypeObject.tp_init` function, if not *NULL*, is called when an instance is + created normally by calling its type, after the type's :c:member:`~PyTypeObject.tp_new` function + has returned an instance of the type. If the :c:member:`~PyTypeObject.tp_new` function returns an instance of some other type that is not a subtype of the original type, no - :attr:`tp_init` function is called; if :attr:`tp_new` returns an instance of a - subtype of the original type, the subtype's :attr:`tp_init` is called. (VERSION + :c:member:`~PyTypeObject.tp_init` function is called; if :c:member:`~PyTypeObject.tp_new` returns an instance of a + subtype of the original type, the subtype's :c:member:`~PyTypeObject.tp_init` is called. (VERSION NOTE: described here is what is implemented in Python 2.2.1 and later. In - Python 2.2, the :attr:`tp_init` of the type of the object returned by - :attr:`tp_new` was always called, if not *NULL*.) + Python 2.2, the :c:member:`~PyTypeObject.tp_init` of the type of the object returned by + :c:member:`~PyTypeObject.tp_new` was always called, if not *NULL*.) This field is inherited by subtypes. @@ -982,14 +982,14 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the initialization. It should return a pointer to a block of memory of adequate length for the instance, suitably aligned, and initialized to zeros, but with :attr:`ob_refcnt` set to ``1`` and :attr:`ob_type` set to the type argument. If - the type's :attr:`tp_itemsize` is non-zero, the object's :attr:`ob_size` field + the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :attr:`ob_size` field should be initialized to *nitems* and the length of the allocated memory block should be ``tp_basicsize + nitems*tp_itemsize``, rounded up to a multiple of ``sizeof(void*)``; otherwise, *nitems* is not used and the length of the block - should be :attr:`tp_basicsize`. + should be :c:member:`~PyTypeObject.tp_basicsize`. Do not use this function to do any other instance initialization, not even to - allocate additional memory; that should be done by :attr:`tp_new`. + allocate additional memory; that should be done by :c:member:`~PyTypeObject.tp_new`. This field is inherited by static subtypes, but not by dynamic subtypes (subtypes created by a class statement); in the latter, this field is always set @@ -1011,20 +1011,20 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the The subtype argument is the type of the object being created; the *args* and *kwds* arguments represent positional and keyword arguments of the call to the - type. Note that subtype doesn't have to equal the type whose :attr:`tp_new` + type. Note that subtype doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new` function is called; it may be a subtype of that type (but not an unrelated type). - The :attr:`tp_new` function should call ``subtype->tp_alloc(subtype, nitems)`` + The :c:member:`~PyTypeObject.tp_new` function should call ``subtype->tp_alloc(subtype, nitems)`` to allocate space for the object, and then do only as much further initialization as is absolutely necessary. Initialization that can safely be - ignored or repeated should be placed in the :attr:`tp_init` handler. A good + ignored or repeated should be placed in the :c:member:`~PyTypeObject.tp_init` handler. A good rule of thumb is that for immutable types, all initialization should take place - in :attr:`tp_new`, while for mutable types, most initialization should be - deferred to :attr:`tp_init`. + in :c:member:`~PyTypeObject.tp_new`, while for mutable types, most initialization should be + deferred to :c:member:`~PyTypeObject.tp_init`. This field is inherited by subtypes, except it is not inherited by static types - whose :attr:`tp_base` is *NULL* or ``&PyBaseObject_Type``. The latter exception + whose :c:member:`~PyTypeObject.tp_base` is *NULL* or ``&PyBaseObject_Type``. The latter exception is a precaution so that old extension types don't become callable simply by being linked with Python 2.2. @@ -1057,7 +1057,7 @@ The next fields, up to and including :attr:`tp_weaklist`, only exist if the The garbage collector needs to know whether a particular object is collectible or not. Normally, it is sufficient to look at the object's type's - :attr:`tp_flags` field, and check the :const:`Py_TPFLAGS_HAVE_GC` flag bit. But + :c:member:`~PyTypeObject.tp_flags` field, and check the :const:`Py_TPFLAGS_HAVE_GC` flag bit. But some types have a mixture of statically and dynamically allocated instances, and the statically allocated instances are not collectible. Such types should define this function; it should return ``1`` for a collectible instance, and @@ -1129,7 +1129,7 @@ subtypes. .. c:member:: PyTypeObject* PyTypeObject.tp_next - Pointer to the next type object with a non-zero :attr:`tp_allocs` field. + Pointer to the next type object with a non-zero :c:member:`~PyTypeObject.tp_allocs` field. Also, note that, in a garbage collected Python, tp_dealloc may be called from any Python thread, not just the thread which created the object (if the object @@ -1227,7 +1227,7 @@ on the flag bit :const:`Py_TPFLAGS_CHECKTYPES`: - If the :const:`Py_TPFLAGS_CHECKTYPES` flag is set, binary and ternary functions must check the type of all their operands, and implement the necessary conversions (at least one of the operands is an instance of the - defined type). This is the recommended way; with Python 3.0 coercion will + defined type). This is the recommended way; with Python 3 coercion will disappear completely. If the operation is not defined for the given operands, binary and ternary @@ -1289,13 +1289,14 @@ Sequence Object Structures This function is used by :c:func:`PySequence_Concat` and has the same signature. It is also used by the ``+`` operator, after trying the numeric - addition via the :attr:`tp_as_number.nb_add` slot. + addition via the :c:member:`~PyTypeObject.tp_as_number.nb_add` slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_repeat This function is used by :c:func:`PySequence_Repeat` and has the same signature. It is also used by the ``*`` operator, after trying numeric - multiplication via the :attr:`tp_as_number.nb_mul` slot. + multiplication via the :c:member:`~PyTypeObject.tp_as_number.nb_multiply` + slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_item @@ -1348,14 +1349,14 @@ data as a set of chunks of data, where each chunk is specified as a pointer/length pair. These chunks are called :dfn:`segments` and are presumed to be non-contiguous in memory. -If an object does not export the buffer interface, then its :attr:`tp_as_buffer` +If an object does not export the buffer interface, then its :c:member:`~PyTypeObject.tp_as_buffer` member in the :c:type:`PyTypeObject` structure should be *NULL*. Otherwise, the -:attr:`tp_as_buffer` will point to a :c:type:`PyBufferProcs` structure. +:c:member:`~PyTypeObject.tp_as_buffer` will point to a :c:type:`PyBufferProcs` structure. .. note:: It is very important that your :c:type:`PyTypeObject` structure uses - :const:`Py_TPFLAGS_DEFAULT` for the value of the :attr:`tp_flags` member rather + :const:`Py_TPFLAGS_DEFAULT` for the value of the :c:member:`~PyTypeObject.tp_flags` member rather than ``0``. This tells the Python runtime that your :c:type:`PyBufferProcs` structure contains the :attr:`bf_getcharbuffer` slot. Older versions of Python did not have this member, so a new Python interpreter using an old extension @@ -1385,7 +1386,7 @@ member in the :c:type:`PyTypeObject` structure should be *NULL*. Otherwise, the The last slot is :attr:`bf_getcharbuffer`, of type :c:type:`getcharbufferproc`. This slot will only be present if the :const:`Py_TPFLAGS_HAVE_GETCHARBUFFER` - flag is present in the :attr:`tp_flags` field of the object's + flag is present in the :c:member:`~PyTypeObject.tp_flags` field of the object's :c:type:`PyTypeObject`. Before using this slot, the caller should test whether it is present by using the :c:func:`PyType_HasFeature` function. If the flag is present, :attr:`bf_getcharbuffer` may be *NULL*, indicating that the object's diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 73f6fe6..ddeaaa2 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -252,6 +252,8 @@ APIs: .. % because not all compilers support the %z width modifier -- we fake it .. % when necessary via interpolating PY_FORMAT_SIZE_T. + .. tabularcolumns:: |l|l|L| + +-------------------+---------------------+--------------------------------+ | Format Characters | Type | Comment | +===================+=====================+================================+ @@ -453,9 +455,9 @@ These are the generic codec APIs: Encode the :c:type:`Py_UNICODE` buffer *s* of the given *size* and return a Python string object. *encoding* and *errors* have the same meaning as the parameters - of the same name in the Unicode :meth:`encode` method. The codec to be used is - looked up using the Python codec registry. Return *NULL* if an exception was - raised by the codec. + of the same name in the Unicode :meth:`~unicode.encode` method. The codec + to be used is looked up using the Python codec registry. Return *NULL* if + an exception was raised by the codec. .. versionchanged:: 2.5 This function used an :c:type:`int` type for *size*. This might require diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 4ce3b03..6107665 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -265,7 +265,7 @@ the same library that the Python runtime is using. frame *f* is executed, interpreting bytecode and executing calls as needed. The additional *throwflag* parameter can mostly be ignored - if true, then it causes an exception to immediately be thrown; this is used for the - :meth:`throw` methods of generator objects. + :meth:`~generator.throw` methods of generator objects. .. c:function:: int PyEval_MergeCompilerFlags(PyCompilerFlags *cf) diff --git a/Doc/copyright.rst b/Doc/copyright.rst index 9a245c8..be47e8a 100644 --- a/Doc/copyright.rst +++ b/Doc/copyright.rst @@ -4,7 +4,7 @@ Copyright Python and this documentation is: -Copyright © 2001-2012 Python Software Foundation. All rights reserved. +Copyright © 2001-2014 Python Software Foundation. All rights reserved. Copyright © 2000 BeOpen.com. All rights reserved. diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat index 1fc896f..06c19d0 100644 --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -932,7 +932,7 @@ PyObject_CallMethod::...:: PyObject_CallMethodObjArgs:PyObject*::+1: PyObject_CallMethodObjArgs:PyObject*:o:0: -PyObject_CallMethodObjArgs:char*:name:: +PyObject_CallMethodObjArgs:PyObject*:name:0: PyObject_CallMethodObjArgs::...:: PyObject_CallObject:PyObject*::+1: diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst index 692d5cf..d1201ef 100644 --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -26,6 +26,8 @@ setup script). Indirectly provides the :class:`distutils.dist.Distribution` and The setup function takes a large number of arguments. These are laid out in the following table. + .. tabularcolumns:: |l|L|L| + +--------------------+--------------------------------+-------------------------------------------------------------+ | argument name | value | type | +====================+================================+=============================================================+ @@ -48,7 +50,10 @@ setup script). Indirectly provides the :class:`distutils.dist.Distribution` and +--------------------+--------------------------------+-------------------------------------------------------------+ | *maintainer* | The name of the current | a string | | | maintainer, if different from | | - | | the author | | + | | the author. Note that if | | + | | the maintainer is provided, | | + | | distutils will use it as the | | + | | author in :file:`PKG-INFO` | | +--------------------+--------------------------------+-------------------------------------------------------------+ | *maintainer_email* | The email address of the | a string | | | current maintainer, if | | @@ -122,6 +127,8 @@ setup script). Indirectly provides the :class:`distutils.dist.Distribution` and *stop_after* tells :func:`setup` when to stop processing; possible values: + .. tabularcolumns:: |l|L| + +---------------+---------------------------------------------+ | value | description | +===============+=============================================+ @@ -162,6 +169,8 @@ the full reference. The Extension class describes a single C or C++extension module in a setup script. It accepts the following keyword arguments in its constructor + .. tabularcolumns:: |l|L|l| + +------------------------+--------------------------------+---------------------------+ | argument name | value | type | +========================+================================+===========================+ @@ -444,7 +453,9 @@ This module provides the following functions. Define a preprocessor macro for all compilations driven by this compiler object. The optional parameter *value* should be a string; if it is not supplied, then the macro will be defined without an explicit value and the exact outcome - depends on the compiler used (XXX true? does ANSI say anything about this?) + depends on the compiler used. + + .. XXX true? does ANSI say anything about this? .. method:: CCompiler.undefine_macro(name) @@ -598,7 +609,9 @@ This module provides the following functions. *output_libname* should be a library name, not a filename; the filename will be inferred from the library name. *output_dir* is the directory where the library - file will be put. XXX defaults to what? + file will be put. + + .. XXX defaults to what? *debug* is a boolean; if true, debugging information will be included in the library (note that on most platforms, it is the compile step where this matters: @@ -716,32 +729,31 @@ This module provides the following functions. .. method:: CCompiler.execute(func, args[, msg=None, level=1]) - Invokes :func:`distutils.util.execute` This method invokes a Python function + Invokes :func:`distutils.util.execute`. This method invokes a Python function *func* with the given arguments *args*, after logging and taking into account - the *dry_run* flag. XXX see also. + the *dry_run* flag. .. method:: CCompiler.spawn(cmd) Invokes :func:`distutils.util.spawn`. This invokes an external process to run - the given command. XXX see also. + the given command. .. method:: CCompiler.mkpath(name[, mode=511]) Invokes :func:`distutils.dir_util.mkpath`. This creates a directory and any - missing ancestor directories. XXX see also. + missing ancestor directories. .. method:: CCompiler.move_file(src, dst) - Invokes :meth:`distutils.file_util.move_file`. Renames *src* to *dst*. XXX see - also. + Invokes :meth:`distutils.file_util.move_file`. Renames *src* to *dst*. .. method:: CCompiler.announce(msg[, level=1]) - Write a message using :func:`distutils.log.debug`. XXX see also. + Write a message using :func:`distutils.log.debug`. .. method:: CCompiler.warn(msg) @@ -869,8 +881,6 @@ tarballs or zipfiles. prefix of all files and directories in the archive. *root_dir* and *base_dir* both default to the current directory. Returns the name of the archive file. - .. XXX This should be changed to support bz2 files. - .. function:: make_tarball(base_name, base_dir[, compress='gzip', verbose=0, dry_run=0]) @@ -882,8 +892,6 @@ tarballs or zipfiles. possibly plus the appropriate compression extension (:file:`.gz`, :file:`.bz2` or :file:`.Z`). Return the output filename. - .. XXX This should be replaced with calls to the :mod:`tarfile` module. - .. function:: make_zipfile(base_name, base_dir[, verbose=0, dry_run=0]) @@ -974,20 +982,28 @@ directories. Copy an entire directory tree *src* to a new location *dst*. Both *src* and *dst* must be directory names. If *src* is not a directory, raise :exc:`DistutilsFileError`. If *dst* does not exist, it is created with - :func:`mkpath`. The end result of the copy is that every file in *src* is - copied to *dst*, and directories under *src* are recursively copied to *dst*. + :func:`mkpath`. The end result of the copy is that every file in *src* is + copied to *dst*, and directories under *src* are recursively copied to *dst*. Return the list of files that were copied or might have been copied, using their output name. The return value is unaffected by *update* or *dry_run*: it is simply the list of all files under *src*, with the names changed to be under *dst*. - *preserve_mode* and *preserve_times* are the same as for :func:`copy_file` in - :mod:`distutils.file_util`; note that they only apply to regular files, not to + *preserve_mode* and *preserve_times* are the same as for + :func:`distutils.file_util.copy_file`; note that they only apply to + regular files, not to directories. If *preserve_symlinks* is true, symlinks will be copied as symlinks (on platforms that support them!); otherwise (the default), the destination of the symlink will be copied. *update* and *verbose* are the same as for :func:`copy_file`. + Files in *src* that begin with :file:`.nfs` are skipped (more information on + these files is available in answer D2 of the `NFS FAQ page + <http://nfs.sourceforge.net/#section_d>`_. + + .. versionchanged:: 2.7.4 + NFS files are ignored. + .. function:: remove_tree(directory[, verbose=0, dry_run=0]) @@ -995,8 +1011,6 @@ directories. errors are ignored (apart from being reported to ``sys.stdout`` if *verbose* is true). -.. XXX Some of this could be replaced with the shutil module? - :mod:`distutils.file_util` --- Single file operations ===================================================== @@ -1110,8 +1124,6 @@ other utility module. * ``macosx-10.6-intel`` - .. % XXX isn't this also provided by some other non-distutils module? - .. function:: convert_path(pathname) @@ -1155,15 +1167,6 @@ other utility module. underscore. No { } or ( ) style quoting is available. -.. function:: grok_environment_error(exc[, prefix='error: ']) - - Generate a useful error message from an :exc:`EnvironmentError` (:exc:`IOError` - or :exc:`OSError`) exception object. Handles Python 1.5.1 and later styles, - and does what it can to deal with exception objects that don't have a filename - (which happens when the error is due to a two-file operation, such as - :func:`rename` or :func:`link`). Returns the error message as a string - prefixed with *prefix*. - .. function:: split_quoted(s) @@ -1246,8 +1249,8 @@ other utility module. built/installed/distributed -This module provides the :class:`Distribution` class, which represents the -module distribution being built/installed/distributed. +This module provides the :class:`~distutils.core.Distribution` class, which +represents the module distribution being built/installed/distributed. :mod:`distutils.extension` --- The Extension class @@ -1311,8 +1314,6 @@ provides the following additional features: the "negative alias" of :option:`--verbose`, then :option:`--quiet` on the command line sets *verbose* to false. -.. XXX Should be replaced with :mod:`optparse`. - .. function:: fancy_getopt(options, negative_opt, object, args) @@ -1329,8 +1330,6 @@ provides the following additional features: Wraps *text* to less than *width* wide. - .. XXX Should be replaced with :mod:`textwrap` (which is available in Python - 2.3 and later). .. class:: FancyGetopt([option_table=None]) @@ -1394,10 +1393,6 @@ filesystem and building lists of files. :synopsis: A simple logging mechanism, 282-style -.. XXX Should be replaced with standard :mod:`logging` module. - - - :mod:`distutils.spawn` --- Spawn a sub-process ============================================== @@ -1559,6 +1554,8 @@ lines, and joining lines with backslashes. The options are all boolean, and affect the values returned by :meth:`readline` + .. tabularcolumns:: |l|L|l| + +------------------+--------------------------------+---------+ | option name | description | default | +==================+================================+=========+ @@ -1701,8 +1698,8 @@ This module supplies the abstract base class :class:`Command`. options, is the :meth:`run` method, which must also be implemented by every command class. - The class constructor takes a single argument *dist*, a :class:`Distribution` - instance. + The class constructor takes a single argument *dist*, a + :class:`~distutils.core.Distribution` instance. Creating a new Distutils command @@ -1894,9 +1891,6 @@ Subclasses of :class:`Command` must define the following methods. :synopsis: Build the .py/.pyc files of a package -.. % todo - - :mod:`distutils.command.build_scripts` --- Build the scripts of a package ========================================================================= @@ -1913,8 +1907,12 @@ Subclasses of :class:`Command` must define the following methods. .. module:: distutils.command.clean :synopsis: Clean a package build area +This command removes the temporary files created by :command:`build` +and its subcommands, like intermediary compiled object files. With +the ``--all`` option, the complete build directory will be removed. -.. % todo +Extension modules built :ref:`in place <distutils-build-ext-inplace>` +will not be cleaned, as they are not in the build directory. :mod:`distutils.command.config` --- Perform package configuration diff --git a/Doc/distutils/configfile.rst b/Doc/distutils/configfile.rst index 890047c..ac79671 100644 --- a/Doc/distutils/configfile.rst +++ b/Doc/distutils/configfile.rst @@ -69,6 +69,8 @@ universal :option:`--help` option, e.g. :: Note that an option spelled :option:`--foo-bar` on the command-line is spelled :option:`foo_bar` in configuration files. +.. _distutils-build-ext-inplace: + For example, say you want your extensions to be built "in-place"---that is, you have an extension :mod:`pkg.ext`, and you want the compiled extension file (:file:`ext.so` on Unix, say) to be put in the same source directory as your diff --git a/Doc/distutils/examples.rst b/Doc/distutils/examples.rst index b495928..3c6c7bc 100644 --- a/Doc/distutils/examples.rst +++ b/Doc/distutils/examples.rst @@ -193,9 +193,6 @@ then the corresponding setup script would be :: packages=['foobar', 'foobar.subfoo'], ) -(Again, the empty string in :option:`package_dir` stands for the current -directory.) - .. _single-ext: diff --git a/Doc/distutils/index.rst b/Doc/distutils/index.rst index ace8280..1a6f04c 100644 --- a/Doc/distutils/index.rst +++ b/Doc/distutils/index.rst @@ -6,14 +6,22 @@ :Authors: Greg Ward, Anthony Baxter :Email: distutils-sig@python.org -:Release: |version| -:Date: |today| This document describes the Python Distribution Utilities ("Distutils") from the module developer's point of view, describing how to use the Distutils to make Python modules and extensions easily available to a wider audience with very little overhead for build/release/install mechanics. +.. note:: + + This guide only covers the basic tools for building and distributing + extensions that are provided as part of this version of Python. Third + party tools offer easier to use and more secure alternatives. Refer to the + `quick recommendations section + <https://python-packaging-user-guide.readthedocs.org/en/latest/current.html>`__ + in the Python Packaging User Guide for more information. + + .. toctree:: :maxdepth: 2 :numbered: @@ -24,7 +32,6 @@ very little overhead for build/release/install mechanics. sourcedist.rst builtdist.rst packageindex.rst - uploading.rst examples.rst extending.rst commandref.rst diff --git a/Doc/distutils/packageindex.rst b/Doc/distutils/packageindex.rst index 1498394..1d724e2 100644 --- a/Doc/distutils/packageindex.rst +++ b/Doc/distutils/packageindex.rst @@ -1,12 +1,33 @@ +.. index:: + single: Python Package Index (PyPI) + single: PyPI; (see Python Package Index (PyPI)) + .. _package-index: -********************************** -Registering with the Package Index -********************************** +******************************* +The Python Package Index (PyPI) +******************************* + +The `Python Package Index (PyPI)`_ holds :ref:`meta-data <meta-data>` +describing distributions packaged with distutils, as well as package data like +distribution files if the package author wishes. + +Distutils exposes two commands for submitting package data to PyPI: the +:ref:`register <package-register>` command for submitting meta-data to PyPI +and the :ref:`upload <package-upload>` command for submitting distribution +files. Both commands read configuration data from a special file called the +:ref:`.pypirc file <pypirc>`. PyPI :ref:`displays a home page +<package-display>` for each package created from the ``long_description`` +submitted by the :command:`register` command. + + +.. _package-register: -The Python Package Index (PyPI) holds meta-data describing distributions -packaged with distutils. The distutils command :command:`register` is used to -submit your distribution's meta-data to the index. It is invoked as follows:: +Registering Packages +==================== + +The distutils command :command:`register` is used to submit your distribution's +meta-data to the index. It is invoked as follows:: python setup.py register @@ -43,10 +64,58 @@ the web interface. They may also designate other users as Owners or Maintainers. Maintainers may edit the package information, but not designate other Owners or Maintainers. -By default PyPI will list all versions of a given package. To hide certain -versions, the Hidden property should be set to yes. This must be edited through -the web interface. +By default PyPI displays only the newest version of a given package. The web +interface lets one change this default behavior and manually select which +versions to display and hide. + + +.. _package-upload: + +Uploading Packages +================== + +.. versionadded:: 2.5 + +The distutils command :command:`upload` pushes the distribution files to PyPI. + +The command is invoked immediately after building one or more distribution +files. For example, the command :: + + python setup.py sdist bdist_wininst upload + +will cause the source distribution and the Windows installer to be uploaded to +PyPI. Note that these will be uploaded even if they are built using an earlier +invocation of :file:`setup.py`, but that only distributions named on the command +line for the invocation including the :command:`upload` command are uploaded. + +The :command:`upload` command uses the username, password, and repository URL +from the :file:`$HOME/.pypirc` file (see section :ref:`pypirc` for more on this +file). If a :command:`register` command was previously called in the same command, +and if the password was entered in the prompt, :command:`upload` will reuse the +entered password. This is useful if you do not want to store a clear text +password in the :file:`$HOME/.pypirc` file. + +You can specify another PyPI server with the ``--repository=url`` option:: + + python setup.py sdist bdist_wininst upload -r http://example.com/pypi + +See section :ref:`pypirc` for more on defining several servers. +You can use the ``--sign`` option to tell :command:`upload` to sign each +uploaded file using GPG (GNU Privacy Guard). The :program:`gpg` program must +be available for execution on the system :envvar:`PATH`. You can also specify +which key to use for signing using the ``--identity=name`` option. + +Other :command:`upload` options include ``--repository=url`` or +``--repository=section`` where *url* is the url of the server and +*section* the name of the section in :file:`$HOME/.pypirc`, and +``--show-response`` (which displays the full response text from the PyPI +server for help in debugging upload problems). + + +.. index:: + single: .pypirc file + single: Python Package Index (PyPI); .pypirc file .. _pypirc: @@ -102,3 +171,45 @@ For convenience, the name of the section that describes the repository may also be used:: python setup.py register -r other + + +.. _package-display: + +PyPI package display +==================== + +The ``long_description`` field plays a special role at PyPI. It is used by +the server to display a home page for the registered package. + +If you use the `reStructuredText <http://docutils.sourceforge.net/rst.html>`_ +syntax for this field, PyPI will parse it and display an HTML output for +the package home page. + +The ``long_description`` field can be attached to a text file located +in the package:: + + from distutils.core import setup + + with open('README.txt') as file: + long_description = file.read() + + setup(name='Distutils', + long_description=long_description) + +In that case, :file:`README.txt` is a regular reStructuredText text file located +in the root of the package besides :file:`setup.py`. + +To prevent registering broken reStructuredText content, you can use the +:program:`rst2html` program that is provided by the :mod:`docutils` package and +check the ``long_description`` from the command line:: + + $ python setup.py --long-description | rst2html.py > output.html + +:mod:`docutils` will display a warning if there's something wrong with your +syntax. Because PyPI applies additional checks (e.g. by passing ``--no-raw`` +to ``rst2html.py`` in the command above), being able to run the command above +without warnings does not guarantee that PyPI will convert the content +successfully. + + +.. _Python Package Index (PyPI): http://pypi.python.org/ diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst index 165bfcd..15f130f 100644 --- a/Doc/distutils/setupscript.rst +++ b/Doc/distutils/setupscript.rst @@ -139,7 +139,8 @@ directories, libraries to link with, etc.). All of this is done through another keyword argument to :func:`setup`, the :option:`ext_modules` option. :option:`ext_modules` is just a list of -:class:`Extension` instances, each of which describes a single extension module. +:class:`~distutils.core.Extension` instances, each of which describes a +single extension module. Suppose your distribution includes a single extension, called :mod:`foo` and implemented by :file:`foo.c`. If no additional instructions to the compiler/linker are needed, describing this extension is quite simple:: @@ -165,8 +166,8 @@ following sections. Extension names and packages ---------------------------- -The first argument to the :class:`Extension` constructor is always the name of -the extension, including any package names. For example, :: +The first argument to the :class:`~distutils.core.Extension` constructor is +always the name of the extension, including any package names. For example, :: Extension('foo', ['src/foo1.c', 'src/foo2.c']) @@ -196,7 +197,8 @@ will compile :file:`foo.c` to the extension :mod:`pkg.foo`, and :file:`bar.c` to Extension source files ---------------------- -The second argument to the :class:`Extension` constructor is a list of source +The second argument to the :class:`~distutils.core.Extension` constructor is +a list of source files. Since the Distutils currently only support C, C++, and Objective-C extensions, these are normally C/C++/Objective-C source files. (Be sure to use appropriate extensions to distinguish C++\ source files: :file:`.cc` and @@ -232,9 +234,9 @@ linked into the executable. Preprocessor options -------------------- -Three optional arguments to :class:`Extension` will help if you need to specify -include directories to search or preprocessor macros to define/undefine: -``include_dirs``, ``define_macros``, and ``undef_macros``. +Three optional arguments to :class:`~distutils.core.Extension` will help if +you need to specify include directories to search or preprocessor macros to +define/undefine: ``include_dirs``, ``define_macros``, and ``undef_macros``. For example, if your extension requires header files in the :file:`include` directory under your distribution root, use the ``include_dirs`` option:: @@ -598,7 +600,8 @@ Notes: It is recommended that versions take the form *major.minor[.patch[.sub]]*. (3) - Either the author or the maintainer must be identified. + Either the author or the maintainer must be identified. If maintainer is + provided, distutils lists it as the author in :file:`PKG-INFO`. (4) These fields should not be used if your package is to be compatible with Python @@ -606,8 +609,9 @@ Notes: <http://pypi.python.org/pypi>`_. (5) - The ``long_description`` field is used by PyPI when you are registering a - package, to build its home page. + The ``long_description`` field is used by PyPI when you are + :ref:`registering <package-register>` a package, to + :ref:`build its home page <package-display>`. (6) The ``license`` field is a text indicating the license covering the @@ -680,6 +684,8 @@ include the following code fragment in your :file:`setup.py` before the DistributionMetadata.download_url = None +.. _debug-setup-script: + Debugging the setup script ========================== @@ -695,7 +701,8 @@ installation is broken because they don't read all the way down to the bottom and see that it's a permission problem. On the other hand, this doesn't help the developer to find the cause of the -failure. For this purpose, the DISTUTILS_DEBUG environment variable can be set +failure. For this purpose, the :envvar:`DISTUTILS_DEBUG` environment variable can be set to anything except an empty string, and distutils will now print detailed -information what it is doing, and prints the full traceback in case an exception -occurs. +information about what it is doing, dump the full traceback when an exception +occurs, and print the whole command line when an external program (like a C +compiler) fails. diff --git a/Doc/distutils/sourcedist.rst b/Doc/distutils/sourcedist.rst index a9858d0..b1695a2 100644 --- a/Doc/distutils/sourcedist.rst +++ b/Doc/distutils/sourcedist.rst @@ -51,8 +51,7 @@ Notes: of the standard Python library since Python 1.6) (4) - requires the :program:`compress` program. Notice that this format is now - pending for deprecation and will be removed in the future versions of Python. + requires the :program:`compress` program. When using any ``tar`` format (``gztar``, ``bztar``, ``ztar`` or ``tar``) under Unix, you can specify the ``owner`` and ``group`` names diff --git a/Doc/distutils/uploading.rst b/Doc/distutils/uploading.rst index 936402b..4bce699 100644 --- a/Doc/distutils/uploading.rst +++ b/Doc/distutils/uploading.rst @@ -1,77 +1,7 @@ -.. _package-upload: +:orphan: *************************************** Uploading Packages to the Package Index *************************************** -.. versionadded:: 2.5 - -The Python Package Index (PyPI) not only stores the package info, but also the -package data if the author of the package wishes to. The distutils command -:command:`upload` pushes the distribution files to PyPI. - -The command is invoked immediately after building one or more distribution -files. For example, the command :: - - python setup.py sdist bdist_wininst upload - -will cause the source distribution and the Windows installer to be uploaded to -PyPI. Note that these will be uploaded even if they are built using an earlier -invocation of :file:`setup.py`, but that only distributions named on the command -line for the invocation including the :command:`upload` command are uploaded. - -The :command:`upload` command uses the username, password, and repository URL -from the :file:`$HOME/.pypirc` file (see section :ref:`pypirc` for more on this -file). If a :command:`register` command was previously called in the same command, -and if the password was entered in the prompt, :command:`upload` will reuse the -entered password. This is useful if you do not want to store a clear text -password in the :file:`$HOME/.pypirc` file. - -You can specify another PyPI server with the :option:`--repository=*url*` option:: - - python setup.py sdist bdist_wininst upload -r http://example.com/pypi - -See section :ref:`pypirc` for more on defining several servers. - -You can use the :option:`--sign` option to tell :command:`upload` to sign each -uploaded file using GPG (GNU Privacy Guard). The :program:`gpg` program must -be available for execution on the system :envvar:`PATH`. You can also specify -which key to use for signing using the :option:`--identity=*name*` option. - -Other :command:`upload` options include :option:`--repository=<url>` or -:option:`--repository=<section>` where *url* is the url of the server and -*section* the name of the section in :file:`$HOME/.pypirc`, and -:option:`--show-response` (which displays the full response text from the PyPI -server for help in debugging upload problems). - -PyPI package display -==================== - -The ``long_description`` field plays a special role at PyPI. It is used by -the server to display a home page for the registered package. - -If you use the `reStructuredText <http://docutils.sourceforge.net/rst.html>`_ -syntax for this field, PyPI will parse it and display an HTML output for -the package home page. - -The ``long_description`` field can be attached to a text file located -in the package:: - - from distutils.core import setup - - with open('README.txt') as file: - long_description = file.read() - - setup(name='Distutils', - long_description=long_description) - -In that case, :file:`README.txt` is a regular reStructuredText text file located -in the root of the package besides :file:`setup.py`. - -To prevent registering broken reStructuredText content, you can use the -:program:`rst2html` program that is provided by the :mod:`docutils` package -and check the ``long_description`` from the command line:: - - $ python setup.py --long-description | rst2html.py > output.html - -:mod:`docutils` will display a warning if there's something wrong with your syntax. +The contents of this page have moved to the section :ref:`package-index`. diff --git a/Doc/extending/building.rst b/Doc/extending/building.rst index f4d95b2..08b0cc2 100644 --- a/Doc/extending/building.rst +++ b/Doc/extending/building.rst @@ -58,8 +58,9 @@ distutils; this section explains building extension modules only. It is common to pre-compute arguments to :func:`setup`, to better structure the driver script. In the example above, the\ ``ext_modules`` argument to :func:`setup` is a list of extension modules, each of which is an instance of -the :class:`Extension`. In the example, the instance defines an extension named -``demo`` which is build by compiling a single source file, :file:`demo.c`. +the :class:`~distutils.extension.Extension`. In the example, the instance +defines an extension named ``demo`` which is build by compiling a single source +file, :file:`demo.c`. In many cases, building an extension is more complex, since additional preprocessor defines and libraries may be needed. This is demonstrated in the diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst index 4bd0199..981e1d5 100644 --- a/Doc/extending/embedding.rst +++ b/Doc/extending/embedding.rst @@ -61,6 +61,7 @@ perform some operation on a file. :: int main(int argc, char *argv[]) { + Py_SetProgramName(argv[0]); /* optional but recommended */ Py_Initialize(); PyRun_SimpleString("from time import time,ctime\n" "print 'Today is',ctime(time())\n"); @@ -68,9 +69,11 @@ perform some operation on a file. :: return 0; } -The above code first initializes the Python interpreter with +The :c:func:`Py_SetProgramName` function should be called before +:c:func:`Py_Initialize` to inform the interpreter about paths to Python run-time +libraries. Next, the Python interpreter is initialized with :c:func:`Py_Initialize`, followed by the execution of a hard-coded Python script -that print the date and time. Afterwards, the :c:func:`Py_Finalize` call shuts +that prints the date and time. Afterwards, the :c:func:`Py_Finalize` call shuts the interpreter down, followed by the end of the program. In a real program, you may want to get the Python script from another source, perhaps a text-editor routine, a file, or a database. Getting the Python code from a file can better @@ -137,7 +140,9 @@ The code to run a function defined in a Python script is: This code loads a Python script using ``argv[1]``, and calls the function named in ``argv[2]``. Its integer arguments are the other values of the ``argv`` array. If you compile and link this program (let's call the finished executable -:program:`call`), and use it to execute a Python script, such as:: +:program:`call`), and use it to execute a Python script, such as: + +.. code-block:: python def multiply(a,b): print "Will compute", a, "times", b @@ -226,7 +231,9 @@ following two statements directly after :c:func:`Py_Initialize`:: These two lines initialize the ``numargs`` variable, and make the :func:`emb.numargs` function accessible to the embedded Python interpreter. -With these extensions, the Python script can do things like :: +With these extensions, the Python script can do things like + +.. code-block:: python import emb print "Number of arguments", emb.numargs() @@ -251,35 +258,55 @@ program. There is no need to recompile Python itself using C++. .. _link-reqs: -Linking Requirements -==================== - -While the :program:`configure` script shipped with the Python sources will -correctly build Python to export the symbols needed by dynamically linked -extensions, this is not automatically inherited by applications which embed the -Python library statically, at least on Unix. This is an issue when the -application is linked to the static runtime library (:file:`libpython.a`) and -needs to load dynamic extensions (implemented as :file:`.so` files). - -The problem is that some entry points are defined by the Python runtime solely -for extension modules to use. If the embedding application does not use any of -these entry points, some linkers will not include those entries in the symbol -table of the finished executable. Some additional options are needed to inform -the linker not to remove these symbols. - -Determining the right options to use for any given platform can be quite -difficult, but fortunately the Python configuration already has those values. -To retrieve them from an installed Python interpreter, start an interactive -interpreter and have a short session like this:: - - >>> import distutils.sysconfig - >>> distutils.sysconfig.get_config_var('LINKFORSHARED') - '-Xlinker -export-dynamic' +Compiling and Linking under Unix-like systems +============================================= -.. index:: module: distutils.sysconfig +It is not necessarily trivial to find the right flags to pass to your +compiler (and linker) in order to embed the Python interpreter into your +application, particularly because Python needs to load library modules +implemented as C dynamic extensions (:file:`.so` files) linked against +it. + +To find out the required compiler and linker flags, you can execute the +:file:`python{X.Y}-config` script which is generated as part of the +installation process (a :file:`python-config` script may also be +available). This script has several options, of which the following will +be directly useful to you: + +* ``pythonX.Y-config --cflags`` will give you the recommended flags when + compiling:: + + $ /opt/bin/python2.7-config --cflags + -I/opt/include/python2.7 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes + +* ``pythonX.Y-config --ldflags`` will give you the recommended flags when + linking:: + + $ /opt/bin/python2.7-config --ldflags + -L/opt/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic + +.. note:: + To avoid confusion between several Python installations (and especially + between the system Python and your own compiled Python), it is recommended + that you use the absolute path to :file:`python{X.Y}-config`, as in the above + example. + +If this procedure doesn't work for you (it is not guaranteed to work for +all Unix-like platforms; however, we welcome :ref:`bug reports <reporting-bugs>`) +you will have to read your system's documentation about dynamic linking and/or +examine Python's :file:`Makefile` (use :func:`sysconfig.get_makefile_filename` +to find its location) and compilation +options. In this case, the :mod:`sysconfig` module is a useful tool to +programmatically extract the configuration values that you will want to +combine together. For example: + +.. code-block:: python + + >>> import sysconfig + >>> sysconfig.get_config_var('LIBS') + '-lpthread -ldl -lutil' + >>> sysconfig.get_config_var('LINKFORSHARED') + '-Xlinker -export-dynamic' -The contents of the string presented will be the options that should be used. -If the string is empty, there's no need to add any additional options. The -:const:`LINKFORSHARED` definition corresponds to the variable of the same name -in Python's top-level :file:`Makefile`. +.. XXX similar documentation for Windows missing diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index eb18a46..8e8c3ab 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -372,6 +372,8 @@ to :c:func:`Py_Initialize`:: /* Add a static module */ initspam(); + ... + An example may be found in the file :file:`Demo/embed/demo.c` in the Python source distribution. @@ -510,7 +512,7 @@ or more format codes between parentheses. For example:: value of the Python function. :c:func:`PyObject_CallObject` is "reference-count-neutral" with respect to its arguments. In the example a new tuple was created to serve as the argument list, which is :c:func:`Py_DECREF`\ --ed immediately after the call. +-ed immediately after the :c:func:`PyObject_CallObject` call. The return value of :c:func:`PyObject_CallObject` is "new": either it is a brand new object, or it is an existing object whose reference count has been @@ -843,9 +845,9 @@ the cycle itself. The cycle detector is able to detect garbage cycles and can reclaim them so long as there are no finalizers implemented in Python (:meth:`__del__` methods). When there are such finalizers, the detector exposes the cycles through the -:mod:`gc` module (specifically, the -``garbage`` variable in that module). The :mod:`gc` module also exposes a way -to run the detector (the :func:`collect` function), as well as configuration +:mod:`gc` module (specifically, the :attr:`~gc.garbage` variable in that module). +The :mod:`gc` module also exposes a way to run the detector (the +:func:`~gc.collect` function), as well as configuration interfaces and the ability to disable the detector at runtime. The cycle detector is considered an optional component; though it is included by default, it can be disabled at build time using the :option:`--without-cycle-gc` option diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst index 92e6132..44a7f92 100644 --- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -4,16 +4,13 @@ Extending and Embedding the Python Interpreter ################################################## -:Release: |version| -:Date: |today| - This document describes how to write modules in C or C++ to extend the Python -interpreter with new modules. Those modules can define new functions but also -new object types and their methods. The document also describes how to embed -the Python interpreter in another application, for use as an extension language. -Finally, it shows how to compile and link extension modules so that they can be -loaded dynamically (at run time) into the interpreter, if the underlying -operating system supports this feature. +interpreter with new modules. Those modules can not only define new functions +but also new object types and their methods. The document also describes how +to embed the Python interpreter in another application, for use as an extension +language. Finally, it shows how to compile and link extension modules so that +they can be loaded dynamically (at run time) into the interpreter, if the +underlying operating system supports this feature. This document assumes basic knowledge about Python. For an informal introduction to the language, see :ref:`tutorial-index`. :ref:`reference-index` @@ -24,6 +21,15 @@ Python) that give the language its wide application range. For a detailed description of the whole Python/C API, see the separate :ref:`c-api-index`. +.. note:: + + This guide only covers the basic tools for creating extensions provided + as part of this version of CPython. Third party tools may offer simpler + alternatives. Refer to the `binary extensions section + <https://python-packaging-user-guide.readthedocs.org/en/latest/extensions.html>`__ + in the Python Packaging User Guide for more information. + + .. toctree:: :maxdepth: 2 :numbered: diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst index f18814f..d76aa24 100644 --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -150,11 +150,11 @@ This is so that Python knows how much memory to allocate when you call .. note:: If you want your type to be subclassable from Python, and your type has the same - :attr:`tp_basicsize` as its base type, you may have problems with multiple + :c:member:`~PyTypeObject.tp_basicsize` as its base type, you may have problems with multiple inheritance. A Python subclass of your type will have to list your type first - in its :attr:`__bases__`, or else it will not be able to call your type's + in its :attr:`~class.__bases__`, or else it will not be able to call your type's :meth:`__new__` method without getting an error. You can avoid this problem by - ensuring that your type has a larger value for :attr:`tp_basicsize` than its + ensuring that your type has a larger value for :c:member:`~PyTypeObject.tp_basicsize` than its base type does. Most of the time, this will be true anyway, because either your base type will be :class:`object`, or else you will be adding data members to your base type, and therefore increasing its size. @@ -174,7 +174,7 @@ to :const:`Py_TPFLAGS_DEFAULT`. :: All types should include this constant in their flags. It enables all of the members defined by the current version of Python. -We provide a doc string for the type in :attr:`tp_doc`. :: +We provide a doc string for the type in :c:member:`~PyTypeObject.tp_doc`. :: "Noddy objects", /* tp_doc */ @@ -183,12 +183,12 @@ from the others. We aren't going to implement any of these in this version of the module. We'll expand this example later to have more interesting behavior. For now, all we want to be able to do is to create new :class:`Noddy` objects. -To enable object creation, we have to provide a :attr:`tp_new` implementation. +To enable object creation, we have to provide a :c:member:`~PyTypeObject.tp_new` implementation. In this case, we can just use the default implementation provided by the API function :c:func:`PyType_GenericNew`. We'd like to just assign this to the -:attr:`tp_new` slot, but we can't, for portability sake, On some platforms or +:c:member:`~PyTypeObject.tp_new` slot, but we can't, for portability sake, On some platforms or compilers, we can't statically initialize a structure member with a function -defined in another C module, so, instead, we'll assign the :attr:`tp_new` slot +defined in another C module, so, instead, we'll assign the :c:member:`~PyTypeObject.tp_new` slot in the module initialization function just before calling :c:func:`PyType_Ready`:: @@ -283,13 +283,13 @@ allocation and deallocation. At a minimum, we need a deallocation method:: self->ob_type->tp_free((PyObject*)self); } -which is assigned to the :attr:`tp_dealloc` member:: +which is assigned to the :c:member:`~PyTypeObject.tp_dealloc` member:: (destructor)Noddy_dealloc, /*tp_dealloc*/ This method decrements the reference counts of the two Python attributes. We use :c:func:`Py_XDECREF` here because the :attr:`first` and :attr:`last` members -could be *NULL*. It then calls the :attr:`tp_free` member of the object's type +could be *NULL*. It then calls the :c:member:`~PyTypeObject.tp_free` member of the object's type to free the object's memory. Note that the object's type might not be :class:`NoddyType`, because the object may be an instance of a subclass. @@ -323,7 +323,7 @@ strings, so we provide a new method:: return (PyObject *)self; } -and install it in the :attr:`tp_new` member:: +and install it in the :c:member:`~PyTypeObject.tp_new` member:: Noddy_new, /* tp_new */ @@ -344,16 +344,16 @@ created. New methods always accept positional and keyword arguments, but they often ignore the arguments, leaving the argument handling to initializer methods. Note that if the type supports subclassing, the type passed may not be the type being defined. The new method calls the tp_alloc slot to allocate -memory. We don't fill the :attr:`tp_alloc` slot ourselves. Rather +memory. We don't fill the :c:member:`~PyTypeObject.tp_alloc` slot ourselves. Rather :c:func:`PyType_Ready` fills it for us by inheriting it from our base class, which is :class:`object` by default. Most types use the default allocation. .. note:: - If you are creating a co-operative :attr:`tp_new` (one that calls a base type's - :attr:`tp_new` or :meth:`__new__`), you must *not* try to determine what method + If you are creating a co-operative :c:member:`~PyTypeObject.tp_new` (one that calls a base type's + :c:member:`~PyTypeObject.tp_new` or :meth:`__new__`), you must *not* try to determine what method to call using method resolution order at runtime. Always statically determine - what type you are going to call, and call its :attr:`tp_new` directly, or via + what type you are going to call, and call its :c:member:`~PyTypeObject.tp_new` directly, or via ``type->tp_base->tp_new``. If you do not do this, Python subclasses of your type that also inherit from other Python-defined classes may not work correctly. (Specifically, you may not be able to create instances of such subclasses @@ -390,11 +390,11 @@ We provide an initialization function:: return 0; } -by filling the :attr:`tp_init` slot. :: +by filling the :c:member:`~PyTypeObject.tp_init` slot. :: (initproc)Noddy_init, /* tp_init */ -The :attr:`tp_init` slot is exposed in Python as the :meth:`__init__` method. It +The :c:member:`~PyTypeObject.tp_init` slot is exposed in Python as the :meth:`__init__` method. It is used to initialize an object after it's created. Unlike the new method, we can't guarantee that the initializer is called. The initializer isn't called when unpickling objects and it can be overridden. Our initializer accepts @@ -424,7 +424,7 @@ reference counts. When don't we have to do this? * when we know that deallocation of the object [#]_ will not cause any calls back into our type's code -* when decrementing a reference count in a :attr:`tp_dealloc` handler when +* when decrementing a reference count in a :c:member:`~PyTypeObject.tp_dealloc` handler when garbage-collections is not supported [#]_ We want to expose our instance variables as attributes. There are a @@ -440,7 +440,7 @@ number of ways to do that. The simplest way is to define member definitions:: {NULL} /* Sentinel */ }; -and put the definitions in the :attr:`tp_members` slot:: +and put the definitions in the :c:member:`~PyTypeObject.tp_members` slot:: Noddy_members, /* tp_members */ @@ -516,7 +516,7 @@ definitions:: {NULL} /* Sentinel */ }; -and assign them to the :attr:`tp_methods` slot:: +and assign them to the :c:member:`~PyTypeObject.tp_methods` slot:: Noddy_methods, /* tp_methods */ @@ -611,7 +611,7 @@ We create an array of :c:type:`PyGetSetDef` structures:: {NULL} /* Sentinel */ }; -and register it in the :attr:`tp_getset` slot:: +and register it in the :c:member:`~PyTypeObject.tp_getset` slot:: Noddy_getseters, /* tp_getset */ @@ -628,7 +628,7 @@ We also remove the member definitions for these attributes:: {NULL} /* Sentinel */ }; -We also need to update the :attr:`tp_init` handler to only allow strings [#]_ to +We also need to update the :c:member:`~PyTypeObject.tp_init` handler to only allow strings [#]_ to be passed:: static int @@ -747,7 +747,7 @@ simplified:: .. note:: - Note that the :attr:`tp_traverse` implementation must name its arguments exactly + Note that the :c:member:`~PyTypeObject.tp_traverse` implementation must name its arguments exactly *visit* and *arg* in order to use :c:func:`Py_VISIT`. This is to encourage uniformity across these boring implementations. @@ -784,7 +784,7 @@ its reference count. We do this because, as was discussed earlier, if the reference count drops to zero, we might cause code to run that calls back into the object. In addition, because we now support garbage collection, we also have to worry about code being run that triggers garbage collection. If garbage -collection is run, our :attr:`tp_traverse` handler could get called. We can't +collection is run, our :c:member:`~PyTypeObject.tp_traverse` handler could get called. We can't take a chance of having :c:func:`Noddy_traverse` called when a member's reference count has dropped to zero and its value hasn't been set to *NULL*. @@ -804,8 +804,8 @@ Finally, we add the :const:`Py_TPFLAGS_HAVE_GC` flag to the class flags:: Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, /*tp_flags*/ -That's pretty much it. If we had written custom :attr:`tp_alloc` or -:attr:`tp_free` slots, we'd need to modify them for cyclic-garbage collection. +That's pretty much it. If we had written custom :c:member:`~PyTypeObject.tp_alloc` or +:c:member:`~PyTypeObject.tp_free` slots, we'd need to modify them for cyclic-garbage collection. Most extensions will use the versions automatically provided. @@ -864,8 +864,8 @@ the :attr:`__init__` method of the base type. This pattern is important when writing a type with custom :attr:`new` and :attr:`dealloc` methods. The :attr:`new` method should not actually create the -memory for the object with :attr:`tp_alloc`, that will be handled by the base -class when calling its :attr:`tp_new`. +memory for the object with :c:member:`~PyTypeObject.tp_alloc`, that will be handled by the base +class when calling its :c:member:`~PyTypeObject.tp_new`. When filling out the :c:func:`PyTypeObject` for the :class:`Shoddy` type, you see a slot for :c:func:`tp_base`. Due to cross platform compiler issues, you can't @@ -890,8 +890,8 @@ the module's :c:func:`init` function. :: } Before calling :c:func:`PyType_Ready`, the type structure must have the -:attr:`tp_base` slot filled in. When we are deriving a new type, it is not -necessary to fill out the :attr:`tp_alloc` slot with :c:func:`PyType_GenericNew` +:c:member:`~PyTypeObject.tp_base` slot filled in. When we are deriving a new type, it is not +necessary to fill out the :c:member:`~PyTypeObject.tp_alloc` slot with :c:func:`PyType_GenericNew` -- the allocate function from the base type will be inherited. After that, calling :c:func:`PyType_Ready` and adding the type object to the @@ -934,7 +934,7 @@ that will be helpful in such a situation! :: These fields tell the runtime how much memory to allocate when new objects of this type are created. Python has some built-in support for variable length -structures (think: strings, lists) which is where the :attr:`tp_itemsize` field +structures (think: strings, lists) which is where the :c:member:`~PyTypeObject.tp_itemsize` field comes in. This will be dealt with later. :: char *tp_doc; @@ -1032,13 +1032,13 @@ that creating a temporary string object to be written to a file is too expensive. These handlers are all optional, and most types at most need to implement the -:attr:`tp_str` and :attr:`tp_repr` handlers. :: +:c:member:`~PyTypeObject.tp_str` and :c:member:`~PyTypeObject.tp_repr` handlers. :: reprfunc tp_repr; reprfunc tp_str; printfunc tp_print; -The :attr:`tp_repr` handler should return a string object containing a +The :c:member:`~PyTypeObject.tp_repr` handler should return a string object containing a representation of the instance for which it is called. Here is a simple example:: @@ -1049,15 +1049,15 @@ example:: obj->obj_UnderlyingDatatypePtr->size); } -If no :attr:`tp_repr` handler is specified, the interpreter will supply a -representation that uses the type's :attr:`tp_name` and a uniquely-identifying +If no :c:member:`~PyTypeObject.tp_repr` handler is specified, the interpreter will supply a +representation that uses the type's :c:member:`~PyTypeObject.tp_name` and a uniquely-identifying value for the object. -The :attr:`tp_str` handler is to :func:`str` what the :attr:`tp_repr` handler +The :c:member:`~PyTypeObject.tp_str` handler is to :func:`str` what the :c:member:`~PyTypeObject.tp_repr` handler described above is to :func:`repr`; that is, it is called when Python code calls :func:`str` on an instance of your object. Its implementation is very similar -to the :attr:`tp_repr` function, but the resulting string is intended for human -consumption. If :attr:`tp_str` is not specified, the :attr:`tp_repr` handler is +to the :c:member:`~PyTypeObject.tp_repr` function, but the resulting string is intended for human +consumption. If :c:member:`~PyTypeObject.tp_str` is not specified, the :c:member:`~PyTypeObject.tp_repr` handler is used instead. Here is a simple example:: @@ -1152,7 +1152,7 @@ type object to create :term:`descriptor`\s which are placed in the dictionary of type object. Each descriptor controls access to one attribute of the instance object. Each of the tables is optional; if all three are *NULL*, instances of the type will only have attributes that are inherited from their base type, and -should leave the :attr:`tp_getattro` and :attr:`tp_setattro` fields *NULL* as +should leave the :c:member:`~PyTypeObject.tp_getattro` and :c:member:`~PyTypeObject.tp_setattro` fields *NULL* as well, allowing the base type to handle attributes. The tables are declared as three fields of the type object:: @@ -1161,7 +1161,7 @@ The tables are declared as three fields of the type object:: struct PyMemberDef *tp_members; struct PyGetSetDef *tp_getset; -If :attr:`tp_methods` is not *NULL*, it must refer to an array of +If :c:member:`~PyTypeObject.tp_methods` is not *NULL*, it must refer to an array of :c:type:`PyMethodDef` structures. Each entry in the table is an instance of this structure:: @@ -1225,13 +1225,13 @@ combined using bitwise-OR. single: WRITE_RESTRICTED single: RESTRICTED -An interesting advantage of using the :attr:`tp_members` table to build +An interesting advantage of using the :c:member:`~PyTypeObject.tp_members` table to build descriptors that are used at runtime is that any attribute defined this way can have an associated doc string simply by providing the text in the table. An application can use the introspection API to retrieve the descriptor from the class object, and get the doc string using its :attr:`__doc__` attribute. -As with the :attr:`tp_methods` table, a sentinel entry with a :attr:`name` value +As with the :c:member:`~PyTypeObject.tp_methods` table, a sentinel entry with a :attr:`name` value of *NULL* is required. .. XXX Descriptors need to be explained in more detail somewhere, but not here. @@ -1257,7 +1257,7 @@ portable to older versions of Python, and explains how the handler functions are called, so that if you do need to extend their functionality, you'll understand what needs to be done. -The :attr:`tp_getattr` handler is called when the object requires an attribute +The :c:member:`~PyTypeObject.tp_getattr` handler is called when the object requires an attribute look-up. It is called in the same situations where the :meth:`__getattr__` method of a class would be called. @@ -1265,7 +1265,7 @@ A likely way to handle this is (1) to implement a set of functions (such as :c:func:`newdatatype_getSize` and :c:func:`newdatatype_setSize` in the example below), (2) provide a method table listing these functions, and (3) provide a getattr function that returns the result of a lookup in that table. The method -table uses the same structure as the :attr:`tp_methods` field of the type +table uses the same structure as the :c:member:`~PyTypeObject.tp_methods` field of the type object. Here is an example:: @@ -1284,11 +1284,11 @@ Here is an example:: return Py_FindMethod(newdatatype_methods, (PyObject *)obj, name); } -The :attr:`tp_setattr` handler is called when the :meth:`__setattr__` or +The :c:member:`~PyTypeObject.tp_setattr` handler is called when the :meth:`__setattr__` or :meth:`__delattr__` method of a class instance would be called. When an attribute should be deleted, the third parameter will be *NULL*. Here is an example that simply raises an exception; if this were really all you wanted, the -:attr:`tp_setattr` handler should be set to *NULL*. :: +:c:member:`~PyTypeObject.tp_setattr` handler should be set to *NULL*. :: static int newdatatype_setattr(newdatatypeobject *obj, char *name, PyObject *v) @@ -1305,7 +1305,7 @@ Object Comparison cmpfunc tp_compare; -The :attr:`tp_compare` handler is called when comparisons are needed and the +The :c:member:`~PyTypeObject.tp_compare` handler is called when comparisons are needed and the object does not implement the specific rich comparison method which matches the requested comparison. (It is always used if defined and the :c:func:`PyObject_Compare` or :c:func:`PyObject_Cmp` functions are used, or if @@ -1316,7 +1316,7 @@ allowed to return arbitrary negative or positive integers for less than and greater than, respectively; as of Python 2.2, this is no longer allowed. In the future, other return values may be assigned a different meaning.) -A :attr:`tp_compare` handler may raise an exception. In this case it should +A :c:member:`~PyTypeObject.tp_compare` handler may raise an exception. In this case it should return a negative value. The caller has to test for the exception using :c:func:`PyErr_Occurred`. @@ -1360,9 +1360,9 @@ that the slots are present and should be checked by the interpreter. (The flag bit does not indicate that the slot values are non-*NULL*. The flag may be set to indicate the presence of a slot, but a slot may still be unfilled.) :: - PyNumberMethods tp_as_number; - PySequenceMethods tp_as_sequence; - PyMappingMethods tp_as_mapping; + PyNumberMethods *tp_as_number; + PySequenceMethods *tp_as_sequence; + PyMappingMethods *tp_as_mapping; If you wish your object to be able to act like a number, a sequence, or a mapping object, then you place the address of a structure that implements the C @@ -1391,7 +1391,7 @@ instance of your data type. Here is a moderately pointless example:: This function is called when an instance of your data type is "called", for example, if ``obj1`` is an instance of your data type and the Python script -contains ``obj1('hello')``, the :attr:`tp_call` handler is invoked. +contains ``obj1('hello')``, the :c:member:`~PyTypeObject.tp_call` handler is invoked. This function takes three arguments: @@ -1480,7 +1480,7 @@ those objects which do not benefit by weak referencing (such as numbers). For an object to be weakly referencable, the extension must include a :c:type:`PyObject\*` field in the instance structure for the use of the weak reference mechanism; it must be initialized to *NULL* by the object's -constructor. It must also set the :attr:`tp_weaklistoffset` field of the +constructor. It must also set the :c:member:`~PyTypeObject.tp_weaklistoffset` field of the corresponding type object to the offset of the field. For example, the instance type is defined with the following structure:: @@ -1521,9 +1521,8 @@ The type constructor is responsible for initializing the weak reference list to } The only further addition is that the destructor needs to call the weak -reference manager to clear any weak references. This should be done before any -other parts of the destruction have occurred, but is only required if the weak -reference list is non-*NULL*:: +reference manager to clear any weak references. This is only required if the +weak reference list is non-*NULL*:: static void instance_dealloc(PyInstanceObject *inst) @@ -1567,7 +1566,7 @@ might be something like the following:: .. [#] This is true when we know that the object is a basic type, like a string or a float. -.. [#] We relied on this in the :attr:`tp_dealloc` handler in this example, because our +.. [#] We relied on this in the :c:member:`~PyTypeObject.tp_dealloc` handler in this example, because our type doesn't support garbage collection. Even if a type supports garbage collection, there are calls that can be made to "untrack" the object from garbage collection, however, these calls are advanced and not covered here. diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst index 962b4ef..017c6d4 100644 --- a/Doc/faq/design.rst +++ b/Doc/faq/design.rst @@ -225,7 +225,7 @@ The major reason is history. Functions were used for those operations that were generic for a group of types and which were intended to work even for objects that didn't have methods at all (e.g. tuples). It is also convenient to have a function that can readily be applied to an amorphous collection of objects when -you use the functional features of Python (``map()``, ``apply()`` et al). +you use the functional features of Python (``map()``, ``zip()`` et al). In fact, implementing ``len()``, ``max()``, ``min()`` as a built-in function is actually less code than implementing them as methods for each type. One can @@ -297,8 +297,9 @@ use the ``join()`` function from the string module, which allows you to write :: How fast are exceptions? ------------------------ -A try/except block is extremely efficient. Actually catching an exception is -expensive. In versions of Python prior to 2.0 it was common to use this idiom:: +A try/except block is extremely efficient if no exceptions are raised. Actually +catching an exception is expensive. In versions of Python prior to 2.0 it was +common to use this idiom:: try: value = mydict[key] @@ -309,11 +310,10 @@ expensive. In versions of Python prior to 2.0 it was common to use this idiom:: This only made sense when you expected the dict to have the key almost all the time. If that wasn't the case, you coded it like this:: - if mydict.has_key(key): + if key in mydict: value = mydict[key] else: - mydict[key] = getvalue(key) - value = mydict[key] + value = mydict[key] = getvalue(key) .. note:: @@ -370,25 +370,22 @@ support for C. Answer 2: Fortunately, there is `Stackless Python <http://www.stackless.com>`_, which has a completely redesigned interpreter loop that avoids the C stack. -It's still experimental but looks very promising. Although it is binary -compatible with standard Python, it's still unclear whether Stackless will make -it into the core -- maybe it's just too revolutionary. -Why can't lambda forms contain statements? ------------------------------------------- +Why can't lambda expressions contain statements? +------------------------------------------------ -Python lambda forms cannot contain statements because Python's syntactic +Python lambda expressions cannot contain statements because Python's syntactic framework can't handle statements nested inside expressions. However, in Python, this is not a serious problem. Unlike lambda forms in other languages, where they add functionality, Python lambdas are only a shorthand notation if you're too lazy to define a function. Functions are already first class objects in Python, and can be declared in a -local scope. Therefore the only advantage of using a lambda form instead of a +local scope. Therefore the only advantage of using a lambda instead of a locally-defined function is that you don't need to invent a name for the function -- but that's just a local variable to which the function object (which -is exactly the same type of object that a lambda form yields) is assigned! +is exactly the same type of object that a lambda expression yields) is assigned! Can Python be compiled to machine code, C or some other language? @@ -685,7 +682,8 @@ Python 2.6 adds an :mod:`abc` module that lets you define Abstract Base Classes (ABCs). You can then use :func:`isinstance` and :func:`issubclass` to check whether an instance or a class implements a particular ABC. The :mod:`collections` module defines a set of useful ABCs such as -:class:`Iterable`, :class:`Container`, and :class:`MutableMapping`. +:class:`~collections.Iterable`, :class:`~collections.Container`, and +:class:`~collections.MutableMapping`. For Python, many of the advantages of interface specifications can be obtained by an appropriate test discipline for components. There is also a tool, @@ -757,7 +755,7 @@ of each call to the function, and return the cached value if the same value is requested again. This is called "memoizing", and can be implemented like this:: # Callers will never provide a third parameter for this function. - def expensive (arg1, arg2, _cache={}): + def expensive(arg1, arg2, _cache={}): if (arg1, arg2) in _cache: return _cache[(arg1, arg2)] @@ -782,7 +780,7 @@ languages. For example:: try: ... - if (condition): raise label() # goto label + if condition: raise label() # goto label ... except label: # where to goto pass @@ -913,8 +911,8 @@ There are several reasons to allow this. When you have a literal value for a list, tuple, or dictionary spread across multiple lines, it's easier to add more elements because you don't have to -remember to add a comma to the previous line. The lines can also be sorted in -your editor without creating a syntax error. +remember to add a comma to the previous line. The lines can also be reordered +without creating a syntax error. Accidentally omitting the comma can lead to errors that are hard to diagnose. For example:: diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst index b79d716..4f3cabf 100644 --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -2,7 +2,9 @@ Extending/Embedding FAQ ======================= -.. contents:: +.. only:: html + + .. contents:: .. highlight:: c diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst index df43196..aa8075c 100644 --- a/Doc/faq/general.rst +++ b/Doc/faq/general.rst @@ -4,7 +4,10 @@ General Python FAQ ================== -.. contents:: +.. only:: html + + .. contents:: + General Information =================== @@ -178,8 +181,8 @@ at http://docs.python.org/. PDF, plain text, and downloadable HTML versions are also available at http://docs.python.org/download.html. The documentation is written in reStructuredText and processed by `the Sphinx -documentation tool <http://sphinx.pocoo.org/>`__. The reStructuredText source -for the documentation is part of the Python source distribution. +documentation tool <http://sphinx-doc.org/>`__. The reStructuredText source for +the documentation is part of the Python source distribution. I've never programmed before. Is there a Python tutorial? @@ -265,9 +268,13 @@ Python references; or perhaps search for "Python" and "language". Where in the world is www.python.org located? --------------------------------------------- -It's currently in Amsterdam, graciously hosted by `XS4ALL -<http://www.xs4all.nl>`_. Thanks to Thomas Wouters for his work in arranging -python.org's hosting. +The Python project's infrastructure is located all over the world. +`www.python.org <http://www.python.org>`_ is currently in Amsterdam, graciously +hosted by `XS4ALL <http://www.xs4all.nl>`_. `Upfront Systems +<http://www.upfrontsystems.co.za>`_ hosts `bugs.python.org +<http://bugs.python.org>`_. Most other Python services like `PyPI +<https://pypi.python.org>`_ and hg.python.org are hosted by `Oregon State +University Open Source Lab <https://osuosl.org>`_. Why is it called Python? @@ -464,7 +471,8 @@ that is written in Python using Tkinter. PythonWin is a Windows-specific IDE. Emacs users will be happy to know that there is a very good Python mode for Emacs. All of these programming environments provide syntax highlighting, auto-indenting, and access to the interactive interpreter while coding. Consult -http://www.python.org/editors/ for a full list of Python editing environments. +`the Python wiki <https://wiki.python.org/moin/PythonEditors>`_ for a full list +of Python editing environments. If you want to discuss Python's use in education, you may be interested in joining `the edu-sig mailing list diff --git a/Doc/faq/gui.rst b/Doc/faq/gui.rst index 50a30b0..42d95bd 100644 --- a/Doc/faq/gui.rst +++ b/Doc/faq/gui.rst @@ -4,7 +4,9 @@ Graphic User Interface FAQ ========================== -.. contents:: +.. only:: html + + .. contents:: What platform-independent GUI toolkits exist for Python? ======================================================== @@ -19,15 +21,15 @@ Tkinter Standard builds of Python include an object-oriented interface to the Tcl/Tk widget set, called Tkinter. This is probably the easiest to install and use. For more info about Tk, including pointers to the source, see the Tcl/Tk home -page at http://www.tcl.tk. Tcl/Tk is fully portable to the MacOS, Windows, and -Unix platforms. +page at http://www.tcl.tk. Tcl/Tk is fully portable to the Mac OS X, Windows, +and Unix platforms. wxWidgets --------- wxWidgets (http://www.wxwidgets.org) is a free, portable GUI class library written in C++ that provides a native look and feel on a -number of platforms, with Windows, MacOS X, GTK, X11, all listed as +number of platforms, with Windows, Mac OS X, GTK, X11, all listed as current stable targets. Language bindings are available for a number of languages including Python, Perl, Ruby, etc. @@ -45,13 +47,15 @@ well as in freeware or shareware. Qt --- -There are bindings available for the Qt toolkit (`PyQt -<http://www.riverbankcomputing.co.uk/software/pyqt/>`_) and for KDE (`PyKDE <http://www.riverbankcomputing.co.uk/software/pykde/intro>`__). If -you're writing open source software, you don't need to pay for PyQt, but if you -want to write proprietary applications, you must buy a PyQt license from -`Riverbank Computing <http://www.riverbankcomputing.co.uk>`_ and (up to Qt 4.4; -Qt 4.5 upwards is licensed under the LGPL license) a Qt license from `Trolltech -<http://www.trolltech.com>`_. +There are bindings available for the Qt toolkit (using either `PyQt +<http://www.riverbankcomputing.co.uk/software/pyqt/>`_ or `PySide +<http://www.pyside.org/>`_) and for KDE (`PyKDE <http://www.riverbankcomputing.co.uk/software/pykde/intro>`_). +PyQt is currently more mature than PySide, but you must buy a PyQt license from +`Riverbank Computing <http://www.riverbankcomputing.co.uk/software/pyqt/license>`_ +if you want to write proprietary applications. PySide is free for all applications. + +Qt 4.5 upwards is licensed under the LGPL license; also, commercial licenses +are available from `Nokia <http://qt.nokia.com/>`_. Gtk+ ---- @@ -84,13 +88,9 @@ For OpenGL bindings, see `PyOpenGL <http://pyopengl.sourceforge.net>`_. What platform-specific GUI toolkits exist for Python? ======================================================== -`The Mac port <http://python.org/download/mac>`_ by Jack Jansen has a rich and -ever-growing set of modules that support the native Mac toolbox calls. The port -supports MacOS X's Carbon libraries. - By installing the `PyObjc Objective-C bridge -<http://pyobjc.sourceforge.net>`_, Python programs can use MacOS X's -Cocoa libraries. See the documentation that comes with the Mac port. +<http://pyobjc.sourceforge.net>`_, Python programs can use Mac OS X's +Cocoa libraries. :ref:`Pythonwin <windows-faq>` by Mark Hammond includes an interface to the Microsoft Foundation Classes and a Python programming environment diff --git a/Doc/faq/index.rst b/Doc/faq/index.rst index caba425..46ed3db 100644 --- a/Doc/faq/index.rst +++ b/Doc/faq/index.rst @@ -1,10 +1,9 @@ +.. _faq-index: + ################################### Python Frequently Asked Questions ################################### -:Release: |version| -:Date: |today| - .. toctree:: :maxdepth: 1 diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst index 5b77eb8..0d80f76 100644 --- a/Doc/faq/library.rst +++ b/Doc/faq/library.rst @@ -4,7 +4,9 @@ Library and Extension FAQ ========================= -.. contents:: +.. only:: html + + .. contents:: General Library Questions ========================= @@ -14,7 +16,7 @@ How do I find a module or application to perform task X? Check :ref:`the Library Reference <library-index>` to see if there's a relevant standard library module. (Eventually you'll learn what's in the standard -library and will able to skip this step.) +library and will be able to skip this step.) For third-party packages, search the `Python Package Index <http://pypi.python.org/pypi>`_ or try `Google <http://www.google.com>`_ or @@ -28,7 +30,7 @@ Where is the math.py (socket.py, regex.py, etc.) source file? If you can't find a source file for a module it may be a built-in or dynamically loaded module implemented in C, C++ or other compiled language. In this case you may not have the source file or it may be something like -mathmodule.c, somewhere in a C source directory (not on the Python Path). +:file:`mathmodule.c`, somewhere in a C source directory (not on the Python Path). There are (at least) three kinds of modules in Python: @@ -60,18 +62,18 @@ as the very first line of your file, using the pathname for where the Python interpreter is installed on your platform. If you would like the script to be independent of where the Python interpreter -lives, you can use the "env" program. Almost all Unix variants support the -following, assuming the Python interpreter is in a directory on the user's -$PATH:: +lives, you can use the :program:`env` program. Almost all Unix variants support +the following, assuming the Python interpreter is in a directory on the user's +:envvar:`PATH`:: #!/usr/bin/env python -*Don't* do this for CGI scripts. The $PATH variable for CGI scripts is often -very minimal, so you need to use the actual absolute pathname of the +*Don't* do this for CGI scripts. The :envvar:`PATH` variable for CGI scripts is +often very minimal, so you need to use the actual absolute pathname of the interpreter. -Occasionally, a user's environment is so full that the /usr/bin/env program -fails; or there's no env program at all. In that case, you can try the +Occasionally, a user's environment is so full that the :program:`/usr/bin/env` +program fails; or there's no env program at all. In that case, you can try the following hack (due to Alex Rezinsky):: #! /bin/sh @@ -91,12 +93,12 @@ Is there a curses/termcap package for Python? .. XXX curses *is* built by default, isn't it? -For Unix variants: The standard Python source distribution comes with a curses -module in the ``Modules/`` subdirectory, though it's not compiled by default -(note that this is not available in the Windows distribution -- there is no -curses module for Windows). +For Unix variants the standard Python source distribution comes with a curses +module in the :source:`Modules` subdirectory, though it's not compiled by default. +(Note that this is not available in the Windows distribution -- there is no +curses module for Windows.) -The curses module supports basic curses features as well as many additional +The :mod:`curses` module supports basic curses features as well as many additional functions from ncurses and SYSV curses such as colour, alternative character set support, pads, and mouse support. This means the module isn't compatible with operating systems that only have BSD curses, but there don't seem to be any @@ -110,7 +112,7 @@ Is there an equivalent to C's onexit() in Python? ------------------------------------------------- The :mod:`atexit` module provides a register function that is similar to C's -onexit. +:c:func:`onexit`. Why don't my signal handlers work? @@ -140,8 +142,8 @@ the expected output given in the docstring. The :mod:`unittest` module is a fancier testing framework modelled on Java and Smalltalk testing frameworks. -For testing, it helps to write the program so that it may be easily tested by -using good modular design. Your program should have almost all functionality +To make testing easier, you should use good modular design in your program. +Your program should have almost all functionality encapsulated in either functions or class methods -- and this sometimes has the surprising and delightful effect of making the program run faster (because local variable accesses are faster than global accesses). Furthermore the program @@ -157,7 +159,7 @@ at the bottom of the main module of your program. Once your program is organized as a tractable collection of functions and class behaviours you should write test functions that exercise the behaviours. A test -suite can be associated with each module which automates a sequence of tests. +suite that automates a sequence of tests can be associated with each module. This sounds like a lot of work, but since Python is so terse and flexible it's surprisingly easy. You can make coding much more pleasant and fun by writing your test functions in parallel with the "production code", since this makes it @@ -186,7 +188,7 @@ docstrings is `epydoc <http://epydoc.sf.net/>`_. `Sphinx How do I get a single keypress at a time? ----------------------------------------- -For Unix variants: There are several solutions. It's straightforward to do this +For Unix variants there are several solutions. It's straightforward to do this using curses, but curses is a fairly large module to learn. Here's a solution without curses:: @@ -273,7 +275,7 @@ A simple fix is to add a tiny sleep to the start of the run function:: time.sleep(10) -Instead of trying to guess how long a :func:`time.sleep` delay will be enough, +Instead of trying to guess a good delay value for :func:`time.sleep`, it's better to use some kind of semaphore mechanism. One idea is to use the :mod:`Queue` module to create a queue object, let each thread append a token to the queue when it finishes, and let the main thread read as many tokens from the @@ -284,10 +286,10 @@ How do I parcel out work among a bunch of worker threads? --------------------------------------------------------- Use the :mod:`Queue` module to create a queue containing a list of jobs. The -:class:`~Queue.Queue` class maintains a list of objects with ``.put(obj)`` to -add an item to the queue and ``.get()`` to return an item. The class will take -care of the locking necessary to ensure that each job is handed out exactly -once. +:class:`~Queue.Queue` class maintains a list of objects and has a ``.put(obj)`` +method that adds items to the queue and a ``.get()`` method to return them. +The class will take care of the locking necessary to ensure that each job is +handed out exactly once. Here's a trivial example:: @@ -296,7 +298,7 @@ Here's a trivial example:: # The worker thread gets jobs off the queue. When the queue is empty, it # assumes there will be no more work and exits. # (Realistically workers will run until terminated.) - def worker (): + def worker(): print 'Running worker' time.sleep(0.1) while True: @@ -329,6 +331,8 @@ Here's a trivial example:: When run, this will produce the following output: +.. code-block:: none + Running worker Running worker Running worker @@ -343,15 +347,15 @@ When run, this will produce the following output: Worker <Thread(worker 1, started)> running with argument 5 ... -Consult the module's documentation for more details; the ``Queue`` class -provides a featureful interface. +Consult the module's documentation for more details; the :class:`~Queue.Queue` +class provides a featureful interface. What kinds of global value mutation are thread-safe? ---------------------------------------------------- -A global interpreter lock (GIL) is used internally to ensure that only one -thread runs in the Python VM at a time. In general, Python offers to switch +A :term:`global interpreter lock` (GIL) is used internally to ensure that only +one thread runs in the Python VM at a time. In general, Python offers to switch among threads only between bytecode instructions; how frequently it switches can be set via :func:`sys.setcheckinterval`. Each bytecode instruction and therefore all the C implementation code reached from each instruction is @@ -396,7 +400,7 @@ Can't we get rid of the Global Interpreter Lock? .. XXX mention multiprocessing .. XXX link to dbeazley's talk about GIL? -The Global Interpreter Lock (GIL) is often seen as a hindrance to Python's +The :term:`global interpreter lock` (GIL) is often seen as a hindrance to Python's deployment on high-end multiprocessor server machines, because a multi-threaded Python program effectively only uses one CPU, due to the insistence that (almost) all Python code can only run while the GIL is held. @@ -459,7 +463,7 @@ To rename a file, use ``os.rename(old_path, new_path)``. To truncate a file, open it using ``f = open(filename, "r+")``, and use ``f.truncate(offset)``; offset defaults to the current seek position. There's also ``os.ftruncate(fd, offset)`` for files opened with :func:`os.open`, where -``fd`` is the file descriptor (a small integer). +*fd* is the file descriptor (a small integer). The :mod:`shutil` module also contains a number of functions to work on files including :func:`~shutil.copyfile`, :func:`~shutil.copytree`, and @@ -493,7 +497,7 @@ The '>' in the format string forces big-endian data; the letter 'h' reads one "short integer" (2 bytes), and 'l' reads one "long integer" (4 bytes) from the string. -For data that is more regular (e.g. a homogeneous list of ints or thefloats), +For data that is more regular (e.g. a homogeneous list of ints or floats), you can also use the :mod:`array` module. @@ -503,7 +507,7 @@ I can't seem to use os.read() on a pipe created with os.popen(); why? :func:`os.read` is a low-level function which takes a file descriptor, a small integer representing the opened file. :func:`os.popen` creates a high-level file object, the same type returned by the built-in :func:`open` function. -Thus, to read n bytes from a pipe p created with :func:`os.popen`, you need to +Thus, to read *n* bytes from a pipe *p* created with :func:`os.popen`, you need to use ``p.read(n)``. @@ -522,9 +526,9 @@ Use the :mod:`popen2` module. For example:: Warning: in general it is unwise to do this because you can easily cause a deadlock where your process is blocked waiting for output from the child while -the child is blocked waiting for input from you. This can be caused because the -parent expects the child to output more text than it does, or it can be caused -by data being stuck in stdio buffers due to lack of flushing. The Python parent +the child is blocked waiting for input from you. This can be caused by the +parent expecting the child to output more text than it does or by data being +stuck in stdio buffers due to lack of flushing. The Python parent can of course explicitly flush the data it sends to the child before it reads any output, but if the child is a naive C program it may have been written to never explicitly flush its output, even if it is interactive, since flushing is @@ -544,8 +548,8 @@ place to insert such a call would be before calling ``popen2`` again. In many cases, all you really need is to run some data through a command and get the result back. Unless the amount of data is very large, the easiest way to do this is to write it to a temporary file and run the command with that temporary -file as input. The standard module :mod:`tempfile` exports a ``mktemp()`` -function to generate unique temporary file names. :: +file as input. The standard module :mod:`tempfile` exports a +:func:`~tempfile.mktemp` function to generate unique temporary file names. :: import tempfile import os @@ -636,7 +640,7 @@ and client-side web systems. .. XXX check if wiki page is still up to date A summary of available frameworks is maintained by Paul Boddie at -http://wiki.python.org/moin/WebProgramming . +http://wiki.python.org/moin/WebProgramming\ . Cameron Laird maintains a useful set of pages about Python web technologies at http://phaseit.net/claird/comp.lang.python/web_python. @@ -673,15 +677,12 @@ Yes. Here's a simple example that uses httplib:: sys.stdout.write(httpobj.getfile().read()) Note that in general for percent-encoded POST operations, query strings must be -quoted using :func:`urllib.quote`. For example to send name="Guy Steele, Jr.":: +quoted using :func:`urllib.urlencode`. For example, to send +``name=Guy Steele, Jr.``:: - >>> from urllib import quote - >>> x = quote("Guy Steele, Jr.") - >>> x - 'Guy%20Steele,%20Jr.' - >>> query_string = "name="+x - >>> query_string - 'name=Guy%20Steele,%20Jr.' + >>> import urllib + >>> urllib.urlencode({'name': 'Guy Steele, Jr.'}) + 'name=Guy+Steele%2C+Jr.' What module should I use to help with generating HTML? @@ -689,19 +690,8 @@ What module should I use to help with generating HTML? .. XXX add modern template languages -There are many different modules available: - -* HTMLgen is a class library of objects corresponding to all the HTML 3.2 markup - tags. It's used when you are writing in Python and wish to synthesize HTML - pages for generating a web or for CGI forms, etc. - -* DocumentTemplate and Zope Page Templates are two different systems that are - part of Zope. - -* Quixote's PTL uses Python syntax to assemble strings of text. - -Consult the `Web Programming wiki pages -<http://wiki.python.org/moin/WebProgramming>`_ for more links. +You can find a collection of useful links on the `Web Programming wiki page +<http://wiki.python.org/moin/WebProgramming>`_. How do I send mail from a Python script? @@ -730,7 +720,7 @@ work on any host that supports an SMTP listener. :: server.quit() A Unix-only alternative uses sendmail. The location of the sendmail program -varies between systems; sometimes it is ``/usr/lib/sendmail``, sometime +varies between systems; sometimes it is ``/usr/lib/sendmail``, sometimes ``/usr/sbin/sendmail``. The sendmail manual page will help you out. Here's some sample code:: @@ -797,7 +787,7 @@ A more awkward way of doing things is to use pickle's little sister, marshal. The :mod:`marshal` module provides very fast ways to store noncircular basic Python types to files and strings, and back again. Although marshal does not do fancy things like store instances or handle shared references properly, it does -run extremely fast. For example loading a half megabyte of data may take less +run extremely fast. For example, loading a half megabyte of data may take less than a third of a second. This often beats doing something more complex and general such as using gdbm with pickle/shelve. @@ -807,9 +797,9 @@ Why is cPickle so slow? .. XXX update this, default protocol is 2/3 -The default format used by the pickle module is a slow one that results in -readable pickles. Making it the default, but it would break backward -compatibility:: +By default :mod:`pickle` uses a relatively old and slow format for backward +compatibility. You can however specify other protocol versions that are +faster:: largeString = 'z' * (100 * 1024) myPickle = cPickle.dumps(largeString, protocol=1) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index a0898c5..0f3a331 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -4,7 +4,9 @@ Programming FAQ =============== -.. contents:: +.. only:: html + + .. contents:: General Questions ================= @@ -147,7 +149,7 @@ There is a page on the wiki devoted to `performance tips <http://wiki.python.org/moin/PythonSpeed/PerformanceTips>`_. Guido van Rossum has written up an anecdote related to optimization at -http://www.python.org/doc/essays/list2str.html. +http://www.python.org/doc/essays/list2str. One thing to notice is that function and (especially) method calls are rather expensive; if you have designed a purely OO interface with lots of tiny @@ -352,6 +354,58 @@ an imported module. This clutter would defeat the usefulness of the ``global`` declaration for identifying side-effects. +Why do lambdas defined in a loop with different values all return the same result? +---------------------------------------------------------------------------------- + +Assume you use a for loop to define a few different lambdas (or even plain +functions), e.g.:: + + >>> squares = [] + >>> for x in range(5): + ... squares.append(lambda: x**2) + +This gives you a list that contains 5 lambdas that calculate ``x**2``. You +might expect that, when called, they would return, respectively, ``0``, ``1``, +``4``, ``9``, and ``16``. However, when you actually try you will see that +they all return ``16``:: + + >>> squares[2]() + 16 + >>> squares[4]() + 16 + +This happens because ``x`` is not local to the lambdas, but is defined in +the outer scope, and it is accessed when the lambda is called --- not when it +is defined. At the end of the loop, the value of ``x`` is ``4``, so all the +functions now return ``4**2``, i.e. ``16``. You can also verify this by +changing the value of ``x`` and see how the results of the lambdas change:: + + >>> x = 8 + >>> squares[2]() + 64 + +In order to avoid this, you need to save the values in variables local to the +lambdas, so that they don't rely on the value of the global ``x``:: + + >>> squares = [] + >>> for x in range(5): + ... squares.append(lambda n=x: n**2) + +Here, ``n=x`` creates a new variable ``n`` local to the lambda and computed +when the lambda is defined so that it has the same value that ``x`` had at +that point in the loop. This means that the value of ``n`` will be ``0`` +in the first lambda, ``1`` in the second, ``2`` in the third, and so on. +Therefore each lambda will now return the correct result:: + + >>> squares[2]() + 4 + >>> squares[4]() + 16 + +Note that this behaviour is not peculiar to lambdas, but applies to regular +functions too. + + How do I share global variables across modules? ------------------------------------------------ @@ -469,6 +523,31 @@ In the unlikely case that you care about Python versions older than 2.0, use apply(g, (x,)+args, kwargs) +.. index:: + single: argument; difference from parameter + single: parameter; difference from argument + +.. _faq-argument-vs-parameter: + +What is the difference between arguments and parameters? +-------------------------------------------------------- + +:term:`Parameters <parameter>` are defined by the names that appear in a +function definition, whereas :term:`arguments <argument>` are the values +actually passed to a function when calling it. Parameters define what types of +arguments a function can accept. For example, given the function definition:: + + def func(foo, bar=None, **kwargs): + pass + +*foo*, *bar* and *kwargs* are parameters of ``func``. However, when calling +``func``, for example:: + + func(42, bar=314, extra=somevar) + +the values ``42``, ``314``, and ``somevar`` are arguments. + + How do I write a function with output parameters (call by reference)? --------------------------------------------------------------------- @@ -669,11 +748,11 @@ Comma is not an operator in Python. Consider this session:: Since the comma is not an operator, but a separator between expressions the above is evaluated as if you had entered:: - >>> ("a" in "b"), "a" + ("a" in "b"), "a" not:: - >>> "a" in ("b", "a") + "a" in ("b", "a") The same is true of the various assignment operators (``=``, ``+=`` etc). They are not truly operators but syntactic delimiters in assignment statements. @@ -692,52 +771,6 @@ Yes, this feature was added in Python 2.5. The syntax would be as follows:: For versions previous to 2.5 the answer would be 'No'. -.. XXX remove rest? - -In many cases you can mimic ``a ? b : c`` with ``a and b or c``, but there's a -flaw: if *b* is zero (or empty, or ``None`` -- anything that tests false) then -*c* will be selected instead. In many cases you can prove by looking at the -code that this can't happen (e.g. because *b* is a constant or has a type that -can never be false), but in general this can be a problem. - -Tim Peters (who wishes it was Steve Majewski) suggested the following solution: -``(a and [b] or [c])[0]``. Because ``[b]`` is a singleton list it is never -false, so the wrong path is never taken; then applying ``[0]`` to the whole -thing gets the *b* or *c* that you really wanted. Ugly, but it gets you there -in the rare cases where it is really inconvenient to rewrite your code using -'if'. - -The best course is usually to write a simple ``if...else`` statement. Another -solution is to implement the ``?:`` operator as a function:: - - def q(cond, on_true, on_false): - if cond: - if not isfunction(on_true): - return on_true - else: - return on_true() - else: - if not isfunction(on_false): - return on_false - else: - return on_false() - -In most cases you'll pass b and c directly: ``q(a, b, c)``. To avoid evaluating -b or c when they shouldn't be, encapsulate them within a lambda function, e.g.: -``q(a, lambda: b, lambda: c)``. - -It has been asked *why* Python has no if-then-else expression. There are -several answers: many languages do just fine without one; it can easily lead to -less readable code; no sufficiently "Pythonic" syntax has been discovered; a -search of the standard library found remarkably few places where using an -if-then-else expression would make the code more understandable. - -In 2002, :pep:`308` was written proposing several possible syntaxes and the -community was asked to vote on the issue. The vote was inconclusive. Most -people liked one of the syntaxes, but also hated other syntaxes; many votes -implied that people preferred no ternary operator rather than having a syntax -they hated. - Is it possible to write obfuscated one-liners in Python? -------------------------------------------------------- @@ -864,6 +897,7 @@ How do I modify a string in place? You can't, because strings are immutable. If you need an object with this ability, try converting the string to a list or use the array module:: + >>> import io >>> s = "Hello, world" >>> a = list(s) >>> print a @@ -876,8 +910,8 @@ ability, try converting the string to a list or use the array module:: >>> a = array.array('c', s) >>> print a array('c', 'Hello, world') - >>> a[0] = 'y' ; print a - array('c', 'yello world') + >>> a[0] = 'y'; print a + array('c', 'yello, world') >>> a.tostring() 'yello, world' @@ -1139,7 +1173,7 @@ How do I create a multidimensional list? You probably tried to make a multidimensional array like this:: - A = [[None] * 2] * 3 + >>> A = [[None] * 2] * 3 This looks correct if you print it:: @@ -1171,7 +1205,7 @@ use a list comprehension:: A = [[None] * w for i in range(h)] Or, you can use an extension that provides a matrix datatype; `Numeric Python -<http://numpy.scipy.org/>`_ is the best known. +<http://www.numpy.org/>`_ is the best known. How do I apply a method to a sequence of objects? @@ -1190,6 +1224,92 @@ More generically, you can try the following function:: return map(apply, methods, [arguments]*nobjects) +Why does a_tuple[i] += ['item'] raise an exception when the addition works? +--------------------------------------------------------------------------- + +This is because of a combination of the fact that augmented assignment +operators are *assignment* operators, and the difference between mutable and +immutable objects in Python. + +This discussion applies in general when augmented assignment operators are +applied to elements of a tuple that point to mutable objects, but we'll use +a ``list`` and ``+=`` as our exemplar. + +If you wrote:: + + >>> a_tuple = (1, 2) + >>> a_tuple[0] += 1 + Traceback (most recent call last): + ... + TypeError: 'tuple' object does not support item assignment + +The reason for the exception should be immediately clear: ``1`` is added to the +object ``a_tuple[0]`` points to (``1``), producing the result object, ``2``, +but when we attempt to assign the result of the computation, ``2``, to element +``0`` of the tuple, we get an error because we can't change what an element of +a tuple points to. + +Under the covers, what this augmented assignment statement is doing is +approximately this:: + + >>> result = a_tuple[0] + 1 + >>> a_tuple[0] = result + Traceback (most recent call last): + ... + TypeError: 'tuple' object does not support item assignment + +It is the assignment part of the operation that produces the error, since a +tuple is immutable. + +When you write something like:: + + >>> a_tuple = (['foo'], 'bar') + >>> a_tuple[0] += ['item'] + Traceback (most recent call last): + ... + TypeError: 'tuple' object does not support item assignment + +The exception is a bit more surprising, and even more surprising is the fact +that even though there was an error, the append worked:: + + >>> a_tuple[0] + ['foo', 'item'] + +To see why this happens, you need to know that (a) if an object implements an +``__iadd__`` magic method, it gets called when the ``+=`` augmented assignment +is executed, and its return value is what gets used in the assignment statement; +and (b) for lists, ``__iadd__`` is equivalent to calling ``extend`` on the list +and returning the list. That's why we say that for lists, ``+=`` is a +"shorthand" for ``list.extend``:: + + >>> a_list = [] + >>> a_list += [1] + >>> a_list + [1] + +This is equivalent to:: + + >>> result = a_list.__iadd__([1]) + >>> a_list = result + +The object pointed to by a_list has been mutated, and the pointer to the +mutated object is assigned back to ``a_list``. The end result of the +assignment is a no-op, since it is a pointer to the same object that ``a_list`` +was previously pointing to, but the assignment still happens. + +Thus, in our tuple example what is happening is equivalent to:: + + >>> result = a_tuple[0].__iadd__(['item']) + >>> a_tuple[0] = result + Traceback (most recent call last): + ... + TypeError: 'tuple' object does not support item assignment + +The ``__iadd__`` succeeds, and thus the list is extended, but even though +``result`` points to the same object that ``a_tuple[0]`` already points to, +that final assignment still results in an error, because tuples are immutable. + + Dictionaries ============ @@ -1604,6 +1724,32 @@ You can program the class's constructor to keep track of all instances by keeping a list of weak references to each instance. +Why does the result of ``id()`` appear to be not unique? +-------------------------------------------------------- + +The :func:`id` builtin returns an integer that is guaranteed to be unique during +the lifetime of the object. Since in CPython, this is the object's memory +address, it happens frequently that after an object is deleted from memory, the +next freshly created object is allocated at the same position in memory. This +is illustrated by this example: + +>>> id(1000) +13901272 +>>> id(2000) +13901272 + +The two ids belong to different integer objects that are created before, and +deleted immediately after execution of the ``id()`` call. To be sure that +objects whose id you want to examine are still alive, create another reference +to the object: + +>>> a = 1000; b = 2000 +>>> id(a) +13901272 +>>> id(b) +13891296 + + Modules ======= @@ -1621,13 +1767,13 @@ file is automatic if you're importing a module and Python has the ability (permissions, free space, etc...) to write the compiled module back to the directory. -Running Python on a top level script is not considered an import and no ``.pyc`` -will be created. For example, if you have a top-level module ``abc.py`` that -imports another module ``xyz.py``, when you run abc, ``xyz.pyc`` will be created -since xyz is imported, but no ``abc.pyc`` file will be created since ``abc.py`` -isn't being imported. +Running Python on a top level script is not considered an import and no +``.pyc`` will be created. For example, if you have a top-level module +``foo.py`` that imports another module ``xyz.py``, when you run ``foo``, +``xyz.pyc`` will be created since ``xyz`` is imported, but no ``foo.pyc`` file +will be created since ``foo.py`` isn't being imported. -If you need to create abc.pyc -- that is, to create a .pyc file for a module +If you need to create ``foo.pyc`` -- that is, to create a ``.pyc`` file for a module that is not imported -- you can, using the :mod:`py_compile` and :mod:`compileall` modules. @@ -1635,9 +1781,9 @@ The :mod:`py_compile` module can manually compile any module. One way is to use the ``compile()`` function in that module interactively:: >>> import py_compile - >>> py_compile.compile('abc.py') + >>> py_compile.compile('foo.py') # doctest: +SKIP -This will write the ``.pyc`` to the same location as ``abc.py`` (or you can +This will write the ``.pyc`` to the same location as ``foo.py`` (or you can override that with the optional parameter ``cfile``). You can also automatically compile all files in a directory or directories using diff --git a/Doc/faq/windows.rst b/Doc/faq/windows.rst index dbb7bb8..0379bac 100644 --- a/Doc/faq/windows.rst +++ b/Doc/faq/windows.rst @@ -6,16 +6,16 @@ Python on Windows FAQ ===================== -.. contents:: +.. only:: html + + .. contents:: How do I run a Python program under Windows? -------------------------------------------- This is not necessarily a straightforward question. If you are already familiar with running programs from the Windows command line then everything will seem -obvious; otherwise, you might need a little more guidance. There are also -differences between Windows 95, 98, NT, ME, 2000 and XP which can add to the -confusion. +obvious; otherwise, you might need a little more guidance. .. sidebar:: |Python Development on XP|_ :subtitle: `Python Development on XP`_ @@ -32,7 +32,7 @@ confusion. Unless you use some sort of integrated development environment, you will end up *typing* Windows commands into what is variously referred to as a "DOS window" or "Command prompt window". Usually you can create such a window from your -Start menu; under Windows 2000 the menu selection is :menuselection:`Start --> +Start menu; under Windows 7 the menu selection is :menuselection:`Start --> Programs --> Accessories --> Command Prompt`. You should be able to recognize when you have started such a window because you will see a Windows "command prompt", which usually looks like this:: @@ -42,23 +42,27 @@ prompt", which usually looks like this:: The letter may be different, and there might be other things after it, so you might just as easily see something like:: - D:\Steve\Projects\Python> + D:\YourName\Projects\Python> depending on how your computer has been set up and what else you have recently done with it. Once you have started such a window, you are well on the way to running Python programs. You need to realize that your Python scripts have to be processed by another -program called the Python interpreter. The interpreter reads your script, +program called the Python *interpreter*. The interpreter reads your script, compiles it into bytecodes, and then executes the bytecodes to run your program. So, how do you arrange for the interpreter to handle your Python? First, you need to make sure that your command window recognises the word "python" as an instruction to start the interpreter. If you have opened a command window, you should try entering the command ``python`` and hitting -return. You should then see something like:: +return.:: + + C:\Users\YourName> python + +You should then see something like:: - Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit (Intel)] on win32 + Python 2.7.3 (default, Apr 10 2012, 22.71:26) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> @@ -78,7 +82,7 @@ key down while you enter a Z, then hit the "Enter" key to get back to your Windows command prompt. You may also find that you have a Start-menu entry such as :menuselection:`Start ---> Programs --> Python 2.2 --> Python (command line)` that results in you +--> Programs --> Python 2.7 --> Python (command line)` that results in you seeing the ``>>>`` prompt in a new window. If so, the window will disappear after you enter the Ctrl-Z character; Windows is running a single "python" command in the window, and closes it when you terminate the interpreter. @@ -86,8 +90,7 @@ command in the window, and closes it when you terminate the interpreter. If the ``python`` command, instead of displaying the interpreter prompt ``>>>``, gives you a message like:: - 'python' is not recognized as an internal or external command, - operable program or batch file. + 'python' is not recognized as an internal or external command, operable program or batch file. .. sidebar:: |Adding Python to DOS Path|_ :subtitle: `Adding Python to DOS Path`_ @@ -116,115 +119,33 @@ then the command :: dir C:\py* will probably tell you where it is installed; the usual location is something -like ``C:\Python23``. Otherwise you will be reduced to a search of your whole +like ``C:\Python27``. Otherwise you will be reduced to a search of your whole disk ... use :menuselection:`Tools --> Find` or hit the :guilabel:`Search` button and look for "python.exe". Supposing you discover that Python is -installed in the ``C:\Python23`` directory (the default at the time of writing), +installed in the ``C:\Python27`` directory (the default at the time of writing), you should make sure that entering the command :: - c:\Python23\python + c:\Python27\python starts up the interpreter as above (and don't forget you'll need a "CTRL-Z" and -an "Enter" to get out of it). Once you have verified the directory, you need to -add it to the start-up routines your computer goes through. For older versions -of Windows the easiest way to do this is to edit the ``C:\AUTOEXEC.BAT`` -file. You would want to add a line like the following to ``AUTOEXEC.BAT``:: - - PATH C:\Python23;%PATH% - -For Windows NT, 2000 and (I assume) XP, you will need to add a string such as :: - - ;C:\Python23 - -to the current setting for the PATH environment variable, which you will find in -the properties window of "My Computer" under the "Advanced" tab. Note that if -you have sufficient privilege you might get a choice of installing the settings -either for the Current User or for System. The latter is preferred if you want -everybody to be able to run Python on the machine. - -If you aren't confident doing any of these manipulations yourself, ask for help! -At this stage you may want to reboot your system to make absolutely sure the new -setting has taken effect. You probably won't need to reboot for Windows NT, XP -or 2000. You can also avoid it in earlier versions by editing the file -``C:\WINDOWS\COMMAND\CMDINIT.BAT`` instead of ``AUTOEXEC.BAT``. - -You should now be able to start a new command window, enter ``python`` at the -``C:\>`` (or whatever) prompt, and see the ``>>>`` prompt that indicates the -Python interpreter is reading interactive commands. - -Let's suppose you have a program called ``pytest.py`` in directory -``C:\Steve\Projects\Python``. A session to run that program might look like -this:: - - C:\> cd \Steve\Projects\Python - C:\Steve\Projects\Python> python pytest.py - -Because you added a file name to the command to start the interpreter, when it -starts up it reads the Python script in the named file, compiles it, executes -it, and terminates, so you see another ``C:\>`` prompt. You might also have -entered :: - - C:\> python \Steve\Projects\Python\pytest.py - -if you hadn't wanted to change your current directory. - -Under NT, 2000 and XP you may well find that the installation process has also -arranged that the command ``pytest.py`` (or, if the file isn't in the current -directory, ``C:\Steve\Projects\Python\pytest.py``) will automatically recognize -the ".py" extension and run the Python interpreter on the named file. Using this -feature is fine, but *some* versions of Windows have bugs which mean that this -form isn't exactly equivalent to using the interpreter explicitly, so be -careful. - -The important things to remember are: - -1. Start Python from the Start Menu, or make sure the PATH is set correctly so - Windows can find the Python interpreter. :: - - python - - should give you a '>>>' prompt from the Python interpreter. Don't forget the - CTRL-Z and ENTER to terminate the interpreter (and, if you started the window - from the Start Menu, make the window disappear). - -2. Once this works, you run programs with commands:: - - python {program-file} - -3. When you know the commands to use you can build Windows shortcuts to run the - Python interpreter on any of your scripts, naming particular working - directories, and adding them to your menus. Take a look at :: - - python --help - - if your needs are complex. - -4. Interactive mode (where you see the ``>>>`` prompt) is best used for checking - that individual statements and expressions do what you think they will, and - for developing code by experiment. +an "Enter" to get out of it). Once you have verified the directory, you can +add it to the system path to make it easier to start Python by just running +the ``python`` command. This is currently an option in the installer as of +CPython 2.7. +More information about environment variables can be found on the +:ref:`Using Python on Windows <setting-envvars>` page. How do I make Python scripts executable? ---------------------------------------- -On Windows 2000, the standard Python installer already associates the .py +On Windows, the standard Python installer already associates the .py extension with a file type (Python.File) and gives that file type an open command that runs the interpreter (``D:\Program Files\Python\python.exe "%1" %*``). This is enough to make scripts executable from the command prompt as 'foo.py'. If you'd rather be able to execute the script by simple typing 'foo' with no extension you need to add .py to the PATHEXT environment variable. -On Windows NT, the steps taken by the installer as described above allow you to -run a script with 'foo.py', but a longtime bug in the NT command processor -prevents you from redirecting the input or output of any script executed in this -way. This is often important. - -The incantation for making a Python script executable under WinNT is to give the -file an extension of .cmd and add the following as the first line:: - - @setlocal enableextensions & python -x %~f0 %* & goto :EOF - - Why does Python sometimes take so long to start? ------------------------------------------------ @@ -242,22 +163,11 @@ McAfee, when configured to scan all file system read activity, is a particular offender. -Where is Freeze for Windows? ----------------------------- - -"Freeze" is a program that allows you to ship a Python program as a single -stand-alone executable file. It is *not* a compiler; your programs don't run -any faster, but they are more easily distributable, at least to platforms with -the same OS and CPU. Read the README file of the freeze program for more -disclaimers. - -You can use freeze on Windows, but you must download the source tree (see -http://www.python.org/download/source). The freeze program is in the -``Tools\freeze`` subdirectory of the source tree. - -You need the Microsoft VC++ compiler, and you probably need to build Python. -The required project files are in the PCbuild directory. +How do I make an executable from a Python script? +------------------------------------------------- +See http://www.py2exe.org/ for a distutils extension that allows you +to create console and GUI executables from Python code. Is a ``*.pyd`` file the same as a DLL? -------------------------------------- @@ -288,7 +198,7 @@ Embedding the Python interpreter in a Windows app can be summarized as follows: be a DLL to handle importing modules that are themselves DLL's. (This is the first key undocumented fact.) Instead, link to :file:`python{NN}.dll`; it is typically installed in ``C:\Windows\System``. *NN* is the Python version, a - number such as "23" for Python 2.3. + number such as "27" for Python 2.7. You can link to Python in two different ways. Load-time linking means linking against :file:`python{NN}.lib`, while run-time linking means linking @@ -333,7 +243,7 @@ Embedding the Python interpreter in a Windows app can be summarized as follows: ... Py_Initialize(); // Initialize Python. initmyAppc(); // Initialize (import) the helper class. - PyRun_SimpleString("import myApp") ; // Import the shadow class. + PyRun_SimpleString("import myApp"); // Import the shadow class. 5. There are two problems with Python's C API which will become apparent if you use a compiler other than MSVC, the compiler used to build pythonNN.dll. @@ -372,47 +282,6 @@ Embedding the Python interpreter in a Windows app can be summarized as follows: object that supports read and write, so all you need is a Python object (defined in your extension module) that contains read() and write() methods. - -How do I use Python for CGI? ----------------------------- - -On the Microsoft IIS server or on the Win95 MS Personal Web Server you set up -Python in the same way that you would set up any other scripting engine. - -Run regedt32 and go to:: - - HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W3SVC\Parameters\ScriptMap - -and enter the following line (making any specific changes that your system may -need):: - - .py :REG_SZ: c:\<path to python>\python.exe -u %s %s - -This line will allow you to call your script with a simple reference like: -``http://yourserver/scripts/yourscript.py`` provided "scripts" is an -"executable" directory for your server (which it usually is by default). The -:option:`-u` flag specifies unbuffered and binary mode for stdin - needed when -working with binary data. - -In addition, it is recommended that using ".py" may not be a good idea for the -file extensions when used in this context (you might want to reserve ``*.py`` -for support modules and use ``*.cgi`` or ``*.cgp`` for "main program" scripts). - -In order to set up Internet Information Services 5 to use Python for CGI -processing, please see the following links: - - http://www.e-coli.net/pyiis_server.html (for Win2k Server) - http://www.e-coli.net/pyiis.html (for Win2k pro) - -Configuring Apache is much simpler. In the Apache configuration file -``httpd.conf``, add the following line at the end of the file:: - - ScriptInterpreterSource Registry - -Then, give your Python CGI-scripts the extension .py and put them in the cgi-bin -directory. - - How do I keep editors from inserting tabs into my Python source? ---------------------------------------------------------------- @@ -456,116 +325,6 @@ with the additional feature of being able to send CTRL+C and CTRL+BREAK to console subprocesses which are designed to handle those signals. See :func:`os.kill` for further details. - -Why does os.path.isdir() fail on NT shared directories? -------------------------------------------------------- - -The solution appears to be always append the "\\" on the end of shared -drives. - - >>> import os - >>> os.path.isdir( '\\\\rorschach\\public') - 0 - >>> os.path.isdir( '\\\\rorschach\\public\\') - 1 - -It helps to think of share points as being like drive letters. Example:: - - k: is not a directory - k:\ is a directory - k:\media is a directory - k:\media\ is not a directory - -The same rules apply if you substitute "k:" with "\\conky\foo":: - - \\conky\foo is not a directory - \\conky\foo\ is a directory - \\conky\foo\media is a directory - \\conky\foo\media\ is not a directory - - -cgi.py (or other CGI programming) doesn't work sometimes on NT or win95! ------------------------------------------------------------------------- - -Be sure you have the latest python.exe, that you are using python.exe rather -than a GUI version of Python and that you have configured the server to execute -:: - - "...\python.exe -u ..." - -for the CGI execution. The :option:`-u` (unbuffered) option on NT and Win95 -prevents the interpreter from altering newlines in the standard input and -output. Without it post/multipart requests will seem to have the wrong length -and binary (e.g. GIF) responses may get garbled (resulting in broken images, PDF -files, and other binary downloads failing). - - -Why doesn't os.popen() work in PythonWin on NT? ------------------------------------------------ - -The reason that os.popen() doesn't work from within PythonWin is due to a bug in -Microsoft's C Runtime Library (CRT). The CRT assumes you have a Win32 console -attached to the process. - -You should use the win32pipe module's popen() instead which doesn't depend on -having an attached Win32 console. - -Example:: - - import win32pipe - f = win32pipe.popen('dir /c c:\\') - print f.readlines() - f.close() - - -Why doesn't os.popen()/win32pipe.popen() work on Win9x? -------------------------------------------------------- - -There is a bug in Win9x that prevents os.popen/win32pipe.popen* from -working. The good news is there is a way to work around this problem. The -Microsoft Knowledge Base article that you need to lookup is: Q150956. You will -find links to the knowledge base at: http://support.microsoft.com/. - - -PyRun_SimpleFile() crashes on Windows but not on Unix; why? ------------------------------------------------------------ - -This is very sensitive to the compiler vendor, version and (perhaps) even -options. If the FILE* structure in your embedding program isn't the same as is -assumed by the Python interpreter it won't work. - -The Python 1.5.* DLLs (``python15.dll``) are all compiled with MS VC++ 5.0 and -with multithreading-DLL options (``/MD``). - -If you can't change compilers or flags, try using :c:func:`Py_RunSimpleString`. -A trick to get it to run an arbitrary file is to construct a call to -:func:`execfile` with the name of your file as argument. - -Also note that you can not mix-and-match Debug and Release versions. If you -wish to use the Debug Multithreaded DLL, then your module *must* have ``_d`` -appended to the base name. - - -Importing _tkinter fails on Windows 95/98: why? ------------------------------------------------- - -Sometimes, the import of _tkinter fails on Windows 95 or 98, complaining with a -message like the following:: - - ImportError: DLL load failed: One of the library files needed - to run this application cannot be found. - -It could be that you haven't installed Tcl/Tk, but if you did install Tcl/Tk, -and the Wish application works correctly, the problem may be that its installer -didn't manage to edit the autoexec.bat file correctly. It tries to add a -statement that changes the PATH environment variable to include the Tcl/Tk 'bin' -subdirectory, but sometimes this edit doesn't quite work. Opening it with -notepad usually reveals what the problem is. - -(One additional hint, noted by David Szafranski: you can't use long filenames -here; e.g. use ``C:\PROGRA~1\Tcl\bin`` instead of ``C:\Program Files\Tcl\bin``.) - - How do I extract the downloaded documentation on Windows? --------------------------------------------------------- @@ -577,38 +336,3 @@ Simply rename the downloaded file to have the .TGZ extension, and WinZip will be able to handle it. (If your copy of WinZip doesn't, get a newer one from http://www.winzip.com.) - -Missing cw3215mt.dll (or missing cw3215.dll) --------------------------------------------- - -Sometimes, when using Tkinter on Windows, you get an error that cw3215mt.dll or -cw3215.dll is missing. - -Cause: you have an old Tcl/Tk DLL built with cygwin in your path (probably -``C:\Windows``). You must use the Tcl/Tk DLLs from the standard Tcl/Tk -installation (Python 1.5.2 comes with one). - - -Warning about CTL3D32 version from installer --------------------------------------------- - -The Python installer issues a warning like this:: - - This version uses CTL3D32.DLL which is not the correct version. - This version is used for windows NT applications only. - -Tim Peters: - - This is a Microsoft DLL, and a notorious source of problems. The message - means what it says: you have the wrong version of this DLL for your operating - system. The Python installation did not cause this -- something else you - installed previous to this overwrote the DLL that came with your OS (probably - older shareware of some sort, but there's no way to tell now). If you search - for "CTL3D32" using any search engine (AltaVista, for example), you'll find - hundreds and hundreds of web pages complaining about the same problem with - all sorts of installation programs. They'll point you to ways to get the - correct version reinstalled on your system (since Python doesn't cause this, - we can't fix it). - -David A Burton has written a little program to fix this. Go to -http://www.burtonsys.com/downloads.html and click on "ctl3dfix.zip". diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 36a912c..b5e8171 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -39,16 +39,34 @@ Glossary create your own ABCs with the :mod:`abc` module. argument - A value passed to a function or method, assigned to a named local - variable in the function body. A function or method may have both - positional arguments and keyword arguments in its definition. - Positional and keyword arguments may be variable-length: ``*`` accepts - or passes (if in the function definition or call) several positional - arguments in a list, while ``**`` does the same for keyword arguments - in a dictionary. + A value passed to a :term:`function` (or :term:`method`) when calling the + function. There are two types of arguments: + + * :dfn:`keyword argument`: an argument preceded by an identifier (e.g. + ``name=``) in a function call or passed as a value in a dictionary + preceded by ``**``. For example, ``3`` and ``5`` are both keyword + arguments in the following calls to :func:`complex`:: + + complex(real=3, imag=5) + complex(**{'real': 3, 'imag': 5}) + + * :dfn:`positional argument`: an argument that is not a keyword argument. + Positional arguments can appear at the beginning of an argument list + and/or be passed as elements of an :term:`iterable` preceded by ``*``. + For example, ``3`` and ``5`` are both positional arguments in the + following calls:: + + complex(3, 5) + complex(*(3, 5)) - Any expression may be used within the argument list, and the evaluated - value is passed to the local variable. + Arguments are assigned to the named local variables in a function body. + See the :ref:`calls` section for the rules governing this assignment. + Syntactically, any expression can be used to represent an argument; the + evaluated value is assigned to the local variable. + + See also the :term:`parameter` glossary entry and the FAQ question on + :ref:`the difference between arguments and parameters + <faq-argument-vs-parameter>`. attribute A value associated with an object which is referenced by name using @@ -59,6 +77,14 @@ Glossary Benevolent Dictator For Life, a.k.a. `Guido van Rossum <http://www.python.org/~guido/>`_, Python's creator. + bytes-like object + An object that supports the :ref:`buffer protocol <bufferobjects>`, + like :class:`str`, :class:`bytearray` or :class:`memoryview`. + Bytes-like objects can be used for various operations that expect + binary data, such as compression, saving to a binary file or sending + over a socket. Some operations need the binary data to be mutable, + in which case not all bytes-like objects can apply. + bytecode Python source code is compiled into bytecode, the internal representation of a Python program in the CPython interpreter. The bytecode is also @@ -80,7 +106,7 @@ Glossary classic class Any class which does not inherit from :class:`object`. See - :term:`new-style class`. Classic classes will be removed in Python 3.0. + :term:`new-style class`. Classic classes have been removed in Python 3. coercion The implicit conversion of an instance of one type to another during an @@ -152,9 +178,9 @@ Glossary For more information about descriptors' methods, see :ref:`descriptors`. dictionary - An associative array, where arbitrary keys are mapped to values. The keys - can be any object with :meth:`__hash__` function and :meth:`__eq__` - methods. Called a hash in Perl. + An associative array, where arbitrary keys are mapped to values. The + keys can be any object with :meth:`__hash__` and :meth:`__eq__` methods. + Called a hash in Perl. docstring A string literal which appears as the first expression in a class, @@ -200,7 +226,7 @@ Glossary An object exposing a file-oriented API (with methods such as :meth:`read()` or :meth:`write()`) to an underlying resource. Depending on the way it was created, a file object can mediate access to a real - on-disk file or to another other type of storage or communication device + on-disk file or to another type of storage or communication device (for example standard input/output, in-memory buffers, sockets, pipes, etc.). File objects are also called :dfn:`file-like objects` or :dfn:`streams`. @@ -227,8 +253,9 @@ Glossary function A series of statements which returns some value to a caller. It can also - be passed zero or more arguments which may be used in the execution of - the body. See also :term:`argument` and :term:`method`. + be passed zero or more :term:`arguments <argument>` which may be used in + the execution of the body. See also :term:`parameter`, :term:`method`, + and the :ref:`function` section. __future__ A pseudo-module which programmers can use to enable new language features @@ -311,7 +338,8 @@ Glossary All of Python's immutable built-in objects are hashable, while no mutable containers (such as lists or dictionaries) are. Objects which are instances of user-defined classes are hashable by default; they all - compare unequal, and their hash value is their :func:`id`. + compare unequal (except with themselves), and their hash value is their + :func:`id`. IDLE An Integrated Development Environment for Python. IDLE is a basic editor @@ -337,6 +365,10 @@ Glossary fraction. Integer division can be forced by using the ``//`` operator instead of the ``/`` operator. See also :term:`__future__`. + importing + The process by which Python code in one module is made available to + Python code in another module. + importer An object that both finds and loads a module; both a :term:`finder` and :term:`loader` object. @@ -359,17 +391,17 @@ Glossary slowly. See also :term:`interactive`. iterable - An object capable of returning its members one at a - time. Examples of iterables include all sequence types (such as - :class:`list`, :class:`str`, and :class:`tuple`) and some non-sequence - types like :class:`dict` and :class:`file` and objects of any classes you - define with an :meth:`__iter__` or :meth:`__getitem__` method. Iterables - can be used in a :keyword:`for` loop and in many other places where a - sequence is needed (:func:`zip`, :func:`map`, ...). When an iterable - object is passed as an argument to the built-in function :func:`iter`, it - returns an iterator for the object. This iterator is good for one pass - over the set of values. When using iterables, it is usually not necessary - to call :func:`iter` or deal with iterator objects yourself. The ``for`` + An object capable of returning its members one at a time. Examples of + iterables include all sequence types (such as :class:`list`, :class:`str`, + and :class:`tuple`) and some non-sequence types like :class:`dict` + and :class:`file` and objects of any classes you define + with an :meth:`__iter__` or :meth:`__getitem__` method. Iterables can be + used in a :keyword:`for` loop and in many other places where a sequence is + needed (:func:`zip`, :func:`map`, ...). When an iterable object is passed + as an argument to the built-in function :func:`iter`, it returns an + iterator for the object. This iterator is good for one pass over the set + of values. When using iterables, it is usually not necessary to call + :func:`iter` or deal with iterator objects yourself. The ``for`` statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop. See also :term:`iterator`, :term:`sequence`, and :term:`generator`. @@ -406,16 +438,13 @@ Glossary :meth:`str.lower` method can serve as a key function for case insensitive sorts. Alternatively, an ad-hoc key function can be built from a :keyword:`lambda` expression such as ``lambda r: (r[0], r[2])``. Also, - the :mod:`operator` module provides three key function constuctors: + the :mod:`operator` module provides three key function constructors: :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and :func:`~operator.methodcaller`. See the :ref:`Sorting HOW TO <sortinghowto>` for examples of how to create and use key functions. keyword argument - Arguments which are preceded with a ``variable_name=`` in the call. - The variable name designates the local name in the function to which the - value is assigned. ``**`` is used to accept or pass a dictionary of - keyword arguments. See :term:`argument`. + See :term:`argument`. lambda An anonymous inline function consisting of a single :term:`expression` @@ -484,6 +513,13 @@ Glossary for a member during lookup. See `The Python 2.3 Method Resolution Order <http://www.python.org/download/releases/2.3/mro/>`_. + module + An object that serves as an organizational unit of Python code. Modules + have a namespace containing arbitrary Python objects. Modules are loaded + into Python by the process of :term:`importing`. + + See also :term:`package`. + MRO See :term:`method resolution order`. @@ -527,7 +563,7 @@ Glossary new-style class Any class which inherits from :class:`object`. This includes all built-in types like :class:`list` and :class:`dict`. Only new-style classes can - use Python's newer, versatile features like :attr:`__slots__`, + use Python's newer, versatile features like :attr:`~object.__slots__`, descriptors, properties, and :meth:`__getattribute__`. More information can be found in :ref:`newstyle`. @@ -537,12 +573,51 @@ Glossary (methods). Also the ultimate base class of any :term:`new-style class`. + package + A Python :term:`module` which can contain submodules or recursively, + subpackages. Technically, a package is a Python module with an + ``__path__`` attribute. + + parameter + A named entity in a :term:`function` (or method) definition that + specifies an :term:`argument` (or in some cases, arguments) that the + function can accept. There are four types of parameters: + + * :dfn:`positional-or-keyword`: specifies an argument that can be passed + either :term:`positionally <argument>` or as a :term:`keyword argument + <argument>`. This is the default kind of parameter, for example *foo* + and *bar* in the following:: + + def func(foo, bar=None): ... + + * :dfn:`positional-only`: specifies an argument that can be supplied only + by position. Python has no syntax for defining positional-only + parameters. However, some built-in functions have positional-only + parameters (e.g. :func:`abs`). + + * :dfn:`var-positional`: specifies that an arbitrary sequence of + positional arguments can be provided (in addition to any positional + arguments already accepted by other parameters). Such a parameter can + be defined by prepending the parameter name with ``*``, for example + *args* in the following:: + + def func(*args, **kwargs): ... + + * :dfn:`var-keyword`: specifies that arbitrarily many keyword arguments + can be provided (in addition to any keyword arguments already accepted + by other parameters). Such a parameter can be defined by prepending + the parameter name with ``**``, for example *kwargs* in the example + above. + + Parameters can specify both optional and required arguments, as well as + default values for some optional arguments. + + See also the :term:`argument` glossary entry, the FAQ question on + :ref:`the difference between arguments and parameters + <faq-argument-vs-parameter>`, and the :ref:`function` section. + positional argument - The arguments assigned to local names inside a function or method, - determined by the order in which they were given in the call. ``*`` is - used to either accept multiple positional arguments (when in the - definition), or pass several arguments as a list to a function. See - :term:`argument`. + See :term:`argument`. Python 3000 Nickname for the Python 3.x release line (coined long ago when the release @@ -605,7 +680,7 @@ Glossary statement A statement is part of a suite (a "block" of code). A statement is either - an :term:`expression` or a one of several constructs with a keyword, such + an :term:`expression` or one of several constructs with a keyword, such as :keyword:`if`, :keyword:`while` or :keyword:`for`. struct sequence @@ -628,7 +703,15 @@ Glossary type The type of a Python object determines what kind of object it is; every object has a type. An object's type is accessible as its - :attr:`__class__` attribute or can be retrieved with ``type(obj)``. + :attr:`~instance.__class__` attribute or can be retrieved with + ``type(obj)``. + + universal newlines + A manner of interpreting text streams in which all of the following are + recognized as ending a line: the Unix end-of-line convention ``'\n'``, + the Windows convention ``'\r\n'``, and the old Macintosh convention + ``'\r'``. See :pep:`278` and :pep:`3116`, as well as + :func:`str.splitlines` for an additional use. view The objects returned from :meth:`dict.viewkeys`, :meth:`dict.viewvalues`, diff --git a/Doc/howto/advocacy.rst b/Doc/howto/advocacy.rst deleted file mode 100644 index e67e201..0000000 --- a/Doc/howto/advocacy.rst +++ /dev/null @@ -1,356 +0,0 @@ -************************* - Python Advocacy HOWTO -************************* - -:Author: A.M. Kuchling -:Release: 0.03 - - -.. topic:: Abstract - - It's usually difficult to get your management to accept open source software, - and Python is no exception to this rule. This document discusses reasons to use - Python, strategies for winning acceptance, facts and arguments you can use, and - cases where you *shouldn't* try to use Python. - - -Reasons to Use Python -===================== - -There are several reasons to incorporate a scripting language into your -development process, and this section will discuss them, and why Python has some -properties that make it a particularly good choice. - - -Programmability ---------------- - -Programs are often organized in a modular fashion. Lower-level operations are -grouped together, and called by higher-level functions, which may in turn be -used as basic operations by still further upper levels. - -For example, the lowest level might define a very low-level set of functions for -accessing a hash table. The next level might use hash tables to store the -headers of a mail message, mapping a header name like ``Date`` to a value such -as ``Tue, 13 May 1997 20:00:54 -0400``. A yet higher level may operate on -message objects, without knowing or caring that message headers are stored in a -hash table, and so forth. - -Often, the lowest levels do very simple things; they implement a data structure -such as a binary tree or hash table, or they perform some simple computation, -such as converting a date string to a number. The higher levels then contain -logic connecting these primitive operations. Using the approach, the primitives -can be seen as basic building blocks which are then glued together to produce -the complete product. - -Why is this design approach relevant to Python? Because Python is well suited -to functioning as such a glue language. A common approach is to write a Python -module that implements the lower level operations; for the sake of speed, the -implementation might be in C, Java, or even Fortran. Once the primitives are -available to Python programs, the logic underlying higher level operations is -written in the form of Python code. The high-level logic is then more -understandable, and easier to modify. - -John Ousterhout wrote a paper that explains this idea at greater length, -entitled "Scripting: Higher Level Programming for the 21st Century". I -recommend that you read this paper; see the references for the URL. Ousterhout -is the inventor of the Tcl language, and therefore argues that Tcl should be -used for this purpose; he only briefly refers to other languages such as Python, -Perl, and Lisp/Scheme, but in reality, Ousterhout's argument applies to -scripting languages in general, since you could equally write extensions for any -of the languages mentioned above. - - -Prototyping ------------ - -In *The Mythical Man-Month*, Fredrick Brooks suggests the following rule when -planning software projects: "Plan to throw one away; you will anyway." Brooks -is saying that the first attempt at a software design often turns out to be -wrong; unless the problem is very simple or you're an extremely good designer, -you'll find that new requirements and features become apparent once development -has actually started. If these new requirements can't be cleanly incorporated -into the program's structure, you're presented with two unpleasant choices: -hammer the new features into the program somehow, or scrap everything and write -a new version of the program, taking the new features into account from the -beginning. - -Python provides you with a good environment for quickly developing an initial -prototype. That lets you get the overall program structure and logic right, and -you can fine-tune small details in the fast development cycle that Python -provides. Once you're satisfied with the GUI interface or program output, you -can translate the Python code into C++, Fortran, Java, or some other compiled -language. - -Prototyping means you have to be careful not to use too many Python features -that are hard to implement in your other language. Using ``eval()``, or regular -expressions, or the :mod:`pickle` module, means that you're going to need C or -Java libraries for formula evaluation, regular expressions, and serialization, -for example. But it's not hard to avoid such tricky code, and in the end the -translation usually isn't very difficult. The resulting code can be rapidly -debugged, because any serious logical errors will have been removed from the -prototype, leaving only more minor slip-ups in the translation to track down. - -This strategy builds on the earlier discussion of programmability. Using Python -as glue to connect lower-level components has obvious relevance for constructing -prototype systems. In this way Python can help you with development, even if -end users never come in contact with Python code at all. If the performance of -the Python version is adequate and corporate politics allow it, you may not need -to do a translation into C or Java, but it can still be faster to develop a -prototype and then translate it, instead of attempting to produce the final -version immediately. - -One example of this development strategy is Microsoft Merchant Server. Version -1.0 was written in pure Python, by a company that subsequently was purchased by -Microsoft. Version 2.0 began to translate the code into C++, shipping with some -C++code and some Python code. Version 3.0 didn't contain any Python at all; all -the code had been translated into C++. Even though the product doesn't contain -a Python interpreter, the Python language has still served a useful purpose by -speeding up development. - -This is a very common use for Python. Past conference papers have also -described this approach for developing high-level numerical algorithms; see -David M. Beazley and Peter S. Lomdahl's paper "Feeding a Large-scale Physics -Application to Python" in the references for a good example. If an algorithm's -basic operations are things like "Take the inverse of this 4000x4000 matrix", -and are implemented in some lower-level language, then Python has almost no -additional performance cost; the extra time required for Python to evaluate an -expression like ``m.invert()`` is dwarfed by the cost of the actual computation. -It's particularly good for applications where seemingly endless tweaking is -required to get things right. GUI interfaces and Web sites are prime examples. - -The Python code is also shorter and faster to write (once you're familiar with -Python), so it's easier to throw it away if you decide your approach was wrong; -if you'd spent two weeks working on it instead of just two hours, you might -waste time trying to patch up what you've got out of a natural reluctance to -admit that those two weeks were wasted. Truthfully, those two weeks haven't -been wasted, since you've learnt something about the problem and the technology -you're using to solve it, but it's human nature to view this as a failure of -some sort. - - -Simplicity and Ease of Understanding ------------------------------------- - -Python is definitely *not* a toy language that's only usable for small tasks. -The language features are general and powerful enough to enable it to be used -for many different purposes. It's useful at the small end, for 10- or 20-line -scripts, but it also scales up to larger systems that contain thousands of lines -of code. - -However, this expressiveness doesn't come at the cost of an obscure or tricky -syntax. While Python has some dark corners that can lead to obscure code, there -are relatively few such corners, and proper design can isolate their use to only -a few classes or modules. It's certainly possible to write confusing code by -using too many features with too little concern for clarity, but most Python -code can look a lot like a slightly-formalized version of human-understandable -pseudocode. - -In *The New Hacker's Dictionary*, Eric S. Raymond gives the following definition -for "compact": - -.. epigraph:: - - Compact *adj.* Of a design, describes the valuable property that it can all be - apprehended at once in one's head. This generally means the thing created from - the design can be used with greater facility and fewer errors than an equivalent - tool that is not compact. Compactness does not imply triviality or lack of - power; for example, C is compact and FORTRAN is not, but C is more powerful than - FORTRAN. Designs become non-compact through accreting features and cruft that - don't merge cleanly into the overall design scheme (thus, some fans of Classic C - maintain that ANSI C is no longer compact). - - (From http://www.catb.org/~esr/jargon/html/C/compact.html) - -In this sense of the word, Python is quite compact, because the language has -just a few ideas, which are used in lots of places. Take namespaces, for -example. Import a module with ``import math``, and you create a new namespace -called ``math``. Classes are also namespaces that share many of the properties -of modules, and have a few of their own; for example, you can create instances -of a class. Instances? They're yet another namespace. Namespaces are currently -implemented as Python dictionaries, so they have the same methods as the -standard dictionary data type: .keys() returns all the keys, and so forth. - -This simplicity arises from Python's development history. The language syntax -derives from different sources; ABC, a relatively obscure teaching language, is -one primary influence, and Modula-3 is another. (For more information about ABC -and Modula-3, consult their respective Web sites at http://www.cwi.nl/~steven/abc/ -and http://www.m3.org.) Other features have come from C, Icon, -Algol-68, and even Perl. Python hasn't really innovated very much, but instead -has tried to keep the language small and easy to learn, building on ideas that -have been tried in other languages and found useful. - -Simplicity is a virtue that should not be underestimated. It lets you learn the -language more quickly, and then rapidly write code -- code that often works the -first time you run it. - - -Java Integration ----------------- - -If you're working with Java, Jython (http://www.jython.org/) is definitely worth -your attention. Jython is a re-implementation of Python in Java that compiles -Python code into Java bytecodes. The resulting environment has very tight, -almost seamless, integration with Java. It's trivial to access Java classes -from Python, and you can write Python classes that subclass Java classes. -Jython can be used for prototyping Java applications in much the same way -CPython is used, and it can also be used for test suites for Java code, or -embedded in a Java application to add scripting capabilities. - - -Arguments and Rebuttals -======================= - -Let's say that you've decided upon Python as the best choice for your -application. How can you convince your management, or your fellow developers, -to use Python? This section lists some common arguments against using Python, -and provides some possible rebuttals. - -**Python is freely available software that doesn't cost anything. How good can -it be?** - -Very good, indeed. These days Linux and Apache, two other pieces of open source -software, are becoming more respected as alternatives to commercial software, -but Python hasn't had all the publicity. - -Python has been around for several years, with many users and developers. -Accordingly, the interpreter has been used by many people, and has gotten most -of the bugs shaken out of it. While bugs are still discovered at intervals, -they're usually either quite obscure (they'd have to be, for no one to have run -into them before) or they involve interfaces to external libraries. The -internals of the language itself are quite stable. - -Having the source code should be viewed as making the software available for -peer review; people can examine the code, suggest (and implement) improvements, -and track down bugs. To find out more about the idea of open source code, along -with arguments and case studies supporting it, go to http://www.opensource.org. - -**Who's going to support it?** - -Python has a sizable community of developers, and the number is still growing. -The Internet community surrounding the language is an active one, and is worth -being considered another one of Python's advantages. Most questions posted to -the comp.lang.python newsgroup are quickly answered by someone. - -Should you need to dig into the source code, you'll find it's clear and -well-organized, so it's not very difficult to write extensions and track down -bugs yourself. If you'd prefer to pay for support, there are companies and -individuals who offer commercial support for Python. - -**Who uses Python for serious work?** - -Lots of people; one interesting thing about Python is the surprising diversity -of applications that it's been used for. People are using Python to: - -* Run Web sites - -* Write GUI interfaces - -* Control number-crunching code on supercomputers - -* Make a commercial application scriptable by embedding the Python interpreter - inside it - -* Process large XML data sets - -* Build test suites for C or Java code - -Whatever your application domain is, there's probably someone who's used Python -for something similar. Yet, despite being useable for such high-end -applications, Python's still simple enough to use for little jobs. - -See http://wiki.python.org/moin/OrganizationsUsingPython for a list of some of -the organizations that use Python. - -**What are the restrictions on Python's use?** - -They're practically nonexistent. Consult the :file:`Misc/COPYRIGHT` file in the -source distribution, or the section :ref:`history-and-license` for the full -language, but it boils down to three conditions: - -* You have to leave the copyright notice on the software; if you don't include - the source code in a product, you have to put the copyright notice in the - supporting documentation. - -* Don't claim that the institutions that have developed Python endorse your - product in any way. - -* If something goes wrong, you can't sue for damages. Practically all software - licenses contain this condition. - -Notice that you don't have to provide source code for anything that contains -Python or is built with it. Also, the Python interpreter and accompanying -documentation can be modified and redistributed in any way you like, and you -don't have to pay anyone any licensing fees at all. - -**Why should we use an obscure language like Python instead of well-known -language X?** - -I hope this HOWTO, and the documents listed in the final section, will help -convince you that Python isn't obscure, and has a healthily growing user base. -One word of advice: always present Python's positive advantages, instead of -concentrating on language X's failings. People want to know why a solution is -good, rather than why all the other solutions are bad. So instead of attacking -a competing solution on various grounds, simply show how Python's virtues can -help. - - -Useful Resources -================ - -http://www.pythonology.com/success - The Python Success Stories are a collection of stories from successful users of - Python, with the emphasis on business and corporate users. - -.. http://www.fsbassociates.com/books/pythonchpt1.htm - The first chapter of \emph{Internet Programming with Python} also - examines some of the reasons for using Python. The book is well worth - buying, but the publishers have made the first chapter available on - the Web. - -http://www.tcl.tk/doc/scripting.html - John Ousterhout's white paper on scripting is a good argument for the utility of - scripting languages, though naturally enough, he emphasizes Tcl, the language he - developed. Most of the arguments would apply to any scripting language. - -http://www.python.org/workshops/1997-10/proceedings/beazley.html - The authors, David M. Beazley and Peter S. Lomdahl, describe their use of - Python at Los Alamos National Laboratory. It's another good example of how - Python can help get real work done. This quotation from the paper has been - echoed by many people: - - .. epigraph:: - - Originally developed as a large monolithic application for massively parallel - processing systems, we have used Python to transform our application into a - flexible, highly modular, and extremely powerful system for performing - simulation, data analysis, and visualization. In addition, we describe how - Python has solved a number of important problems related to the development, - debugging, deployment, and maintenance of scientific software. - -http://pythonjournal.cognizor.com/pyj1/Everitt-Feit_interview98-V1.html - This interview with Andy Feit, discussing Infoseek's use of Python, can be used - to show that choosing Python didn't introduce any difficulties into a company's - development process, and provided some substantial benefits. - -.. http://www.python.org/psa/Commercial.html - Robin Friedrich wrote this document on how to support Python's use in - commercial projects. - -http://www.python.org/workshops/1997-10/proceedings/stein.ps - For the 6th Python conference, Greg Stein presented a paper that traced Python's - adoption and usage at a startup called eShop, and later at Microsoft. - -http://www.opensource.org - Management may be doubtful of the reliability and usefulness of software that - wasn't written commercially. This site presents arguments that show how open - source software can have considerable advantages over closed-source software. - -http://www.faqs.org/docs/Linux-mini/Advocacy.html - The Linux Advocacy mini-HOWTO was the inspiration for this document, and is also - well worth reading for general suggestions on winning acceptance for a new - technology, such as Linux or Python. In general, you won't make much progress - by simply attacking existing systems and complaining about their inadequacies; - this often ends up looking like unfocused whining. It's much better to point - out some of the many areas where Python is an improvement over other systems. - diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst new file mode 100644 index 0000000..f110a55 --- /dev/null +++ b/Doc/howto/argparse.rst @@ -0,0 +1,764 @@ +***************** +Argparse Tutorial +***************** + +:author: Tshepang Lekhonkhobe + +.. _argparse-tutorial: + +This tutorial is intended to be a gentle introduction to :mod:`argparse`, the +recommended command-line parsing module in the Python standard library. + +.. note:: + + There are two other modules that fulfill the same task, namely + :mod:`getopt` (an equivalent for :c:func:`getopt` from the C + language) and the deprecated :mod:`optparse`. + Note also that :mod:`argparse` is based on :mod:`optparse`, + and therefore very similar in terms of usage. + + +Concepts +======== + +Let's show the sort of functionality that we are going to explore in this +introductory tutorial by making use of the :command:`ls` command: + +.. code-block:: sh + + $ ls + cpython devguide prog.py pypy rm-unused-function.patch + $ ls pypy + ctypes_configure demo dotviewer include lib_pypy lib-python ... + $ ls -l + total 20 + drwxr-xr-x 19 wena wena 4096 Feb 18 18:51 cpython + drwxr-xr-x 4 wena wena 4096 Feb 8 12:04 devguide + -rwxr-xr-x 1 wena wena 535 Feb 19 00:05 prog.py + drwxr-xr-x 14 wena wena 4096 Feb 7 00:59 pypy + -rw-r--r-- 1 wena wena 741 Feb 18 01:01 rm-unused-function.patch + $ ls --help + Usage: ls [OPTION]... [FILE]... + List information about the FILEs (the current directory by default). + Sort entries alphabetically if none of -cftuvSUX nor --sort is specified. + ... + +A few concepts we can learn from the four commands: + +* The :command:`ls` command is useful when run without any options at all. It defaults + to displaying the contents of the current directory. + +* If we want beyond what it provides by default, we tell it a bit more. In + this case, we want it to display a different directory, ``pypy``. + What we did is specify what is known as a positional argument. It's named so + because the program should know what to do with the value, solely based on + where it appears on the command line. This concept is more relevant + to a command like :command:`cp`, whose most basic usage is ``cp SRC DEST``. + The first position is *what you want copied,* and the second + position is *where you want it copied to*. + +* Now, say we want to change behaviour of the program. In our example, + we display more info for each file instead of just showing the file names. + The ``-l`` in that case is known as an optional argument. + +* That's a snippet of the help text. It's very useful in that you can + come across a program you have never used before, and can figure out + how it works simply by reading its help text. + + +The basics +========== + +Let us start with a very simple example which does (almost) nothing:: + + import argparse + parser = argparse.ArgumentParser() + parser.parse_args() + +Following is a result of running the code: + +.. code-block:: sh + + $ python prog.py + $ python prog.py --help + usage: prog.py [-h] + + optional arguments: + -h, --help show this help message and exit + $ python prog.py --verbose + usage: prog.py [-h] + prog.py: error: unrecognized arguments: --verbose + $ python prog.py foo + usage: prog.py [-h] + prog.py: error: unrecognized arguments: foo + +Here is what is happening: + +* Running the script without any options results in nothing displayed to + stdout. Not so useful. + +* The second one starts to display the usefulness of the :mod:`argparse` + module. We have done almost nothing, but already we get a nice help message. + +* The ``--help`` option, which can also be shortened to ``-h``, is the only + option we get for free (i.e. no need to specify it). Specifying anything + else results in an error. But even then, we do get a useful usage message, + also for free. + + +Introducing Positional arguments +================================ + +An example:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("echo") + args = parser.parse_args() + print args.echo + +And running the code: + +.. code-block:: sh + + $ python prog.py + usage: prog.py [-h] echo + prog.py: error: the following arguments are required: echo + $ python prog.py --help + usage: prog.py [-h] echo + + positional arguments: + echo + + optional arguments: + -h, --help show this help message and exit + $ python prog.py foo + foo + +Here is what's happening: + +* We've added the :meth:`add_argument` method, which is what we use to specify + which command-line options the program is willing to accept. In this case, + I've named it ``echo`` so that it's in line with its function. + +* Calling our program now requires us to specify an option. + +* The :meth:`parse_args` method actually returns some data from the + options specified, in this case, ``echo``. + +* The variable is some form of 'magic' that :mod:`argparse` performs for free + (i.e. no need to specify which variable that value is stored in). + You will also notice that its name matches the string argument given + to the method, ``echo``. + +Note however that, although the help display looks nice and all, it currently +is not as helpful as it can be. For example we see that we got ``echo`` as a +positional argument, but we don't know what it does, other than by guessing or +by reading the source code. So, let's make it a bit more useful:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("echo", help="echo the string you use here") + args = parser.parse_args() + print args.echo + +And we get: + +.. code-block:: sh + + $ python prog.py -h + usage: prog.py [-h] echo + + positional arguments: + echo echo the string you use here + + optional arguments: + -h, --help show this help message and exit + +Now, how about doing something even more useful:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", help="display a square of a given number") + args = parser.parse_args() + print args.square**2 + +Following is a result of running the code: + +.. code-block:: sh + + $ python prog.py 4 + Traceback (most recent call last): + File "prog.py", line 5, in <module> + print args.square**2 + TypeError: unsupported operand type(s) for ** or pow(): 'str' and 'int' + +That didn't go so well. That's because :mod:`argparse` treats the options we +give it as strings, unless we tell it otherwise. So, let's tell +:mod:`argparse` to treat that input as an integer:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", help="display a square of a given number", + type=int) + args = parser.parse_args() + print args.square**2 + +Following is a result of running the code: + +.. code-block:: sh + + $ python prog.py 4 + 16 + $ python prog.py four + usage: prog.py [-h] square + prog.py: error: argument square: invalid int value: 'four' + +That went well. The program now even helpfully quits on bad illegal input +before proceeding. + + +Introducing Optional arguments +============================== + +So far we, have been playing with positional arguments. Let us +have a look on how to add optional ones:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("--verbosity", help="increase output verbosity") + args = parser.parse_args() + if args.verbosity: + print "verbosity turned on" + +And the output: + +.. code-block:: sh + + $ python prog.py --verbosity 1 + verbosity turned on + $ python prog.py + $ python prog.py --help + usage: prog.py [-h] [--verbosity VERBOSITY] + + optional arguments: + -h, --help show this help message and exit + --verbosity VERBOSITY + increase output verbosity + $ python prog.py --verbosity + usage: prog.py [-h] [--verbosity VERBOSITY] + prog.py: error: argument --verbosity: expected one argument + +Here is what is happening: + +* The program is written so as to display something when ``--verbosity`` is + specified and display nothing when not. + +* To show that the option is actually optional, there is no error when running + the program without it. Note that by default, if an optional argument isn't + used, the relevant variable, in this case :attr:`args.verbosity`, is + given ``None`` as a value, which is the reason it fails the truth + test of the :keyword:`if` statement. + +* The help message is a bit different. + +* When using the ``--verbosity`` option, one must also specify some value, + any value. + +The above example accepts arbitrary integer values for ``--verbosity``, but for +our simple program, only two values are actually useful, ``True`` or ``False``. +Let's modify the code accordingly:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("--verbose", help="increase output verbosity", + action="store_true") + args = parser.parse_args() + if args.verbose: + print "verbosity turned on" + +And the output: + +.. code-block:: sh + + $ python prog.py --verbose + verbosity turned on + $ python prog.py --verbose 1 + usage: prog.py [-h] [--verbose] + prog.py: error: unrecognized arguments: 1 + $ python prog.py --help + usage: prog.py [-h] [--verbose] + + optional arguments: + -h, --help show this help message and exit + --verbose increase output verbosity + +Here is what is happening: + +* The option is now more of a flag than something that requires a value. + We even changed the name of the option to match that idea. + Note that we now specify a new keyword, ``action``, and give it the value + ``"store_true"``. This means that, if the option is specified, + assign the value ``True`` to :data:`args.verbose`. + Not specifying it implies ``False``. + +* It complains when you specify a value, in true spirit of what flags + actually are. + +* Notice the different help text. + + +Short options +------------- + +If you are familiar with command line usage, +you will notice that I haven't yet touched on the topic of short +versions of the options. It's quite simple:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("-v", "--verbose", help="increase output verbosity", + action="store_true") + args = parser.parse_args() + if args.verbose: + print "verbosity turned on" + +And here goes: + +.. code-block:: sh + + $ python prog.py -v + verbosity turned on + $ python prog.py --help + usage: prog.py [-h] [-v] + + optional arguments: + -h, --help show this help message and exit + -v, --verbose increase output verbosity + +Note that the new ability is also reflected in the help text. + + +Combining Positional and Optional arguments +=========================================== + +Our program keeps growing in complexity:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbose", action="store_true", + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbose: + print "the square of {} equals {}".format(args.square, answer) + else: + print answer + +And now the output: + +.. code-block:: sh + + $ python prog.py + usage: prog.py [-h] [-v] square + prog.py: error: the following arguments are required: square + $ python prog.py 4 + 16 + $ python prog.py 4 --verbose + the square of 4 equals 16 + $ python prog.py --verbose 4 + the square of 4 equals 16 + +* We've brought back a positional argument, hence the complaint. + +* Note that the order does not matter. + +How about we give this program of ours back the ability to have +multiple verbosity values, and actually get to use them:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbosity", type=int, + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbosity == 2: + print "the square of {} equals {}".format(args.square, answer) + elif args.verbosity == 1: + print "{}^2 == {}".format(args.square, answer) + else: + print answer + +And the output: + +.. code-block:: sh + + $ python prog.py 4 + 16 + $ python prog.py 4 -v + usage: prog.py [-h] [-v VERBOSITY] square + prog.py: error: argument -v/--verbosity: expected one argument + $ python prog.py 4 -v 1 + 4^2 == 16 + $ python prog.py 4 -v 2 + the square of 4 equals 16 + $ python prog.py 4 -v 3 + 16 + +These all look good except the last one, which exposes a bug in our program. +Let's fix it by restricting the values the ``--verbosity`` option can accept:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbosity", type=int, choices=[0, 1, 2], + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbosity == 2: + print "the square of {} equals {}".format(args.square, answer) + elif args.verbosity == 1: + print "{}^2 == {}".format(args.square, answer) + else: + print answer + +And the output: + +.. code-block:: sh + + $ python prog.py 4 -v 3 + usage: prog.py [-h] [-v {0,1,2}] square + prog.py: error: argument -v/--verbosity: invalid choice: 3 (choose from 0, 1, 2) + $ python prog.py 4 -h + usage: prog.py [-h] [-v {0,1,2}] square + + positional arguments: + square display a square of a given number + + optional arguments: + -h, --help show this help message and exit + -v {0,1,2}, --verbosity {0,1,2} + increase output verbosity + +Note that the change also reflects both in the error message as well as the +help string. + +Now, let's use a different approach of playing with verbosity, which is pretty +common. It also matches the way the CPython executable handles its own +verbosity argument (check the output of ``python --help``):: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display the square of a given number") + parser.add_argument("-v", "--verbosity", action="count", + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbosity == 2: + print "the square of {} equals {}".format(args.square, answer) + elif args.verbosity == 1: + print "{}^2 == {}".format(args.square, answer) + else: + print answer + +We have introduced another action, "count", +to count the number of occurrences of a specific optional arguments: + +.. code-block:: sh + + $ python prog.py 4 + 16 + $ python prog.py 4 -v + 4^2 == 16 + $ python prog.py 4 -vv + the square of 4 equals 16 + $ python prog.py 4 --verbosity --verbosity + the square of 4 equals 16 + $ python prog.py 4 -v 1 + usage: prog.py [-h] [-v] square + prog.py: error: unrecognized arguments: 1 + $ python prog.py 4 -h + usage: prog.py [-h] [-v] square + + positional arguments: + square display a square of a given number + + optional arguments: + -h, --help show this help message and exit + -v, --verbosity increase output verbosity + $ python prog.py 4 -vvv + 16 + +* Yes, it's now more of a flag (similar to ``action="store_true"``) in the + previous version of our script. That should explain the complaint. + +* It also behaves similar to "store_true" action. + +* Now here's a demonstration of what the "count" action gives. You've probably + seen this sort of usage before. + +* And, just like the "store_true" action, if you don't specify the ``-v`` flag, + that flag is considered to have ``None`` value. + +* As should be expected, specifying the long form of the flag, we should get + the same output. + +* Sadly, our help output isn't very informative on the new ability our script + has acquired, but that can always be fixed by improving the documentation for + out script (e.g. via the ``help`` keyword argument). + +* That last output exposes a bug in our program. + + +Let's fix:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbosity", action="count", + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + + # bugfix: replace == with >= + if args.verbosity >= 2: + print "the square of {} equals {}".format(args.square, answer) + elif args.verbosity >= 1: + print "{}^2 == {}".format(args.square, answer) + else: + print answer + +And this is what it gives: + +.. code-block:: sh + + $ python prog.py 4 -vvv + the square of 4 equals 16 + $ python prog.py 4 -vvvv + the square of 4 equals 16 + $ python prog.py 4 + Traceback (most recent call last): + File "prog.py", line 11, in <module> + if args.verbosity >= 2: + TypeError: unorderable types: NoneType() >= int() + +* First output went well, and fixes the bug we had before. + That is, we want any value >= 2 to be as verbose as possible. + +* Third output not so good. + +Let's fix that bug:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("square", type=int, + help="display a square of a given number") + parser.add_argument("-v", "--verbosity", action="count", default=0, + help="increase output verbosity") + args = parser.parse_args() + answer = args.square**2 + if args.verbosity >= 2: + print "the square of {} equals {}".format(args.square, answer) + elif args.verbosity >= 1: + print "{}^2 == {}".format(args.square, answer) + else: + print answer + +We've just introduced yet another keyword, ``default``. +We've set it to ``0`` in order to make it comparable to the other int values. +Remember that by default, +if an optional argument isn't specified, +it gets the ``None`` value, and that cannot be compared to an int value +(hence the :exc:`TypeError` exception). + +And: + +.. code-block:: sh + + $ python prog.py 4 + 16 + +You can go quite far just with what we've learned so far, +and we have only scratched the surface. +The :mod:`argparse` module is very powerful, +and we'll explore a bit more of it before we end this tutorial. + + +Getting a little more advanced +============================== + +What if we wanted to expand our tiny program to perform other powers, +not just squares:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("x", type=int, help="the base") + parser.add_argument("y", type=int, help="the exponent") + parser.add_argument("-v", "--verbosity", action="count", default=0) + args = parser.parse_args() + answer = args.x**args.y + if args.verbosity >= 2: + print "{} to the power {} equals {}".format(args.x, args.y, answer) + elif args.verbosity >= 1: + print "{}^{} == {}".format(args.x, args.y, answer) + else: + print answer + +Output: + +.. code-block:: sh + + $ python prog.py + usage: prog.py [-h] [-v] x y + prog.py: error: the following arguments are required: x, y + $ python prog.py -h + usage: prog.py [-h] [-v] x y + + positional arguments: + x the base + y the exponent + + optional arguments: + -h, --help show this help message and exit + -v, --verbosity + $ python prog.py 4 2 -v + 4^2 == 16 + + +Notice that so far we've been using verbosity level to *change* the text +that gets displayed. The following example instead uses verbosity level +to display *more* text instead:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("x", type=int, help="the base") + parser.add_argument("y", type=int, help="the exponent") + parser.add_argument("-v", "--verbosity", action="count", default=0) + args = parser.parse_args() + answer = args.x**args.y + if args.verbosity >= 2: + print "Running '{}'".format(__file__) + if args.verbosity >= 1: + print "{}^{} ==".format(args.x, args.y), + print answer + +Output: + +.. code-block:: sh + + $ python prog.py 4 2 + 16 + $ python prog.py 4 2 -v + 4^2 == 16 + $ python prog.py 4 2 -vv + Running 'prog.py' + 4^2 == 16 + + +Conflicting options +------------------- + +So far, we have been working with two methods of an +:class:`argparse.ArgumentParser` instance. Let's introduce a third one, +:meth:`add_mutually_exclusive_group`. It allows for us to specify options that +conflict with each other. Let's also change the rest of the program so that +the new functionality makes more sense: +we'll introduce the ``--quiet`` option, +which will be the opposite of the ``--verbose`` one:: + + import argparse + + parser = argparse.ArgumentParser() + group = parser.add_mutually_exclusive_group() + group.add_argument("-v", "--verbose", action="store_true") + group.add_argument("-q", "--quiet", action="store_true") + parser.add_argument("x", type=int, help="the base") + parser.add_argument("y", type=int, help="the exponent") + args = parser.parse_args() + answer = args.x**args.y + + if args.quiet: + print answer + elif args.verbose: + print "{} to the power {} equals {}".format(args.x, args.y, answer) + else: + print "{}^{} == {}".format(args.x, args.y, answer) + +Our program is now simpler, and we've lost some functionality for the sake of +demonstration. Anyways, here's the output: + +.. code-block:: sh + + $ python prog.py 4 2 + 4^2 == 16 + $ python prog.py 4 2 -q + 16 + $ python prog.py 4 2 -v + 4 to the power 2 equals 16 + $ python prog.py 4 2 -vq + usage: prog.py [-h] [-v | -q] x y + prog.py: error: argument -q/--quiet: not allowed with argument -v/--verbose + $ python prog.py 4 2 -v --quiet + usage: prog.py [-h] [-v | -q] x y + prog.py: error: argument -q/--quiet: not allowed with argument -v/--verbose + +That should be easy to follow. I've added that last output so you can see the +sort of flexibility you get, i.e. mixing long form options with short form +ones. + +Before we conclude, you probably want to tell your users the main purpose of +your program, just in case they don't know:: + + import argparse + + parser = argparse.ArgumentParser(description="calculate X to the power of Y") + group = parser.add_mutually_exclusive_group() + group.add_argument("-v", "--verbose", action="store_true") + group.add_argument("-q", "--quiet", action="store_true") + parser.add_argument("x", type=int, help="the base") + parser.add_argument("y", type=int, help="the exponent") + args = parser.parse_args() + answer = args.x**args.y + + if args.quiet: + print answer + elif args.verbose: + print "{} to the power {} equals {}".format(args.x, args.y, answer) + else: + print "{}^{} == {}".format(args.x, args.y, answer) + +Note that slight difference in the usage text. Note the ``[-v | -q]``, +which tells us that we can either use ``-v`` or ``-q``, +but not both at the same time: + +.. code-block:: sh + + $ python prog.py --help + usage: prog.py [-h] [-v | -q] x y + + calculate X to the power of Y + + positional arguments: + x the base + y the exponent + + optional arguments: + -h, --help show this help message and exit + -v, --verbose + -q, --quiet + + +Conclusion +========== + +The :mod:`argparse` module offers a lot more than shown here. +Its docs are quite detailed and thorough, and full of examples. +Having gone through this tutorial, you should easily digest them +without feeling overwhelmed. diff --git a/Doc/howto/cporting.rst b/Doc/howto/cporting.rst index 7ef7537..1ad77d6 100644 --- a/Doc/howto/cporting.rst +++ b/Doc/howto/cporting.rst @@ -2,27 +2,28 @@ .. _cporting-howto: -******************************** -Porting Extension Modules to 3.0 -******************************** +************************************* +Porting Extension Modules to Python 3 +************************************* :author: Benjamin Peterson .. topic:: Abstract - Although changing the C-API was not one of Python 3.0's objectives, the many - Python level changes made leaving 2.x's API intact impossible. In fact, some - changes such as :func:`int` and :func:`long` unification are more obvious on - the C level. This document endeavors to document incompatibilities and how - they can be worked around. + Although changing the C-API was not one of Python 3's objectives, + the many Python-level changes made leaving Python 2's API intact + impossible. In fact, some changes such as :func:`int` and + :func:`long` unification are more obvious on the C level. This + document endeavors to document incompatibilities and how they can + be worked around. Conditional compilation ======================= -The easiest way to compile only some code for 3.0 is to check if -:c:macro:`PY_MAJOR_VERSION` is greater than or equal to 3. :: +The easiest way to compile only some code for Python 3 is to check +if :c:macro:`PY_MAJOR_VERSION` is greater than or equal to 3. :: #if PY_MAJOR_VERSION >= 3 #define IS_PY3K @@ -35,7 +36,7 @@ conditional blocks. Changes to Object APIs ====================== -Python 3.0 merged together some types with similar functions while cleanly +Python 3 merged together some types with similar functions while cleanly separating others. @@ -43,14 +44,14 @@ str/unicode Unification ----------------------- -Python 3.0's :func:`str` (``PyString_*`` functions in C) type is equivalent to -2.x's :func:`unicode` (``PyUnicode_*``). The old 8-bit string type has become -:func:`bytes`. Python 2.6 and later provide a compatibility header, +Python 3's :func:`str` (``PyString_*`` functions in C) type is equivalent to +Python 2's :func:`unicode` (``PyUnicode_*``). The old 8-bit string type has +become :func:`bytes`. Python 2.6 and later provide a compatibility header, :file:`bytesobject.h`, mapping ``PyBytes`` names to ``PyString`` ones. For best -compatibility with 3.0, :c:type:`PyUnicode` should be used for textual data and +compatibility with Python 3, :c:type:`PyUnicode` should be used for textual data and :c:type:`PyBytes` for binary data. It's also important to remember that -:c:type:`PyBytes` and :c:type:`PyUnicode` in 3.0 are not interchangeable like -:c:type:`PyString` and :c:type:`PyUnicode` are in 2.x. The following example +:c:type:`PyBytes` and :c:type:`PyUnicode` in Python 3 are not interchangeable like +:c:type:`PyString` and :c:type:`PyUnicode` are in Python 2. The following example shows best practices with regards to :c:type:`PyUnicode`, :c:type:`PyString`, and :c:type:`PyBytes`. :: @@ -94,36 +95,20 @@ and :c:type:`PyBytes`. :: long/int Unification -------------------- -In Python 3.0, there is only one integer type. It is called :func:`int` on the -Python level, but actually corresponds to 2.x's :func:`long` type. In the -C-API, ``PyInt_*`` functions are replaced by their ``PyLong_*`` neighbors. The -best course of action here is using the ``PyInt_*`` functions aliased to -``PyLong_*`` found in :file:`intobject.h`. The abstract ``PyNumber_*`` APIs -can also be used in some cases. :: - - #include "Python.h" - #include "intobject.h" - - static PyObject * - add_ints(PyObject *self, PyObject *args) { - int one, two; - PyObject *result; - - if (!PyArg_ParseTuple(args, "ii:add_ints", &one, &two)) - return NULL; - - return PyInt_FromLong(one + two); - } - +Python 3 has only one integer type, :func:`int`. But it actually +corresponds to Python 2's :func:`long` type--the :func:`int` type +used in Python 2 was removed. In the C-API, ``PyInt_*`` functions +are replaced by their ``PyLong_*`` equivalents. Module initialization and state =============================== -Python 3.0 has a revamped extension module initialization system. (See -:pep:`3121`.) Instead of storing module state in globals, they should be stored -in an interpreter specific structure. Creating modules that act correctly in -both 2.x and 3.0 is tricky. The following simple example demonstrates how. :: +Python 3 has a revamped extension module initialization system. (See +:pep:`3121`.) Instead of storing module state in globals, they should +be stored in an interpreter specific structure. Creating modules that +act correctly in both Python 2 and Python 3 is tricky. The following +simple example demonstrates how. :: #include "Python.h" @@ -223,15 +208,18 @@ If you're currently using CObjects, and you want to migrate to 3.1 or newer, you'll need to switch to Capsules. :c:type:`CObject` was deprecated in 3.1 and 2.7 and completely removed in Python 3.2. If you only support 2.7, or 3.1 and above, you -can simply switch to :c:type:`Capsule`. If you need to support 3.0 or -versions of Python earlier than 2.7 you'll have to support both CObjects -and Capsules. +can simply switch to :c:type:`Capsule`. If you need to support Python 3.0, +or versions of Python earlier than 2.7, +you'll have to support both CObjects and Capsules. +(Note that Python 3.0 is no longer supported, and it is not recommended +for production use.) The following example header file :file:`capsulethunk.h` may -solve the problem for you; -simply write your code against the :c:type:`Capsule` API, include -this header file after ``"Python.h"``, and you'll automatically use CObjects -in Python 3.0 or versions earlier than 2.7. +solve the problem for you. Simply write your code against the +:c:type:`Capsule` API and include this header file after +:file:`Python.h`. Your code will automatically use Capsules +in versions of Python with Capsules, and switch to CObjects +when Capsules are unavailable. :file:`capsulethunk.h` simulates Capsules using CObjects. However, :c:type:`CObject` provides no place to store the capsule's "name". As a @@ -246,16 +234,16 @@ behave slightly differently from real Capsules. Specifically: * :c:func:`PyCapsule_GetName` always returns NULL. - * :c:func:`PyCapsule_SetName` always throws an exception and + * :c:func:`PyCapsule_SetName` always raises an exception and returns failure. (Since there's no way to store a name in a CObject, noisy failure of :c:func:`PyCapsule_SetName` was deemed preferable to silent failure here. If this is - inconveient, feel free to modify your local + inconvenient, feel free to modify your local copy as you see fit.) You can find :file:`capsulethunk.h` in the Python source distribution -in the :file:`Doc/includes` directory. We also include it here for -your reference; here is :file:`capsulethunk.h`: +as :source:`Doc/includes/capsulethunk.h`. We also include it here for +your convenience: .. literalinclude:: ../includes/capsulethunk.h @@ -266,5 +254,5 @@ Other options If you are writing a new extension module, you might consider `Cython <http://www.cython.org>`_. It translates a Python-like language to C. The -extension modules it creates are compatible with Python 3.x and 2.x. +extension modules it creates are compatible with Python 3 and Python 2. diff --git a/Doc/howto/curses.rst b/Doc/howto/curses.rst index 71e640c..74c1f2a 100644 --- a/Doc/howto/curses.rst +++ b/Doc/howto/curses.rst @@ -118,7 +118,7 @@ function to restore the terminal to its original operating mode. :: A common problem when debugging a curses application is to get your terminal messed up when the application dies without restoring the terminal to its previous state. In Python this commonly happens when your code is buggy and -raises an uncaught exception. Keys are no longer be echoed to the screen when +raises an uncaught exception. Keys are no longer echoed to the screen when you type them, for example, which makes using the shell difficult. In Python you can avoid these complications and make debugging much easier by @@ -144,8 +144,8 @@ window, but you might wish to divide the screen into smaller windows, in order to redraw or clear them separately. The :func:`newwin` function creates a new window of a given size, returning the new window object. :: - begin_x = 20 ; begin_y = 7 - height = 5 ; width = 40 + begin_x = 20; begin_y = 7 + height = 5; width = 40 win = curses.newwin(height, width, begin_y, begin_x) A word about the coordinate system used in curses: coordinates are always passed @@ -184,11 +184,13 @@ displayed. :: # explained in the next section for y in range(0, 100): for x in range(0, 100): - try: pad.addch(y,x, ord('a') + (x*x+y*y) % 26 ) - except curses.error: pass + try: + pad.addch(y,x, ord('a') + (x*x+y*y) % 26) + except curses.error: + pass # Displays a section of the pad in the middle of the screen - pad.refresh( 0,0, 5,5, 20,75) + pad.refresh(0,0, 5,5, 20,75) The :func:`refresh` call displays a section of the pad in the rectangle extending from coordinate (5,5) to coordinate (20,75) on the screen; the upper @@ -271,7 +273,7 @@ application are commonly shown in reverse video; a text viewer may need to highlight certain words. curses supports this by allowing you to specify an attribute for each cell on the screen. -An attribute is a integer, each bit representing a different attribute. You can +An attribute is an integer, each bit representing a different attribute. You can try to display text with multiple attribute bits set, but curses doesn't guarantee that all the possible combinations are available, or that they're all visually distinct. That depends on the ability of the terminal being used, so @@ -300,7 +302,7 @@ could code:: curses.A_REVERSE) stdscr.refresh() -The curses library also supports color on those terminals that provide it, The +The curses library also supports color on those terminals that provide it. The most common such terminal is probably the Linux console, followed by color xterms. @@ -321,7 +323,7 @@ again, such combinations are not guaranteed to work on all terminals. An example, which displays a line of text using color pair 1:: - stdscr.addstr( "Pretty text", curses.color_pair(1) ) + stdscr.addstr("Pretty text", curses.color_pair(1)) stdscr.refresh() As I said before, a color pair consists of a foreground and background color. @@ -343,7 +345,7 @@ When you change a color pair, any text already displayed using that color pair will change to the new colors. You can also display new text in this color with:: - stdscr.addstr(0,0, "RED ALERT!", curses.color_pair(1) ) + stdscr.addstr(0,0, "RED ALERT!", curses.color_pair(1)) Very fancy terminals can change the definitions of the actual colors to a given RGB value. This lets you change color 1, which is usually red, to purple or @@ -381,9 +383,12 @@ your program will look something like this:: while 1: c = stdscr.getch() - if c == ord('p'): PrintDocument() - elif c == ord('q'): break # Exit the while() - elif c == curses.KEY_HOME: x = y = 0 + if c == ord('p'): + PrintDocument() + elif c == ord('q'): + break # Exit the while() + elif c == curses.KEY_HOME: + x = y = 0 The :mod:`curses.ascii` module supplies ASCII class membership functions that take either integer or 1-character-string arguments; these may be useful in @@ -433,4 +438,3 @@ If you write an interesting little program, feel free to contribute it as another demo. We can always use more of them! The ncurses FAQ: http://invisible-island.net/ncurses/ncurses.faq.html - diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index ce4b6bb..2a323c7 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -124,7 +124,7 @@ The important points to remember are: The object returned by ``super()`` also has a custom :meth:`__getattribute__` method for invoking descriptors. The call ``super(B, obj).m()`` searches ``obj.__class__.__mro__`` for the base class ``A`` immediately following ``B`` -and then returns ``A.__dict__['m'].__get__(obj, A)``. If not a descriptor, +and then returns ``A.__dict__['m'].__get__(obj, B)``. If not a descriptor, ``m`` is returned unchanged. If not in the dictionary, ``m`` reverts to a search using :meth:`object.__getattribute__`. @@ -167,7 +167,7 @@ descriptor is useful for monitoring just a few chosen attributes:: return self.val def __set__(self, obj, val): - print 'Updating' , self.name + print 'Updating', self.name self.val = val >>> class MyClass(object): @@ -218,25 +218,36 @@ here is a pure Python equivalent:: self.fget = fget self.fset = fset self.fdel = fdel + if doc is None and fget is not None: + doc = fget.__doc__ self.__doc__ = doc def __get__(self, obj, objtype=None): if obj is None: return self if self.fget is None: - raise AttributeError, "unreadable attribute" + raise AttributeError("unreadable attribute") return self.fget(obj) def __set__(self, obj, value): if self.fset is None: - raise AttributeError, "can't set attribute" + raise AttributeError("can't set attribute") self.fset(obj, value) def __delete__(self, obj): if self.fdel is None: - raise AttributeError, "can't delete attribute" + raise AttributeError("can't delete attribute") self.fdel(obj) + def getter(self, fget): + return type(self)(fget, self.fset, self.fdel, self.__doc__) + + def setter(self, fset): + return type(self)(self.fget, fset, self.fdel, self.__doc__) + + def deleter(self, fdel): + return type(self)(self.fget, self.fset, fdel, self.__doc__) + The :func:`property` builtin helps whenever a user interface has granted attribute access and then subsequent changes require the intervention of a method. @@ -398,7 +409,7 @@ is to create alternate class constructors. In Python 2.3, the classmethod :func:`dict.fromkeys` creates a new dictionary from a list of keys. The pure Python equivalent is:: - class Dict: + class Dict(object): . . . def fromkeys(klass, iterable, value=None): "Emulate dict_fromkeys() in Objects/dictobject.c" diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst index 9636c6c..c6d6a56 100644 --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -244,9 +244,9 @@ Built-in functions such as :func:`max` and :func:`min` can take a single iterator argument and will return the largest or smallest element. The ``"in"`` and ``"not in"`` operators also support iterators: ``X in iterator`` is true if X is found in the stream returned by the iterator. You'll run into obvious -problems if the iterator is infinite; ``max()``, ``min()``, and ``"not in"`` +problems if the iterator is infinite; ``max()``, ``min()`` will never return, and if the element X never appears in the stream, the -``"in"`` operator won't return either. +``"in"`` and ``"not in"`` operators won't return either. Note that you can only go forward in an iterator; there's no way to get the previous element, reset the iterator, or make a copy of it. Iterator objects @@ -587,7 +587,8 @@ And here's an example of changing the counter: Because ``yield`` will often be returning ``None``, you should always check for this case. Don't just use its value in expressions unless you're sure that the -``send()`` method will be the only method used resume your generator function. +``send()`` method will be the only method used to resume your generator +function. In addition to ``send()``, there are two other new methods on generators: @@ -743,8 +744,8 @@ the constructed list's ``.sort()`` method. :: Python wiki at http://wiki.python.org/moin/HowTo/Sorting.) The ``any(iter)`` and ``all(iter)`` built-ins look at the truth values of an -iterable's contents. :func:`any` returns True if any element in the iterable is -a true value, and :func:`all` returns True if all of the elements are true +iterable's contents. :func:`any` returns ``True`` if any element in the iterable is +a true value, and :func:`all` returns ``True`` if all of the elements are true values: >>> any([0,1,0]) diff --git a/Doc/howto/index.rst b/Doc/howto/index.rst index 94ecc9a..e4c95b1 100644 --- a/Doc/howto/index.rst +++ b/Doc/howto/index.rst @@ -13,7 +13,6 @@ Currently, the HOWTOs are: .. toctree:: :maxdepth: 1 - advocacy.rst pyporting.rst cporting.rst curses.rst @@ -28,4 +27,5 @@ Currently, the HOWTOs are: unicode.rst urllib2.rst webservers.rst + argparse.rst diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst index 5e02fdb..07b04f9 100644 --- a/Doc/howto/logging-cookbook.rst +++ b/Doc/howto/logging-cookbook.rst @@ -97,11 +97,11 @@ The output looks like this:: Multiple handlers and formatters -------------------------------- -Loggers are plain Python objects. The :func:`addHandler` method has no minimum -or maximum quota for the number of handlers you may add. Sometimes it will be -beneficial for an application to log all messages of all severities to a text -file while simultaneously logging errors or above to the console. To set this -up, simply configure the appropriate handlers. The logging calls in the +Loggers are plain Python objects. The :meth:`~Logger.addHandler` method has no +minimum or maximum quota for the number of handlers you may add. Sometimes it +will be beneficial for an application to log all messages of all severities to a +text file while simultaneously logging errors or above to the console. To set +this up, simply configure the appropriate handlers. The logging calls in the application code will remain unchanged. Here is a slight modification to the previous simple module-based configuration example:: @@ -295,17 +295,17 @@ the receiving end. A simple way of doing this is attaching a logger2.warning('Jail zesty vixen who grabbed pay from quack.') logger2.error('The five boxing wizards jump quickly.') -At the receiving end, you can set up a receiver using the :mod:`socketserver` +At the receiving end, you can set up a receiver using the :mod:`SocketServer` module. Here is a basic working example:: import pickle import logging import logging.handlers - import socketserver + import SocketServer import struct - class LogRecordStreamHandler(socketserver.StreamRequestHandler): + class LogRecordStreamHandler(SocketServer.StreamRequestHandler): """Handler for a streaming logging request. This basically logs the record using whatever logging policy is @@ -347,7 +347,7 @@ module. Here is a basic working example:: # cycles and network bandwidth! logger.handle(record) - class LogRecordSocketReceiver(socketserver.ThreadingTCPServer): + class LogRecordSocketReceiver(SocketServer.ThreadingTCPServer): """ Simple TCP socket-based logging receiver suitable for testing. """ @@ -357,7 +357,7 @@ module. Here is a basic working example:: def __init__(self, host='localhost', port=logging.handlers.DEFAULT_TCP_LOGGING_PORT, handler=LogRecordStreamHandler): - socketserver.ThreadingTCPServer.__init__(self, (host, port), handler) + SocketServer.ThreadingTCPServer.__init__(self, (host, port), handler) self.abort = 0 self.timeout = 1 self.logname = None @@ -395,8 +395,9 @@ printed on the console; on the server side, you should see something like:: Note that there are some security issues with pickle in some scenarios. If these affect you, you can use an alternative serialization scheme by overriding -the :meth:`makePickle` method and implementing your alternative there, as -well as adapting the above script to use your alternative serialization. +the :meth:`~handlers.SocketHandler.makePickle` method and implementing your +alternative there, as well as adapting the above script to use your alternative +serialization. .. _context-info: @@ -404,6 +405,8 @@ well as adapting the above script to use your alternative serialization. Adding contextual information to your logging output ---------------------------------------------------- +.. currentmodule:: logging + Sometimes you want logging output to contain contextual information in addition to the parameters passed to the logging call. For example, in a networked application, it may be desirable to log client-specific information @@ -445,9 +448,9 @@ information in the delegated call. Here's a snippet from the code of msg, kwargs = self.process(msg, kwargs) self.logger.debug(msg, *args, **kwargs) -The :meth:`process` method of :class:`LoggerAdapter` is where the contextual -information is added to the logging output. It's passed the message and -keyword arguments of the logging call, and it passes back (potentially) +The :meth:`~LoggerAdapter.process` method of :class:`LoggerAdapter` is where the +contextual information is added to the logging output. It's passed the message +and keyword arguments of the logging call, and it passes back (potentially) modified versions of these to use in the call to the underlying logger. The default implementation of this method leaves the message alone, but inserts an 'extra' key in the keyword argument whose value is the dict-like object @@ -459,70 +462,32 @@ merged into the :class:`LogRecord` instance's __dict__, allowing you to use customized strings with your :class:`Formatter` instances which know about the keys of the dict-like object. If you need a different method, e.g. if you want to prepend or append the contextual information to the message string, -you just need to subclass :class:`LoggerAdapter` and override :meth:`process` -to do what you need. Here's an example script which uses this class, which -also illustrates what dict-like behaviour is needed from an arbitrary -'dict-like' object for use in the constructor:: +you just need to subclass :class:`LoggerAdapter` and override +:meth:`~LoggerAdapter.process` to do what you need. Here is a simple example:: - import logging + class CustomAdapter(logging.LoggerAdapter): + """ + This example adapter expects the passed in dict-like object to have a + 'connid' key, whose value in brackets is prepended to the log message. + """ + def process(self, msg, kwargs): + return '[%s] %s' % (self.extra['connid'], msg), kwargs - class ConnInfo: - """ - An example class which shows how an arbitrary class can be used as - the 'extra' context information repository passed to a LoggerAdapter. - """ +which you can use like this:: - def __getitem__(self, name): - """ - To allow this instance to look like a dict. - """ - from random import choice - if name == 'ip': - result = choice(['127.0.0.1', '192.168.0.1']) - elif name == 'user': - result = choice(['jim', 'fred', 'sheila']) - else: - result = self.__dict__.get(name, '?') - return result + logger = logging.getLogger(__name__) + adapter = CustomAdapter(logger, {'connid': some_conn_id}) - def __iter__(self): - """ - To allow iteration over keys, which will be merged into - the LogRecord dict before formatting and output. - """ - keys = ['ip', 'user'] - keys.extend(self.__dict__.keys()) - return keys.__iter__() +Then any events that you log to the adapter will have the value of +``some_conn_id`` prepended to the log messages. - if __name__ == '__main__': - from random import choice - levels = (logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL) - a1 = logging.LoggerAdapter(logging.getLogger('a.b.c'), - { 'ip' : '123.231.231.123', 'user' : 'sheila' }) - logging.basicConfig(level=logging.DEBUG, - format='%(asctime)-15s %(name)-5s %(levelname)-8s IP: %(ip)-15s User: %(user)-8s %(message)s') - a1.debug('A debug message') - a1.info('An info message with %s', 'some parameters') - a2 = logging.LoggerAdapter(logging.getLogger('d.e.f'), ConnInfo()) - for x in range(10): - lvl = choice(levels) - lvlname = logging.getLevelName(lvl) - a2.log(lvl, 'A message at %s level with %d %s', lvlname, 2, 'parameters') - -When this script is run, the output should look something like this:: +Using objects other than dicts to pass contextual information +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - 2008-01-18 14:49:54,023 a.b.c DEBUG IP: 123.231.231.123 User: sheila A debug message - 2008-01-18 14:49:54,023 a.b.c INFO IP: 123.231.231.123 User: sheila An info message with some parameters - 2008-01-18 14:49:54,023 d.e.f CRITICAL IP: 192.168.0.1 User: jim A message at CRITICAL level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f INFO IP: 192.168.0.1 User: jim A message at INFO level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f WARNING IP: 192.168.0.1 User: sheila A message at WARNING level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f ERROR IP: 127.0.0.1 User: fred A message at ERROR level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f ERROR IP: 127.0.0.1 User: sheila A message at ERROR level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f WARNING IP: 192.168.0.1 User: sheila A message at WARNING level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f WARNING IP: 192.168.0.1 User: jim A message at WARNING level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f INFO IP: 192.168.0.1 User: fred A message at INFO level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f WARNING IP: 192.168.0.1 User: sheila A message at WARNING level with 2 parameters - 2008-01-18 14:49:54,033 d.e.f WARNING IP: 127.0.0.1 User: jim A message at WARNING level with 2 parameters +You don't need to pass an actual dict to a :class:`LoggerAdapter` - you could +pass an instance of a class which implements ``__getitem__`` and ``__iter__`` so +that it looks like a dict to logging. This would be useful if you want to +generate values dynamically (whereas the values in a dict would be constant). .. _filters-contextual: @@ -607,25 +572,23 @@ threads in a single process *is* supported, logging to a single file from *multiple processes* is *not* supported, because there is no standard way to serialize access to a single file across multiple processes in Python. If you need to log to a single file from multiple processes, one way of doing this is -to have all the processes log to a :class:`SocketHandler`, and have a separate -process which implements a socket server which reads from the socket and logs -to file. (If you prefer, you can dedicate one thread in one of the existing -processes to perform this function.) :ref:`This section <network-logging>` -documents this approach in more detail and includes a working socket receiver -which can be used as a starting point for you to adapt in your own -applications. +to have all the processes log to a :class:`~handlers.SocketHandler`, and have a +separate process which implements a socket server which reads from the socket +and logs to file. (If you prefer, you can dedicate one thread in one of the +existing processes to perform this function.) +:ref:`This section <network-logging>` documents this approach in more detail and +includes a working socket receiver which can be used as a starting point for you +to adapt in your own applications. If you are using a recent version of Python which includes the :mod:`multiprocessing` module, you could write your own handler which uses the -:class:`Lock` class from this module to serialize access to the file from -your processes. The existing :class:`FileHandler` and subclasses do not make -use of :mod:`multiprocessing` at present, though they may do so in the future. -Note that at present, the :mod:`multiprocessing` module does not provide +:class:`~multiprocessing.Lock` class from this module to serialize access to the +file from your processes. The existing :class:`FileHandler` and subclasses do +not make use of :mod:`multiprocessing` at present, though they may do so in the +future. Note that at present, the :mod:`multiprocessing` module does not provide working lock functionality on all platforms (see http://bugs.python.org/issue3770). -.. currentmodule:: logging.handlers - Using file rotation ------------------- @@ -637,7 +600,7 @@ Sometimes you want to let a log file grow to a certain size, then open a new file and log to that. You may want to keep a certain number of these files, and when that many files have been created, rotate the files so that the number of files and the size of the files both remain bounded. For this usage pattern, the -logging package provides a :class:`RotatingFileHandler`:: +logging package provides a :class:`~handlers.RotatingFileHandler`:: import glob import logging @@ -688,7 +651,7 @@ An example dictionary-based configuration Below is an example of a logging configuration dictionary - it's taken from the `documentation on the Django project <https://docs.djangoproject.com/en/1.3/topics/logging/#configuring-logging>`_. -This dictionary is passed to :func:`~logging.config.dictConfig` to put the configuration into effect:: +This dictionary is passed to :func:`~config.dictConfig` to put the configuration into effect:: LOGGING = { 'version': 1, @@ -743,5 +706,351 @@ This dictionary is passed to :func:`~logging.config.dictConfig` to put the confi } For more information about this configuration, you can see the `relevant -section <https://docs.djangoproject.com/en/1.3/topics/logging/#configuring-logging>`_ +section <https://docs.djangoproject.com/en/1.6/topics/logging/#configuring-logging>`_ of the Django documentation. + +Inserting a BOM into messages sent to a SysLogHandler +----------------------------------------------------- + +`RFC 5424 <http://tools.ietf.org/html/rfc5424>`_ requires that a +Unicode message be sent to a syslog daemon as a set of bytes which have the +following structure: an optional pure-ASCII component, followed by a UTF-8 Byte +Order Mark (BOM), followed by Unicode encoded using UTF-8. (See the `relevant +section of the specification <http://tools.ietf.org/html/rfc5424#section-6>`_.) + +In Python 2.6 and 2.7, code was added to +:class:`~logging.handlers.SysLogHandler` to insert a BOM into the message, but +unfortunately, it was implemented incorrectly, with the BOM appearing at the +beginning of the message and hence not allowing any pure-ASCII component to +appear before it. + +As this behaviour is broken, the incorrect BOM insertion code is being removed +from Python 2.7.4 and later. However, it is not being replaced, and if you +want to produce RFC 5424-compliant messages which include a BOM, an optional +pure-ASCII sequence before it and arbitrary Unicode after it, encoded using +UTF-8, then you need to do the following: + +#. Attach a :class:`~logging.Formatter` instance to your + :class:`~logging.handlers.SysLogHandler` instance, with a format string + such as:: + + u'ASCII section\ufeffUnicode section' + + The Unicode code point ``u'\ufeff'``, when encoded using UTF-8, will be + encoded as a UTF-8 BOM -- the byte-string ``'\xef\xbb\xbf'``. + +#. Replace the ASCII section with whatever placeholders you like, but make sure + that the data that appears in there after substitution is always ASCII (that + way, it will remain unchanged after UTF-8 encoding). + +#. Replace the Unicode section with whatever placeholders you like; if the data + which appears there after substitution contains characters outside the ASCII + range, that's fine -- it will be encoded using UTF-8. + +If the formatted message is Unicode, it *will* be encoded using UTF-8 encoding +by ``SysLogHandler``. If you follow the above rules, you should be able to +produce RFC 5424-compliant messages. If you don't, logging may not complain, +but your messages will not be RFC 5424-compliant, and your syslog daemon may +complain. + + +Implementing structured logging +------------------------------- + +Although most logging messages are intended for reading by humans, and thus not +readily machine-parseable, there might be cirumstances where you want to output +messages in a structured format which *is* capable of being parsed by a program +(without needing complex regular expressions to parse the log message). This is +straightforward to achieve using the logging package. There are a number of +ways in which this could be achieved, but the following is a simple approach +which uses JSON to serialise the event in a machine-parseable manner:: + + import json + import logging + + class StructuredMessage(object): + def __init__(self, message, **kwargs): + self.message = message + self.kwargs = kwargs + + def __str__(self): + return '%s >>> %s' % (self.message, json.dumps(self.kwargs)) + + _ = StructuredMessage # optional, to improve readability + + logging.basicConfig(level=logging.INFO, format='%(message)s') + logging.info(_('message 1', foo='bar', bar='baz', num=123, fnum=123.456)) + +If the above script is run, it prints:: + + message 1 >>> {"fnum": 123.456, "num": 123, "bar": "baz", "foo": "bar"} + +Note that the order of items might be different according to the version of +Python used. + +If you need more specialised processing, you can use a custom JSON encoder, +as in the following complete example:: + + from __future__ import unicode_literals + + import json + import logging + + # This next bit is to ensure the script runs unchanged on 2.x and 3.x + try: + unicode + except NameError: + unicode = str + + class Encoder(json.JSONEncoder): + def default(self, o): + if isinstance(o, set): + return tuple(o) + elif isinstance(o, unicode): + return o.encode('unicode_escape').decode('ascii') + return super(Encoder, self).default(o) + + class StructuredMessage(object): + def __init__(self, message, **kwargs): + self.message = message + self.kwargs = kwargs + + def __str__(self): + s = Encoder().encode(self.kwargs) + return '%s >>> %s' % (self.message, s) + + _ = StructuredMessage # optional, to improve readability + + def main(): + logging.basicConfig(level=logging.INFO, format='%(message)s') + logging.info(_('message 1', set_value=set([1, 2, 3]), snowman='\u2603')) + + if __name__ == '__main__': + main() + +When the above script is run, it prints:: + + message 1 >>> {"snowman": "\u2603", "set_value": [1, 2, 3]} + +Note that the order of items might be different according to the version of +Python used. + + +.. _custom-handlers: + +.. currentmodule:: logging.config + +Customizing handlers with :func:`dictConfig` +-------------------------------------------- + +There are times when you want to customize logging handlers in particular ways, +and if you use :func:`dictConfig` you may be able to do this without +subclassing. As an example, consider that you may want to set the ownership of a +log file. On POSIX, this is easily done using :func:`shutil.chown`, but the file +handlers in the stdlib don't offer built-in support. You can customize handler +creation using a plain function such as:: + + def owned_file_handler(filename, mode='a', encoding=None, owner=None): + if owner: + if not os.path.exists(filename): + open(filename, 'a').close() + shutil.chown(filename, *owner) + return logging.FileHandler(filename, mode, encoding) + +You can then specify, in a logging configuration passed to :func:`dictConfig`, +that a logging handler be created by calling this function:: + + LOGGING = { + 'version': 1, + 'disable_existing_loggers': False, + 'formatters': { + 'default': { + 'format': '%(asctime)s %(levelname)s %(name)s %(message)s' + }, + }, + 'handlers': { + 'file':{ + # The values below are popped from this dictionary and + # used to create the handler, set the handler's level and + # its formatter. + '()': owned_file_handler, + 'level':'DEBUG', + 'formatter': 'default', + # The values below are passed to the handler creator callable + # as keyword arguments. + 'owner': ['pulse', 'pulse'], + 'filename': 'chowntest.log', + 'mode': 'w', + 'encoding': 'utf-8', + }, + }, + 'root': { + 'handlers': ['file'], + 'level': 'DEBUG', + }, + } + +In this example I am setting the ownership using the ``pulse`` user and group, +just for the purposes of illustration. Putting it together into a working +script, ``chowntest.py``:: + + import logging, logging.config, os, shutil + + def owned_file_handler(filename, mode='a', encoding=None, owner=None): + if owner: + if not os.path.exists(filename): + open(filename, 'a').close() + shutil.chown(filename, *owner) + return logging.FileHandler(filename, mode, encoding) + + LOGGING = { + 'version': 1, + 'disable_existing_loggers': False, + 'formatters': { + 'default': { + 'format': '%(asctime)s %(levelname)s %(name)s %(message)s' + }, + }, + 'handlers': { + 'file':{ + # The values below are popped from this dictionary and + # used to create the handler, set the handler's level and + # its formatter. + '()': owned_file_handler, + 'level':'DEBUG', + 'formatter': 'default', + # The values below are passed to the handler creator callable + # as keyword arguments. + 'owner': ['pulse', 'pulse'], + 'filename': 'chowntest.log', + 'mode': 'w', + 'encoding': 'utf-8', + }, + }, + 'root': { + 'handlers': ['file'], + 'level': 'DEBUG', + }, + } + + logging.config.dictConfig(LOGGING) + logger = logging.getLogger('mylogger') + logger.debug('A debug message') + +To run this, you will probably need to run as ``root``:: + + $ sudo python3.3 chowntest.py + $ cat chowntest.log + 2013-11-05 09:34:51,128 DEBUG mylogger A debug message + $ ls -l chowntest.log + -rw-r--r-- 1 pulse pulse 55 2013-11-05 09:34 chowntest.log + +Note that this example uses Python 3.3 because that's where :func:`shutil.chown` +makes an appearance. This approach should work with any Python version that +supports :func:`dictConfig` - namely, Python 2.7, 3.2 or later. With pre-3.3 +versions, you would need to implement the actual ownership change using e.g. +:func:`os.chown`. + +In practice, the handler-creating function may be in a utility module somewhere +in your project. Instead of the line in the configuration:: + + '()': owned_file_handler, + +you could use e.g.:: + + '()': 'ext://project.util.owned_file_handler', + +where ``project.util`` can be replaced with the actual name of the package +where the function resides. In the above working script, using +``'ext://__main__.owned_file_handler'`` should work. Here, the actual callable +is resolved by :func:`dictConfig` from the ``ext://`` specification. + +This example hopefully also points the way to how you could implement other +types of file change - e.g. setting specific POSIX permission bits - in the +same way, using :func:`os.chmod`. + +Of course, the approach could also be extended to types of handler other than a +:class:`~logging.FileHandler` - for example, one of the rotating file handlers, +or a different type of handler altogether. + + +.. _filters-dictconfig: + +Configuring filters with :func:`dictConfig` +------------------------------------------- + +You *can* configure filters using :func:`~logging.config.dictConfig`, though it +might not be obvious at first glance how to do it (hence this recipe). Since +:class:`~logging.Filter` is the only filter class included in the standard +library, and it is unlikely to cater to many requirements (it's only there as a +base class), you will typically need to define your own :class:`~logging.Filter` +subclass with an overridden :meth:`~logging.Filter.filter` method. To do this, +specify the ``()`` key in the configuration dictionary for the filter, +specifying a callable which will be used to create the filter (a class is the +most obvious, but you can provide any callable which returns a +:class:`~logging.Filter` instance). Here is a complete example:: + + import logging + import logging.config + import sys + + class MyFilter(logging.Filter): + def __init__(self, param=None): + self.param = param + + def filter(self, record): + if self.param is None: + allow = True + else: + allow = self.param not in record.msg + if allow: + record.msg = 'changed: ' + record.msg + return allow + + LOGGING = { + 'version': 1, + 'filters': { + 'myfilter': { + '()': MyFilter, + 'param': 'noshow', + } + }, + 'handlers': { + 'console': { + 'class': 'logging.StreamHandler', + 'filters': ['myfilter'] + } + }, + 'root': { + 'level': 'DEBUG', + 'handlers': ['console'] + }, + } + + if __name__ == '__main__': + logging.config.dictConfig(LOGGING) + logging.debug('hello') + logging.debug('hello - noshow') + +This example shows how you can pass configuration data to the callable which +constructs the instance, in the form of keyword parameters. When run, the above +script will print:: + + changed: hello + +which shows that the filter is working as configured. + +A couple of extra points to note: + +* If you can't refer to the callable directly in the configuration (e.g. if it + lives in a different module, and you can't import it directly where the + configuration dictionary is), you can use the form ``ext://...`` as described + in :ref:`logging-config-dict-externalobj`. For example, you could have used + the text ``'ext://__main__.MyFilter'`` instead of ``MyFilter`` in the above + example. + +* As well as for filters, this technique can also be used to configure custom + handlers and formatters. See :ref:`logging-config-dict-userdef` for more + information on how logging supports using user-defined objects in its + configuration, and see the other cookbook recipe :ref:`custom-handlers` above. + diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst index 029a0ab..fdb6c53 100644 --- a/Doc/howto/logging.rst +++ b/Doc/howto/logging.rst @@ -63,6 +63,8 @@ The logging functions are named after the level or severity of the events they are used to track. The standard levels and their applicability are described below (in increasing order of severity): +.. tabularcolumns:: |l|L| + +--------------+---------------------------------------------+ | Level | When it's used | +==============+=============================================+ @@ -120,7 +122,8 @@ Logging to a file ^^^^^^^^^^^^^^^^^ A very common situation is that of recording logging events in a file, so let's -look at that next:: +look at that next. Be sure to try the following in a newly-started Python +interpreter, and don't just continue from the session described above:: import logging logging.basicConfig(filename='example.log',level=logging.DEBUG) @@ -330,6 +333,9 @@ of components: loggers, handlers, filters, and formatters. to output. * Formatters specify the layout of log records in the final output. +Log event information is passed between loggers, handlers, filters and +formatters in a :class:`LogRecord` instance. + Logging is performed by calling methods on instances of the :class:`Logger` class (hereafter called :dfn:`loggers`). Each instance has a name, and they are conceptually arranged in a namespace hierarchy using dots (periods) as @@ -374,6 +380,13 @@ You can change this by passing a format string to :func:`basicConfig` with the *format* keyword argument. For all options regarding how a format string is constructed, see :ref:`formatter-objects`. +Logging Flow +^^^^^^^^^^^^ + +The flow of log event information in loggers and handlers is illustrated in the +following diagram. + +.. image:: logging_flow.png Loggers ^^^^^^^ @@ -457,12 +470,13 @@ Handlers :class:`~logging.Handler` objects are responsible for dispatching the appropriate log messages (based on the log messages' severity) to the handler's -specified destination. Logger objects can add zero or more handler objects to -themselves with an :func:`addHandler` method. As an example scenario, an -application may want to send all log messages to a log file, all log messages -of error or higher to stdout, and all messages of critical to an email address. -This scenario requires three individual handlers where each handler is -responsible for sending messages of a specific severity to a specific location. +specified destination. :class:`Logger` objects can add zero or more handler +objects to themselves with an :meth:`~Logger.addHandler` method. As an example +scenario, an application may want to send all log messages to a log file, all +log messages of error or higher to stdout, and all messages of critical to an +email address. This scenario requires three individual handlers where each +handler is responsible for sending messages of a specific severity to a specific +location. The standard library includes quite a few handler types (see :ref:`useful-handlers`); the tutorials use mainly :class:`StreamHandler` and @@ -473,16 +487,17 @@ themselves with. The only handler methods that seem relevant for application developers who are using the built-in handler objects (that is, not creating custom handlers) are the following configuration methods: -* The :meth:`Handler.setLevel` method, just as in logger objects, specifies the +* The :meth:`~Handler.setLevel` method, just as in logger objects, specifies the lowest severity that will be dispatched to the appropriate destination. Why are there two :func:`setLevel` methods? The level set in the logger determines which severity of messages it will pass to its handlers. The level set in each handler determines which messages that handler will send on. -* :func:`setFormatter` selects a Formatter object for this handler to use. +* :meth:`~Handler.setFormatter` selects a Formatter object for this handler to + use. -* :func:`addFilter` and :func:`removeFilter` respectively configure and - deconfigure filter objects on handlers. +* :meth:`~Handler.addFilter` and :meth:`~Handler.removeFilter` respectively + configure and deconfigure filter objects on handlers. Application code should not directly instantiate and use instances of :class:`Handler`. Instead, the :class:`Handler` class is a base class that @@ -642,6 +657,21 @@ You can see that the config file approach has a few advantages over the Python code approach, mainly separation of configuration and code and the ability of noncoders to easily modify the logging properties. +.. warning:: The :func:`fileConfig` function takes a default parameter, + ``disable_existing_loggers``, which defaults to ``True`` for reasons of + backward compatibility. This may or may not be what you want, since it + will cause any loggers existing before the :func:`fileConfig` call to + be disabled unless they (or an ancestor) are explicitly named in the + configuration. Please refer to the reference documentation for more + information, and specify ``False`` for this parameter if you wish. + + The dictionary passed to :func:`dictConfig` can also specify a Boolean + value with key ``disable_existing_loggers``, which if not specified + explicitly in the dictionary also defaults to being interpreted as + ``True``. This leads to the logger-disabling behaviour described above, + which may not be what you want - in which case, provide the key + explicitly with a value of ``False``. + .. currentmodule:: logging Note that the class names referenced in config files need to be either relative @@ -713,12 +743,11 @@ Configuring Logging for a Library When developing a library which uses logging, you should take care to document how the library uses logging - for example, the names of loggers used. Some consideration also needs to be given to its logging configuration. -If the using application does not use logging, and library code makes logging -calls, then (as described in the previous section) events of severity -``WARNING`` and greater will be printed to ``sys.stderr``. This is regarded as -the best default behaviour. +If the using application does not configure logging, and library code makes +logging calls, then (as described in the previous section) an error message +will be printed to ``sys.stderr``. -If for some reason you *don't* want these messages printed in the absence of +If for some reason you *don't* want this message printed in the absence of any logging configuration, you can attach a do-nothing handler to the top-level logger for your library. This avoids the message being printed, since a handler will be always be found for the library's events: it just doesn't produce any @@ -730,7 +759,7 @@ handlers, as normal. A do-nothing handler is included in the logging package: :class:`~logging.NullHandler` (since Python 2.7). An instance of this handler could be added to the top-level logger of the logging namespace used by the -library (*if* you want to prevent your library's logged events being output to +library (*if* you want to prevent an error message being output to ``sys.stderr`` in the absence of logging configuration). If all logging by a library *foo* is done using loggers with names matching 'foo.x', 'foo.x.y', etc. then the code:: @@ -742,13 +771,14 @@ should have the desired effect. If an organisation produces a number of libraries, then the logger name specified can be 'orgname.foo' rather than just 'foo'. -**PLEASE NOTE:** It is strongly advised that you *do not add any handlers other -than* :class:`~logging.NullHandler` *to your library's loggers*. This is -because the configuration of handlers is the prerogative of the application -developer who uses your library. The application developer knows their target -audience and what handlers are most appropriate for their application: if you -add handlers 'under the hood', you might well interfere with their ability to -carry out unit tests and deliver logs which suit their requirements. +.. note:: It is strongly advised that you *do not add any handlers other + than* :class:`~logging.NullHandler` *to your library's loggers*. This is + because the configuration of handlers is the prerogative of the application + developer who uses your library. The application developer knows their + target audience and what handlers are most appropriate for their + application: if you add handlers 'under the hood', you might well interfere + with their ability to carry out unit tests and deliver logs which suit their + requirements. Logging Levels @@ -891,16 +921,16 @@ Logged messages are formatted for presentation through instances of the use with the % operator and a dictionary. For formatting multiple messages in a batch, instances of -:class:`BufferingFormatter` can be used. In addition to the format string (which -is applied to each message in the batch), there is provision for header and -trailer format strings. +:class:`~handlers.BufferingFormatter` can be used. In addition to the format +string (which is applied to each message in the batch), there is provision for +header and trailer format strings. When filtering based on logger level and/or handler level is not enough, instances of :class:`Filter` can be added to both :class:`Logger` and -:class:`Handler` instances (through their :meth:`addFilter` method). Before -deciding to process a message further, both loggers and handlers consult all -their filters for permission. If any filter returns a false value, the message -is not processed further. +:class:`Handler` instances (through their :meth:`~Handler.addFilter` method). +Before deciding to process a message further, both loggers and handlers consult +all their filters for permission. If any filter returns a false value, the +message is not processed further. The basic :class:`Filter` functionality allows filtering by specific logger name. If this feature is used, messages sent to the named logger and its @@ -918,19 +948,20 @@ in production. This is so that errors which occur while handling logging events cause the application using logging to terminate prematurely. :class:`SystemExit` and :class:`KeyboardInterrupt` exceptions are never -swallowed. Other exceptions which occur during the :meth:`emit` method of a -:class:`Handler` subclass are passed to its :meth:`handleError` method. +swallowed. Other exceptions which occur during the :meth:`~Handler.emit` method +of a :class:`Handler` subclass are passed to its :meth:`~Handler.handleError` +method. -The default implementation of :meth:`handleError` in :class:`Handler` checks -to see if a module-level variable, :data:`raiseExceptions`, is set. If set, a -traceback is printed to :data:`sys.stderr`. If not set, the exception is swallowed. +The default implementation of :meth:`~Handler.handleError` in :class:`Handler` +checks to see if a module-level variable, :data:`raiseExceptions`, is set. If +set, a traceback is printed to :data:`sys.stderr`. If not set, the exception is +swallowed. -**Note:** The default value of :data:`raiseExceptions` is ``True``. This is because -during development, you typically want to be notified of any exceptions that -occur. It's advised that you set :data:`raiseExceptions` to ``False`` for production -usage. +.. note:: The default value of :data:`raiseExceptions` is ``True``. This is + because during development, you typically want to be notified of any + exceptions that occur. It's advised that you set :data:`raiseExceptions` to + ``False`` for production usage. -.. currentmodule:: logging .. _arbitrary-object-messages: @@ -940,11 +971,11 @@ Using arbitrary objects as messages In the preceding sections and examples, it has been assumed that the message passed when logging the event is a string. However, this is not the only possibility. You can pass an arbitrary object as a message, and its -:meth:`__str__` method will be called when the logging system needs to convert -it to a string representation. In fact, if you want to, you can avoid +:meth:`~object.__str__` method will be called when the logging system needs to +convert it to a string representation. In fact, if you want to, you can avoid computing a string representation altogether - for example, the -:class:`SocketHandler` emits an event by pickling it and sending it over the -wire. +:class:`~handlers.SocketHandler` emits an event by pickling it and sending it +over the wire. Optimization @@ -953,9 +984,10 @@ Optimization Formatting of message arguments is deferred until it cannot be avoided. However, computing the arguments passed to the logging method can also be expensive, and you may want to avoid doing it if the logger will just throw -away your event. To decide what to do, you can call the :meth:`isEnabledFor` -method which takes a level argument and returns true if the event would be -created by the Logger for that level of call. You can write code like this:: +away your event. To decide what to do, you can call the +:meth:`~Logger.isEnabledFor` method which takes a level argument and returns +true if the event would be created by the Logger for that level of call. +You can write code like this:: if logger.isEnabledFor(logging.DEBUG): logger.debug('Message with %s, %s', expensive_func1(), @@ -964,6 +996,15 @@ created by the Logger for that level of call. You can write code like this:: so that if the logger's threshold is set above ``DEBUG``, the calls to :func:`expensive_func1` and :func:`expensive_func2` are never made. +.. note:: In some cases, :meth:`~Logger.isEnabledFor` can itself be more + expensive than you'd like (e.g. for deeply nested loggers where an explicit + level is only set high up in the logger hierarchy). In such cases (or if you + want to avoid calling a method in tight loops), you can cache the result of a + call to :meth:`~Logger.isEnabledFor` in a local or instance variable, and use + that instead of calling the method each time. Such a cached value would only + need to be recomputed when the logging configuration changes dynamically + while the application is running (which is not all that common). + There are other optimizations which can be made for specific applications which need more precise control over what logging information is collected. Here's a list of things you can do to avoid processing during logging which you don't @@ -973,6 +1014,11 @@ need: | What you don't want to collect | How to avoid collecting it | +===============================================+========================================+ | Information about where calls were made from. | Set ``logging._srcfile`` to ``None``. | +| | This avoids calling | +| | :func:`sys._getframe`, which may help | +| | to speed up your code in environments | +| | like PyPy (which can't speed up code | +| | that uses :func:`sys._getframe`). | +-----------------------------------------------+----------------------------------------+ | Threading information. | Set ``logging.logThreads`` to ``0``. | +-----------------------------------------------+----------------------------------------+ diff --git a/Doc/howto/logging_flow.png b/Doc/howto/logging_flow.png Binary files differnew file mode 100755 index 0000000..a883823 --- /dev/null +++ b/Doc/howto/logging_flow.png diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst index 309f3f7..9d7e859 100644 --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -10,238 +10,211 @@ Porting Python 2 Code to Python 3 With Python 3 being the future of Python while Python 2 is still in active use, it is good to have your project available for both major releases of - Python. This guide is meant to help you choose which strategy works best - for your project to support both Python 2 & 3 along with how to execute - that strategy. + Python. This guide is meant to help you figure out how best to support both + Python 2 & 3 simultaneously. If you are looking to port an extension module instead of pure Python code, please see :ref:`cporting-howto`. + If you would like to read one core Python developer's take on why Python 3 + came into existence, you can read Nick Coghlan's `Python 3 Q & A`_. -Choosing a Strategy -=================== - -When a project makes the decision that it's time to support both Python 2 & 3, -a decision needs to be made as to how to go about accomplishing that goal. -The chosen strategy will depend on how large the project's existing -codebase is and how much divergence you want from your Python 2 codebase from -your Python 3 one (e.g., starting a new version with Python 3). - -If your project is brand-new or does not have a large codebase, then you may -want to consider writing/porting :ref:`all of your code for Python 3 -and use 3to2 <use_3to2>` to port your code for Python 2. - -If you would prefer to maintain a codebase which is semantically **and** -syntactically compatible with Python 2 & 3 simultaneously, you can write -:ref:`use_same_source`. While this tends to lead to somewhat non-idiomatic -code, it does mean you keep a rapid development process for you, the developer. - -Finally, you do have the option of :ref:`using 2to3 <use_2to3>` to translate -Python 2 code into Python 3 code (with some manual help). This can take the -form of branching your code and using 2to3 to start a Python 3 branch. You can -also have users perform the translation as installation time automatically so -that you only have to maintain a Python 2 codebase. - -Regardless of which approach you choose, porting is not as hard or -time-consuming as you might initially think. You can also tackle the problem -piece-meal as a good portion of porting is simply updating your code to follow -current best practices in a Python 2/3 compatible way. - - -Universal Bits of Advice ------------------------- - -Regardless of what strategy you pick, there are a few things you should -consider. - -One is make sure you have a robust test suite. You need to make sure everything -continues to work, just like when you support a new minor version of Python. -This means making sure your test suite is thorough and is ported properly -between Python 2 & 3. You will also most likely want to use something like tox_ -to automate testing between both a Python 2 and Python 3 VM. - -Two, once your project has Python 3 support, make sure to add the proper -classifier on the Cheeseshop_ (PyPI_). To have your project listed as Python 3 -compatible it must have the -`Python 3 classifier <http://pypi.python.org/pypi?:action=browse&c=533>`_ -(from -http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting/):: - - setup( - name='Your Library', - version='1.0', - classifiers=[ - # make sure to use :: Python *and* :: Python :: 3 so - # that pypi can list the package on the python 3 page - 'Programming Language :: Python', - 'Programming Language :: Python :: 3' - ], - packages=['yourlibrary'], - # make sure to add custom_fixers to the MANIFEST.in - include_package_data=True, - # ... - ) - - -Doing so will cause your project to show up in the -`Python 3 packages list -<http://pypi.python.org/pypi?:action=browse&c=533&show=all>`_. You will know -you set the classifier properly as visiting your project page on the Cheeseshop -will show a Python 3 logo in the upper-left corner of the page. - -Three, the six_ project provides a library which helps iron out differences -between Python 2 & 3. If you find there is a sticky point that is a continual -point of contention in your translation or maintenance of code, consider using -a source-compatible solution relying on six. If you have to create your own -Python 2/3 compatible solution, you can use ``sys.version_info[0] >= 3`` as a -guard. - -Four, read all the approaches. Just because some bit of advice applies to one -approach more than another doesn't mean that some advice doesn't apply to other -strategies. - -Five, drop support for older Python versions if possible. `Python 2.5`_ + If you prefer to read a (free) book on porting a project to Python 3, + consider reading `Porting to Python 3`_ by Lennart Regebro which should cover + much of what is discussed in this HOWTO. + + For help with porting, you can email the python-porting_ mailing list with + questions. + +The Short Version +================= + +* Decide what's the oldest version of Python 2 you want to support (if at all) +* Make sure you have a thorough test suite and use continuous integration + testing to make sure you stay compatible with the versions of Python you care + about +* If you have dependencies, check their Python 3 status using caniusepython3 + (`command-line tool <https://pypi.python.org/pypi/caniusepython3>`__, + `web app <https://caniusepython3.com/>`__) + +With that done, your options are: + +* If you are dropping Python 2 support, use 2to3_ to port to Python 3 +* If you are keeping Python 2 support, then start writing Python 2/3-compatible + code starting **TODAY** + + + If you have dependencies that have not been ported, reach out to them to port + their project while working to make your code compatible with Python 3 so + you're ready when your dependencies are all ported + + If all your dependencies have been ported (or you have none), go ahead and + port to Python 3 + +* If you are creating a new project that wants to have 2/3 compatibility, + code in Python 3 and then backport to Python 2 + + +Before You Begin +================ + +If your project is on the Cheeseshop_/PyPI_, make sure it has the proper +`trove classifiers`_ to signify what versions of Python it **currently** +supports. At minimum you should specify the major version(s), e.g. +``Programming Language :: Python :: 2`` if your project currently only supports +Python 2. It is preferrable that you be as specific as possible by listing every +major/minor version of Python that you support, e.g. if your project supports +Python 2.6 and 2.7, then you want the classifiers of:: + + Programming Language :: Python :: 2 + Programming Language :: Python :: 2.6 + Programming Language :: Python :: 2.7 + +Once your project supports Python 3 you will want to go back and add the +appropriate classifiers for Python 3 as well. This is important as setting the +``Programming Language :: Python :: 3`` classifier will lead to your project +being listed under the `Python 3 Packages`_ section of PyPI. + +Make sure you have a robust test suite. You need to +make sure everything continues to work, just like when you support a new +minor/feature release of Python. This means making sure your test suite is +thorough and is ported properly between Python 2 & 3 (consider using coverage_ +to measure that you have effective test coverage). You will also most likely +want to use something like tox_ to automate testing between all of your +supported versions of Python. You will also want to **port your tests first** so +that you can make sure that you detect breakage during the transition. Tests also +tend to be simpler than the code they are testing so it gives you an idea of how +easy it can be to port code. + +Drop support for older Python versions if possible. `Python 2.5`_ introduced a lot of useful syntax and libraries which have become idiomatic in Python 3. `Python 2.6`_ introduced future statements which makes compatibility much easier if you are going from Python 2 to 3. -`Python 2.7`_ continues the trend in the stdlib. So choose the newest version +`Python 2.7`_ continues the trend in the stdlib. Choose the newest version of Python which you believe can be your minimum support version and work from there. +Target the newest version of Python 3 that you can. Beyond just the usual +bugfixes, compatibility has continued to improve between Python 2 and 3 as time +has passed. E.g. Python 3.3 added back the ``u`` prefix for +strings, making source-compatible Python code easier to write. -.. _tox: http://codespeak.net/tox/ -.. _Cheeseshop: -.. _PyPI: http://pypi.python.org/ -.. _six: http://packages.python.org/six -.. _Python 2.7: http://www.python.org/2.7.x -.. _Python 2.6: http://www.python.org/2.6.x -.. _Python 2.5: http://www.python.org/2.5.x -.. _Python 2.4: http://www.python.org/2.4.x -.. _Python 2.3: http://www.python.org/2.3.x -.. _Python 2.2: http://www.python.org/2.2.x +Writing Source-Compatible Python 2/3 Code +========================================= -.. _use_3to2: +Over the years the Python community has discovered that the easiest way to +support both Python 2 and 3 in parallel is to write Python code that works in +either version. While this might sound counter-intuitive at first, it actually +is not difficult and typically only requires following some select +(non-idiomatic) practices and using some key projects to help make bridging +between Python 2 and 3 easier. -Python 3 and 3to2 -================= +Projects to Consider +-------------------- -If you are starting a new project or your codebase is small enough, you may -want to consider writing your code for Python 3 and backporting to Python 2 -using 3to2_. Thanks to Python 3 being more strict about things than Python 2 -(e.g., bytes vs. strings), the source translation can be easier and more -straightforward than from Python 2 to 3. Plus it gives you more direct -experience developing in Python 3 which, since it is the future of Python, is a -good thing long-term. +The lowest level library for supporting Python 2 & 3 simultaneously is six_. +Reading through its documentation will give you an idea of where exactly the +Python language changed between versions 2 & 3 and thus what you will want the +library to help you continue to support. -A drawback of this approach is that 3to2 is a third-party project. This means -that the Python core developers (and thus this guide) can make no promises -about how well 3to2 works at any time. There is nothing to suggest, though, -that 3to2 is not a high-quality project. +To help automate porting your code over to using six, you can use +modernize_. This project will attempt to rewrite your code to be as modern as +possible while using six to smooth out any differences between Python 2 & 3. +If you want to write your compatible code to feel more like Python 3 there is +the future_ project. It tries to provide backports of objects from Python 3 so +that you can use them from Python 2-compatible code, e.g. replacing the +``bytes`` type from Python 2 with the one from Python 3. +It also provides a translation script like modernize (its translation code is +actually partially based on it) to help start working with a pre-existing code +base. It is also unique in that its translation script will also port Python 3 +code backwards as well as Python 2 code forwards. -.. _3to2: https://bitbucket.org/amentajo/lib3to2/overview +Tips & Tricks +------------- -.. _use_2to3: - -Python 2 and 2to3 -================= - -Included with Python since 2.6, the 2to3_ tool (and :mod:`lib2to3` module) -helps with porting Python 2 to Python 3 by performing various source -translations. This is a perfect solution for projects which wish to branch -their Python 3 code from their Python 2 codebase and maintain them as -independent codebases. You can even begin preparing to use this approach -today by writing future-compatible Python code which works cleanly in -Python 2 in conjunction with 2to3; all steps outlined below will work -with Python 2 code up to the point when the actual use of 2to3 occurs. - -Use of 2to3 as an on-demand translation step at install time is also possible, -preventing the need to maintain a separate Python 3 codebase, but this approach -does come with some drawbacks. While users will only have to pay the -translation cost once at installation, you as a developer will need to pay the -cost regularly during development. If your codebase is sufficiently large -enough then the translation step ends up acting like a compilation step, -robbing you of the rapid development process you are used to with Python. -Obviously the time required to translate a project will vary, so do an -experimental translation just to see how long it takes to evaluate whether you -prefer this approach compared to using :ref:`use_same_source` or simply keeping -a separate Python 3 codebase. - -Below are the typical steps taken by a project which uses a 2to3-based approach -to supporting Python 2 & 3. - +To help with writing source-compatible code using one of the projects mentioned +in `Projects to Consider`_, consider following the below suggestions. Some of +them are handled by the suggested projects, so if you do use one of them then +read their documentation first to see which suggestions below will taken care of +for you. Support Python 2.7 ------------------- +////////////////// As a first step, make sure that your project is compatible with `Python 2.7`_. This is just good to do as Python 2.7 is the last release of Python 2 and thus will be used for a rather long time. It also allows for use of the ``-3`` flag -to Python to help discover places in your code which 2to3 cannot handle but are -known to cause issues. +to Python to help discover places in your code where compatibility might be an +issue (the ``-3`` flag is in Python 2.6 but Python 2.7 adds more warnings). Try to Support `Python 2.6`_ and Newer Only -------------------------------------------- +/////////////////////////////////////////// While not possible for all projects, if you can support `Python 2.6`_ and newer **only**, your life will be much easier. Various future statements, stdlib additions, etc. exist only in Python 2.6 and later which greatly assist in -porting to Python 3. But if you project must keep support for `Python 2.5`_ (or -even `Python 2.4`_) then it is still possible to port to Python 3. +supporting Python 3. But if you project must keep support for `Python 2.5`_ then +it is still possible to simultaneously support Python 3. Below are the benefits you gain if you only have to support Python 2.6 and newer. Some of these options are personal choice while others are **strongly** recommended (the ones that are more for personal choice are labeled as such). If you continue to support older versions of Python then you -at least need to watch out for situations that these solutions fix. +at least need to watch out for situations that these solutions fix and handle +them appropriately (which is where library help from e.g. six_ comes in handy). ``from __future__ import print_function`` ''''''''''''''''''''''''''''''''''''''''' -This is a personal choice. 2to3 handles the translation from the print -statement to the print function rather well so this is an optional step. This -future statement does help, though, with getting used to typing -``print('Hello, World')`` instead of ``print 'Hello, World'``. +It will not only get you used to typing ``print()`` as a function instead of a +statement, but it will also give you the various benefits the function has over +the Python 2 statement (six_ provides a function if you support Python 2.5 or +older). ``from __future__ import unicode_literals`` ''''''''''''''''''''''''''''''''''''''''''' -Another personal choice. You can always mark what you want to be a (unicode) -string with a ``u`` prefix to get the same effect. But regardless of whether -you use this future statement or not, you **must** make sure you know exactly -which Python 2 strings you want to be bytes, and which are to be strings. This -means you should, **at minimum** mark all strings that are meant to be text -strings with a ``u`` prefix if you do not use this future statement. +If you choose to use this future statement then all string literals in +Python 2 will be assumed to be Unicode (as is already the case in Python 3). +If you choose not to use this future statement then you should mark all of your +text strings with a ``u`` prefix and only support Python 3.3 or newer. But you +are **strongly** advised to do one or the other (six_ provides a function in +case you don't want to use the future statement **and** you want to support +Python 3.2 or older). -Bytes literals -'''''''''''''' +Bytes/string literals +''''''''''''''''''''' -This is a **very** important one. The ability to prefix Python 2 strings that -are meant to contain bytes with a ``b`` prefix help to very clearly delineate -what is and is not a Python 3 string. When you run 2to3 on code, all Python 2 -strings become Python 3 strings **unless** they are prefixed with ``b``. +This is a **very** important one. Prefix Python 2 strings that +are meant to contain bytes with a ``b`` prefix to very clearly delineate +what is and is not a Python 3 text string (six_ provides a function to use for +Python 2.5 compatibility). + +This point cannot be stressed enough: make sure you know what all of your string +literals in Python 2 are meant to be in Python 3. Any string literal that +should be treated as bytes should have the ``b`` prefix. Any string literal +that should be Unicode/text in Python 2 should either have the ``u`` literal +(supported, but ignored, in Python 3.3 and later) or you should have +``from __future__ import unicode_literals`` at the top of the file. But the key +point is you should know how Python 3 will treat every one one of your string +literals and you should mark them as appropriate. There are some differences between byte literals in Python 2 and those in Python 3 thanks to the bytes type just being an alias to ``str`` in Python 2. -Probably the biggest "gotcha" is that indexing results in different values. In -Python 2, the value of ``b'py'[1]`` is ``'y'``, while in Python 3 it's ``121``. -You can avoid this disparity by always slicing at the size of a single element: -``b'py'[1:2]`` is ``'y'`` in Python 2 and ``b'y'`` in Python 3 (i.e., close -enough). +See the `Handle Common "Gotchas"`_ section for what to watch out for. -You cannot concatenate bytes and strings in Python 3. But since in Python -2 has bytes aliased to ``str``, it will succeed: ``b'a' + u'b'`` works in -Python 2, but ``b'a' + 'b'`` in Python 3 is a :exc:`TypeError`. A similar issue -also comes about when doing comparisons between bytes and strings. +``from __future__ import absolute_import`` +'''''''''''''''''''''''''''''''''''''''''' +Discussed in more detail below, but you should use this future statement to +prevent yourself from accidentally using implicit relative imports. Supporting `Python 2.5`_ and Newer Only ---------------------------------------- +/////////////////////////////////////// If you are supporting `Python 2.5`_ and newer there are still some features of Python that you can utilize. @@ -251,7 +224,7 @@ Python that you can utilize. '''''''''''''''''''''''''''''''''''''''''' Implicit relative imports (e.g., importing ``spam.bacon`` from within -``spam.eggs`` with the statement ``import bacon``) does not work in Python 3. +``spam.eggs`` with the statement ``import bacon``) do not work in Python 3. This future statement moves away from that and allows the use of explicit relative imports (e.g., ``from . import bacon``). @@ -261,16 +234,74 @@ implicit ones. In `Python 2.6`_ explicit relative imports are available without the statement, but you still want the __future__ statement to prevent implicit relative imports. In `Python 2.7`_ the __future__ statement is not needed. In other words, unless you are only supporting Python 2.7 or a version earlier -than Python 2.5, use the __future__ statement. +than Python 2.5, use this __future__ statement. + +Mark all Unicode strings with a ``u`` prefix +''''''''''''''''''''''''''''''''''''''''''''' + +While Python 2.6 has a ``__future__`` statement to automatically cause Python 2 +to treat all string literals as Unicode, Python 2.5 does not have that shortcut. +This means you should go through and mark all string literals with a ``u`` +prefix to turn them explicitly into text strings where appropriate and only +support Python 3.3 or newer. Otherwise use a project like six_ which provides a +function to pass all text string literals through. + + +Capturing the Currently Raised Exception +'''''''''''''''''''''''''''''''''''''''' + +In Python 2.5 and earlier the syntax to access the current exception is:: + + try: + raise Exception() + except Exception, exc: + # Current exception is 'exc'. + pass + +This syntax changed in Python 3 (and backported to `Python 2.6`_ and later) +to:: + + try: + raise Exception() + except Exception as exc: + # Current exception is 'exc'. + # In Python 3, 'exc' is restricted to the block; in Python 2.6/2.7 it will "leak". + pass + +Because of this syntax change you must change how you capture the current +exception in Python 2.5 and earlier to:: + + try: + raise Exception() + except Exception: + import sys + exc = sys.exc_info()[1] + # Current exception is 'exc'. + pass + +You can get more information about the raised exception from +:func:`sys.exc_info` than simply the current exception instance, but you most +likely don't need it. + +.. note:: + In Python 3, the traceback is attached to the exception instance + through the ``__traceback__`` attribute. If the instance is saved in + a local variable that persists outside of the ``except`` block, the + traceback will create a reference cycle with the current frame and its + dictionary of local variables. This will delay reclaiming dead + resources until the next cyclic :term:`garbage collection` pass. + + In Python 2, this problem only occurs if you save the traceback itself + (e.g. the third element of the tuple returned by :func:`sys.exc_info`) + in a variable. Handle Common "Gotchas" ------------------------ +/////////////////////// -There are a few things that just consistently come up as sticking points for -people which 2to3 cannot handle automatically or can easily be done in Python 2 -to help modernize your code. +These are things to watch out for no matter what version of Python 2 you are +supporting which are not syntactic considerations. ``from __future__ import division`` @@ -327,9 +358,9 @@ One of the biggest issues people have when porting code to Python 3 is handling the bytes/string dichotomy. Because Python 2 allowed the ``str`` type to hold textual data, people have over the years been rather loose in their delineation of what ``str`` instances held text compared to bytes. In Python 3 you cannot -be so care-free anymore and need to properly handle the difference. The key -handling this issue to make sure that **every** string literal in your -Python 2 code is either syntactically of functionally marked as either bytes or +be so care-free anymore and need to properly handle the difference. The key to +handling this issue is to make sure that **every** string literal in your +Python 2 code is either syntactically or functionally marked as either bytes or text data. After this is done you then need to make sure your APIs are designed to either handle a specific type or made to be properly polymorphic. @@ -343,7 +374,7 @@ newer, this can be accomplished by marking bytes literals with a ``b`` prefix and then designating textual data with a ``u`` prefix or using the ``unicode_literals`` future statement. -If your project supports versions of Python pre-dating 2.6, then you should use +If your project supports versions of Python predating 2.6, then you should use the six_ project and its ``b()`` function to denote bytes literals. For text literals you can either use six's ``u()`` function or use a ``u`` prefix. @@ -436,14 +467,7 @@ methods which have unpredictable results (e.g., infinite recursion if you happen to use the ``unicode(self).encode('utf8')`` idiom as the body of your ``__str__()`` method). -There are two ways to solve this issue. One is to use a custom 2to3 fixer. The -blog post at http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ -specifies how to do this. That will allow 2to3 to change all instances of ``def -__unicode(self): ...`` to ``def __str__(self): ...``. This does require you -define your ``__str__()`` method in Python 2 before your ``__unicode__()`` -method. - -The other option is to use a mixin class. This allows you to only define a +You can use a mixin class to work around this. This allows you to only define a ``__unicode__()`` method for your class and let the mixin derive ``__str__()`` for you (code from http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/):: @@ -486,6 +510,7 @@ sequence containing all arguments passed to the :meth:`__init__` method. Even better is to use the documented attributes the exception provides. + Don't use ``__getslice__`` & Friends '''''''''''''''''''''''''''''''''''' @@ -497,189 +522,62 @@ friends. Updating doctests ''''''''''''''''' -2to3_ will attempt to generate fixes for doctests that it comes across. It's -not perfect, though. If you wrote a monolithic set of doctests (e.g., a single -docstring containing all of your doctests), you should at least consider -breaking the doctests up into smaller pieces to make it more manageable to fix. -Otherwise it might very well be worth your time and effort to port your tests -to :mod:`unittest`. +Don't forget to make them Python 2/3 compatible as well. If you wrote a +monolithic set of doctests (e.g., a single docstring containing all of your +doctests), you should at least consider breaking the doctests up into smaller +pieces to make it more manageable to fix. Otherwise it might very well be worth +your time and effort to port your tests to :mod:`unittest`. + + +Update ``map`` for imbalanced input sequences +''''''''''''''''''''''''''''''''''''''''''''' + +With Python 2, when ``map`` was given more than one input sequence it would pad +the shorter sequences with `None` values, returning a sequence as long as the +longest input sequence. +With Python 3, if the input sequences to ``map`` are of unequal length, ``map`` +will stop at the termination of the shortest of the sequences. For full +compatibility with ``map`` from Python 2.x, wrap the sequence arguments in +:func:`itertools.zip_longest`, e.g. ``map(func, *sequences)`` becomes +``list(map(func, itertools.zip_longest(*sequences)))``. Eliminate ``-3`` Warnings ------------------------- When you run your application's test suite, run it using the ``-3`` flag passed to Python. This will cause various warnings to be raised during execution about -things that 2to3 cannot handle automatically (e.g., modules that have been -removed). Try to eliminate those warnings to make your code even more portable -to Python 3. - - -Run 2to3 --------- - -Once you have made your Python 2 code future-compatible with Python 3, it's -time to use 2to3_ to actually port your code. - - -Manually -'''''''' - -To manually convert source code using 2to3_, you use the ``2to3`` script that -is installed with Python 2.6 and later.:: - - 2to3 <directory or file to convert> - -This will cause 2to3 to write out a diff with all of the fixers applied for the -converted source code. If you would like 2to3 to go ahead and apply the changes -you can pass it the ``-w`` flag:: - - 2to3 -w <stuff to convert> +things that are semantic changes between Python 2 and 3. Try to eliminate those +warnings to make your code even more portable to Python 3. -There are other flags available to control exactly which fixers are applied, -etc. +Alternative Approaches +====================== -During Installation -''''''''''''''''''' - -When a user installs your project for Python 3, you can have either -:mod:`distutils` or Distribute_ run 2to3_ on your behalf. -For distutils, use the following idiom:: - - try: # Python 3 - from distutils.command.build_py import build_py_2to3 as build_py - except ImportError: # Python 2 - from distutils.command.build_py import build_py - - setup(cmdclass = {'build_py': build_py}, - # ... - ) - -For Distribute:: - - setup(use_2to3=True, - # ... - ) - -This will allow you to not have to distribute a separate Python 3 version of -your project. It does require, though, that when you perform development that -you at least build your project and use the built Python 3 source for testing. - - -Verify & Test -------------- - -At this point you should (hopefully) have your project converted in such a way -that it works in Python 3. Verify it by running your unit tests and making sure -nothing has gone awry. If you miss something then figure out how to fix it in -Python 3, backport to your Python 2 code, and run your code through 2to3 again -to verify the fix transforms properly. - - -.. _2to3: http://docs.python.org/py3k/library/2to3.html -.. _Distribute: http://packages.python.org/distribute/ - - -.. _use_same_source: - -Python 2/3 Compatible Source -============================ - -While it may seem counter-intuitive, you can write Python code which is -source-compatible between Python 2 & 3. It does lead to code that is not -entirely idiomatic Python (e.g., having to extract the currently raised -exception from ``sys.exc_info()[1]``), but it can be run under Python 2 -**and** Python 3 without using 2to3_ as a translation step (although the tool -should be used to help find potential portability problems). This allows you to -continue to have a rapid development process regardless of whether you are -developing under Python 2 or Python 3. Whether this approach or using -:ref:`use_2to3` works best for you will be a per-project decision. - -To get a complete idea of what issues you will need to deal with, see the -`What's New in Python 3.0`_. Others have reorganized the data in other formats -such as http://docs.pythonsprints.com/python3_porting/py-porting.html . - -The following are some steps to take to try to support both Python 2 & 3 from -the same source code. +While supporting Python 2 & 3 simultaneously is typically the preferred choice +by people so that they can continue to improve code and have it work for the +most number of users, your life may be easier if you only have to support one +major version of Python going forward. +Supporting Only Python 3 Going Forward From Python 2 Code +--------------------------------------------------------- -.. _What's New in Python 3.0: http://docs.python.org/release/3.0/whatsnew/3.0.html +If you have Python 2 code but going forward only want to improve it as Python 3 +code, then you can use 2to3_ to translate your Python 2 code to Python 3 code. +This is only recommended, though, if your current version of your project is +going into maintenance mode and you want all new features to be exclusive to +Python 3. -Follow The Steps for Using 2to3_ --------------------------------- +Backporting Python 3 code to Python 2 +------------------------------------- -All of the steps outlined in how to -:ref:`port Python 2 code with 2to3 <use_2to3>` apply -to creating a Python 2/3 codebase. This includes trying only support Python 2.6 -or newer (the :mod:`__future__` statements work in Python 3 without issue), -eliminating warnings that are triggered by ``-3``, etc. - -You should even consider running 2to3_ over your code (without committing the -changes). This will let you know where potential pain points are within your -code so that you can fix them properly before they become an issue. - - -Use six_ --------- - -The six_ project contains many things to help you write portable Python code. -You should make sure to read its documentation from beginning to end and use -any and all features it provides. That way you will minimize any mistakes you -might make in writing cross-version code. - - -Capturing the Currently Raised Exception ----------------------------------------- - -One change between Python 2 and 3 that will require changing how you code (if -you support `Python 2.5`_ and earlier) is -accessing the currently raised exception. In Python 2.5 and earlier the syntax -to access the current exception is:: - - try: - raise Exception() - except Exception, exc: - # Current exception is 'exc' - pass - -This syntax changed in Python 3 (and backported to `Python 2.6`_ and later) -to:: - - try: - raise Exception() - except Exception as exc: - # Current exception is 'exc' - # In Python 3, 'exc' is restricted to the block; Python 2.6 will "leak" - pass - -Because of this syntax change you must change to capturing the current -exception to:: - - try: - raise Exception() - except Exception: - import sys - exc = sys.exc_info()[1] - # Current exception is 'exc' - pass - -You can get more information about the raised exception from -:func:`sys.exc_info` than simply the current exception instance, but you most -likely don't need it. - -.. note:: - In Python 3, the traceback is attached to the exception instance - through the ``__traceback__`` attribute. If the instance is saved in - a local variable that persists outside of the ``except`` block, the - traceback will create a reference cycle with the current frame and its - dictionary of local variables. This will delay reclaiming dead - resources until the next cyclic :term:`garbage collection` pass. - - In Python 2, this problem only occurs if you save the traceback itself - (e.g. the third element of the tuple returned by :func:`sys.exc_info`) - in a variable. +If you have Python 3 code and have little interest in supporting Python 2 you +can use 3to2_ to translate from Python 3 code to Python 2 code. This is only +recommended if you don't plan to heavily support Python 2 users. Otherwise +write your code for Python 3 and then backport as far back as you want. This +is typically easier than going from Python 2 to 3 as you will have worked out +any difficulties with e.g. bytes/strings, etc. Other Resources @@ -687,17 +585,41 @@ Other Resources The authors of the following blog posts, wiki pages, and books deserve special thanks for making public their tips for porting Python 2 code to Python 3 (and -thus helping provide information for this document): +thus helping provide information for this document and its various revisions +over the years): +* http://wiki.python.org/moin/PortingPythonToPy3k * http://python3porting.com/ * http://docs.pythonsprints.com/python3_porting/py-porting.html * http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting/ * http://dabeaz.blogspot.com/2011/01/porting-py65-and-my-superboard-to.html * http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ * http://lucumr.pocoo.org/2010/2/11/porting-to-python-3-a-guide/ -* http://wiki.python.org/moin/PortingPythonToPy3k +* https://wiki.ubuntu.com/Python/3 If you feel there is something missing from this document that should be added, please email the python-porting_ mailing list. + + +.. _2to3: http://docs.python.org/2/library/2to3.html +.. _3to2: https://pypi.python.org/pypi/3to2 +.. _Cheeseshop: PyPI_ +.. _coverage: https://pypi.python.org/pypi/coverage +.. _future: http://python-future.org/ +.. _modernize: https://github.com/mitsuhiko/python-modernize +.. _Porting to Python 3: http://python3porting.com/ +.. _PyPI: http://pypi.python.org/ +.. _Python 2.2: http://www.python.org/2.2.x +.. _Python 2.5: http://www.python.org/2.5.x +.. _Python 2.6: http://www.python.org/2.6.x +.. _Python 2.7: http://www.python.org/2.7.x +.. _Python 2.5: http://www.python.org/2.5.x +.. _Python 3.3: http://www.python.org/3.3.x +.. _Python 3 Packages: https://pypi.python.org/pypi?:action=browse&c=533&show=all +.. _Python 3 Q & A: http://ncoghlan-devs-python-notes.readthedocs.org/en/latest/python3/questions_and_answers.html .. _python-porting: http://mail.python.org/mailman/listinfo/python-porting +.. _six: https://pypi.python.org/pypi/six +.. _tox: https://pypi.python.org/pypi/tox +.. _trove classifiers: https://pypi.python.org/pypi?%3Aaction=list_classifiers + diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst index 1523c48..2f552e3 100644 --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -265,7 +265,7 @@ performing string substitutions. :: >>> import re >>> p = re.compile('ab*') - >>> print p + >>> p #doctest: +ELLIPSIS <_sre.SRE_Pattern object at 0x...> :func:`re.compile` also accepts an optional *flags* argument, used to enable @@ -359,13 +359,13 @@ for a complete listing. +------------------+-----------------------------------------------+ :meth:`match` and :meth:`search` return ``None`` if no match can be found. If -they're successful, a ``MatchObject`` instance is returned, containing -information about the match: where it starts and ends, the substring it matched, -and more. +they're successful, a :ref:`match object <match-objects>` instance is returned, +containing information about the match: where it starts and ends, the substring +it matched, and more. You can learn about this by interactively experimenting with the :mod:`re` module. If you have Tkinter available, you may also want to look at -:file:`Tools/scripts/redemo.py`, a demonstration program included with the +:source:`Tools/scripts/redemo.py`, a demonstration program included with the Python distribution. It allows you to enter REs and strings, and displays whether the RE matches or fails. :file:`redemo.py` can be quite useful when trying to debug a complicated RE. Phil Schwartz's `Kodos @@ -378,7 +378,7 @@ Python interpreter, import the :mod:`re` module, and compile a RE:: Python 2.2.2 (#1, Feb 10 2003, 12:57:01) >>> import re >>> p = re.compile('[a-z]+') - >>> p + >>> p #doctest: +ELLIPSIS <_sre.SRE_Pattern object at 0x...> Now, you can try matching various strings against the RE ``[a-z]+``. An empty @@ -392,16 +392,16 @@ interpreter to print no output. You can explicitly print the result of None Now, let's try it on a string that it should match, such as ``tempo``. In this -case, :meth:`match` will return a :class:`MatchObject`, so you should store the -result in a variable for later use. :: +case, :meth:`match` will return a :ref:`match object <match-objects>`, so you +should store the result in a variable for later use. :: >>> m = p.match('tempo') - >>> print m + >>> m #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> -Now you can query the :class:`MatchObject` for information about the matching -string. :class:`MatchObject` instances also have several methods and -attributes; the most important ones are: +Now you can query the :ref:`match object <match-objects>` for information +about the matching string. :ref:`match object <match-objects>` instances +also have several methods and attributes; the most important ones are: +------------------+--------------------------------------------+ | Method/Attribute | Purpose | @@ -435,15 +435,16 @@ case. :: >>> print p.match('::: message') None - >>> m = p.search('::: message') ; print m + >>> m = p.search('::: message'); print m #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> >>> m.group() 'message' >>> m.span() (4, 11) -In actual programs, the most common style is to store the :class:`MatchObject` -in a variable, and then check if it was ``None``. This usually looks like:: +In actual programs, the most common style is to store the +:ref:`match object <match-objects>` in a variable, and then check if it was +``None``. This usually looks like:: p = re.compile( ... ) m = p.match( 'string goes here' ) @@ -460,12 +461,12 @@ Two pattern methods return all of the matches for a pattern. ['12', '11', '10'] :meth:`findall` has to create the entire list before it can be returned as the -result. The :meth:`finditer` method returns a sequence of :class:`MatchObject` -instances as an :term:`iterator`. [#]_ :: +result. The :meth:`finditer` method returns a sequence of +:ref:`match object <match-objects>` instances as an :term:`iterator`. [#]_ :: >>> iterator = p.finditer('12 drummers drumming, 11 ... 10 ...') - >>> iterator - <callable-iterator object at 0x401833ac> + >>> iterator #doctest: +ELLIPSIS + <callable-iterator object at 0x...> >>> for match in iterator: ... print match.span() ... @@ -482,11 +483,11 @@ You don't have to create a pattern object and call its methods; the :func:`search`, :func:`findall`, :func:`sub`, and so forth. These functions take the same arguments as the corresponding pattern method, with the RE string added as the first argument, and still return either ``None`` or a -:class:`MatchObject` instance. :: +:ref:`match object <match-objects>` instance. :: >>> print re.match(r'From\s+', 'Fromage amk') None - >>> re.match(r'From\s+', 'From amk Thu May 14 19:12:10 1998') + >>> re.match(r'From\s+', 'From amk Thu May 14 19:12:10 1998') #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> Under the hood, these functions simply create a pattern object for you @@ -501,7 +502,7 @@ more convenient. If a program contains a lot of regular expressions, or re-uses the same ones in several locations, then it might be worthwhile to collect all the definitions in one place, in a section of code that compiles all the REs ahead of time. To take an example from the standard library, here's an extract -from :file:`xmllib.py`:: +from the deprecated :mod:`xmllib` module:: ref = re.compile( ... ) entityref = re.compile( ... ) @@ -687,7 +688,7 @@ given location, they can obviously be matched an infinite number of times. For example, if you wish to match the word ``From`` only at the beginning of a line, the RE to use is ``^From``. :: - >>> print re.search('^From', 'From Here to Eternity') + >>> print re.search('^From', 'From Here to Eternity') #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> >>> print re.search('^From', 'Reciting From Memory') None @@ -699,11 +700,11 @@ given location, they can obviously be matched an infinite number of times. Matches at the end of a line, which is defined as either the end of the string, or any location followed by a newline character. :: - >>> print re.search('}$', '{block}') + >>> print re.search('}$', '{block}') #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> >>> print re.search('}$', '{block} ') None - >>> print re.search('}$', '{block}\n') + >>> print re.search('}$', '{block}\n') #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> To match a literal ``'$'``, use ``\$`` or enclose it inside a character class, @@ -728,7 +729,7 @@ given location, they can obviously be matched an infinite number of times. match when it's contained inside another word. :: >>> p = re.compile(r'\bclass\b') - >>> print p.search('no class at all') + >>> print p.search('no class at all') #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> >>> print p.search('the declassified algorithm') None @@ -746,7 +747,7 @@ given location, they can obviously be matched an infinite number of times. >>> p = re.compile('\bclass\b') >>> print p.search('no class at all') None - >>> print p.search('\b' + 'class' + '\b') + >>> print p.search('\b' + 'class' + '\b') #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> Second, inside a character class, where there's no use for this assertion, @@ -791,9 +792,9 @@ Groups indicated with ``'('``, ``')'`` also capture the starting and ending index of the text that they match; this can be retrieved by passing an argument to :meth:`group`, :meth:`start`, :meth:`end`, and :meth:`span`. Groups are numbered starting with 0. Group 0 is always present; it's the whole RE, so -:class:`MatchObject` methods all have group 0 as their default argument. Later -we'll see how to express groups that don't capture the span of text that they -match. :: +:ref:`match object <match-objects>` methods all have group 0 as their default +argument. Later we'll see how to express groups that don't capture the span +of text that they match. :: >>> p = re.compile('(a)b') >>> m = p.match('ab') @@ -913,10 +914,10 @@ numbers, groups can be referenced by a name. The syntax for a named group is one of the Python-specific extensions: ``(?P<name>...)``. *name* is, obviously, the name of the group. Named groups also behave exactly like capturing groups, and additionally associate a name -with a group. The :class:`MatchObject` methods that deal with capturing groups -all accept either integers that refer to the group by number or strings that -contain the desired group's name. Named groups are still given numbers, so you -can retrieve information about a group in two ways:: +with a group. The :ref:`match object <match-objects>` methods that deal with +capturing groups all accept either integers that refer to the group by number +or strings that contain the desired group's name. Named groups are still +given numbers, so you can retrieve information about a group in two ways:: >>> p = re.compile(r'(?P<word>\b\w+\b)') >>> m = p.search( '(((( Lots of punctuation )))' ) @@ -1180,16 +1181,16 @@ three variations of the replacement string. :: *replacement* can also be a function, which gives you even more control. If *replacement* is a function, the function is called for every non-overlapping -occurrence of *pattern*. On each call, the function is passed a -:class:`MatchObject` argument for the match and can use this information to -compute the desired replacement string and return it. +occurrence of *pattern*. On each call, the function is passed a +:ref:`match object <match-objects>` argument for the match and can use this +information to compute the desired replacement string and return it. -In the following example, the replacement function translates decimals into +In the following example, the replacement function translates decimals into hexadecimal:: - >>> def hexrepl( match ): + >>> def hexrepl(match): ... "Return the hex string for a decimal number" - ... value = int( match.group() ) + ... value = int(match.group()) ... return hex(value) ... >>> p = re.compile(r'\d+') diff --git a/Doc/howto/sockets.rst b/Doc/howto/sockets.rst index f15d659..f8ac348 100644 --- a/Doc/howto/sockets.rst +++ b/Doc/howto/sockets.rst @@ -19,12 +19,6 @@ Sockets ======= -Sockets are used nearly everywhere, but are one of the most severely -misunderstood technologies around. This is a 10,000 foot overview of sockets. -It's not really a tutorial - you'll still have work to do in getting things -working. It doesn't cover the fine points (and there are a lot of them), but I -hope it will give you enough background to begin using them decently. - I'm only going to talk about INET sockets, but they account for at least 99% of the sockets in use. And I'll only talk about STREAM sockets - unless you really know what you're doing (in which case this HOWTO isn't for you!), you'll get @@ -88,9 +82,11 @@ creates a "server socket":: serversocket.listen(5) A couple things to notice: we used ``socket.gethostname()`` so that the socket -would be visible to the outside world. If we had used ``s.bind(('', 80))`` or -``s.bind(('localhost', 80))`` or ``s.bind(('127.0.0.1', 80))`` we would still -have a "server" socket, but one that was only visible within the same machine. +would be visible to the outside world. If we had used ``s.bind(('localhost', +80))`` or ``s.bind(('127.0.0.1', 80))`` we would still have a "server" socket, +but one that was only visible within the same machine. ``s.bind(('', 80))`` +specifies that the socket is reachable by any address the machine happens to +have. A second thing to note: low number ports are usually reserved for "well known" services (HTTP, SNMP etc). If you're playing around, use a nice high number (4 @@ -156,7 +152,7 @@ I'm not going to talk about it here, except to warn you that you need to use there, you may wait forever for the reply, because the request may still be in your output buffer. -Now we come the major stumbling block of sockets - ``send`` and ``recv`` operate +Now we come to the major stumbling block of sockets - ``send`` and ``recv`` operate on the network buffers. They do not necessarily handle all the bytes you hand them (or expect from them), because their major focus is handling the network buffers. In general, they return when the associated network buffers have been @@ -167,7 +163,7 @@ been completely dealt with. When a ``recv`` returns 0 bytes, it means the other side has closed (or is in the process of closing) the connection. You will not receive any more data on this connection. Ever. You may be able to send data successfully; I'll talk -about that some on the next page. +more about this later. A protocol like HTTP uses a socket for only one transfer. The client sends a request, then reads a reply. That's it. The socket is discarded. This means that @@ -211,13 +207,15 @@ length message:: totalsent = totalsent + sent def myreceive(self): - msg = '' - while len(msg) < MSGLEN: - chunk = self.sock.recv(MSGLEN-len(msg)) + chunks = [] + bytes_recd = 0 + while bytes_recd < MSGLEN: + chunk = self.sock.recv(min(MSGLEN - bytes_recd, 2048)) if chunk == '': raise RuntimeError("socket connection broken") - msg = msg + chunk - return msg + chunks.append(chunk) + bytes_recd = bytes_recd + len(chunk) + return ''.join(chunks) The sending code here is usable for almost any messaging scheme - in Python you send strings, and you can use ``len()`` to determine its length (even if it has diff --git a/Doc/howto/sorting.rst b/Doc/howto/sorting.rst index 9aa39f7..56b65b0 100644 --- a/Doc/howto/sorting.rst +++ b/Doc/howto/sorting.rst @@ -124,7 +124,7 @@ Ascending and Descending ======================== Both :meth:`list.sort` and :func:`sorted` accept a *reverse* parameter with a -boolean value. This is using to flag descending sorts. For example, to get the +boolean value. This is used to flag descending sorts. For example, to get the student data in reverse *age* order: >>> sorted(student_tuples, key=itemgetter(2), reverse=True) @@ -210,11 +210,11 @@ there was no :func:`sorted` builtin and :meth:`list.sort` took no keyword arguments. Instead, all of the Py2.x versions supported a *cmp* parameter to handle user specified comparison functions. -In Py3.0, the *cmp* parameter was removed entirely (as part of a larger effort to +In Python 3, the *cmp* parameter was removed entirely (as part of a larger effort to simplify and unify the language, eliminating the conflict between rich comparisons and the :meth:`__cmp__` magic method). -In Py2.x, sort allowed an optional function which can be called for doing the +In Python 2, :meth:`~list.sort` allowed an optional function which can be called for doing the comparisons. That function should take two arguments to be compared and then return a negative value for less-than, return zero if they are equal, or return a positive value for greater-than. For example, we can do: diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst index ff3c721..297e87e 100644 --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -6,8 +6,8 @@ This HOWTO discusses Python 2.x's support for Unicode, and explains various problems that people commonly encounter when trying to work -with Unicode. (This HOWTO has not yet been updated to cover the 3.x -versions of Python.) +with Unicode. For the Python 3 version, see +<http://docs.python.org/py3k/howto/unicode.html>. Introduction to Unicode ======================= @@ -49,7 +49,7 @@ another and managed to catch on. 255 characters aren't very many. For example, you can't fit both the accented characters used in Western Europe and the Cyrillic alphabet used for Russian -into the 128-255 range because there are more than 127 such characters. +into the 128-255 range because there are more than 128 such characters. You could write files using different codes (all your Russian files in a coding system called KOI8, all your French files in a different coding system called @@ -253,11 +253,11 @@ characters greater than 127 will be treated as errors:: >>> s = unicode('abcdef') >>> type(s) <type 'unicode'> - >>> unicode('abcdef' + chr(255)) + >>> unicode('abcdef' + chr(255)) #doctest: +NORMALIZE_WHITESPACE Traceback (most recent call last): - File "<stdin>", line 1, in ? + ... UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 6: - ordinal not in range(128) + ordinal not in range(128) The ``errors`` argument specifies the response when the input string can't be converted according to the encoding's rules. Legal values for this argument are @@ -265,11 +265,11 @@ converted according to the encoding's rules. Legal values for this argument are 'REPLACEMENT CHARACTER'), or 'ignore' (just leave the character out of the Unicode result). The following examples show the differences:: - >>> unicode('\x80abc', errors='strict') + >>> unicode('\x80abc', errors='strict') #doctest: +NORMALIZE_WHITESPACE Traceback (most recent call last): - File "<stdin>", line 1, in ? + ... UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: - ordinal not in range(128) + ordinal not in range(128) >>> unicode('\x80abc', errors='replace') u'\ufffdabc' >>> unicode('\x80abc', errors='ignore') @@ -312,10 +312,11 @@ strings. 8-bit strings will be converted to Unicode before carrying out the operation; Python's default ASCII encoding will be used, so characters greater than 127 will cause an exception:: - >>> s.find('Was\x9f') + >>> s.find('Was\x9f') #doctest: +NORMALIZE_WHITESPACE Traceback (most recent call last): - File "<stdin>", line 1, in ? - UnicodeDecodeError: 'ascii' codec can't decode byte 0x9f in position 3: ordinal not in range(128) + ... + UnicodeDecodeError: 'ascii' codec can't decode byte 0x9f in position 3: + ordinal not in range(128) >>> s.find(u'Was\x9f') -1 @@ -333,10 +334,11 @@ character references. The following example shows the different results:: >>> u = unichr(40960) + u'abcd' + unichr(1972) >>> u.encode('utf-8') '\xea\x80\x80abcd\xde\xb4' - >>> u.encode('ascii') + >>> u.encode('ascii') #doctest: +NORMALIZE_WHITESPACE Traceback (most recent call last): - File "<stdin>", line 1, in ? - UnicodeEncodeError: 'ascii' codec can't encode character '\ua000' in position 0: ordinal not in range(128) + ... + UnicodeEncodeError: 'ascii' codec can't encode character u'\ua000' in + position 0: ordinal not in range(128) >>> u.encode('ascii', 'ignore') 'abcd' >>> u.encode('ascii', 'replace') @@ -384,9 +386,9 @@ arbitrary code point. Octal escapes can go up to U+01ff, which is octal 777. :: >>> s = u"a\xac\u1234\u20ac\U00008000" - ^^^^ two-digit hex escape - ^^^^^^ four-digit Unicode escape - ^^^^^^^^^^ eight-digit Unicode escape + ... # ^^^^ two-digit hex escape + ... # ^^^^^^ four-digit Unicode escape + ... # ^^^^^^^^^^ eight-digit Unicode escape >>> for c in s: print ord(c), ... 97 172 4660 8364 32768 diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst index 6c80c77..d13f174 100644 --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -18,7 +18,7 @@ Introduction .. sidebar:: Related Articles You may also find useful the following article on fetching web resources - with Python : + with Python: * `Basic Authentication <http://www.voidspace.org.uk/python/articles/authentication.shtml>`_ @@ -134,7 +134,7 @@ This is done as follows:: >>> data['location'] = 'Northampton' >>> data['language'] = 'Python' >>> url_values = urllib.urlencode(data) - >>> print url_values + >>> print url_values # The order may differ. #doctest: +SKIP name=Somebody+Here&language=Python&location=Northampton >>> url = 'http://www.example.com/example.cgi' >>> full_url = url + '?' + url_values @@ -150,7 +150,7 @@ We'll discuss here one particular HTTP header, to illustrate how to add headers to your HTTP request. Some websites [#]_ dislike being browsed by programs, or send different versions -to different browsers [#]_ . By default urllib2 identifies itself as +to different browsers [#]_. By default urllib2 identifies itself as ``Python-urllib/x.y`` (where ``x`` and ``y`` are the major and minor version numbers of the Python release, e.g. ``Python-urllib/2.5``), which may confuse the site, or just plain @@ -201,9 +201,9 @@ e.g. :: >>> req = urllib2.Request('http://www.pretend_server.org') >>> try: urllib2.urlopen(req) - >>> except URLError, e: - >>> print e.reason - >>> + ... except URLError as e: + ... print e.reason #doctest: +SKIP + ... (4, 'getaddrinfo failed') @@ -309,18 +309,18 @@ geturl, and info, methods. :: >>> req = urllib2.Request('http://www.python.org/fish.html') >>> try: - >>> urllib2.urlopen(req) - >>> except HTTPError, e: - >>> print e.code - >>> print e.read() - >>> + ... urllib2.urlopen(req) + ... except urllib2.HTTPError as e: + ... print e.code + ... print e.read() #doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE + ... 404 - <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" - "http://www.w3.org/TR/html4/loose.dtd"> - <?xml-stylesheet href="./css/ht2html.css" - type="text/css"?> - <html><head><title>Error 404: File Not Found</title> - ...... etc... + <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" + "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> + ... + <title>Page Not Found</title> + ... + Wrapping it Up -------------- @@ -338,10 +338,10 @@ Number 1 req = Request(someurl) try: response = urlopen(req) - except HTTPError, e: + except HTTPError as e: print 'The server couldn\'t fulfill the request.' print 'Error code: ', e.code - except URLError, e: + except URLError as e: print 'We failed to reach a server.' print 'Reason: ', e.reason else: @@ -362,7 +362,7 @@ Number 2 req = Request(someurl) try: response = urlopen(req) - except URLError, e: + except URLError as e: if hasattr(e, 'reason'): print 'We failed to reach a server.' print 'Reason: ', e.reason @@ -439,12 +439,12 @@ Authentication Tutorial When authentication is required, the server sends a header (as well as the 401 error code) requesting authentication. This specifies the authentication scheme -and a 'realm'. The header looks like : ``Www-authenticate: SCHEME +and a 'realm'. The header looks like: ``WWW-Authenticate: SCHEME realm="REALM"``. e.g. :: - Www-authenticate: Basic realm="cPanel Users" + WWW-Authenticate: Basic realm="cPanel Users" The client should then retry the request with the appropriate name and password @@ -489,7 +489,8 @@ than the URL you pass to .add_password() will also match. :: In the above example we only supplied our ``HTTPBasicAuthHandler`` to ``build_opener``. By default openers have the handlers for normal situations - -- ``ProxyHandler``, ``UnknownHandler``, ``HTTPHandler``, + -- ``ProxyHandler`` (if a proxy setting such as an :envvar:`http_proxy` + environment variable is set), ``UnknownHandler``, ``HTTPHandler``, ``HTTPDefaultErrorHandler``, ``HTTPRedirectHandler``, ``FTPHandler``, ``FileHandler``, ``HTTPErrorProcessor``. @@ -506,10 +507,11 @@ Proxies ======= **urllib2** will auto-detect your proxy settings and use those. This is through -the ``ProxyHandler`` which is part of the normal handler chain. Normally that's -a good thing, but there are occasions when it may not be helpful [#]_. One way -to do this is to setup our own ``ProxyHandler``, with no proxies defined. This -is done using similar steps to setting up a `Basic Authentication`_ handler : :: +the ``ProxyHandler``, which is part of the normal handler chain when a proxy +setting is detected. Normally that's a good thing, but there are occasions +when it may not be helpful [#]_. One way to do this is to setup our own +``ProxyHandler``, with no proxies defined. This is done using similar steps to +setting up a `Basic Authentication`_ handler: :: >>> proxy_support = urllib2.ProxyHandler({}) >>> opener = urllib2.build_opener(proxy_support) diff --git a/Doc/howto/webservers.rst b/Doc/howto/webservers.rst index fbc9fd9..c3b79e4 100644 --- a/Doc/howto/webservers.rst +++ b/Doc/howto/webservers.rst @@ -691,7 +691,7 @@ published, which is a good starting point. The newest version of TurboGears, version 2.0, moves even further in direction of WSGI support and a component-based architecture. TurboGears 2 is based on the WSGI stack of another popular component-based web framework, `Pylons -<http://pylonshq.com/>`_. +<http://www.pylonsproject.org/>`_. Zope diff --git a/Doc/includes/email-unpack.py b/Doc/includes/email-unpack.py index 8f99ded..a8f712d 100644 --- a/Doc/includes/email-unpack.py +++ b/Doc/includes/email-unpack.py @@ -35,7 +35,7 @@ Usage: %prog [options] msgfile try: os.mkdir(opts.directory) - except OSError, e: + except OSError as e: # Ignore directory exists error if e.errno != errno.EEXIST: raise diff --git a/Doc/includes/sqlite3/complete_statement.py b/Doc/includes/sqlite3/complete_statement.py index 22525e3..76ea7f6 100644 --- a/Doc/includes/sqlite3/complete_statement.py +++ b/Doc/includes/sqlite3/complete_statement.py @@ -23,7 +23,7 @@ while True: if buffer.lstrip().upper().startswith("SELECT"): print cur.fetchall() - except sqlite3.Error, e: + except sqlite3.Error as e: print "An error occurred:", e.args[0] buffer = "" diff --git a/Doc/includes/sqlite3/execute_1.py b/Doc/includes/sqlite3/execute_1.py index fb3784f..763167c 100644 --- a/Doc/includes/sqlite3/execute_1.py +++ b/Doc/includes/sqlite3/execute_1.py @@ -1,11 +1,16 @@ import sqlite3 -con = sqlite3.connect("mydb") - +con = sqlite3.connect(":memory:") cur = con.cursor() +cur.execute("create table people (name_last, age)") who = "Yeltsin" age = 72 -cur.execute("select name_last, age from people where name_last=? and age=?", (who, age)) +# This is the qmark style: +cur.execute("insert into people values (?, ?)", (who, age)) + +# And this is the named style: +cur.execute("select * from people where name_last=:who and age=:age", {"who": who, "age": age}) + print cur.fetchone() diff --git a/Doc/includes/sqlite3/execute_2.py b/Doc/includes/sqlite3/execute_2.py deleted file mode 100644 index df6c894..0000000 --- a/Doc/includes/sqlite3/execute_2.py +++ /dev/null @@ -1,12 +0,0 @@ -import sqlite3 - -con = sqlite3.connect("mydb") - -cur = con.cursor() - -who = "Yeltsin" -age = 72 - -cur.execute("select name_last, age from people where name_last=:who and age=:age", - {"who": who, "age": age}) -print cur.fetchone() diff --git a/Doc/includes/sqlite3/executemany_2.py b/Doc/includes/sqlite3/executemany_2.py index 05857c0..0b12688 100644 --- a/Doc/includes/sqlite3/executemany_2.py +++ b/Doc/includes/sqlite3/executemany_2.py @@ -1,8 +1,8 @@ import sqlite3 +import string def char_generator(): - import string - for c in string.letters[:26]: + for c in string.lowercase: yield (c,) con = sqlite3.connect(":memory:") diff --git a/Doc/includes/sqlite3/rowclass.py b/Doc/includes/sqlite3/rowclass.py index 3fa0b87..92b5ad6 100644 --- a/Doc/includes/sqlite3/rowclass.py +++ b/Doc/includes/sqlite3/rowclass.py @@ -1,12 +1,12 @@ import sqlite3 -con = sqlite3.connect("mydb") +con = sqlite3.connect(":memory:") con.row_factory = sqlite3.Row cur = con.cursor() -cur.execute("select name_last, age from people") +cur.execute("select 'John' as name, 42 as age") for row in cur: - assert row[0] == row["name_last"] - assert row["name_last"] == row["nAmE_lAsT"] + assert row[0] == row["name"] + assert row["name"] == row["nAmE"] assert row[1] == row["age"] assert row[1] == row["AgE"] diff --git a/Doc/includes/sqlite3/text_factory.py b/Doc/includes/sqlite3/text_factory.py index 1959498..577378f 100644 --- a/Doc/includes/sqlite3/text_factory.py +++ b/Doc/includes/sqlite3/text_factory.py @@ -3,9 +3,6 @@ import sqlite3 con = sqlite3.connect(":memory:") cur = con.cursor() -# Create the table -con.execute("create table person(lastname, firstname)") - AUSTRIA = u"\xd6sterreich" # by default, rows are returned as Unicode @@ -17,7 +14,7 @@ assert row[0] == AUSTRIA con.text_factory = str cur.execute("select ?", (AUSTRIA,)) row = cur.fetchone() -assert type(row[0]) == str +assert type(row[0]) is str # the bytestrings will be encoded in UTF-8, unless you stored garbage in the # database ... assert row[0] == AUSTRIA.encode("utf-8") @@ -29,15 +26,15 @@ con.text_factory = lambda x: unicode(x, "utf-8", "ignore") cur.execute("select ?", ("this is latin1 and would normally create errors" + u"\xe4\xf6\xfc".encode("latin1"),)) row = cur.fetchone() -assert type(row[0]) == unicode +assert type(row[0]) is unicode # sqlite3 offers a built-in optimized text_factory that will return bytestring # objects, if the data is in ASCII only, and otherwise return unicode objects con.text_factory = sqlite3.OptimizedUnicode cur.execute("select ?", (AUSTRIA,)) row = cur.fetchone() -assert type(row[0]) == unicode +assert type(row[0]) is unicode cur.execute("select ?", ("Germany",)) row = cur.fetchone() -assert type(row[0]) == str +assert type(row[0]) is str diff --git a/Doc/install/index.rst b/Doc/install/index.rst index abce88d..ad939ab 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -7,8 +7,6 @@ ***************************** :Author: Greg Ward -:Release: |version| -:Date: |today| .. TODO: Fill in XXX comments @@ -22,12 +20,20 @@ Finally, it might be useful to include all the material from my "Care and Feeding of a Python Installation" talk in here somewhere. Yow! -.. topic:: Abstract +This document describes the Python Distribution Utilities ("Distutils") from the +end-user's point-of-view, describing how to extend the capabilities of a +standard Python installation by building and installing third-party Python +modules and extensions. - This document describes the Python Distribution Utilities ("Distutils") from the - end-user's point-of-view, describing how to extend the capabilities of a - standard Python installation by building and installing third-party Python - modules and extensions. + +.. note:: + + This guide only covers the basic tools for installing extensions that are + provided as part of this version of Python. Third party tools offer easier + to use and more secure alternatives. Refer to the + `quick recommendations section + <https://python-packaging-user-guide.readthedocs.org/en/latest/current.html>`__ + in the Python Packaging User Guide for more information. .. _inst-intro: @@ -52,7 +58,9 @@ new goodies to their toolbox. You don't need to know Python to read this document; there will be some brief forays into using Python's interactive mode to explore your installation, but that's it. If you're looking for information on how to distribute your own Python modules so that others may use them, see -the :ref:`distutils-index` manual. +the :ref:`distutils-index` manual. :ref:`debug-setup-script` may also be of +interest. + .. _inst-trivial-install: @@ -191,7 +199,7 @@ under the distribution root; if you're excessively concerned with speed, or want to keep the source tree pristine, you can change the build directory with the :option:`--build-base` option. For example:: - python setup.py build --build-base=/tmp/pybuild/foo-1.0 + python setup.py build --build-base=/path/to/pybuild/foo-1.0 (Or you could do this permanently with a directive in your system or personal Distutils configuration file; see section :ref:`inst-config-files`.) Normally, this @@ -237,6 +245,8 @@ by how you built/installed Python itself. On Unix (and Mac OS X, which is also Unix-based), it also depends on whether the module distribution being installed is pure Python or contains extensions ("non-pure"): +.. tabularcolumns:: |l|l|l|l| + +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ | Platform | Standard installation location | Default value | Notes | +=================+=====================================================+==================================================+=======+ @@ -1042,7 +1052,7 @@ These compilers require some special libraries. This task is more complex than for Borland's C++, because there is no program to convert the library. First you have to create a list of symbols which the Python DLL exports. (You can find a good program for this task at -http://www.emmestech.com/software/pexports-0.43/download_pexports.html). +http://sourceforge.net/projects/mingw/files/MinGW/Extension/pexports/). .. I don't understand what the next line means. --amk .. (inclusive the references on data structures.) diff --git a/Doc/library/2to3.rst b/Doc/library/2to3.rst index d08c853..a927510 100644 --- a/Doc/library/2to3.rst +++ b/Doc/library/2to3.rst @@ -23,7 +23,7 @@ Using 2to3 also located in the :file:`Tools/scripts` directory of the Python root. 2to3's basic arguments are a list of files or directories to transform. The -directories are to recursively traversed for Python sources. +directories are recursively traversed for Python sources. Here is a sample Python 2.x source file, :file:`example.py`:: @@ -264,8 +264,7 @@ and off individually. They are described here in more detail. .. 2to3fixer:: long - Strips the ``L`` suffix on long literals and renames :class:`long` to - :class:`int`. + Renames :class:`long` to :class:`int`. .. 2to3fixer:: map @@ -291,11 +290,11 @@ and off individually. They are described here in more detail. Converts the use of iterator's :meth:`~iterator.next` methods to the :func:`next` function. It also renames :meth:`next` methods to - :meth:`~object.__next__`. + :meth:`~iterator.__next__`. .. 2to3fixer:: nonzero - Renames :meth:`~object.__nonzero__` to :meth:`~object.__bool__`. + Renames :meth:`__nonzero__` to :meth:`~object.__bool__`. .. 2to3fixer:: numliterals @@ -314,7 +313,7 @@ and off individually. They are described here in more detail. Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise E(V).with_traceback(T)``. If ``E`` is a tuple, the translation will be - incorrect because substituting tuples for exceptions has been removed in 3.0. + incorrect because substituting tuples for exceptions has been removed in Python 3. .. 2to3fixer:: raw_input @@ -337,7 +336,7 @@ and off individually. They are described here in more detail. Replaces use of the :class:`set` constructor with set literals. This fixer is optional. -.. 2to3fixer:: standard_error +.. 2to3fixer:: standarderror Renames :exc:`StandardError` to :exc:`Exception`. diff --git a/Doc/library/__future__.rst b/Doc/library/__future__.rst index 87e1205..329e411 100644 --- a/Doc/library/__future__.rst +++ b/Doc/library/__future__.rst @@ -75,7 +75,7 @@ language using this mechanism: | division | 2.2.0a2 | 3.0 | :pep:`238`: | | | | | *Changing the Division Operator* | +------------------+-------------+--------------+---------------------------------------------+ -| absolute_import | 2.5.0a1 | 2.7 | :pep:`328`: | +| absolute_import | 2.5.0a1 | 3.0 | :pep:`328`: | | | | | *Imports: Multi-Line and Absolute/Relative* | +------------------+-------------+--------------+---------------------------------------------+ | with_statement | 2.5.0a1 | 2.6 | :pep:`343`: | diff --git a/Doc/library/_winreg.rst b/Doc/library/_winreg.rst index 825ce1f..f82d1c5 100644 --- a/Doc/library/_winreg.rst +++ b/Doc/library/_winreg.rst @@ -7,9 +7,9 @@ .. sectionauthor:: Mark Hammond <MarkH@ActiveState.com> .. note:: - The :mod:`_winreg` module has been renamed to :mod:`winreg` in Python 3.0. + The :mod:`_winreg` module has been renamed to :mod:`winreg` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. .. versionadded:: 2.0 diff --git a/Doc/library/abc.rst b/Doc/library/abc.rst index ef8b08e..3a00a9c 100644 --- a/Doc/library/abc.rst +++ b/Doc/library/abc.rst @@ -110,19 +110,19 @@ This module provides the following class: MyIterable.register(Foo) The ABC ``MyIterable`` defines the standard iterable method, - :meth:`__iter__`, as an abstract method. The implementation given here can - still be called from subclasses. The :meth:`get_iterator` method is also - part of the ``MyIterable`` abstract base class, but it does not have to be - overridden in non-abstract derived classes. + :meth:`~iterator.__iter__`, as an abstract method. The implementation given + here can still be called from subclasses. The :meth:`get_iterator` method + is also part of the ``MyIterable`` abstract base class, but it does not have + to be overridden in non-abstract derived classes. The :meth:`__subclasshook__` class method defined here says that any class - that has an :meth:`__iter__` method in its :attr:`__dict__` (or in that of - one of its base classes, accessed via the :attr:`__mro__` list) is - considered a ``MyIterable`` too. + that has an :meth:`~iterator.__iter__` method in its + :attr:`~object.__dict__` (or in that of one of its base classes, accessed + via the :attr:`~class.__mro__` list) is considered a ``MyIterable`` too. Finally, the last line makes ``Foo`` a virtual subclass of ``MyIterable``, - even though it does not define an :meth:`__iter__` method (it uses the - old-style iterable protocol, defined in terms of :meth:`__len__` and + even though it does not define an :meth:`~iterator.__iter__` method (it uses + the old-style iterable protocol, defined in terms of :meth:`__len__` and :meth:`__getitem__`). Note that this will not make ``get_iterator`` available as a method of ``Foo``, so it is provided separately. diff --git a/Doc/library/aifc.rst b/Doc/library/aifc.rst index 1265423..de4144c 100644 --- a/Doc/library/aifc.rst +++ b/Doc/library/aifc.rst @@ -30,8 +30,8 @@ sampling rate or frame rate is the number of times per second the sound is sampled. The number of channels indicate if the audio is mono, stereo, or quadro. Each frame consists of one sample per channel. The sample size is the size in bytes of each sample. Thus a frame consists of -*nchannels*\**samplesize* bytes, and a second's worth of audio consists of -*nchannels*\**samplesize*\**framerate* bytes. +*nchannels*\*\ *samplesize* bytes, and a second's worth of audio consists of +*nchannels*\*\ *samplesize*\*\ *framerate* bytes. For example, CD quality audio has a sample size of two bytes (16 bits), uses two channels (stereo) and has a frame rate of 44,100 frames/second. This gives a diff --git a/Doc/library/al.rst b/Doc/library/al.rst index ad2eaea..f796c5c 100644 --- a/Doc/library/al.rst +++ b/Doc/library/al.rst @@ -8,7 +8,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`al` module has been deprecated for removal in Python 3.0. + The :mod:`al` module has been removed in Python 3. This module provides access to the audio facilities of the SGI Indy and Indigo @@ -201,7 +201,7 @@ Port objects, as returned by :func:`openport`, have the following methods: :deprecated: .. deprecated:: 2.6 - The :mod:`AL` module has been deprecated for removal in Python 3.0. + The :mod:`AL` module has been removed in Python 3. This module defines symbolic constants needed to use the built-in module diff --git a/Doc/library/anydbm.rst b/Doc/library/anydbm.rst index 7c6f99f..38e01f3 100644 --- a/Doc/library/anydbm.rst +++ b/Doc/library/anydbm.rst @@ -6,9 +6,9 @@ .. note:: - The :mod:`anydbm` module has been renamed to :mod:`dbm` in Python 3.0. The + The :mod:`anydbm` module has been renamed to :mod:`dbm` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. .. index:: module: dbhash @@ -92,6 +92,14 @@ then prints out the contents of the database:: db.close() +In addition to the dictionary-like methods, ``anydbm`` objects +provide the following method: + +.. function:: close() + + Close the ``anydbm`` database. + + .. seealso:: Module :mod:`dbhash` diff --git a/Doc/library/archiving.rst b/Doc/library/archiving.rst index 7d0df5f..472c617 100644 --- a/Doc/library/archiving.rst +++ b/Doc/library/archiving.rst @@ -7,6 +7,7 @@ Data Compression and Archiving The modules described in this chapter support data compression with the zlib, gzip, and bzip2 algorithms, and the creation of ZIP- and tar-format archives. +See also :ref:`archiving-operations` provided by the :mod:`shutil` module. .. toctree:: diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index c76fe60..4aa2918 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -12,6 +12,12 @@ -------------- +.. sidebar:: Tutorial + + This page contains the API reference information. For a more gentle + introduction to Python command-line parsing, have a look at the + :ref:`argparse tutorial <argparse-tutorial>`. + The :mod:`argparse` module makes it easy to write user-friendly command-line interfaces. The program defines what arguments it requires, and :mod:`argparse` will figure out how to parse those out of :data:`sys.argv`. The :mod:`argparse` @@ -40,7 +46,7 @@ produces either the sum or the max:: Assuming the Python code above is saved into a file called ``prog.py``, it can be run at the command line and provides useful help messages:: - $ prog.py -h + $ python prog.py -h usage: prog.py [-h] [--sum] N [N ...] Process some integers. @@ -55,15 +61,15 @@ be run at the command line and provides useful help messages:: When run with the appropriate arguments, it prints either the sum or the max of the command-line integers:: - $ prog.py 1 2 3 4 + $ python prog.py 1 2 3 4 4 - $ prog.py 1 2 3 4 --sum + $ python prog.py 1 2 3 4 --sum 10 If invalid arguments are passed in, it will issue an error:: - $ prog.py a b c + $ python prog.py a b c usage: prog.py [-h] [--sum] N [N ...] prog.py: error: argument N: invalid int value: 'a' @@ -124,44 +130,143 @@ command-line arguments from :data:`sys.argv`. ArgumentParser objects ---------------------- -.. class:: ArgumentParser([description], [epilog], [prog], [usage], [add_help], \ - [argument_default], [parents], [prefix_chars], \ - [conflict_handler], [formatter_class]) +.. class:: ArgumentParser(prog=None, usage=None, description=None, \ + epilog=None, parents=[], \ + formatter_class=argparse.HelpFormatter, \ + prefix_chars='-', fromfile_prefix_chars=None, \ + argument_default=None, conflict_handler='error', \ + add_help=True) - Create a new :class:`ArgumentParser` object. Each parameter has its own more - detailed description below, but in short they are: + Create a new :class:`ArgumentParser` object. All parameters should be passed + as keyword arguments. Each parameter has its own more detailed description + below, but in short they are: - * description_ - Text to display before the argument help. + * prog_ - The name of the program (default: ``sys.argv[0]``) - * epilog_ - Text to display after the argument help. + * usage_ - The string describing the program usage (default: generated from + arguments added to parser) - * add_help_ - Add a -h/--help option to the parser. (default: ``True``) + * description_ - Text to display before the argument help (default: none) - * argument_default_ - Set the global default value for arguments. - (default: ``None``) + * epilog_ - Text to display after the argument help (default: none) * parents_ - A list of :class:`ArgumentParser` objects whose arguments should - also be included. + also be included + + * formatter_class_ - A class for customizing the help output - * prefix_chars_ - The set of characters that prefix optional arguments. + * prefix_chars_ - The set of characters that prefix optional arguments (default: '-') * fromfile_prefix_chars_ - The set of characters that prefix files from - which additional arguments should be read. (default: ``None``) + which additional arguments should be read (default: ``None``) - * formatter_class_ - A class for customizing the help output. - - * conflict_handler_ - Usually unnecessary, defines strategy for resolving - conflicting optionals. + * argument_default_ - The global default value for arguments + (default: ``None``) - * prog_ - The name of the program (default: - ``sys.argv[0]``) + * conflict_handler_ - The strategy for resolving conflicting optionals + (usually unnecessary) - * usage_ - The string describing the program usage (default: generated) + * add_help_ - Add a -h/--help option to the parser (default: ``True``) The following sections describe how each of these are used. +prog +^^^^ + +By default, :class:`ArgumentParser` objects uses ``sys.argv[0]`` to determine +how to display the name of the program in help messages. This default is almost +always desirable because it will make the help messages match how the program was +invoked on the command line. For example, consider a file named +``myprogram.py`` with the following code:: + + import argparse + parser = argparse.ArgumentParser() + parser.add_argument('--foo', help='foo help') + args = parser.parse_args() + +The help for this program will display ``myprogram.py`` as the program name +(regardless of where the program was invoked from):: + + $ python myprogram.py --help + usage: myprogram.py [-h] [--foo FOO] + + optional arguments: + -h, --help show this help message and exit + --foo FOO foo help + $ cd .. + $ python subdir\myprogram.py --help + usage: myprogram.py [-h] [--foo FOO] + + optional arguments: + -h, --help show this help message and exit + --foo FOO foo help + +To change this default behavior, another value can be supplied using the +``prog=`` argument to :class:`ArgumentParser`:: + + >>> parser = argparse.ArgumentParser(prog='myprogram') + >>> parser.print_help() + usage: myprogram [-h] + + optional arguments: + -h, --help show this help message and exit + +Note that the program name, whether determined from ``sys.argv[0]`` or from the +``prog=`` argument, is available to help messages using the ``%(prog)s`` format +specifier. + +:: + + >>> parser = argparse.ArgumentParser(prog='myprogram') + >>> parser.add_argument('--foo', help='foo of the %(prog)s program') + >>> parser.print_help() + usage: myprogram [-h] [--foo FOO] + + optional arguments: + -h, --help show this help message and exit + --foo FOO foo of the myprogram program + + +usage +^^^^^ + +By default, :class:`ArgumentParser` calculates the usage message from the +arguments it contains:: + + >>> parser = argparse.ArgumentParser(prog='PROG') + >>> parser.add_argument('--foo', nargs='?', help='foo help') + >>> parser.add_argument('bar', nargs='+', help='bar help') + >>> parser.print_help() + usage: PROG [-h] [--foo [FOO]] bar [bar ...] + + positional arguments: + bar bar help + + optional arguments: + -h, --help show this help message and exit + --foo [FOO] foo help + +The default message can be overridden with the ``usage=`` keyword argument:: + + >>> parser = argparse.ArgumentParser(prog='PROG', usage='%(prog)s [options]') + >>> parser.add_argument('--foo', nargs='?', help='foo help') + >>> parser.add_argument('bar', nargs='+', help='bar help') + >>> parser.print_help() + usage: PROG [options] + + positional arguments: + bar bar help + + optional arguments: + -h, --help show this help message and exit + --foo [FOO] foo help + +The ``%(prog)s`` format specifier is available to fill in the program name in +your usage messages. + + description ^^^^^^^^^^^ @@ -209,122 +314,6 @@ line-wrapped, but this behavior can be adjusted with the formatter_class_ argument to :class:`ArgumentParser`. -add_help -^^^^^^^^ - -By default, ArgumentParser objects add an option which simply displays -the parser's help message. For example, consider a file named -``myprogram.py`` containing the following code:: - - import argparse - parser = argparse.ArgumentParser() - parser.add_argument('--foo', help='foo help') - args = parser.parse_args() - -If ``-h`` or ``--help`` is supplied at the command line, the ArgumentParser -help will be printed:: - - $ python myprogram.py --help - usage: myprogram.py [-h] [--foo FOO] - - optional arguments: - -h, --help show this help message and exit - --foo FOO foo help - -Occasionally, it may be useful to disable the addition of this help option. -This can be achieved by passing ``False`` as the ``add_help=`` argument to -:class:`ArgumentParser`:: - - >>> parser = argparse.ArgumentParser(prog='PROG', add_help=False) - >>> parser.add_argument('--foo', help='foo help') - >>> parser.print_help() - usage: PROG [--foo FOO] - - optional arguments: - --foo FOO foo help - -The help option is typically ``-h/--help``. The exception to this is -if the ``prefix_chars=`` is specified and does not include ``-``, in -which case ``-h`` and ``--help`` are not valid options. In -this case, the first character in ``prefix_chars`` is used to prefix -the help options:: - - >>> parser = argparse.ArgumentParser(prog='PROG', prefix_chars='+/') - >>> parser.print_help() - usage: PROG [+h] - - optional arguments: - +h, ++help show this help message and exit - - -prefix_chars -^^^^^^^^^^^^ - -Most command-line options will use ``-`` as the prefix, e.g. ``-f/--foo``. -Parsers that need to support different or additional prefix -characters, e.g. for options -like ``+f`` or ``/foo``, may specify them using the ``prefix_chars=`` argument -to the ArgumentParser constructor:: - - >>> parser = argparse.ArgumentParser(prog='PROG', prefix_chars='-+') - >>> parser.add_argument('+f') - >>> parser.add_argument('++bar') - >>> parser.parse_args('+f X ++bar Y'.split()) - Namespace(bar='Y', f='X') - -The ``prefix_chars=`` argument defaults to ``'-'``. Supplying a set of -characters that does not include ``-`` will cause ``-f/--foo`` options to be -disallowed. - - -fromfile_prefix_chars -^^^^^^^^^^^^^^^^^^^^^ - -Sometimes, for example when dealing with a particularly long argument lists, it -may make sense to keep the list of arguments in a file rather than typing it out -at the command line. If the ``fromfile_prefix_chars=`` argument is given to the -:class:`ArgumentParser` constructor, then arguments that start with any of the -specified characters will be treated as files, and will be replaced by the -arguments they contain. For example:: - - >>> with open('args.txt', 'w') as fp: - ... fp.write('-f\nbar') - >>> parser = argparse.ArgumentParser(fromfile_prefix_chars='@') - >>> parser.add_argument('-f') - >>> parser.parse_args(['-f', 'foo', '@args.txt']) - Namespace(f='bar') - -Arguments read from a file must by default be one per line (but see also -:meth:`~ArgumentParser.convert_arg_line_to_args`) and are treated as if they -were in the same place as the original file referencing argument on the command -line. So in the example above, the expression ``['-f', 'foo', '@args.txt']`` -is considered equivalent to the expression ``['-f', 'foo', '-f', 'bar']``. - -The ``fromfile_prefix_chars=`` argument defaults to ``None``, meaning that -arguments will never be treated as file references. - - -argument_default -^^^^^^^^^^^^^^^^ - -Generally, argument defaults are specified either by passing a default to -:meth:`~ArgumentParser.add_argument` or by calling the -:meth:`~ArgumentParser.set_defaults` methods with a specific set of name-value -pairs. Sometimes however, it may be useful to specify a single parser-wide -default for arguments. This can be accomplished by passing the -``argument_default=`` keyword argument to :class:`ArgumentParser`. For example, -to globally suppress attribute creation on :meth:`~ArgumentParser.parse_args` -calls, we supply ``argument_default=SUPPRESS``:: - - >>> parser = argparse.ArgumentParser(argument_default=argparse.SUPPRESS) - >>> parser.add_argument('--foo') - >>> parser.add_argument('bar', nargs='?') - >>> parser.parse_args(['--foo', '1', 'BAR']) - Namespace(bar='BAR', foo='1') - >>> parser.parse_args([]) - Namespace() - - parents ^^^^^^^ @@ -443,6 +432,74 @@ will add information about the default value of each of the arguments:: --foo FOO FOO! (default: 42) +prefix_chars +^^^^^^^^^^^^ + +Most command-line options will use ``-`` as the prefix, e.g. ``-f/--foo``. +Parsers that need to support different or additional prefix +characters, e.g. for options +like ``+f`` or ``/foo``, may specify them using the ``prefix_chars=`` argument +to the ArgumentParser constructor:: + + >>> parser = argparse.ArgumentParser(prog='PROG', prefix_chars='-+') + >>> parser.add_argument('+f') + >>> parser.add_argument('++bar') + >>> parser.parse_args('+f X ++bar Y'.split()) + Namespace(bar='Y', f='X') + +The ``prefix_chars=`` argument defaults to ``'-'``. Supplying a set of +characters that does not include ``-`` will cause ``-f/--foo`` options to be +disallowed. + + +fromfile_prefix_chars +^^^^^^^^^^^^^^^^^^^^^ + +Sometimes, for example when dealing with a particularly long argument lists, it +may make sense to keep the list of arguments in a file rather than typing it out +at the command line. If the ``fromfile_prefix_chars=`` argument is given to the +:class:`ArgumentParser` constructor, then arguments that start with any of the +specified characters will be treated as files, and will be replaced by the +arguments they contain. For example:: + + >>> with open('args.txt', 'w') as fp: + ... fp.write('-f\nbar') + >>> parser = argparse.ArgumentParser(fromfile_prefix_chars='@') + >>> parser.add_argument('-f') + >>> parser.parse_args(['-f', 'foo', '@args.txt']) + Namespace(f='bar') + +Arguments read from a file must by default be one per line (but see also +:meth:`~ArgumentParser.convert_arg_line_to_args`) and are treated as if they +were in the same place as the original file referencing argument on the command +line. So in the example above, the expression ``['-f', 'foo', '@args.txt']`` +is considered equivalent to the expression ``['-f', 'foo', '-f', 'bar']``. + +The ``fromfile_prefix_chars=`` argument defaults to ``None``, meaning that +arguments will never be treated as file references. + + +argument_default +^^^^^^^^^^^^^^^^ + +Generally, argument defaults are specified either by passing a default to +:meth:`~ArgumentParser.add_argument` or by calling the +:meth:`~ArgumentParser.set_defaults` methods with a specific set of name-value +pairs. Sometimes however, it may be useful to specify a single parser-wide +default for arguments. This can be accomplished by passing the +``argument_default=`` keyword argument to :class:`ArgumentParser`. For example, +to globally suppress attribute creation on :meth:`~ArgumentParser.parse_args` +calls, we supply ``argument_default=SUPPRESS``:: + + >>> parser = argparse.ArgumentParser(argument_default=argparse.SUPPRESS) + >>> parser.add_argument('--foo') + >>> parser.add_argument('bar', nargs='?') + >>> parser.parse_args(['--foo', '1', 'BAR']) + Namespace(bar='BAR', foo='1') + >>> parser.parse_args([]) + Namespace() + + conflict_handler ^^^^^^^^^^^^^^^^ @@ -480,22 +537,20 @@ action is retained as the ``-f`` action, because only the ``--foo`` option string was overridden. -prog -^^^^ +add_help +^^^^^^^^ -By default, :class:`ArgumentParser` objects uses ``sys.argv[0]`` to determine -how to display the name of the program in help messages. This default is almost -always desirable because it will make the help messages match how the program was -invoked on the command line. For example, consider a file named -``myprogram.py`` with the following code:: +By default, ArgumentParser objects add an option which simply displays +the parser's help message. For example, consider a file named +``myprogram.py`` containing the following code:: import argparse parser = argparse.ArgumentParser() parser.add_argument('--foo', help='foo help') args = parser.parse_args() -The help for this program will display ``myprogram.py`` as the program name -(regardless of where the program was invoked from):: +If ``-h`` or ``--help`` is supplied at the command line, the ArgumentParser +help will be printed:: $ python myprogram.py --help usage: myprogram.py [-h] [--foo FOO] @@ -503,76 +558,31 @@ The help for this program will display ``myprogram.py`` as the program name optional arguments: -h, --help show this help message and exit --foo FOO foo help - $ cd .. - $ python subdir\myprogram.py --help - usage: myprogram.py [-h] [--foo FOO] - - optional arguments: - -h, --help show this help message and exit - --foo FOO foo help -To change this default behavior, another value can be supplied using the -``prog=`` argument to :class:`ArgumentParser`:: - - >>> parser = argparse.ArgumentParser(prog='myprogram') - >>> parser.print_help() - usage: myprogram [-h] - - optional arguments: - -h, --help show this help message and exit - -Note that the program name, whether determined from ``sys.argv[0]`` or from the -``prog=`` argument, is available to help messages using the ``%(prog)s`` format -specifier. - -:: - - >>> parser = argparse.ArgumentParser(prog='myprogram') - >>> parser.add_argument('--foo', help='foo of the %(prog)s program') - >>> parser.print_help() - usage: myprogram [-h] [--foo FOO] - - optional arguments: - -h, --help show this help message and exit - --foo FOO foo of the myprogram program - - -usage -^^^^^ - -By default, :class:`ArgumentParser` calculates the usage message from the -arguments it contains:: +Occasionally, it may be useful to disable the addition of this help option. +This can be achieved by passing ``False`` as the ``add_help=`` argument to +:class:`ArgumentParser`:: - >>> parser = argparse.ArgumentParser(prog='PROG') - >>> parser.add_argument('--foo', nargs='?', help='foo help') - >>> parser.add_argument('bar', nargs='+', help='bar help') + >>> parser = argparse.ArgumentParser(prog='PROG', add_help=False) + >>> parser.add_argument('--foo', help='foo help') >>> parser.print_help() - usage: PROG [-h] [--foo [FOO]] bar [bar ...] - - positional arguments: - bar bar help + usage: PROG [--foo FOO] optional arguments: - -h, --help show this help message and exit - --foo [FOO] foo help + --foo FOO foo help -The default message can be overridden with the ``usage=`` keyword argument:: +The help option is typically ``-h/--help``. The exception to this is +if the ``prefix_chars=`` is specified and does not include ``-``, in +which case ``-h`` and ``--help`` are not valid options. In +this case, the first character in ``prefix_chars`` is used to prefix +the help options:: - >>> parser = argparse.ArgumentParser(prog='PROG', usage='%(prog)s [options]') - >>> parser.add_argument('--foo', nargs='?', help='foo help') - >>> parser.add_argument('bar', nargs='+', help='bar help') + >>> parser = argparse.ArgumentParser(prog='PROG', prefix_chars='+/') >>> parser.print_help() - usage: PROG [options] - - positional arguments: - bar bar help + usage: PROG [+h] optional arguments: - -h, --help show this help message and exit - --foo [FOO] foo help - -The ``%(prog)s`` format specifier is available to fill in the program name in -your usage messages. + +h, ++help show this help message and exit The add_argument() method @@ -743,7 +753,7 @@ the Action API. The easiest way to do this is to extend * ``values`` - The associated command-line arguments, with any type conversions applied. (Type conversions are specified with the type_ keyword argument to - :meth:`~ArgumentParser.add_argument`. + :meth:`~ArgumentParser.add_argument`.) * ``option_string`` - The option string that was used to invoke this action. The ``option_string`` argument is optional, and will be absent if the action @@ -897,6 +907,17 @@ was not present at the command line:: >>> parser.parse_args(''.split()) Namespace(foo=42) +If the ``default`` value is a string, the parser parses the value as if it +were a command-line argument. In particular, the parser applies any type_ +conversion argument, if provided, before setting the attribute on the +:class:`Namespace` return value. Otherwise, the parser uses the value as is:: + + >>> parser = argparse.ArgumentParser() + >>> parser.add_argument('--length', default='10', type=int) + >>> parser.add_argument('--width', default=10.5, type=int) + >>> parser.parse_args() + Namespace(length=10, width=10.5) + For positional arguments with nargs_ equal to ``?`` or ``*``, the ``default`` value is used when no command-line argument was present:: @@ -935,6 +956,9 @@ types and functions can be used directly as the value of the ``type`` argument:: >>> parser.parse_args('2 temp.txt'.split()) Namespace(bar=<open file 'temp.txt', mode 'r' at 0x...>, foo=2) +See the section on the default_ keyword argument for information on when the +``type`` argument is applied to default arguments. + To ease the use of various types of files, the argparse module provides the factory FileType which takes the ``mode=`` and ``bufsize=`` arguments of the ``file`` object. For example, ``FileType('w')`` can be used to create a @@ -982,32 +1006,33 @@ choices ^^^^^^^ Some command-line arguments should be selected from a restricted set of values. -These can be handled by passing a container object as the ``choices`` keyword +These can be handled by passing a container object as the *choices* keyword argument to :meth:`~ArgumentParser.add_argument`. When the command line is -parsed, argument values will be checked, and an error message will be displayed if -the argument was not one of the acceptable values:: - - >>> parser = argparse.ArgumentParser(prog='PROG') - >>> parser.add_argument('foo', choices='abc') - >>> parser.parse_args('c'.split()) - Namespace(foo='c') - >>> parser.parse_args('X'.split()) - usage: PROG [-h] {a,b,c} - PROG: error: argument foo: invalid choice: 'X' (choose from 'a', 'b', 'c') - -Note that inclusion in the ``choices`` container is checked after any type_ -conversions have been performed, so the type of the objects in the ``choices`` +parsed, argument values will be checked, and an error message will be displayed +if the argument was not one of the acceptable values:: + + >>> parser = argparse.ArgumentParser(prog='game.py') + >>> parser.add_argument('move', choices=['rock', 'paper', 'scissors']) + >>> parser.parse_args(['rock']) + Namespace(move='rock') + >>> parser.parse_args(['fire']) + usage: game.py [-h] {rock,paper,scissors} + game.py: error: argument move: invalid choice: 'fire' (choose from 'rock', + 'paper', 'scissors') + +Note that inclusion in the *choices* container is checked after any type_ +conversions have been performed, so the type of the objects in the *choices* container should match the type_ specified:: - >>> parser = argparse.ArgumentParser(prog='PROG') - >>> parser.add_argument('foo', type=complex, choices=[1, 1j]) - >>> parser.parse_args('1j'.split()) - Namespace(foo=1j) - >>> parser.parse_args('-- -4'.split()) - usage: PROG [-h] {1,1j} - PROG: error: argument foo: invalid choice: (-4+0j) (choose from 1, 1j) - -Any object that supports the ``in`` operator can be passed as the ``choices`` + >>> parser = argparse.ArgumentParser(prog='doors.py') + >>> parser.add_argument('door', type=int, choices=range(1, 4)) + >>> print(parser.parse_args(['3'])) + Namespace(door=3) + >>> parser.parse_args(['4']) + usage: doors.py [-h] {1,2,3} + doors.py: error: argument door: invalid choice: 4 (choose from 1, 2, 3) + +Any object that supports the ``in`` operator can be passed as the *choices* value, so :class:`dict` objects, :class:`set` objects, custom containers, etc. are all supported. @@ -1093,7 +1118,7 @@ setting the ``help`` value to ``argparse.SUPPRESS``:: metavar ^^^^^^^ -When :class:`ArgumentParser` generates help messages, it need some way to refer +When :class:`ArgumentParser` generates help messages, it needs some way to refer to each expected argument. By default, ArgumentParser objects use the dest_ value as the "name" of each object. By default, for positional argument actions, the dest_ value is used directly, and for optional argument actions, @@ -1326,12 +1351,14 @@ argument:: >>> parser.parse_args(['--', '-f']) Namespace(foo='-f', one=None) +.. _prefix-matching: -Argument abbreviations -^^^^^^^^^^^^^^^^^^^^^^ +Argument abbreviations (prefix matching) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :meth:`~ArgumentParser.parse_args` method allows long options to be -abbreviated if the abbreviation is unambiguous:: +abbreviated to a prefix, if the abbreviation is unambiguous (the prefix matches +a unique option):: >>> parser = argparse.ArgumentParser(prog='PROG') >>> parser.add_argument('-bacon') @@ -1407,7 +1434,10 @@ Other utilities Sub-commands ^^^^^^^^^^^^ -.. method:: ArgumentParser.add_subparsers() +.. method:: ArgumentParser.add_subparsers([title], [description], [prog], \ + [parser_class], [action], \ + [option_string], [dest], [help], \ + [metavar]) Many programs split up their functionality into a number of sub-commands, for example, the ``svn`` program can invoke sub-commands like ``svn @@ -1416,11 +1446,35 @@ Sub-commands different functions which require different kinds of command-line arguments. :class:`ArgumentParser` supports the creation of such sub-commands with the :meth:`add_subparsers` method. The :meth:`add_subparsers` method is normally - called with no arguments and returns an special action object. This object + called with no arguments and returns a special action object. This object has a single method, :meth:`~ArgumentParser.add_parser`, which takes a command name and any :class:`ArgumentParser` constructor arguments, and returns an :class:`ArgumentParser` object that can be modified as usual. + Description of parameters: + + * title - title for the sub-parser group in help output; by default + "subcommands" if description is provided, otherwise uses title for + positional arguments + + * description - description for the sub-parser group in help output, by + default None + + * prog - usage information that will be displayed with sub-command help, + by default the name of the program and any positional arguments before the + subparser argument + + * parser_class - class which will be used to create sub-parser instances, by + default the class of the current parser (e.g. ArgumentParser) + + * dest - name of the attribute under which sub-command name will be + stored; by default None and no value is stored + + * help - help for sub-parser group in help output, by default None + + * metavar - string presenting available sub-commands in help; by default it + is None and presents sub-commands in form {cmd1, cmd2, ..} + Some example usage:: >>> # create the top-level parser @@ -1462,8 +1516,8 @@ Sub-commands positional arguments: {a,b} sub-command help - a a help - b b help + a a help + b b help optional arguments: -h, --help show this help message and exit @@ -1634,14 +1688,14 @@ Argument groups --bar BAR bar help - Note that any arguments not your user defined groups will end up back in the - usual "positional arguments" and "optional arguments" sections. + Note that any arguments not in your user-defined groups will end up back + in the usual "positional arguments" and "optional arguments" sections. Mutual exclusion ^^^^^^^^^^^^^^^^ -.. method:: add_mutually_exclusive_group(required=False) +.. method:: ArgumentParser.add_mutually_exclusive_group(required=False) Create a mutually exclusive group. :mod:`argparse` will make sure that only one of the arguments in the mutually exclusive group was present on the @@ -1770,6 +1824,12 @@ the populated namespace and the list of remaining argument strings. >>> parser.parse_known_args(['--foo', '--badger', 'BAR', 'spam']) (Namespace(bar='BAR', foo=True), ['--badger', 'spam']) +.. warning:: + :ref:`Prefix matching <prefix-matching>` rules apply to + :meth:`parse_known_args`. The parser may consume an option even if it's just + a prefix of one of its known options, instead of leaving it in the remaining + arguments list. + Customizing file parsing ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1826,9 +1886,10 @@ A partial upgrade path from :mod:`optparse` to :mod:`argparse`: * Replace all :meth:`optparse.OptionParser.add_option` calls with :meth:`ArgumentParser.add_argument` calls. -* Replace ``options, args = parser.parse_args()`` with ``args = +* Replace ``(options, args) = parser.parse_args()`` with ``args = parser.parse_args()`` and add additional :meth:`ArgumentParser.add_argument` - calls for the positional arguments. + calls for the positional arguments. Keep in mind that what was previously + called ``options``, now in :mod:`argparse` context is called ``args``. * Replace callback actions and the ``callback_*`` keyword arguments with ``type`` or ``action`` arguments. diff --git a/Doc/library/array.rst b/Doc/library/array.rst index d34cf38..1766d47 100644 --- a/Doc/library/array.rst +++ b/Doc/library/array.rst @@ -268,9 +268,7 @@ Examples:: Packing and unpacking of External Data Representation (XDR) data as used in some remote procedure call systems. - `The Numerical Python Manual <http://numpy.sourceforge.net/numdoc/HTML/numdoc.htm>`_ + `The Numerical Python Documentation <http://docs.scipy.org/doc/>`_ The Numeric Python extension (NumPy) defines another array type; see - http://numpy.sourceforge.net/ for further information about Numerical Python. - (A PDF version of the NumPy manual is available at - http://numpy.sourceforge.net/numdoc/numdoc.pdf). + http://www.numpy.org/ for further information about Numerical Python. diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 5130d00..8adc88f 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -131,10 +131,10 @@ and classes for traversing abstract syntax trees: .. function:: literal_eval(node_or_string) - Safely evaluate an expression node or a string containing a Python - expression. The string or node provided may only consist of the following - Python literal structures: strings, numbers, tuples, lists, dicts, booleans, - and ``None``. + Safely evaluate an expression node or a Unicode or *Latin-1* encoded string + containing a Python expression. The string or node provided may only consist + of the following Python literal structures: strings, numbers, tuples, lists, + dicts, booleans, and ``None``. This can be used for safely evaluating strings containing Python expressions from untrusted sources without the need to parse the values oneself. @@ -257,6 +257,6 @@ and classes for traversing abstract syntax trees: Return a formatted dump of the tree in *node*. This is mainly useful for debugging purposes. The returned string will show the names and the values for fields. This makes the code impossible to evaluate, so if evaluation is - wanted *annotate_fields* must be set to False. Attributes such as line + wanted *annotate_fields* must be set to ``False``. Attributes such as line numbers and column offsets are not dumped by default. If this is wanted, *include_attributes* can be set to ``True``. diff --git a/Doc/library/asyncore.rst b/Doc/library/asyncore.rst index c108450..70ca8e7 100644 --- a/Doc/library/asyncore.rst +++ b/Doc/library/asyncore.rst @@ -53,10 +53,10 @@ any that have been added to the map during asynchronous service) is closed. channels have been closed. All arguments are optional. The *count* parameter defaults to None, resulting in the loop terminating only when all channels have been closed. The *timeout* argument sets the timeout - parameter for the appropriate :func:`select` or :func:`poll` call, measured - in seconds; the default is 30 seconds. The *use_poll* parameter, if true, - indicates that :func:`poll` should be used in preference to :func:`select` - (the default is ``False``). + parameter for the appropriate :func:`~select.select` or :func:`~select.poll` + call, measured in seconds; the default is 30 seconds. The *use_poll* + parameter, if true, indicates that :func:`~select.poll` should be used in + preference to :func:`~select.select` (the default is ``False``). The *map* parameter is a dictionary whose items are the channels to watch. As channels are closed they are deleted from their map. If *map* is @@ -318,13 +318,10 @@ connections and dispatches the incoming connections to a handler:: def handle_accept(self): pair = self.accept() - if pair is None: - pass - else: + if pair is not None: sock, addr = pair print 'Incoming connection from %s' % repr(addr) handler = EchoHandler(sock) server = EchoServer('localhost', 8080) asyncore.loop() - diff --git a/Doc/library/atexit.rst b/Doc/library/atexit.rst index 6ac36b2..0b5e121 100644 --- a/Doc/library/atexit.rst +++ b/Doc/library/atexit.rst @@ -15,13 +15,14 @@ The :mod:`atexit` module defines a single function to register cleanup functions. Functions thus registered are automatically executed upon normal -interpreter termination. The order in which the functions are called is not -defined; if you have cleanup operations that depend on each other, you should -wrap them in a function and register that one. This keeps :mod:`atexit` simple. +interpreter termination. :mod:`atexit` runs these functions in the *reverse* +order in which they were registered; if you register ``A``, ``B``, and ``C``, +at interpreter termination time they will be run in the order ``C``, ``B``, +``A``. -Note: the functions registered via this module are not called when the program -is killed by a signal not handled by Python, when a Python fatal internal error -is detected, or when :func:`os._exit` is called. +**Note:** The functions registered via this module are not called when the +program is killed by a signal not handled by Python, when a Python fatal +internal error is detected, or when :func:`os._exit` is called. .. index:: single: exitfunc (in sys) @@ -76,7 +77,7 @@ automatically when the program terminates without relying on the application making an explicit call into this module at termination. :: try: - _count = int(open("/tmp/counter").read()) + _count = int(open("counter").read()) except IOError: _count = 0 @@ -85,7 +86,7 @@ making an explicit call into this module at termination. :: _count = _count + n def savecounter(): - open("/tmp/counter", "w").write("%d" % _count) + open("counter", "w").write("%d" % _count) import atexit atexit.register(savecounter) diff --git a/Doc/library/audioop.rst b/Doc/library/audioop.rst index 29c3914..e747ba1 100644 --- a/Doc/library/audioop.rst +++ b/Doc/library/audioop.rst @@ -38,7 +38,7 @@ The module defines the following variables and functions: Return a fragment which is the addition of the two samples passed as parameters. *width* is the sample width in bytes, either ``1``, ``2`` or ``4``. Both - fragments should have the same length. + fragments should have the same length. Samples are truncated in case of overflow. .. function:: adpcm2lin(adpcmfragment, width, state) @@ -71,7 +71,7 @@ The module defines the following variables and functions: .. function:: bias(fragment, width, bias) Return a fragment that is the original fragment with a bias added to each - sample. + sample. Samples wrap around in case of overflow. .. function:: cross(fragment, width) @@ -162,12 +162,6 @@ The module defines the following variables and functions: hardware, among others. -.. function:: minmax(fragment, width) - - Return a tuple consisting of the minimum and maximum values of all samples in - the sound fragment. - - .. function:: max(fragment, width) Return the maximum of the *absolute value* of all samples in a fragment. @@ -178,10 +172,16 @@ The module defines the following variables and functions: Return the maximum peak-peak value in the sound fragment. +.. function:: minmax(fragment, width) + + Return a tuple consisting of the minimum and maximum values of all samples in + the sound fragment. + + .. function:: mul(fragment, width, factor) Return a fragment that has all samples in the original fragment multiplied by - the floating-point value *factor*. Overflow is silently ignored. + the floating-point value *factor*. Samples are truncated in case of overflow. .. function:: ratecv(fragment, width, nchannels, inrate, outrate, state[, weightA[, weightB]]) @@ -247,7 +247,7 @@ to be stateless (i.e. to be able to tolerate packet loss) you should not only transmit the data but also the state. Note that you should send the *initial* state (the one you passed to :func:`lin2adpcm`) along to the decoder, not the final state (as returned by the coder). If you want to use -:func:`struct.struct` to store the state in binary you can code the first +:class:`struct.Struct` to store the state in binary you can code the first element (the predicted value) in 16 bits and the second (the delta index) in 8. The ADPCM coders have never been tried against other ADPCM coders, only against diff --git a/Doc/library/base64.rst b/Doc/library/base64.rst index ab85436..c9ce28b 100644 --- a/Doc/library/base64.rst +++ b/Doc/library/base64.rst @@ -93,7 +93,7 @@ The modern interface, which was introduced in Python 2.4, provides: digit 0 is always mapped to the letter O). For security purposes the default is ``None``, so that 0 and 1 are not allowed in the input. - The decoded string is returned. A :exc:`TypeError` is raised if *s* were + The decoded string is returned. A :exc:`TypeError` is raised if *s* is incorrectly padded or if there are non-alphabet characters present in the string. diff --git a/Doc/library/basehttpserver.rst b/Doc/library/basehttpserver.rst index 98b5ce4..01776af 100644 --- a/Doc/library/basehttpserver.rst +++ b/Doc/library/basehttpserver.rst @@ -6,8 +6,8 @@ .. note:: The :mod:`BaseHTTPServer` module has been merged into :mod:`http.server` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. .. index:: @@ -240,7 +240,7 @@ to a handler. Code to create and run the server looks like this:: to create custom error logging mechanisms. The *format* argument is a standard printf-style format string, where the additional arguments to :meth:`log_message` are applied as inputs to the formatting. The client - address and current date and time are prefixed to every message logged. + ip address and current date and time are prefixed to every message logged. .. method:: version_string() diff --git a/Doc/library/bastion.rst b/Doc/library/bastion.rst index 8f103e7..2e3efcd 100644 --- a/Doc/library/bastion.rst +++ b/Doc/library/bastion.rst @@ -7,7 +7,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`Bastion` module has been removed in Python 3.0. + The :mod:`Bastion` module has been removed in Python 3. .. moduleauthor:: Barry Warsaw <bwarsaw@python.org> diff --git a/Doc/library/bdb.rst b/Doc/library/bdb.rst index bd2b16f..82ef37e 100644 --- a/Doc/library/bdb.rst +++ b/Doc/library/bdb.rst @@ -20,7 +20,7 @@ The following exception is defined: The :mod:`bdb` module also defines two classes: -.. class:: Breakpoint(self, file, line[, temporary=0[, cond=None [, funcname=None]]]) +.. class:: Breakpoint(self, file, line, temporary=0, cond=None, funcname=None) This class implements temporary breakpoints, ignore counts, disabling and (re-)enabling, and conditionals. @@ -186,17 +186,17 @@ The :mod:`bdb` module also defines two classes: .. method:: user_line(frame) This method is called from :meth:`dispatch_line` when either - :meth:`stop_here` or :meth:`break_here` yields True. + :meth:`stop_here` or :meth:`break_here` yields ``True``. .. method:: user_return(frame, return_value) This method is called from :meth:`dispatch_return` when :meth:`stop_here` - yields True. + yields ``True``. .. method:: user_exception(frame, exc_info) This method is called from :meth:`dispatch_exception` when - :meth:`stop_here` yields True. + :meth:`stop_here` yields ``True``. .. method:: do_clear(arg) @@ -237,7 +237,7 @@ The :mod:`bdb` module also defines two classes: .. method:: set_quit() - Set the :attr:`quitting` attribute to True. This raises :exc:`BdbQuit` in + Set the :attr:`quitting` attribute to ``True``. This raises :exc:`BdbQuit` in the next call to one of the :meth:`dispatch_\*` methods. @@ -245,7 +245,7 @@ The :mod:`bdb` module also defines two classes: breakpoints. These methods return a string containing an error message if something went wrong, or ``None`` if all is well. - .. method:: set_break(filename, lineno[, temporary=0[, cond[, funcname]]]) + .. method:: set_break(filename, lineno, temporary=0, cond=None, funcname=None) Set a new breakpoint. If the *lineno* line doesn't exist for the *filename* passed as argument, return an error message. The *filename* diff --git a/Doc/library/binascii.rst b/Doc/library/binascii.rst index 23939de..0f8a3de 100644 --- a/Doc/library/binascii.rst +++ b/Doc/library/binascii.rst @@ -127,7 +127,7 @@ The :mod:`binascii` module defines the following functions: The return value is in the range [-2**31, 2**31-1] regardless of platform. In the past the value would be signed on some platforms and unsigned on others. Use & 0xffffffff on the - value if you want it to match 3.0 behavior. + value if you want it to match Python 3 behavior. .. versionchanged:: 3.0 The return value is unsigned and in the range [0, 2**32-1] diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index 27ab526..64a362e 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -123,9 +123,9 @@ based on a set of ordered numeric breakpoints: 90 and up is an 'A', 80 to 89 is a 'B', and so on:: >>> def grade(score, breakpoints=[60, 70, 80, 90], grades='FDCBA'): - ... i = bisect(breakpoints, score) - ... return grades[i] - ... + i = bisect(breakpoints, score) + return grades[i] + >>> [grade(score) for score in [33, 99, 77, 70, 89, 90, 100]] ['F', 'A', 'C', 'C', 'B', 'A', 'A'] diff --git a/Doc/library/bsddb.rst b/Doc/library/bsddb.rst index cdd380a..7551f10 100644 --- a/Doc/library/bsddb.rst +++ b/Doc/library/bsddb.rst @@ -7,7 +7,7 @@ .. sectionauthor:: Skip Montanaro <skip@pobox.com> .. deprecated:: 2.6 - The :mod:`bsddb` module has been deprecated for removal in Python 3.0. + The :mod:`bsddb` module has been removed in Python 3. The :mod:`bsddb` module provides an interface to the Berkeley DB library. Users @@ -52,7 +52,7 @@ arguments should be used in most instances. Open the hash format file named *filename*. Files never intended to be preserved on disk may be created by passing ``None`` as the *filename*. The optional *flag* identifies the mode used to open the file. It may be ``'r'`` - (read only), ``'w'`` (read-write) , ``'c'`` (read-write - create if necessary; + (read only), ``'w'`` (read-write), ``'c'`` (read-write - create if necessary; the default) or ``'n'`` (read-write - truncate to zero length). The other arguments are rarely used and are just passed to the low-level :c:func:`dbopen` function. Consult the Berkeley DB documentation for their use and @@ -86,7 +86,7 @@ arguments should be used in most instances. This is present *only* to allow backwards compatibility with systems which ship with the old Berkeley DB 1.85 database library. The :mod:`bsddb185` module should never be used directly in new code. The module has been removed in - Python 3.0. If you find you still need it look in PyPI. + Python 3. If you find you still need it look in PyPI. .. seealso:: @@ -170,7 +170,7 @@ dictionaries. In addition, they support the methods listed below. Example:: >>> import bsddb - >>> db = bsddb.btopen('/tmp/spam.db', 'c') + >>> db = bsddb.btopen('spam.db', 'c') >>> for i in range(10): db['%d'%i] = '%d'% (i*i) ... >>> db['3'] diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst index 20dc765..6793c44 100644 --- a/Doc/library/bz2.rst +++ b/Doc/library/bz2.rst @@ -14,9 +14,6 @@ This module provides a comprehensive interface for the bz2 compression library. It implements a complete file interface, one-shot (de)compression functions, and types for sequential (de)compression. -For other archive formats, see the :mod:`gzip`, :mod:`zipfile`, and -:mod:`tarfile` modules. - Here is a summary of the features offered by the bz2 module: * :class:`BZ2File` class implements a complete file interface, including @@ -45,6 +42,9 @@ Here is a summary of the features offered by the bz2 module: Handling of compressed files is offered by the :class:`BZ2File` class. +.. index:: + single: universal newlines; bz2.BZ2File class + .. class:: BZ2File(filename[, mode[, buffering[, compresslevel]]]) Open a bz2 file. Mode can be either ``'r'`` or ``'w'``, for reading (default) @@ -53,7 +53,7 @@ Handling of compressed files is offered by the :class:`BZ2File` class. unbuffered, and larger numbers specify the buffer size; the default is ``0``. If *compresslevel* is given, it must be a number between ``1`` and ``9``; the default is ``9``. Add a ``'U'`` to mode to open the file for input - with universal newline support. Any line ending in the input file will be + in :term:`universal newlines` mode. Any line ending in the input file will be seen as a ``'\n'`` in Python. Also, a file so opened gains the attribute :attr:`newlines`; the value for this attribute is one of ``None`` (no newline read yet), ``'\r'``, ``'\n'``, ``'\r\n'`` or a tuple containing all the diff --git a/Doc/library/calendar.rst b/Doc/library/calendar.rst index f4f4693..a04505f 100644 --- a/Doc/library/calendar.rst +++ b/Doc/library/calendar.rst @@ -294,10 +294,11 @@ For simple text calendars this module provides the following functions. .. function:: timegm(tuple) - An unrelated but handy function that takes a time tuple such as returned by the - :func:`gmtime` function in the :mod:`time` module, and returns the corresponding - Unix timestamp value, assuming an epoch of 1970, and the POSIX encoding. In - fact, :func:`time.gmtime` and :func:`timegm` are each others' inverse. + An unrelated but handy function that takes a time tuple such as returned by + the :func:`~time.gmtime` function in the :mod:`time` module, and returns the + corresponding Unix timestamp value, assuming an epoch of 1970, and the POSIX + encoding. In fact, :func:`time.gmtime` and :func:`timegm` are each others' + inverse. .. versionadded:: 2.0 diff --git a/Doc/library/carbon.rst b/Doc/library/carbon.rst index 4abb495..40b76ef 100644 --- a/Doc/library/carbon.rst +++ b/Doc/library/carbon.rst @@ -5,7 +5,7 @@ Mac OS Toolbox Modules ********************** -There are a set of modules that provide interfaces to various Mac OS toolboxes. +These are a set of modules that provide interfaces to various legacy Mac OS toolboxes. If applicable the module will define a number of Python objects for the various structures declared by the toolbox, and operations will be implemented as methods of the object. Other operations will be implemented as functions in the @@ -24,7 +24,10 @@ framework and Qt is in the QuickTime framework. The normal use pattern is :: .. note:: - The Carbon modules have been removed in Python 3.0. + Most of the OS X APIs that these modules use are deprecated or removed + in recent versions of OS X. Many are not available when Python is + executing in 64-bit mode. The Carbon modules have been removed in + Python 3. You should avoid using them in Python 2. :mod:`Carbon.AE` --- Apple Events diff --git a/Doc/library/cd.rst b/Doc/library/cd.rst index f1d9763..40b8ce6 100644 --- a/Doc/library/cd.rst +++ b/Doc/library/cd.rst @@ -9,7 +9,7 @@ .. deprecated:: 2.6 - The :mod:`cd` module has been deprecated for removal in Python 3.0. + The :mod:`cd` module has been removed in Python 3. This module provides an interface to the Silicon Graphics CD library. It is diff --git a/Doc/library/cgi.rst b/Doc/library/cgi.rst index b95f131..6f7c265 100644 --- a/Doc/library/cgi.rst +++ b/Doc/library/cgi.rst @@ -81,7 +81,7 @@ program to users of your script, you can have the reports saved to files instead, with code like this:: import cgitb - cgitb.enable(display=0, logdir="/tmp") + cgitb.enable(display=0, logdir="/path/to/logdir") It's very helpful to use this feature during script development. The reports produced by :mod:`cgitb` provide information that can save you a lot of time in @@ -97,7 +97,7 @@ consume standard input, it should be instantiated only once. The :class:`FieldStorage` instance can be indexed like a Python dictionary. It allows membership testing with the :keyword:`in` operator, and also supports -the standard dictionary method :meth:`keys` and the built-in function +the standard dictionary method :meth:`~dict.keys` and the built-in function :func:`len`. Form fields containing empty strings are ignored and do not appear in the dictionary; to keep such values, provide a true value for the optional *keep_blank_values* keyword parameter when creating the :class:`FieldStorage` @@ -119,28 +119,29 @@ string:: Here the fields, accessed through ``form[key]``, are themselves instances of :class:`FieldStorage` (or :class:`MiniFieldStorage`, depending on the form -encoding). The :attr:`value` attribute of the instance yields the string value -of the field. The :meth:`getvalue` method returns this string value directly; -it also accepts an optional second argument as a default to return if the -requested key is not present. +encoding). The :attr:`~FieldStorage.value` attribute of the instance yields +the string value of the field. The :meth:`~FieldStorage.getvalue` method +returns this string value directly; it also accepts an optional second argument +as a default to return if the requested key is not present. If the submitted form data contains more than one field with the same name, the object retrieved by ``form[key]`` is not a :class:`FieldStorage` or :class:`MiniFieldStorage` instance but a list of such instances. Similarly, in this situation, ``form.getvalue(key)`` would return a list of strings. If you expect this possibility (when your HTML form contains multiple fields with the -same name), use the :func:`getlist` function, which always returns a list of -values (so that you do not need to special-case the single item case). For -example, this code concatenates any number of username fields, separated by -commas:: +same name), use the :meth:`~FieldStorage.getlist` method, which always returns +a list of values (so that you do not need to special-case the single item +case). For example, this code concatenates any number of username fields, +separated by commas:: value = form.getlist("username") usernames = ",".join(value) If a field represents an uploaded file, accessing the value via the -:attr:`value` attribute or the :func:`getvalue` method reads the entire file in -memory as a string. This may not be what you want. You can test for an uploaded -file by testing either the :attr:`filename` attribute or the :attr:`!file` +:attr:`~FieldStorage.value` attribute or the :func:`~FieldStorage.getvalue` +method reads the entire file in memory as a string. This may not be what you +want. You can test for an uploaded file by testing either the +:attr:`~FieldStorage.filename` attribute or the :attr:`~FieldStorage.file` attribute. You can then read the data at leisure from the :attr:`!file` attribute:: @@ -155,8 +156,8 @@ attribute:: If an error is encountered when obtaining the contents of an uploaded file (for example, when the user interrupts the form submission by clicking on -a Back or Cancel button) the :attr:`done` attribute of the object for the -field will be set to the value -1. +a Back or Cancel button) the :attr:`~FieldStorage.done` attribute of the +object for the field will be set to the value -1. The file upload draft standard entertains the possibility of uploading multiple files from one field (using a recursive :mimetype:`multipart/\*` encoding). @@ -225,8 +226,8 @@ Therefore, the appropriate way to read form data values was to always use the code which checks whether the obtained value is a single value or a list of values. That's annoying and leads to less readable scripts. -A more convenient approach is to use the methods :meth:`getfirst` and -:meth:`getlist` provided by this higher level interface. +A more convenient approach is to use the methods :meth:`~FieldStorage.getfirst` +and :meth:`~FieldStorage.getlist` provided by this higher level interface. .. method:: FieldStorage.getfirst(name[, default]) @@ -284,10 +285,10 @@ These are useful if you want more control, or if you want to employ some of the algorithms implemented in this module in other circumstances. -.. function:: parse(fp[, keep_blank_values[, strict_parsing]]) +.. function:: parse(fp[, environ[, keep_blank_values[, strict_parsing]]]) Parse a query in the environment or from a file (the file defaults to - ``sys.stdin``). The *keep_blank_values* and *strict_parsing* parameters are + ``sys.stdin`` and environment defaults to ``os.environ``). The *keep_blank_values* and *strict_parsing* parameters are passed to :func:`urlparse.parse_qs` unchanged. diff --git a/Doc/library/cgihttpserver.rst b/Doc/library/cgihttpserver.rst index 390b2a6..013ee82 100644 --- a/Doc/library/cgihttpserver.rst +++ b/Doc/library/cgihttpserver.rst @@ -8,8 +8,8 @@ .. note:: The :mod:`CGIHTTPServer` module has been merged into :mod:`http.server` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. The :mod:`CGIHTTPServer` module defines a request-handler class, interface diff --git a/Doc/library/chunk.rst b/Doc/library/chunk.rst index 64ce4e2..04c7e27 100644 --- a/Doc/library/chunk.rst +++ b/Doc/library/chunk.rst @@ -55,8 +55,9 @@ instance will fail with a :exc:`EOFError` exception. Class which represents a chunk. The *file* argument is expected to be a file-like object. An instance of this class is specifically allowed. The - only method that is needed is :meth:`read`. If the methods :meth:`seek` and - :meth:`tell` are present and don't raise an exception, they are also used. + only method that is needed is :meth:`~file.read`. If the methods + :meth:`~file.seek` and :meth:`~file.tell` are present and don't + raise an exception, they are also used. If these methods are present and raise an exception, they are expected to not have altered the object. If the optional argument *align* is true, chunks are assumed to be aligned on 2-byte boundaries. If *align* is false, no diff --git a/Doc/library/code.rst b/Doc/library/code.rst index 38e26bc..698fb11 100644 --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -33,11 +33,11 @@ build applications which provide an interactive interpreter prompt. Convenience function to run a read-eval-print loop. This creates a new instance of :class:`InteractiveConsole` and sets *readfunc* to be used as the - :meth:`raw_input` method, if provided. If *local* is provided, it is passed to - the :class:`InteractiveConsole` constructor for use as the default namespace for - the interpreter loop. The :meth:`interact` method of the instance is then run - with *banner* passed as the banner to use, if provided. The console object is - discarded after use. + :meth:`InteractiveConsole.raw_input` method, if provided. If *local* is + provided, it is passed to the :class:`InteractiveConsole` constructor for + use as the default namespace for the interpreter loop. The :meth:`interact` + method of the instance is then run with *banner* passed as the banner to + use, if provided. The console object is discarded after use. .. function:: compile_command(source[, filename[, symbol]]) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst index 62bb504..05c7156 100644 --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -23,6 +23,31 @@ manages the codec and error handling lookup process. It defines the following functions: +.. function:: encode(obj, [encoding[, errors]]) + + Encodes *obj* using the codec registered for *encoding*. The default + encoding is ``'ascii'``. + + *Errors* may be given to set the desired error handling scheme. The + default error handler is ``'strict'`` meaning that encoding errors raise + :exc:`ValueError` (or a more codec specific subclass, such as + :exc:`UnicodeEncodeError`). Refer to :ref:`codec-base-classes` for more + information on codec error handling. + + .. versionadded:: 2.4 + +.. function:: decode(obj, [encoding[, errors]]) + + Decodes *obj* using the codec registered for *encoding*. The default + encoding is ``'ascii'``. + + *Errors* may be given to set the desired error handling scheme. The + default error handler is ``'strict'`` meaning that decoding errors raise + :exc:`ValueError` (or a more codec specific subclass, such as + :exc:`UnicodeDecodeError`). Refer to :ref:`codec-base-classes` for more + information on codec error handling. + + .. versionadded:: 2.4 .. function:: register(search_function) @@ -47,9 +72,9 @@ It defines the following functions: The various functions or classes take the following arguments: *encode* and *decode*: These must be functions or methods which have the same - interface as the :meth:`encode`/:meth:`decode` methods of Codec instances (see - Codec Interface). The functions/methods are expected to work in a stateless - mode. + interface as the :meth:`~Codec.encode`/:meth:`~Codec.decode` methods of Codec + instances (see :ref:`Codec Interface <codec-objects>`). The functions/methods + are expected to work in a stateless mode. *incrementalencoder* and *incrementaldecoder*: These have to be factory functions providing the following interface: @@ -66,7 +91,7 @@ It defines the following functions: ``factory(stream, errors='strict')`` The factory functions must return objects providing the interfaces defined by - the base classes :class:`StreamWriter` and :class:`StreamReader`, respectively. + the base classes :class:`StreamReader` and :class:`StreamWriter`, respectively. Stream codecs can maintain state. Possible values for errors are @@ -315,11 +340,13 @@ implement the file protocols. The :class:`Codec` class defines the interface for stateless encoders/decoders. -To simplify and standardize error handling, the :meth:`encode` and -:meth:`decode` methods may implement different error handling schemes by +To simplify and standardize error handling, the :meth:`~Codec.encode` and +:meth:`~Codec.decode` methods may implement different error handling schemes by providing the *errors* string argument. The following string values are defined and implemented by all standard Python codecs: +.. tabularcolumns:: |l|L| + +-------------------------+-----------------------------------------------+ | Value | Meaning | +=========================+===============================================+ @@ -395,12 +422,14 @@ interfaces of the stateless encoder and decoder: The :class:`IncrementalEncoder` and :class:`IncrementalDecoder` classes provide the basic interface for incremental encoding and decoding. Encoding/decoding the input isn't done with one call to the stateless encoder/decoder function, but -with multiple calls to the :meth:`encode`/:meth:`decode` method of the -incremental encoder/decoder. The incremental encoder/decoder keeps track of the -encoding/decoding process during method calls. - -The joined output of calls to the :meth:`encode`/:meth:`decode` method is the -same as if all the single inputs were joined into one, and this input was +with multiple calls to the +:meth:`~IncrementalEncoder.encode`/:meth:`~IncrementalDecoder.decode` method of +the incremental encoder/decoder. The incremental encoder/decoder keeps track of +the encoding/decoding process during method calls. + +The joined output of calls to the +:meth:`~IncrementalEncoder.encode`/:meth:`~IncrementalDecoder.decode` method is +the same as if all the single inputs were joined into one, and this input was encoded/decoded with the stateless encoder/decoder. @@ -651,7 +680,7 @@ compatible with the Python codec registry. Read one line from the input stream and return the decoded data. *size*, if given, is passed as size argument to the stream's - :meth:`readline` method. + :meth:`read` method. If *keepends* is false line-endings will be stripped from the lines returned. @@ -887,6 +916,8 @@ particular, the following variants typically exist: * an IBM PC code page, which is ASCII compatible +.. tabularcolumns:: |l|p{0.3\linewidth}|p{0.3\linewidth}| + +-----------------+--------------------------------+--------------------------------+ | Codec | Aliases | Languages | +=================+================================+================================+ @@ -1094,84 +1125,112 @@ particular, the following variants typically exist: | utf_8_sig | | all languages | +-----------------+--------------------------------+--------------------------------+ -A number of codecs are specific to Python, so their codec names have no meaning -outside Python. Some of them don't convert from Unicode strings to byte strings, -but instead use the property of the Python codecs machinery that any bijective -function with one argument can be considered as an encoding. - -For the codecs listed below, the result in the "encoding" direction is always a -byte string. The result of the "decoding" direction is listed as operand type in -the table. - -+--------------------+---------------------------+----------------+---------------------------+ -| Codec | Aliases | Operand type | Purpose | -+====================+===========================+================+===========================+ -| base64_codec | base64, base-64 | byte string | Convert operand to MIME | -| | | | base64 | -+--------------------+---------------------------+----------------+---------------------------+ -| bz2_codec | bz2 | byte string | Compress the operand | -| | | | using bz2 | -+--------------------+---------------------------+----------------+---------------------------+ -| hex_codec | hex | byte string | Convert operand to | -| | | | hexadecimal | -| | | | representation, with two | -| | | | digits per byte | -+--------------------+---------------------------+----------------+---------------------------+ -| idna | | Unicode string | Implements :rfc:`3490`, | -| | | | see also | -| | | | :mod:`encodings.idna` | -+--------------------+---------------------------+----------------+---------------------------+ -| mbcs | dbcs | Unicode string | Windows only: Encode | -| | | | operand according to the | -| | | | ANSI codepage (CP_ACP) | -+--------------------+---------------------------+----------------+---------------------------+ -| palmos | | Unicode string | Encoding of PalmOS 3.5 | -+--------------------+---------------------------+----------------+---------------------------+ -| punycode | | Unicode string | Implements :rfc:`3492` | -+--------------------+---------------------------+----------------+---------------------------+ -| quopri_codec | quopri, quoted-printable, | byte string | Convert operand to MIME | -| | quotedprintable | | quoted printable | -+--------------------+---------------------------+----------------+---------------------------+ -| raw_unicode_escape | | Unicode string | Produce a string that is | -| | | | suitable as raw Unicode | -| | | | literal in Python source | -| | | | code | -+--------------------+---------------------------+----------------+---------------------------+ -| rot_13 | rot13 | Unicode string | Returns the Caesar-cypher | -| | | | encryption of the operand | -+--------------------+---------------------------+----------------+---------------------------+ -| string_escape | | byte string | Produce a string that is | -| | | | suitable as string | -| | | | literal in Python source | -| | | | code | -+--------------------+---------------------------+----------------+---------------------------+ -| undefined | | any | Raise an exception for | -| | | | all conversions. Can be | -| | | | used as the system | -| | | | encoding if no automatic | -| | | | :term:`coercion` between | -| | | | byte and Unicode strings | -| | | | is desired. | -+--------------------+---------------------------+----------------+---------------------------+ -| unicode_escape | | Unicode string | Produce a string that is | -| | | | suitable as Unicode | -| | | | literal in Python source | -| | | | code | -+--------------------+---------------------------+----------------+---------------------------+ -| unicode_internal | | Unicode string | Return the internal | -| | | | representation of the | -| | | | operand | -+--------------------+---------------------------+----------------+---------------------------+ -| uu_codec | uu | byte string | Convert the operand using | -| | | | uuencode | -+--------------------+---------------------------+----------------+---------------------------+ -| zlib_codec | zip, zlib | byte string | Compress the operand | -| | | | using gzip | -+--------------------+---------------------------+----------------+---------------------------+ +Python Specific Encodings +------------------------- + +A number of predefined codecs are specific to Python, so their codec names have +no meaning outside Python. These are listed in the tables below based on the +expected input and output types (note that while text encodings are the most +common use case for codecs, the underlying codec infrastructure supports +arbitrary data transforms rather than just text encodings). For asymmetric +codecs, the stated purpose describes the encoding direction. + +The following codecs provide unicode-to-str encoding [#encoding-note]_ and +str-to-unicode decoding [#decoding-note]_, similar to the Unicode text +encodings. + +.. tabularcolumns:: |l|L|L| + ++--------------------+---------------------------+---------------------------+ +| Codec | Aliases | Purpose | ++====================+===========================+===========================+ +| idna | | Implements :rfc:`3490`, | +| | | see also | +| | | :mod:`encodings.idna` | ++--------------------+---------------------------+---------------------------+ +| mbcs | dbcs | Windows only: Encode | +| | | operand according to the | +| | | ANSI codepage (CP_ACP) | ++--------------------+---------------------------+---------------------------+ +| palmos | | Encoding of PalmOS 3.5 | ++--------------------+---------------------------+---------------------------+ +| punycode | | Implements :rfc:`3492` | ++--------------------+---------------------------+---------------------------+ +| raw_unicode_escape | | Produce a string that is | +| | | suitable as raw Unicode | +| | | literal in Python source | +| | | code | ++--------------------+---------------------------+---------------------------+ +| rot_13 | rot13 | Returns the Caesar-cypher | +| | | encryption of the operand | ++--------------------+---------------------------+---------------------------+ +| undefined | | Raise an exception for | +| | | all conversions. Can be | +| | | used as the system | +| | | encoding if no automatic | +| | | :term:`coercion` between | +| | | byte and Unicode strings | +| | | is desired. | ++--------------------+---------------------------+---------------------------+ +| unicode_escape | | Produce a string that is | +| | | suitable as Unicode | +| | | literal in Python source | +| | | code | ++--------------------+---------------------------+---------------------------+ +| unicode_internal | | Return the internal | +| | | representation of the | +| | | operand | ++--------------------+---------------------------+---------------------------+ .. versionadded:: 2.3 The ``idna`` and ``punycode`` encodings. +The following codecs provide str-to-str encoding and decoding +[#decoding-note]_. + +.. tabularcolumns:: |l|L|L|L| + ++--------------------+---------------------------+---------------------------+------------------------------+ +| Codec | Aliases | Purpose | Encoder/decoder | ++====================+===========================+===========================+==============================+ +| base64_codec | base64, base-64 | Convert operand to MIME | :meth:`base64.b64encode`, | +| | | base64 (the result always | :meth:`base64.b64decode` | +| | | includes a trailing | | +| | | ``'\n'``) | | ++--------------------+---------------------------+---------------------------+------------------------------+ +| bz2_codec | bz2 | Compress the operand | :meth:`bz2.compress`, | +| | | using bz2 | :meth:`bz2.decompress` | ++--------------------+---------------------------+---------------------------+------------------------------+ +| hex_codec | hex | Convert operand to | :meth:`base64.b16encode`, | +| | | hexadecimal | :meth:`base64.b16decode` | +| | | representation, with two | | +| | | digits per byte | | ++--------------------+---------------------------+---------------------------+------------------------------+ +| quopri_codec | quopri, quoted-printable, | Convert operand to MIME | :meth:`quopri.encodestring`, | +| | quotedprintable | quoted printable | :meth:`quopri.decodestring` | ++--------------------+---------------------------+---------------------------+------------------------------+ +| string_escape | | Produce a string that is | | +| | | suitable as string | | +| | | literal in Python source | | +| | | code | | ++--------------------+---------------------------+---------------------------+------------------------------+ +| uu_codec | uu | Convert the operand using | :meth:`uu.encode`, | +| | | uuencode | :meth:`uu.decode` | ++--------------------+---------------------------+---------------------------+------------------------------+ +| zlib_codec | zip, zlib | Compress the operand | :meth:`zlib.compress`, | +| | | using gzip | :meth:`zlib.decompress` | ++--------------------+---------------------------+---------------------------+------------------------------+ + +.. [#encoding-note] str objects are also accepted as input in place of unicode + objects. They are implicitly converted to unicode by decoding them using + the default encoding. If this conversion fails, it may lead to encoding + operations raising :exc:`UnicodeDecodeError`. + +.. [#decoding-note] unicode objects are also accepted as input in place of str + objects. They are implicitly converted to str by encoding them using the + default encoding. If this conversion fails, it may lead to decoding + operations raising :exc:`UnicodeEncodeError`. + :mod:`encodings.idna` --- Internationalized Domain Names in Applications ------------------------------------------------------------------------ diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index abb36db..5bb3569 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -51,7 +51,7 @@ For example:: >>> # Find the ten most common words in Hamlet >>> import re - >>> words = re.findall('\w+', open('hamlet.txt').read().lower()) + >>> words = re.findall(r'\w+', open('hamlet.txt').read().lower()) >>> Counter(words).most_common(10) [('the', 1143), ('and', 966), ('to', 762), ('of', 669), ('i', 631), ('you', 554), ('a', 546), ('my', 514), ('hamlet', 471), ('in', 451)] @@ -120,6 +120,7 @@ For example:: >>> c = Counter(a=4, b=2, c=0, d=-2) >>> d = Counter(a=1, b=2, c=3, d=4) >>> c.subtract(d) + >>> c Counter({'a': 3, 'b': 0, 'c': -3, 'd': -6}) The usual dictionary methods are available for :class:`Counter` objects @@ -145,7 +146,7 @@ Common patterns for working with :class:`Counter` objects:: dict(c) # convert to a regular dictionary c.items() # convert to a list of (elem, cnt) pairs Counter(dict(list_of_pairs)) # convert from a list of (elem, cnt) pairs - c.most_common()[:-n:-1] # n least common elements + c.most_common()[:-n-1:-1] # n least common elements c += Counter() # remove zero and negative counts Several mathematical operations are provided for combining :class:`Counter` @@ -601,47 +602,53 @@ Example: >>> Point = namedtuple('Point', ['x', 'y'], verbose=True) class Point(tuple): - 'Point(x, y)' + 'Point(x, y)' <BLANKLINE> - __slots__ = () + __slots__ = () <BLANKLINE> - _fields = ('x', 'y') + _fields = ('x', 'y') <BLANKLINE> - def __new__(_cls, x, y): - 'Create a new instance of Point(x, y)' - return _tuple.__new__(_cls, (x, y)) + def __new__(_cls, x, y): + 'Create a new instance of Point(x, y)' + return _tuple.__new__(_cls, (x, y)) <BLANKLINE> - @classmethod - def _make(cls, iterable, new=tuple.__new__, len=len): - 'Make a new Point object from a sequence or iterable' - result = new(cls, iterable) - if len(result) != 2: - raise TypeError('Expected 2 arguments, got %d' % len(result)) - return result + @classmethod + def _make(cls, iterable, new=tuple.__new__, len=len): + 'Make a new Point object from a sequence or iterable' + result = new(cls, iterable) + if len(result) != 2: + raise TypeError('Expected 2 arguments, got %d' % len(result)) + return result <BLANKLINE> - def __repr__(self): - 'Return a nicely formatted representation string' - return 'Point(x=%r, y=%r)' % self + def __repr__(self): + 'Return a nicely formatted representation string' + return 'Point(x=%r, y=%r)' % self <BLANKLINE> - def _asdict(self): - 'Return a new OrderedDict which maps field names to their values' - return OrderedDict(zip(self._fields, self)) + def _asdict(self): + 'Return a new OrderedDict which maps field names to their values' + return OrderedDict(zip(self._fields, self)) <BLANKLINE> - __dict__ = property(_asdict) + def _replace(_self, **kwds): + 'Return a new Point object replacing specified fields with new values' + result = _self._make(map(kwds.pop, ('x', 'y'), _self)) + if kwds: + raise ValueError('Got unexpected field names: %r' % kwds.keys()) + return result <BLANKLINE> - def _replace(_self, **kwds): - 'Return a new Point object replacing specified fields with new values' - result = _self._make(map(kwds.pop, ('x', 'y'), _self)) - if kwds: - raise ValueError('Got unexpected field names: %r' % kwds.keys()) - return result + def __getnewargs__(self): + 'Return self as a plain tuple. Used by copy and pickle.' + return tuple(self) <BLANKLINE> - def __getnewargs__(self): - 'Return self as a plain tuple. Used by copy and pickle.' - return tuple(self) + __dict__ = _property(_asdict) + <BLANKLINE> + def __getstate__(self): + 'Exclude the OrderedDict from pickling' + pass + <BLANKLINE> + x = _property(_itemgetter(0), doc='Alias for field number 0') + <BLANKLINE> + y = _property(_itemgetter(1), doc='Alias for field number 1') <BLANKLINE> - x = _property(_itemgetter(0), doc='Alias for field number 0') - y = _property(_itemgetter(1), doc='Alias for field number 1') >>> p = Point(11, y=22) # instantiate with positional or keyword arguments >>> p[0] + p[1] # indexable like the plain tuple (11, 22) @@ -811,9 +818,9 @@ reverse iteration using :func:`reversed`. Equality tests between :class:`OrderedDict` objects are order-sensitive and are implemented as ``list(od1.items())==list(od2.items())``. Equality tests between :class:`OrderedDict` objects and other -:class:`Mapping` objects are order-insensitive like regular dictionaries. -This allows :class:`OrderedDict` objects to be substituted anywhere a -regular dictionary is used. +:class:`Mapping` objects are order-insensitive like regular +dictionaries. This allows :class:`OrderedDict` objects to be substituted +anywhere a regular dictionary is used. The :class:`OrderedDict` constructor and :meth:`update` method both accept keyword arguments, but their order is lost because Python's function call @@ -850,7 +857,7 @@ are deleted. But when new keys are added, the keys are appended to the end and the sort is not maintained. It is also straight-forward to create an ordered dictionary variant -that the remembers the order the keys were *last* inserted. +that remembers the order the keys were *last* inserted. If a new entry overwrites an existing entry, the original insertion position is changed and moved to the end:: @@ -892,29 +899,35 @@ ABC Inherits from Abstract Methods Mixin :class:`Sized` ``__len__`` :class:`Callable` ``__call__`` -:class:`Sequence` :class:`Sized`, ``__getitem__`` ``__contains__``, ``__iter__``, ``__reversed__``, - :class:`Iterable`, ``index``, and ``count`` - :class:`Container` - -:class:`MutableSequence` :class:`Sequence` ``__setitem__``, Inherited :class:`Sequence` methods and - ``__delitem__``, ``append``, ``reverse``, ``extend``, ``pop``, - ``insert`` ``remove``, and ``__iadd__`` - -:class:`Set` :class:`Sized`, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``, - :class:`Iterable`, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``, - :class:`Container` ``__sub__``, ``__xor__``, and ``isdisjoint`` - -:class:`MutableSet` :class:`Set` ``add``, Inherited :class:`Set` methods and - ``discard`` ``clear``, ``pop``, ``remove``, ``__ior__``, - ``__iand__``, ``__ixor__``, and ``__isub__`` - -:class:`Mapping` :class:`Sized`, ``__getitem__`` ``__contains__``, ``keys``, ``items``, ``values``, - :class:`Iterable`, ``get``, ``__eq__``, and ``__ne__`` +:class:`Sequence` :class:`Sized`, ``__getitem__``, ``__contains__``, ``__iter__``, ``__reversed__``, + :class:`Iterable`, ``__len__`` ``index``, and ``count`` :class:`Container` -:class:`MutableMapping` :class:`Mapping` ``__setitem__``, Inherited :class:`Mapping` methods and - ``__delitem__`` ``pop``, ``popitem``, ``clear``, ``update``, - and ``setdefault`` +:class:`MutableSequence` :class:`Sequence` ``__getitem__``, Inherited :class:`Sequence` methods and + ``__setitem__``, ``append``, ``reverse``, ``extend``, ``pop``, + ``__delitem__``, ``remove``, and ``__iadd__`` + ``__len__``, + ``insert`` + +:class:`Set` :class:`Sized`, ``__contains__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``, + :class:`Iterable`, ``__iter__``, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``, + :class:`Container` ``__len__`` ``__sub__``, ``__xor__``, and ``isdisjoint`` + +:class:`MutableSet` :class:`Set` ``__contains__``, Inherited :class:`Set` methods and + ``__iter__``, ``clear``, ``pop``, ``remove``, ``__ior__``, + ``__len__``, ``__iand__``, ``__ixor__``, and ``__isub__`` + ``add``, + ``discard`` + +:class:`Mapping` :class:`Sized`, ``__getitem__``, ``__contains__``, ``keys``, ``items``, ``values``, + :class:`Iterable`, ``__iter__``, ``get``, ``__eq__``, and ``__ne__`` + :class:`Container` ``__len__`` + +:class:`MutableMapping` :class:`Mapping` ``__getitem__``, Inherited :class:`Mapping` methods and + ``__setitem__``, ``pop``, ``popitem``, ``clear``, ``update``, + ``__delitem__``, and ``setdefault`` + ``__iter__``, + ``__len__`` :class:`MappingView` :class:`Sized` ``__len__`` diff --git a/Doc/library/colorsys.rst b/Doc/library/colorsys.rst index dbab706..225306c 100644 --- a/Doc/library/colorsys.rst +++ b/Doc/library/colorsys.rst @@ -58,7 +58,7 @@ The :mod:`colorsys` module defines the following functions: Example:: >>> import colorsys - >>> colorsys.rgb_to_hsv(.3, .4, .2) - (0.25, 0.5, 0.4) - >>> colorsys.hsv_to_rgb(0.25, 0.5, 0.4) - (0.3, 0.4, 0.2) + >>> colorsys.rgb_to_hsv(0.2, 0.4, 0.4) + (0.5, 0.5, 0.4) + >>> colorsys.hsv_to_rgb(0.5, 0.5, 0.4) + (0.2, 0.4, 0.4) diff --git a/Doc/library/commands.rst b/Doc/library/commands.rst index 46ff823..0b73e42 100644 --- a/Doc/library/commands.rst +++ b/Doc/library/commands.rst @@ -8,7 +8,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`commands` module has been removed in Python 3.0. Use the + The :mod:`commands` module has been removed in Python 3. Use the :mod:`subprocess` module instead. .. sectionauthor:: Sue Williams <sbw@provis.com> diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst index cf0d5f8..0e79a5a 100644 --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -127,7 +127,7 @@ subdirectory and all its subdirectories:: # Perform same compilation, excluding files in .svn directories. import re - compileall.compile_dir('Lib/', rx=re.compile('/[.]svn'), force=True) + compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) .. seealso:: diff --git a/Doc/library/compiler.rst b/Doc/library/compiler.rst index 458e653..494902e 100644 --- a/Doc/library/compiler.rst +++ b/Doc/library/compiler.rst @@ -6,7 +6,7 @@ Python compiler package *********************** .. deprecated:: 2.6 - The :mod:`compiler` package has been removed in Python 3.0. + The :mod:`compiler` package has been removed in Python 3. .. sectionauthor:: Jeremy Hylton <jeremy@zope.com> @@ -540,7 +540,7 @@ examples demonstrate how to use the :func:`parse` function, what the repr of an AST looks like, and how to access attributes of an AST node. The first module defines a single function. Assume it is stored in -:file:`/tmp/doublelib.py`. :: +:file:`doublelib.py`. :: """This is an example module. @@ -557,7 +557,7 @@ to create an instance from a repr, you must import the class names from the :mod:`compiler.ast` module. :: >>> import compiler - >>> mod = compiler.parseFile("/tmp/doublelib.py") + >>> mod = compiler.parseFile("doublelib.py") >>> mod Module('This is an example module.\n\nThis is the docstring.\n', Stmt([Function(None, 'double', ['x'], [], 0, diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 3536f3e..515074a 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -12,8 +12,8 @@ .. note:: The :mod:`ConfigParser` module has been renamed to :mod:`configparser` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. .. index:: pair: .ini; file @@ -451,9 +451,9 @@ An example of writing to a configuration file:: # when attempting to write to a file or when you get it in non-raw # mode. SafeConfigParser does not allow such assignments to take place. config.add_section('Section1') - config.set('Section1', 'int', '15') - config.set('Section1', 'bool', 'true') - config.set('Section1', 'float', '3.1415') + config.set('Section1', 'an_int', '15') + config.set('Section1', 'a_bool', 'true') + config.set('Section1', 'a_float', '3.1415') config.set('Section1', 'baz', 'fun') config.set('Section1', 'bar', 'Python') config.set('Section1', 'foo', '%(bar)s is %(baz)s!') @@ -471,13 +471,13 @@ An example of reading the configuration file again:: # getfloat() raises an exception if the value is not a float # getint() and getboolean() also do this for their respective types - float = config.getfloat('Section1', 'float') - int = config.getint('Section1', 'int') - print float + int + a_float = config.getfloat('Section1', 'a_float') + an_int = config.getint('Section1', 'an_int') + print a_float + an_int # Notice that the next output does not interpolate '%(bar)s' or '%(baz)s'. # This is because we are using a RawConfigParser(). - if config.getboolean('Section1', 'bool'): + if config.getboolean('Section1', 'a_bool'): print config.get('Section1', 'foo') To get interpolation, you will need to use a :class:`ConfigParser` or diff --git a/Doc/library/cookie.rst b/Doc/library/cookie.rst index 480dffa..19786f7 100644 --- a/Doc/library/cookie.rst +++ b/Doc/library/cookie.rst @@ -8,8 +8,8 @@ .. note:: The :mod:`Cookie` module has been renamed to :mod:`http.cookies` in Python - 3.0. The :term:`2to3` tool will automatically adapt imports when converting - your sources to 3.0. + 3. The :term:`2to3` tool will automatically adapt imports when converting + your sources to Python 3. **Source code:** :source:`Lib/Cookie.py` @@ -22,8 +22,14 @@ cookie value. The module formerly strictly applied the parsing rules described in the :rfc:`2109` and :rfc:`2068` specifications. It has since been discovered that -MSIE 3.0x doesn't follow the character rules outlined in those specs. As a -result, the parsing rules used are a bit less strict. +MSIE 3.0x doesn't follow the character rules outlined in those specs and also +many current day browsers and servers have relaxed parsing rules when comes to +Cookie handling. As a result, the parsing rules used are a bit less strict. + +The character set, :data:`string.ascii_letters`, :data:`string.digits` and +``!#$%&'*+-.^_`|~`` denote the set of valid characters allowed by this module +in Cookie name (as :attr:`~Morsel.key`). + .. note:: diff --git a/Doc/library/cookielib.rst b/Doc/library/cookielib.rst index 77d6624..b1baef1 100644 --- a/Doc/library/cookielib.rst +++ b/Doc/library/cookielib.rst @@ -8,8 +8,8 @@ .. note:: The :mod:`cookielib` module has been renamed to :mod:`http.cookiejar` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. .. versionadded:: 2.4 @@ -98,7 +98,7 @@ The following classes are provided: Netscape and RFC 2965 cookies. By default, RFC 2109 cookies (ie. cookies received in a :mailheader:`Set-Cookie` header with a version cookie-attribute of 1) are treated according to the RFC 2965 rules. However, if RFC 2965 handling - is turned off or :attr:`rfc2109_as_netscape` is True, RFC 2109 cookies are + is turned off or :attr:`rfc2109_as_netscape` is ``True``, RFC 2109 cookies are 'downgraded' by the :class:`CookieJar` instance to Netscape cookies, by setting the :attr:`version` attribute of the :class:`Cookie` instance to 0. :class:`DefaultCookiePolicy` also provides some parameters to allow some @@ -308,7 +308,7 @@ FileCookieJar subclasses and co-operation with web browsers ----------------------------------------------------------- The following :class:`CookieJar` subclasses are provided for reading and -writing . +writing. .. class:: MozillaCookieJar(filename, delayload=None, policy=None) @@ -652,7 +652,7 @@ internal consistency, so you should know what you're doing if you do that. .. attribute:: Cookie.secure - True if cookie should only be returned over a secure connection. + ``True`` if cookie should only be returned over a secure connection. .. attribute:: Cookie.expires @@ -663,7 +663,7 @@ internal consistency, so you should know what you're doing if you do that. .. attribute:: Cookie.discard - True if this is a session cookie. + ``True`` if this is a session cookie. .. attribute:: Cookie.comment @@ -680,7 +680,7 @@ internal consistency, so you should know what you're doing if you do that. .. attribute:: Cookie.rfc2109 - True if this cookie was received as an RFC 2109 cookie (ie. the cookie + ``True`` if this cookie was received as an RFC 2109 cookie (ie. the cookie arrived in a :mailheader:`Set-Cookie` header, and the value of the Version cookie-attribute in that header was 1). This attribute is provided because :mod:`cookielib` may 'downgrade' RFC 2109 cookies to Netscape cookies, in @@ -691,18 +691,18 @@ internal consistency, so you should know what you're doing if you do that. .. attribute:: Cookie.port_specified - True if a port or set of ports was explicitly specified by the server (in the + ``True`` if a port or set of ports was explicitly specified by the server (in the :mailheader:`Set-Cookie` / :mailheader:`Set-Cookie2` header). .. attribute:: Cookie.domain_specified - True if a domain was explicitly specified by the server. + ``True`` if a domain was explicitly specified by the server. .. attribute:: Cookie.domain_initial_dot - True if the domain explicitly specified by the server began with a dot + ``True`` if the domain explicitly specified by the server began with a dot (``'.'``). Cookies may have additional non-standard cookie-attributes. These may be @@ -729,7 +729,7 @@ The :class:`Cookie` class also defines the following method: .. method:: Cookie.is_expired([now=None]) - True if cookie has passed the time at which the server requested it should + ``True`` if cookie has passed the time at which the server requested it should expire. If *now* is given (in seconds since the epoch), return whether the cookie has expired at the specified time. diff --git a/Doc/library/copy_reg.rst b/Doc/library/copy_reg.rst index 609ded0..3d8ef77 100644 --- a/Doc/library/copy_reg.rst +++ b/Doc/library/copy_reg.rst @@ -5,20 +5,20 @@ :synopsis: Register pickle support functions. .. note:: - The :mod:`copy_reg` module has been renamed to :mod:`copyreg` in Python 3.0. + The :mod:`copy_reg` module has been renamed to :mod:`copyreg` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. .. index:: module: pickle module: cPickle module: copy -The :mod:`copy_reg` module provides support for the :mod:`pickle` and -:mod:`cPickle` modules. The :mod:`copy` module is likely to use this in the -future as well. It provides configuration information about object constructors -which are not classes. Such constructors may be factory functions or class -instances. +The :mod:`copy_reg` module offers a way to define fuctions used while pickling +specific objects. The :mod:`pickle`, :mod:`cPickle`, and :mod:`copy` modules +use those functions when pickling/copying those objects. The module provides +configuration information about object constructors which are not classes. +Such constructors may be factory functions or class instances. .. function:: constructor(object) @@ -43,3 +43,24 @@ instances. See the :mod:`pickle` module for more details on the interface expected of *function* and *constructor*. +Example +------- + +The example below would like to show how to register a pickle function and how +it will be used: + + >>> import copy_reg, copy, pickle + >>> class C(object): + ... def __init__(self, a): + ... self.a = a + ... + >>> def pickle_c(c): + ... print("pickling a C instance...") + ... return C, (c.a,) + ... + >>> copy_reg.pickle(C, pickle_c) + >>> c = C(1) + >>> d = copy.copy(c) + pickling a C instance... + >>> p = pickle.dumps(c) + pickling a C instance... diff --git a/Doc/library/csv.rst b/Doc/library/csv.rst index f4e9d7c..2054ce6 100644 --- a/Doc/library/csv.rst +++ b/Doc/library/csv.rst @@ -40,7 +40,7 @@ using the :class:`DictReader` and :class:`DictWriter` classes. This version of the :mod:`csv` module doesn't support Unicode input. Also, there are currently some issues regarding ASCII NUL characters. Accordingly, all input should be UTF-8 or printable ASCII to be safe; see the examples in - section :ref:`csv-examples`. These restrictions will be removed in the future. + section :ref:`csv-examples`. .. seealso:: @@ -57,7 +57,7 @@ Module Contents The :mod:`csv` module defines the following functions: -.. function:: reader(csvfile[, dialect='excel'][, fmtparam]) +.. function:: reader(csvfile, dialect='excel', **fmtparams) Return a reader object which will iterate over lines in the given *csvfile*. *csvfile* can be any object which supports the :term:`iterator` protocol and returns a @@ -67,7 +67,7 @@ The :mod:`csv` module defines the following functions: *dialect* parameter can be given which is used to define a set of parameters specific to a particular CSV dialect. It may be an instance of a subclass of the :class:`Dialect` class or one of the strings returned by the - :func:`list_dialects` function. The other optional *fmtparam* keyword arguments + :func:`list_dialects` function. The other optional *fmtparams* keyword arguments can be given to override individual formatting parameters in the current dialect. For full details about the dialect and formatting parameters, see section :ref:`csv-fmt-params`. @@ -78,9 +78,10 @@ The :mod:`csv` module defines the following functions: A short usage example:: >>> import csv - >>> spamReader = csv.reader(open('eggs.csv', 'rb'), delimiter=' ', quotechar='|') - >>> for row in spamReader: - ... print ', '.join(row) + >>> with open('eggs.csv', 'rb') as csvfile: + ... spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|') + ... for row in spamreader: + ... print ', '.join(row) Spam, Spam, Spam, Spam, Spam, Baked Beans Spam, Lovely Spam, Wonderful Spam @@ -94,7 +95,7 @@ The :mod:`csv` module defines the following functions: be split into lines in a manner which preserves the newline characters. -.. function:: writer(csvfile[, dialect='excel'][, fmtparam]) +.. function:: writer(csvfile, dialect='excel', **fmtparams) Return a writer object responsible for converting the user's data into delimited strings on the given file-like object. *csvfile* can be any object with a @@ -103,7 +104,7 @@ The :mod:`csv` module defines the following functions: parameter can be given which is used to define a set of parameters specific to a particular CSV dialect. It may be an instance of a subclass of the :class:`Dialect` class or one of the strings returned by the - :func:`list_dialects` function. The other optional *fmtparam* keyword arguments + :func:`list_dialects` function. The other optional *fmtparams* keyword arguments can be given to override individual formatting parameters in the current dialect. For full details about the dialect and formatting parameters, see section :ref:`csv-fmt-params`. To make it @@ -115,18 +116,19 @@ The :mod:`csv` module defines the following functions: A short usage example:: - >>> import csv - >>> spamWriter = csv.writer(open('eggs.csv', 'wb'), delimiter=' ', - ... quotechar='|', quoting=csv.QUOTE_MINIMAL) - >>> spamWriter.writerow(['Spam'] * 5 + ['Baked Beans']) - >>> spamWriter.writerow(['Spam', 'Lovely Spam', 'Wonderful Spam']) + import csv + with open('eggs.csv', 'wb') as csvfile: + spamwriter = csv.writer(csvfile, delimiter=' ', + quotechar='|', quoting=csv.QUOTE_MINIMAL) + spamwriter.writerow(['Spam'] * 5 + ['Baked Beans']) + spamwriter.writerow(['Spam', 'Lovely Spam', 'Wonderful Spam']) -.. function:: register_dialect(name[, dialect][, fmtparam]) +.. function:: register_dialect(name[, dialect], **fmtparams) Associate *dialect* with *name*. *name* must be a string or Unicode object. The dialect can be specified either by passing a sub-class of :class:`Dialect`, or - by *fmtparam* keyword arguments, or both, with keyword arguments overriding + by *fmtparams* keyword arguments, or both, with keyword arguments overriding parameters of the dialect. For full details about the dialect and formatting parameters, see section :ref:`csv-fmt-params`. @@ -162,36 +164,43 @@ The :mod:`csv` module defines the following functions: The :mod:`csv` module defines the following classes: -.. class:: DictReader(csvfile[, fieldnames=None[, restkey=None[, restval=None[, dialect='excel'[, *args, **kwds]]]]]) - - Create an object which operates like a regular reader but maps the information - read into a dict whose keys are given by the optional *fieldnames* parameter. - If the *fieldnames* parameter is omitted, the values in the first row of the - *csvfile* will be used as the fieldnames. If the row read has more fields - than the fieldnames sequence, the remaining data is added as a sequence - keyed by the value of *restkey*. If the row read has fewer fields than the - fieldnames sequence, the remaining keys take the value of the optional - *restval* parameter. Any other optional or keyword arguments are passed to - the underlying :class:`reader` instance. - - -.. class:: DictWriter(csvfile, fieldnames[, restval=''[, extrasaction='raise'[, dialect='excel'[, *args, **kwds]]]]) - - Create an object which operates like a regular writer but maps dictionaries onto - output rows. The *fieldnames* parameter identifies the order in which values in - the dictionary passed to the :meth:`writerow` method are written to the - *csvfile*. The optional *restval* parameter specifies the value to be written - if the dictionary is missing a key in *fieldnames*. If the dictionary passed to - the :meth:`writerow` method contains a key not found in *fieldnames*, the - optional *extrasaction* parameter indicates what action to take. If it is set - to ``'raise'`` a :exc:`ValueError` is raised. If it is set to ``'ignore'``, - extra values in the dictionary are ignored. Any other optional or keyword - arguments are passed to the underlying :class:`writer` instance. - - Note that unlike the :class:`DictReader` class, the *fieldnames* parameter of - the :class:`DictWriter` is not optional. Since Python's :class:`dict` objects - are not ordered, there is not enough information available to deduce the order - in which the row should be written to the *csvfile*. +.. class:: DictReader(csvfile, fieldnames=None, restkey=None, restval=None, \ + dialect='excel', *args, **kwds) + + Create an object which operates like a regular reader but maps the + information read into a dict whose keys are given by the optional + *fieldnames* parameter. The *fieldnames* parameter is a :ref:`sequence + <collections-abstract-base-classes>` whose elements are associated with the + fields of the input data in order. These elements become the keys of the + resulting dictionary. If the *fieldnames* parameter is omitted, the values + in the first row of the *csvfile* will be used as the fieldnames. If the + row read has more fields than the fieldnames sequence, the remaining data is + added as a sequence keyed by the value of *restkey*. If the row read has + fewer fields than the fieldnames sequence, the remaining keys take the value + of the optional *restval* parameter. Any other optional or keyword + arguments are passed to the underlying :class:`reader` instance. + + +.. class:: DictWriter(csvfile, fieldnames, restval='', extrasaction='raise', \ + dialect='excel', *args, **kwds) + + Create an object which operates like a regular writer but maps dictionaries + onto output rows. The *fieldnames* parameter is a :ref:`sequence + <collections-abstract-base-classes>` of keys that identify the order in + which values in the dictionary passed to the :meth:`writerow` method are + written to the *csvfile*. The optional *restval* parameter specifies the + value to be written if the dictionary is missing a key in *fieldnames*. If + the dictionary passed to the :meth:`writerow` method contains a key not + found in *fieldnames*, the optional *extrasaction* parameter indicates what + action to take. If it is set to ``'raise'`` a :exc:`ValueError` is raised. + If it is set to ``'ignore'``, extra values in the dictionary are ignored. + Any other optional or keyword arguments are passed to the underlying + :class:`writer` instance. + + Note that unlike the :class:`DictReader` class, the *fieldnames* parameter + of the :class:`DictWriter` is not optional. Since Python's :class:`dict` + objects are not ordered, there is not enough information available to deduce + the order in which the row should be written to the *csvfile*. .. class:: Dialect @@ -219,7 +228,7 @@ The :mod:`csv` module defines the following classes: The :class:`Sniffer` class provides two methods: - .. method:: sniff(sample[, delimiters=None]) + .. method:: sniff(sample, delimiters=None) Analyze the given *sample* and return a :class:`Dialect` subclass reflecting the parameters found. If the optional *delimiters* parameter @@ -234,11 +243,11 @@ The :mod:`csv` module defines the following classes: An example for :class:`Sniffer` use:: - csvfile = open("example.csv", "rb") - dialect = csv.Sniffer().sniff(csvfile.read(1024)) - csvfile.seek(0) - reader = csv.reader(csvfile, dialect) - # ... process CSV file contents here ... + with open('example.csv', 'rb') as csvfile: + dialect = csv.Sniffer().sniff(csvfile.read(1024)) + csvfile.seek(0) + reader = csv.reader(csvfile, dialect) + # ... process CSV file contents here ... The :mod:`csv` module defines the following constants: @@ -353,6 +362,11 @@ Dialects support the following attributes: The default is :const:`False`. +.. attribute:: Dialect.strict + + When ``True``, raise exception :exc:`Error` on bad CSV input. + The default is ``False``. + Reader Objects -------------- @@ -478,7 +492,7 @@ A slightly more advanced use of the reader --- catching and reporting errors:: try: for row in reader: print row - except csv.Error, e: + except csv.Error as e: sys.exit('file %s, line %d: %s' % (filename, reader.line_num, e)) And while the module doesn't directly support parsing strings, it can easily be diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst index 4913b82..31d5b31 100644 --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -215,7 +215,7 @@ more about :mod:`ctypes` data types. Fundamental data types ^^^^^^^^^^^^^^^^^^^^^^ -:mod:`ctypes` defines a number of primitive C compatible data types : +:mod:`ctypes` defines a number of primitive C compatible data types: +----------------------+------------------------------------------+----------------------------+ | ctypes type | C type | Python type | @@ -568,8 +568,8 @@ Here is a simple example of a POINT structure, which contains two integers named ValueError: too many initializers >>> -You can, however, build much more complicated structures. Structures can itself -contain other structures by using a structure as a field type. +You can, however, build much more complicated structures. A structure can +itself contain other structures by using a structure as a field type. Here is a RECT structure which contains two POINTs named *upperleft* and *lowerright*:: @@ -602,6 +602,13 @@ for debugging because they can provide useful information:: .. _ctypes-structureunion-alignment-byte-order: +.. warning:: + + :mod:`ctypes` does not support passing unions or structures with bit-fields + to functions by value. While this may work on 32-bit x86, it's not + guaranteed by the library to work in the general case. Unions and + structures with bit-fields should always be passed to functions by pointer. + Structure/union alignment and byte order ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -816,6 +823,11 @@ pointer types. So, for ``POINTER(c_int)``, ctypes accepts an array of c_int:: 3 >>> +In addition, if a function argument is explicitly declared to be a pointer type +(such as ``POINTER(c_int)``) in :attr:`argtypes`, an object of the pointed +type (``c_int`` in this case) can be passed to the function. ctypes will apply +the required :func:`byref` conversion in this case automatically. + To set a POINTER type field to ``NULL``, you can assign ``None``:: >>> bar.values = None @@ -1068,12 +1080,18 @@ As we can easily check, our array is sorted now:: 1 5 7 33 99 >>> -**Important note for callback functions:** +.. note:: -Make sure you keep references to CFUNCTYPE objects as long as they are used from -C code. :mod:`ctypes` doesn't, and if you don't, they may be garbage collected, -crashing your program when a callback is made. + Make sure you keep references to :func:`CFUNCTYPE` objects as long as they + are used from C code. :mod:`ctypes` doesn't, and if you don't, they may be + garbage collected, crashing your program when a callback is made. + Also, note that if the callback function is called in a thread created + outside of Python's control (e.g. by the foreign code that calls the + callback), ctypes creates a new dummy Python thread on every invocation. This + behavior is correct for most purposes, but it means that values stored with + `threading.local` will *not* survive across different callbacks, even when + those calls are made from the same C thread. .. _ctypes-accessing-values-exported-from-dlls: @@ -1153,8 +1171,8 @@ testing. Try it out with ``import __hello__`` for example. Surprises ^^^^^^^^^ -There are some edges in :mod:`ctypes` where you may be expect something else than -what actually happens. +There are some edge cases in :mod:`ctypes` where you might expect something +other than what actually happens. Consider the following example:: @@ -1321,7 +1339,7 @@ returns the full pathname, but since there is no predefined naming scheme a call like ``find_library("c")`` will fail and return ``None``. If wrapping a shared library with :mod:`ctypes`, it *may* be better to determine -the shared library name at development type, and hardcode that into the wrapper +the shared library name at development time, and hardcode that into the wrapper module instead of using :func:`find_library` to locate the library at runtime. @@ -1702,7 +1720,7 @@ the windows header file is this:: WINUSERAPI int WINAPI MessageBoxA( - HWND hWnd , + HWND hWnd, LPCSTR lpText, LPCSTR lpCaption, UINT uType); @@ -1994,8 +2012,8 @@ Utility functions .. function:: sizeof(obj_or_type) - Returns the size in bytes of a ctypes type or instance memory buffer. Does the - same as the C ``sizeof()`` function. + Returns the size in bytes of a ctypes type or instance memory buffer. + Does the same as the C ``sizeof`` operator. .. function:: string_at(address[, size]) @@ -2339,7 +2357,7 @@ These are the fundamental ctypes data types: .. class:: c_bool Represent the C :c:type:`bool` datatype (more accurately, :c:type:`_Bool` from - C99). Its value can be True or False, and the constructor accepts any object + C99). Its value can be ``True`` or ``False``, and the constructor accepts any object that has a truth value. .. versionadded:: 2.6 diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst index ecdac9a..642d25b 100644 --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -48,7 +48,7 @@ Linux and the BSD variants of Unix. Tutorial material on using curses with Python, by Andrew Kuchling and Eric Raymond. - The :file:`Demo/curses/` directory in the Python source distribution contains + The :source:`Demo/curses/` directory in the Python source distribution contains some example programs using the curses bindings provided by this module. @@ -380,7 +380,8 @@ The module :mod:`curses` defines the following functions: is to be displayed. -.. function:: newwin([nlines, ncols,] begin_y, begin_x) +.. function:: newwin(nlines, ncols) + newwin(nlines, ncols, begin_y, begin_x) Return a new window, whose left-upper corner is at ``(begin_y, begin_x)``, and whose height/width is *nlines*/*ncols*. @@ -648,7 +649,8 @@ Window objects, as returned by :func:`initscr` and :func:`newwin` above, have the following methods: -.. method:: window.addch([y, x,] ch[, attr]) +.. method:: window.addch(ch[, attr]) + window.addch(y, x, ch[, attr]) .. note:: @@ -662,13 +664,15 @@ the following methods: position and attributes are the current settings for the window object. -.. method:: window.addnstr([y, x,] str, n[, attr]) +.. method:: window.addnstr(str, n[, attr]) + window.addnstr(y, x, str, n[, attr]) Paint at most *n* characters of the string *str* at ``(y, x)`` with attributes *attr*, overwriting anything previously on the display. -.. method:: window.addstr([y, x,] str[, attr]) +.. method:: window.addstr(str[, attr]) + window.addstr(y, x, str[, attr]) Paint the string *str* at ``(y, x)`` with attributes *attr*, overwriting anything previously on the display. @@ -755,7 +759,10 @@ the following methods: *bs* are *horch*. The default corner characters are always used by this function. -.. method:: window.chgat([y, x, ] [num,] attr) +.. method:: window.chgat(attr) + window.chgat(num, attr) + window.chgat(y, x, attr) + window.chgat(y, x, num, attr) Set the attributes of *num* characters at the current cursor position, or at position ``(y, x)`` if supplied. If no value of *num* is given or *num* = -1, @@ -804,7 +811,8 @@ the following methods: Delete the line under the cursor. All following lines are moved up by one line. -.. method:: window.derwin([nlines, ncols,] begin_y, begin_x) +.. method:: window.derwin(begin_y, begin_x) + window.derwin(nlines, ncols, begin_y, begin_x) An abbreviation for "derive window", :meth:`derwin` is the same as calling :meth:`subwin`, except that *begin_y* and *begin_x* are relative to the origin @@ -879,7 +887,8 @@ the following methods: upper-left corner. -.. method:: window.hline([y, x,] ch, n) +.. method:: window.hline(ch, n) + window.hline(y, x, ch, n) Display a horizontal line starting at ``(y, x)`` with length *n* consisting of the character *ch*. @@ -913,7 +922,8 @@ the following methods: the character proper, and upper bits are the attributes. -.. method:: window.insch([y, x,] ch[, attr]) +.. method:: window.insch(ch[, attr]) + window.insch(y, x, ch[, attr]) Paint character *ch* at ``(y, x)`` with attributes *attr*, moving the line from position *x* right by one character. @@ -934,7 +944,8 @@ the following methods: line. -.. method:: window.insnstr([y, x,] str, n [, attr]) +.. method:: window.insnstr(str, n[, attr]) + window.insnstr(y, x, str, n[, attr]) Insert a character string (as many characters as will fit on the line) before the character under the cursor, up to *n* characters. If *n* is zero or @@ -943,7 +954,8 @@ the following methods: The cursor position does not change (after moving to *y*, *x*, if specified). -.. method:: window.insstr([y, x, ] str [, attr]) +.. method:: window.insstr(str[, attr]) + window.insstr(y, x, str[, attr]) Insert a character string (as many characters as will fit on the line) before the character under the cursor. All characters to the right of the cursor are @@ -951,7 +963,8 @@ the following methods: position does not change (after moving to *y*, *x*, if specified). -.. method:: window.instr([y, x] [, n]) +.. method:: window.instr([n]) + window.instr(y, x[, n]) Return a string of characters, extracted from the window starting at the current cursor position, or at *y*, *x* if specified. Attributes are stripped @@ -1126,13 +1139,15 @@ the following methods: Turn on attribute *A_STANDOUT*. -.. method:: window.subpad([nlines, ncols,] begin_y, begin_x) +.. method:: window.subpad(begin_y, begin_x) + window.subpad(nlines, ncols, begin_y, begin_x) Return a sub-window, whose upper-left corner is at ``(begin_y, begin_x)``, and whose width/height is *ncols*/*nlines*. -.. method:: window.subwin([nlines, ncols,] begin_y, begin_x) +.. method:: window.subwin(begin_y, begin_x) + window.subwin(nlines, ncols, begin_y, begin_x) Return a sub-window, whose upper-left corner is at ``(begin_y, begin_x)``, and whose width/height is *ncols*/*nlines*. @@ -1189,7 +1204,8 @@ the following methods: :meth:`refresh`. -.. method:: window.vline([y, x,] ch, n) +.. method:: window.vline(ch, n) + window.vline(y, x, ch, n) Display a vertical line starting at ``(y, x)`` with length *n* consisting of the character *ch*. diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst index f3b0870..db95269 100644 --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -14,27 +14,34 @@ The :mod:`datetime` module supplies classes for manipulating dates and times in both simple and complex ways. While date and time arithmetic is supported, the focus of the implementation is on efficient attribute extraction for output -formatting and manipulation. For related -functionality, see also the :mod:`time` and :mod:`calendar` modules. - -There are two kinds of date and time objects: "naive" and "aware". This -distinction refers to whether the object has any notion of time zone, daylight -saving time, or other kind of algorithmic or political time adjustment. Whether -a naive :class:`.datetime` object represents Coordinated Universal Time (UTC), -local time, or time in some other timezone is purely up to the program, just -like it's up to the program whether a particular number represents metres, -miles, or mass. Naive :class:`.datetime` objects are easy to understand and to -work with, at the cost of ignoring some aspects of reality. - -For applications requiring more, :class:`.datetime` and :class:`.time` objects -have an optional time zone information attribute, :attr:`tzinfo`, that can be -set to an instance of a subclass of the abstract :class:`tzinfo` class. These -:class:`tzinfo` objects capture information about the offset from UTC time, the -time zone name, and whether Daylight Saving Time is in effect. Note that no -concrete :class:`tzinfo` classes are supplied by the :mod:`datetime` module. -Supporting timezones at whatever level of detail is required is up to the -application. The rules for time adjustment across the world are more political -than rational, and there is no standard suitable for every application. +formatting and manipulation. For related functionality, see also the +:mod:`time` and :mod:`calendar` modules. + +There are two kinds of date and time objects: "naive" and "aware". + +An aware object has sufficient knowledge of applicable algorithmic and +political time adjustments, such as time zone and daylight saving time +information, to locate itself relative to other aware objects. An aware object +is used to represent a specific moment in time that is not open to +interpretation [#]_. + +A naive object does not contain enough information to unambiguously locate +itself relative to other date/time objects. Whether a naive object represents +Coordinated Universal Time (UTC), local time, or time in some other timezone is +purely up to the program, just like it's up to the program whether a particular +number represents metres, miles, or mass. Naive objects are easy to understand +and to work with, at the cost of ignoring some aspects of reality. + +For applications requiring aware objects, :class:`.datetime` and :class:`.time` +objects have an optional time zone information attribute, :attr:`tzinfo`, that +can be set to an instance of a subclass of the abstract :class:`tzinfo` class. +These :class:`tzinfo` objects capture information about the offset from UTC +time, the time zone name, and whether Daylight Saving Time is in effect. Note +that no concrete :class:`tzinfo` classes are supplied by the :mod:`datetime` +module. Supporting timezones at whatever level of detail is required is up to +the application. The rules for time adjustment across the world are more +political than rational, and there is no standard suitable for every +application. The :mod:`datetime` module exports the following constants: @@ -105,10 +112,13 @@ Objects of these types are immutable. Objects of the :class:`date` type are always naive. -An object *d* of type :class:`.time` or :class:`.datetime` may be naive or aware. -*d* is aware if ``d.tzinfo`` is not ``None`` and ``d.tzinfo.utcoffset(d)`` does -not return ``None``. If ``d.tzinfo`` is ``None``, or if ``d.tzinfo`` is not -``None`` but ``d.tzinfo.utcoffset(d)`` returns ``None``, *d* is naive. +An object of type :class:`.time` or :class:`.datetime` may be naive or aware. +A :class:`.datetime` object *d* is aware if ``d.tzinfo`` is not ``None`` and +``d.tzinfo.utcoffset(d)`` does not return ``None``. If ``d.tzinfo`` is +``None``, or if ``d.tzinfo`` is not ``None`` but ``d.tzinfo.utcoffset(d)`` +returns ``None``, *d* is naive. A :class:`.time` object *t* is aware +if ``t.tzinfo`` is not ``None`` and ``t.tzinfo.utcoffset(None)`` does not return +``None``. Otherwise, *t* is naive. The distinction between naive and aware doesn't apply to :class:`timedelta` objects. @@ -541,8 +551,16 @@ Instance methods: .. method:: date.strftime(format) Return a string representing the date, controlled by an explicit format string. - Format codes referring to hours, minutes or seconds will see 0 values. See - section :ref:`strftime-strptime-behavior`. + Format codes referring to hours, minutes or seconds will see 0 values. For a + complete list of formatting directives, see section + :ref:`strftime-strptime-behavior`. + + +.. method:: date.__format__(format) + + Same as :meth:`.date.strftime`. This makes it possible to specify format + string for a :class:`.date` object when using :meth:`str.format`. + See section :ref:`strftime-strptime-behavior`. Example of counting days to an event:: @@ -595,6 +613,8 @@ Example of working with :class:`date`: '11/03/02' >>> d.strftime("%A %d. %B %Y") 'Monday 11. March 2002' + >>> 'The {1} is {0:%d}, the {2} is {0:%B}.'.format(d, "day", "month") + 'The day is 11, the month is March.' .. _datetime-datetime: @@ -711,7 +731,8 @@ Other constructors, all class methods: *format*. This is equivalent to ``datetime(*(time.strptime(date_string, format)[0:6]))``. :exc:`ValueError` is raised if the date_string and format can't be parsed by :func:`time.strptime` or if it returns a value which isn't a - time tuple. See section :ref:`strftime-strptime-behavior`. + time tuple. For a complete list of formatting directives, see section + :ref:`strftime-strptime-behavior`. .. versionadded:: 2.5 @@ -1031,7 +1052,15 @@ Instance methods: .. method:: datetime.strftime(format) Return a string representing the date and time, controlled by an explicit format - string. See section :ref:`strftime-strptime-behavior`. + string. For a complete list of formatting directives, see section + :ref:`strftime-strptime-behavior`. + + +.. method:: datetime.__format__(format) + + Same as :meth:`.datetime.strftime`. This makes it possible to specify format + string for a :class:`.datetime` object when using :meth:`str.format`. + See section :ref:`strftime-strptime-behavior`. Examples of working with datetime objects: @@ -1078,19 +1107,21 @@ Examples of working with datetime objects: >>> # Formatting datetime >>> dt.strftime("%A, %d. %B %Y %I:%M%p") 'Tuesday, 21. November 2006 04:30PM' + >>> 'The {1} is {0:%d}, the {2} is {0:%B}, the {3} is {0:%I:%M%p}.'.format(dt, "day", "month", "time") + 'The day is 21, the month is November, the time is 04:30PM.' Using datetime with tzinfo: >>> from datetime import timedelta, datetime, tzinfo >>> class GMT1(tzinfo): - ... def __init__(self): # DST starts last Sunday in March + ... def utcoffset(self, dt): + ... return timedelta(hours=1) + self.dst(dt) + ... def dst(self, dt): + ... # DST starts last Sunday in March ... d = datetime(dt.year, 4, 1) # ends last Sunday in October ... self.dston = d - timedelta(days=d.weekday() + 1) ... d = datetime(dt.year, 11, 1) ... self.dstoff = d - timedelta(days=d.weekday() + 1) - ... def utcoffset(self, dt): - ... return timedelta(hours=1) + self.dst(dt) - ... def dst(self, dt): ... if self.dston <= dt.replace(tzinfo=None) < self.dstoff: ... return timedelta(hours=1) ... else: @@ -1099,16 +1130,15 @@ Using datetime with tzinfo: ... return "GMT +1" ... >>> class GMT2(tzinfo): - ... def __init__(self): + ... def utcoffset(self, dt): + ... return timedelta(hours=2) + self.dst(dt) + ... def dst(self, dt): ... d = datetime(dt.year, 4, 1) ... self.dston = d - timedelta(days=d.weekday() + 1) ... d = datetime(dt.year, 11, 1) ... self.dstoff = d - timedelta(days=d.weekday() + 1) - ... def utcoffset(self, dt): - ... return timedelta(hours=1) + self.dst(dt) - ... def dst(self, dt): ... if self.dston <= dt.replace(tzinfo=None) < self.dstoff: - ... return timedelta(hours=2) + ... return timedelta(hours=1) ... else: ... return timedelta(0) ... def tzname(self,dt): @@ -1145,7 +1175,7 @@ Using datetime with tzinfo: A time object represents a (local) time of day, independent of any particular day, and subject to adjustment via a :class:`tzinfo` object. -.. class:: time(hour[, minute[, second[, microsecond[, tzinfo]]]]) +.. class:: time([hour[, minute[, second[, microsecond[, tzinfo]]]]]) All arguments are optional. *tzinfo* may be ``None``, or an instance of a :class:`tzinfo` subclass. The remaining arguments may be ints or longs, in the @@ -1256,6 +1286,14 @@ Instance methods: .. method:: time.strftime(format) Return a string representing the time, controlled by an explicit format string. + For a complete list of formatting directives, see section + :ref:`strftime-strptime-behavior`. + + +.. method:: time.__format__(format) + + Same as :meth:`.time.strftime`. This makes it possible to specify format string + for a :class:`.time` object when using :meth:`str.format`. See section :ref:`strftime-strptime-behavior`. @@ -1305,6 +1343,8 @@ Example: 'Europe/Prague' >>> t.strftime("%H:%M:%S %Z") '12:10:30 Europe/Prague' + >>> 'The {} is {:%H:%M}.'.format("time", t) + 'The time is 12:10.' .. _datetime-tzinfo: @@ -1521,6 +1561,21 @@ Applications that can't bear such ambiguities should avoid using hybrid other fixed-offset :class:`tzinfo` subclass (such as a class representing only EST (fixed offset -5 hours), or only EDT (fixed offset -4 hours)). +.. seealso:: + + `pytz <http://pypi.python.org/pypi/pytz/>`_ + The standard library has no :class:`tzinfo` instances, but + there exists a third-party library which brings the *IANA timezone + database* (also known as the Olson database) to Python: *pytz*. + + *pytz* contains up-to-date information and its usage is recommended. + + `IANA timezone database <http://www.iana.org/time-zones>`_ + The Time Zone Database (often called tz or zoneinfo) contains code and + data that represent the history of local time for many representative + locations around the globe. It is updated periodically to reflect changes + made by political bodies to time zone boundaries, UTC offsets, and + daylight-saving rules. .. _strftime-strptime-behavior: @@ -1546,30 +1601,10 @@ For :class:`date` objects, the format codes for hours, minutes, seconds, and microseconds should not be used, as :class:`date` objects have no such values. If they're used anyway, ``0`` is substituted for them. -.. versionadded:: 2.6 - :class:`.time` and :class:`.datetime` objects support a ``%f`` format code - which expands to the number of microseconds in the object, zero-padded on - the left to six places. - -For a naive object, the ``%z`` and ``%Z`` format codes are replaced by empty -strings. - -For an aware object: - -``%z`` - :meth:`utcoffset` is transformed into a 5-character string of the form +HHMM or - -HHMM, where HH is a 2-digit string giving the number of UTC offset hours, and - MM is a 2-digit string giving the number of UTC offset minutes. For example, if - :meth:`utcoffset` returns ``timedelta(hours=-3, minutes=-30)``, ``%z`` is - replaced with the string ``'-0330'``. - -``%Z`` - If :meth:`tzname` returns ``None``, ``%Z`` is replaced by an empty string. - Otherwise ``%Z`` is replaced by the returned value, which must be a string. - The full set of format codes supported varies across platforms, because Python calls the platform C library's :func:`strftime` function, and platform -variations are common. +variations are common. To see the full set of format codes supported on your +platform, consult the :manpage:`strftime(3)` documentation. The following is a list of all the format codes that the C standard (1989 version) requires, and these work on all platforms with a standard C @@ -1579,116 +1614,156 @@ format codes. The exact range of years for which :meth:`strftime` works also varies across platforms. Regardless of platform, years before 1900 cannot be used. -+-----------+--------------------------------+-------+ -| Directive | Meaning | Notes | -+===========+================================+=======+ -| ``%a`` | Locale's abbreviated weekday | | -| | name. | | -+-----------+--------------------------------+-------+ -| ``%A`` | Locale's full weekday name. | | -+-----------+--------------------------------+-------+ -| ``%b`` | Locale's abbreviated month | | -| | name. | | -+-----------+--------------------------------+-------+ -| ``%B`` | Locale's full month name. | | -+-----------+--------------------------------+-------+ -| ``%c`` | Locale's appropriate date and | | -| | time representation. | | -+-----------+--------------------------------+-------+ -| ``%d`` | Day of the month as a decimal | | -| | number [01,31]. | | -+-----------+--------------------------------+-------+ -| ``%f`` | Microsecond as a decimal | \(1) | -| | number [0,999999], zero-padded | | -| | on the left | | -+-----------+--------------------------------+-------+ -| ``%H`` | Hour (24-hour clock) as a | | -| | decimal number [00,23]. | | -+-----------+--------------------------------+-------+ -| ``%I`` | Hour (12-hour clock) as a | | -| | decimal number [01,12]. | | -+-----------+--------------------------------+-------+ -| ``%j`` | Day of the year as a decimal | | -| | number [001,366]. | | -+-----------+--------------------------------+-------+ -| ``%m`` | Month as a decimal number | | -| | [01,12]. | | -+-----------+--------------------------------+-------+ -| ``%M`` | Minute as a decimal number | | -| | [00,59]. | | -+-----------+--------------------------------+-------+ -| ``%p`` | Locale's equivalent of either | \(2) | -| | AM or PM. | | -+-----------+--------------------------------+-------+ -| ``%S`` | Second as a decimal number | \(3) | -| | [00,61]. | | -+-----------+--------------------------------+-------+ -| ``%U`` | Week number of the year | \(4) | -| | (Sunday as the first day of | | -| | the week) as a decimal number | | -| | [00,53]. All days in a new | | -| | year preceding the first | | -| | Sunday are considered to be in | | -| | week 0. | | -+-----------+--------------------------------+-------+ -| ``%w`` | Weekday as a decimal number | | -| | [0(Sunday),6]. | | -+-----------+--------------------------------+-------+ -| ``%W`` | Week number of the year | \(4) | -| | (Monday as the first day of | | -| | the week) as a decimal number | | -| | [00,53]. All days in a new | | -| | year preceding the first | | -| | Monday are considered to be in | | -| | week 0. | | -+-----------+--------------------------------+-------+ -| ``%x`` | Locale's appropriate date | | -| | representation. | | -+-----------+--------------------------------+-------+ -| ``%X`` | Locale's appropriate time | | -| | representation. | | -+-----------+--------------------------------+-------+ -| ``%y`` | Year without century as a | | -| | decimal number [00,99]. | | -+-----------+--------------------------------+-------+ -| ``%Y`` | Year with century as a decimal | | -| | number. | | -+-----------+--------------------------------+-------+ -| ``%z`` | UTC offset in the form +HHMM | \(5) | -| | or -HHMM (empty string if the | | -| | the object is naive). | | -+-----------+--------------------------------+-------+ -| ``%Z`` | Time zone name (empty string | | -| | if the object is naive). | | -+-----------+--------------------------------+-------+ -| ``%%`` | A literal ``'%'`` character. | | -+-----------+--------------------------------+-------+ ++-----------+--------------------------------+------------------------+-------+ +| Directive | Meaning | Example | Notes | ++===========+================================+========================+=======+ +| ``%a`` | Weekday as locale's || Sun, Mon, ..., Sat | \(1) | +| | abbreviated name. | (en_US); | | +| | || So, Mo, ..., Sa | | +| | | (de_DE) | | ++-----------+--------------------------------+------------------------+-------+ +| ``%A`` | Weekday as locale's full name. || Sunday, Monday, ..., | \(1) | +| | | Saturday (en_US); | | +| | || Sonntag, Montag, ..., | | +| | | Samstag (de_DE) | | ++-----------+--------------------------------+------------------------+-------+ +| ``%w`` | Weekday as a decimal number, | 0, 1, ..., 6 | | +| | where 0 is Sunday and 6 is | | | +| | Saturday. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%d`` | Day of the month as a | 01, 02, ..., 31 | | +| | zero-padded decimal number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%b`` | Month as locale's abbreviated || Jan, Feb, ..., Dec | \(1) | +| | name. | (en_US); | | +| | || Jan, Feb, ..., Dez | | +| | | (de_DE) | | ++-----------+--------------------------------+------------------------+-------+ +| ``%B`` | Month as locale's full name. || January, February, | \(1) | +| | | ..., December (en_US);| | +| | || Januar, Februar, ..., | | +| | | Dezember (de_DE) | | ++-----------+--------------------------------+------------------------+-------+ +| ``%m`` | Month as a zero-padded | 01, 02, ..., 12 | | +| | decimal number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%y`` | Year without century as a | 00, 01, ..., 99 | | +| | zero-padded decimal number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%Y`` | Year with century as a decimal | 1970, 1988, 2001, 2013 | | +| | number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%H`` | Hour (24-hour clock) as a | 00, 01, ..., 23 | | +| | zero-padded decimal number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%I`` | Hour (12-hour clock) as a | 01, 02, ..., 12 | | +| | zero-padded decimal number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%p`` | Locale's equivalent of either || AM, PM (en_US); | \(1), | +| | AM or PM. || am, pm (de_DE) | \(2) | ++-----------+--------------------------------+------------------------+-------+ +| ``%M`` | Minute as a zero-padded | 00, 01, ..., 59 | | +| | decimal number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%S`` | Second as a zero-padded | 00, 01, ..., 59 | \(3) | +| | decimal number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%f`` | Microsecond as a decimal | 000000, 000001, ..., | \(4) | +| | number, zero-padded on the | 999999 | | +| | left. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%z`` | UTC offset in the form +HHMM | (empty), +0000, -0400, | \(5) | +| | or -HHMM (empty string if the | +1030 | | +| | the object is naive). | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%Z`` | Time zone name (empty string | (empty), UTC, EST, CST | | +| | if the object is naive). | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%j`` | Day of the year as a | 001, 002, ..., 366 | | +| | zero-padded decimal number. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%U`` | Week number of the year | 00, 01, ..., 53 | \(6) | +| | (Sunday as the first day of | | | +| | the week) as a zero padded | | | +| | decimal number. All days in a | | | +| | new year preceding the first | | | +| | Sunday are considered to be in | | | +| | week 0. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%W`` | Week number of the year | 00, 01, ..., 53 | \(6) | +| | (Monday as the first day of | | | +| | the week) as a decimal number. | | | +| | All days in a new year | | | +| | preceding the first Monday | | | +| | are considered to be in | | | +| | week 0. | | | ++-----------+--------------------------------+------------------------+-------+ +| ``%c`` | Locale's appropriate date and || Tue Aug 16 21:30:00 | \(1) | +| | time representation. | 1988 (en_US); | | +| | || Di 16 Aug 21:30:00 | | +| | | 1988 (de_DE) | | ++-----------+--------------------------------+------------------------+-------+ +| ``%x`` | Locale's appropriate date || 08/16/88 (None); | \(1) | +| | representation. || 08/16/1988 (en_US); | | +| | || 16.08.1988 (de_DE) | | ++-----------+--------------------------------+------------------------+-------+ +| ``%X`` | Locale's appropriate time || 21:30:00 (en_US); | \(1) | +| | representation. || 21:30:00 (de_DE) | | ++-----------+--------------------------------+------------------------+-------+ +| ``%%`` | A literal ``'%'`` character. | % | | ++-----------+--------------------------------+------------------------+-------+ Notes: (1) - When used with the :meth:`strptime` method, the ``%f`` directive - accepts from one to six digits and zero pads on the right. ``%f`` is - an extension to the set of format characters in the C standard (but - implemented separately in datetime objects, and therefore always - available). + Because the format depends on the current locale, care should be taken when + making assumptions about the output value. Field orderings will vary (for + example, "month/day/year" versus "day/month/year"), and the output may + contain Unicode characters encoded using the locale's default encoding (for + example, if the current locale is ``ja_JP``, the default encoding could be + any one of ``eucJP``, ``SJIS``, or ``utf-8``; use :meth:`locale.getlocale` + to determine the current locale's encoding). (2) When used with the :meth:`strptime` method, the ``%p`` directive only affects the output hour field if the ``%I`` directive is used to parse the hour. (3) - The range really is ``0`` to ``61``; according to the Posix standard this - accounts for leap seconds and the (very rare) double leap seconds. - The :mod:`time` module may produce and does accept leap seconds since - it is based on the Posix standard, but the :mod:`datetime` module - does not accept leap seconds in :meth:`strptime` input nor will it - produce them in :func:`strftime` output. + Unlike the :mod:`time` module, the :mod:`datetime` module does not support + leap seconds. (4) - When used with the :meth:`strptime` method, ``%U`` and ``%W`` are only used in - calculations when the day of the week and the year are specified. + ``%f`` is an extension to the set of format characters in the C standard + (but implemented separately in datetime objects, and therefore always + available). When used with the :meth:`strptime` method, the ``%f`` + directive accepts from one to six digits and zero pads on the right. + + .. versionadded:: 2.6 (5) - For example, if :meth:`utcoffset` returns ``timedelta(hours=-3, minutes=-30)``, - ``%z`` is replaced with the string ``'-0330'``. + For a naive object, the ``%z`` and ``%Z`` format codes are replaced by empty + strings. + + For an aware object: + + ``%z`` + :meth:`utcoffset` is transformed into a 5-character string of the form + +HHMM or -HHMM, where HH is a 2-digit string giving the number of UTC + offset hours, and MM is a 2-digit string giving the number of UTC offset + minutes. For example, if :meth:`utcoffset` returns + ``timedelta(hours=-3, minutes=-30)``, ``%z`` is replaced with the string + ``'-0330'``. + + ``%Z`` + If :meth:`tzname` returns ``None``, ``%Z`` is replaced by an empty + string. Otherwise ``%Z`` is replaced by the returned value, which must + be a string. + +(6) + When used with the :meth:`strptime` method, ``%U`` and ``%W`` are only used + in calculations when the day of the week and the year are specified. + + +.. rubric:: Footnotes + +.. [#] If, that is, we ignore the effects of Relativity diff --git a/Doc/library/dbhash.rst b/Doc/library/dbhash.rst index 0b440ab..ed965e1 100644 --- a/Doc/library/dbhash.rst +++ b/Doc/library/dbhash.rst @@ -6,7 +6,7 @@ .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> .. deprecated:: 2.6 - The :mod:`dbhash` module has been deprecated for removal in Python 3.0. + The :mod:`dbhash` module has been removed in Python 3. .. index:: module: bsddb diff --git a/Doc/library/dbm.rst b/Doc/library/dbm.rst index 8747789..a03c7c5 100644 --- a/Doc/library/dbm.rst +++ b/Doc/library/dbm.rst @@ -6,9 +6,9 @@ :synopsis: The standard "database" interface, based on ndbm. .. note:: - The :mod:`dbm` module has been renamed to :mod:`dbm.ndbm` in Python 3.0. The + The :mod:`dbm` module has been renamed to :mod:`dbm.ndbm` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. The :mod:`dbm` module provides an interface to the Unix "(n)dbm" library. Dbm @@ -64,6 +64,14 @@ The module defines the following: database has to be created. It defaults to octal ``0666`` (and will be modified by the prevailing umask). + In addition to the dictionary-like methods, ``dbm`` objects + provide the following method: + + + .. function:: close() + + Close the ``dbm`` database. + .. seealso:: diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst index f9e70de..d5ba4f0 100644 --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -375,6 +375,29 @@ Decimal objects compared, sorted, and coerced to another type (such as :class:`float` or :class:`long`). + There are some small differences between arithmetic on Decimal objects and + arithmetic on integers and floats. When the remainder operator ``%`` is + applied to Decimal objects, the sign of the result is the sign of the + *dividend* rather than the sign of the divisor:: + + >>> (-7) % 4 + 1 + >>> Decimal(-7) % Decimal(4) + Decimal('-3') + + The integer division operator ``//`` behaves analogously, returning the + integer part of the true quotient (truncating towards zero) rather than its + floor, so as to preserve the usual identity ``x == (x // y) * y + x % y``:: + + >>> -7 // 4 + -2 + >>> Decimal(-7) // Decimal(4) + Decimal('-1') + + The ``%`` and ``//`` operators implement the ``remainder`` and + ``divide-integer`` operations (respectively) as described in the + specification. + Decimal objects cannot generally be combined with floats in arithmetic operations: an attempt to add a :class:`Decimal` to a :class:`float`, for example, will raise a :exc:`TypeError`. @@ -802,12 +825,21 @@ Decimal objects .. method:: remainder_near(other[, context]) - Compute the modulo as either a positive or negative value depending on - which is closest to zero. For instance, ``Decimal(10).remainder_near(6)`` - returns ``Decimal('-2')`` which is closer to zero than ``Decimal('4')``. + Return the remainder from dividing *self* by *other*. This differs from + ``self % other`` in that the sign of the remainder is chosen so as to + minimize its absolute value. More precisely, the return value is + ``self - n * other`` where ``n`` is the integer nearest to the exact + value of ``self / other``, and if two integers are equally near then the + even one is chosen. + + If the result is zero then its sign will be the sign of *self*. - If both are equally close, the one chosen will have the same sign as - *self*. + >>> Decimal(18).remainder_near(Decimal(10)) + Decimal('-2') + >>> Decimal(25).remainder_near(Decimal(10)) + Decimal('5') + >>> Decimal(35).remainder_near(Decimal(10)) + Decimal('-5') .. method:: rotate(other[, context]) @@ -944,6 +976,10 @@ the :func:`localcontext` function to temporarily change the active context. s = calculate_something() s = +s # Round the final result back to the default precision + with localcontext(BasicContext): # temporarily use the BasicContext + print Decimal(1) / Decimal(7) + print Decimal(355) / Decimal(113) + New contexts can also be created using the :class:`Context` constructor described below. In addition, the module provides three pre-made contexts: @@ -1190,52 +1226,52 @@ In addition to the three supplied contexts, new contexts can be created with the .. method:: is_canonical(x) - Returns True if *x* is canonical; otherwise returns False. + Returns ``True`` if *x* is canonical; otherwise returns ``False``. .. method:: is_finite(x) - Returns True if *x* is finite; otherwise returns False. + Returns ``True`` if *x* is finite; otherwise returns ``False``. .. method:: is_infinite(x) - Returns True if *x* is infinite; otherwise returns False. + Returns ``True`` if *x* is infinite; otherwise returns ``False``. .. method:: is_nan(x) - Returns True if *x* is a qNaN or sNaN; otherwise returns False. + Returns ``True`` if *x* is a qNaN or sNaN; otherwise returns ``False``. .. method:: is_normal(x) - Returns True if *x* is a normal number; otherwise returns False. + Returns ``True`` if *x* is a normal number; otherwise returns ``False``. .. method:: is_qnan(x) - Returns True if *x* is a quiet NaN; otherwise returns False. + Returns ``True`` if *x* is a quiet NaN; otherwise returns ``False``. .. method:: is_signed(x) - Returns True if *x* is negative; otherwise returns False. + Returns ``True`` if *x* is negative; otherwise returns ``False``. .. method:: is_snan(x) - Returns True if *x* is a signaling NaN; otherwise returns False. + Returns ``True`` if *x* is a signaling NaN; otherwise returns ``False``. .. method:: is_subnormal(x) - Returns True if *x* is subnormal; otherwise returns False. + Returns ``True`` if *x* is subnormal; otherwise returns ``False``. .. method:: is_zero(x) - Returns True if *x* is a zero; otherwise returns False. + Returns ``True`` if *x* is a zero; otherwise returns ``False``. .. method:: ln(x) @@ -1395,7 +1431,7 @@ In addition to the three supplied contexts, new contexts can be created with the .. method:: same_quantum(x, y) - Returns True if the two operands have the same exponent. + Returns ``True`` if the two operands have the same exponent. .. method:: scaleb (x, y) diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index 225b486..878d8e6 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -84,7 +84,7 @@ diffs. For comparing directories and files, see also, the :mod:`filecmp` module. The constructor for this class is: - .. function:: __init__([tabsize][, wrapcolumn][, linejunk][, charjunk]) + .. function:: __init__(tabsize=8, wrapcolumn=None, linejunk=None, charjunk=IS_CHARACTER_JUNK) Initializes instance of :class:`HtmlDiff`. @@ -344,7 +344,7 @@ SequenceMatcher Objects The :class:`SequenceMatcher` class has this constructor: -.. class:: SequenceMatcher([isjunk[, a[, b[, autojunk=True]]]]) +.. class:: SequenceMatcher(isjunk=None, a='', b='', autojunk=True) Optional argument *isjunk* must be ``None`` (the default) or a one-argument function that takes a sequence element and returns true if and only if the @@ -632,10 +632,12 @@ The :class:`Differ` class has this constructor: Compare two sequences of lines, and generate the delta (a sequence of lines). - Each sequence must contain individual single-line strings ending with newlines. - Such sequences can be obtained from the :meth:`readlines` method of file-like - objects. The delta generated also consists of newline-terminated strings, ready - to be printed as-is via the :meth:`writelines` method of a file-like object. + Each sequence must contain individual single-line strings ending with + newlines. Such sequences can be obtained from the + :meth:`~file.readlines` method of file-like objects. The delta + generated also consists of newline-terminated strings, ready to be + printed as-is via the :meth:`~file.writelines` method of a + file-like object. .. _differ-examples: @@ -645,7 +647,7 @@ Differ Example This example compares two texts. First we set up the texts, sequences of individual single-line strings ending with newlines (such sequences can also be -obtained from the :meth:`readlines` method of file-like objects): +obtained from the :meth:`~file.readlines` method of file-like objects): >>> text1 = ''' 1. Beautiful is better than ugly. ... 2. Explicit is better than implicit. diff --git a/Doc/library/dircache.rst b/Doc/library/dircache.rst index 71a8abe..632ddd5 100644 --- a/Doc/library/dircache.rst +++ b/Doc/library/dircache.rst @@ -7,7 +7,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`dircache` module has been removed in Python 3.0. + The :mod:`dircache` module has been removed in Python 3. .. sectionauthor:: Moshe Zadka <moshez@zadka.site.co.il> diff --git a/Doc/library/dl.rst b/Doc/library/dl.rst index 13510c5..40556cc 100644 --- a/Doc/library/dl.rst +++ b/Doc/library/dl.rst @@ -8,7 +8,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`dl` module has been removed in Python 3.0. Use the :mod:`ctypes` + The :mod:`dl` module has been removed in Python 3. Use the :mod:`ctypes` module instead. .. sectionauthor:: Moshe Zadka <moshez@zadka.site.co.il> diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst index b6d1756..a1e270d 100644 --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -1,3 +1,5 @@ +:keepdoctest: + :mod:`doctest` --- Test interactive Python examples =================================================== @@ -339,7 +341,8 @@ The fine print: Tabs in output generated by the tested code are not modified. Because any hard tabs in the sample output *are* expanded, this means that if the code output includes hard tabs, the only way the doctest can pass is if the - :const:`NORMALIZE_WHITESPACE` option or directive is in effect. + :const:`NORMALIZE_WHITESPACE` option or :ref:`directive <doctest-directives>` + is in effect. Alternatively, the test can be rewritten to capture the output and compare it to an expected value as part of the test. This handling of tabs in the source was arrived at through trial and error, and has proven to be the least @@ -363,7 +366,7 @@ The fine print: Backslashes in a raw docstring: m\n Otherwise, the backslash will be interpreted as part of the string. For example, - the "\\" above would be interpreted as a newline character. Alternatively, you + the ``\n`` above would be interpreted as a newline character. Alternatively, you can double each backslash in the doctest version (and not use a raw string):: >>> def f(x): @@ -511,15 +514,16 @@ Some details you should read once, but won't need to remember: SyntaxError: invalid syntax +.. _option-flags-and-directives: .. _doctest-options: -Option Flags and Directives -^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Option Flags +^^^^^^^^^^^^ A number of option flags control various aspects of doctest's behavior. Symbolic names for the flags are supplied as module constants, which can be or'ed together and passed to various functions. The names can also be used in -doctest directives (see below). +:ref:`doctest directives <doctest-directives>`. The first group of options define test semantics, controlling aspects of how doctest decides whether actual output matches an example's expected output: @@ -573,14 +577,14 @@ doctest decides whether actual output matches an example's expected output: :exc:`TypeError` is raised. It will also ignore the module name used in Python 3 doctest reports. Hence - both these variations will work regardless of whether the test is run under - Python 2.7 or Python 3.2 (or later versions): + both of these variations will work with the flag specified, regardless of + whether the test is run under Python 2.7 or Python 3.2 (or later versions):: - >>> raise CustomError('message') #doctest: +IGNORE_EXCEPTION_DETAIL + >>> raise CustomError('message') Traceback (most recent call last): CustomError: message - >>> raise CustomError('message') #doctest: +IGNORE_EXCEPTION_DETAIL + >>> raise CustomError('message') Traceback (most recent call last): my_module.CustomError: message @@ -590,15 +594,16 @@ doctest decides whether actual output matches an example's expected output: exception name. Using :const:`IGNORE_EXCEPTION_DETAIL` and the details from Python 2.3 is also the only clear way to write a doctest that doesn't care about the exception detail yet continues to pass under Python 2.3 or - earlier (those releases do not support doctest directives and ignore them - as irrelevant comments). For example, :: + earlier (those releases do not support :ref:`doctest directives + <doctest-directives>` and ignore them as irrelevant comments). For example:: - >>> (1, 2)[3] = 'moo' #doctest: +IGNORE_EXCEPTION_DETAIL + >>> (1, 2)[3] = 'moo' Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: object doesn't support item assignment - passes under Python 2.3 and later Python versions, even though the detail + passes under Python 2.3 and later Python versions with the flag specified, + even though the detail changed in Python 2.4 to say "does not" instead of "doesn't". .. versionchanged:: 2.7 @@ -662,9 +667,40 @@ The second group of options controls how test failures are reported: A bitmask or'ing together all the reporting flags above. -"Doctest directives" may be used to modify the option flags for individual -examples. Doctest directives are expressed as a special Python comment -following an example's source code: + +.. versionadded:: 2.4 + The constants + :const:`DONT_ACCEPT_BLANKLINE`, :const:`NORMALIZE_WHITESPACE`, + :const:`ELLIPSIS`, :const:`IGNORE_EXCEPTION_DETAIL`, :const:`REPORT_UDIFF`, + :const:`REPORT_CDIFF`, :const:`REPORT_NDIFF`, + :const:`REPORT_ONLY_FIRST_FAILURE`, :const:`COMPARISON_FLAGS` and + :const:`REPORTING_FLAGS` were added. + +There's also a way to register new option flag names, although this isn't useful +unless you intend to extend :mod:`doctest` internals via subclassing: + + +.. function:: register_optionflag(name) + + Create a new option flag with a given name, and return the new flag's integer + value. :func:`register_optionflag` can be used when subclassing + :class:`OutputChecker` or :class:`DocTestRunner` to create new options that are + supported by your subclasses. :func:`register_optionflag` should always be + called using the following idiom:: + + MY_FLAG = register_optionflag('MY_FLAG') + + .. versionadded:: 2.4 + + +.. _doctest-directives: + +Directives +^^^^^^^^^^ + +Doctest directives may be used to modify the :ref:`option flags +<doctest-options>` for an individual example. Doctest directives are +special Python comments following an example's source code: .. productionlist:: doctest directive: "#" "doctest:" `directive_options` @@ -682,7 +718,7 @@ example. Use ``+`` to enable the named behavior, or ``-`` to disable it. For example, this test passes:: - >>> print range(20) #doctest: +NORMALIZE_WHITESPACE + >>> print range(20) # doctest: +NORMALIZE_WHITESPACE [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] @@ -691,10 +727,11 @@ two blanks before the single-digit list elements, and because the actual output is on a single line. This test also passes, and also requires a directive to do so:: - >>> print range(20) # doctest:+ELLIPSIS + >>> print range(20) # doctest: +ELLIPSIS [0, 1, ..., 18, 19] -Multiple directives can be used on a single physical line, separated by commas:: +Multiple directives can be used on a single physical line, separated by +commas:: >>> print range(20) # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE [0, 1, ..., 18, 19] @@ -721,28 +758,7 @@ functions that run doctests, establishing different defaults. In such cases, disabling an option via ``-`` in a directive can be useful. .. versionadded:: 2.4 - Doctest directives and the associated constants - :const:`DONT_ACCEPT_BLANKLINE`, :const:`NORMALIZE_WHITESPACE`, - :const:`ELLIPSIS`, :const:`IGNORE_EXCEPTION_DETAIL`, :const:`REPORT_UDIFF`, - :const:`REPORT_CDIFF`, :const:`REPORT_NDIFF`, - :const:`REPORT_ONLY_FIRST_FAILURE`, :const:`COMPARISON_FLAGS` and - :const:`REPORTING_FLAGS` were added. - -There's also a way to register new option flag names, although this isn't useful -unless you intend to extend :mod:`doctest` internals via subclassing: - - -.. function:: register_optionflag(name) - - Create a new option flag with a given name, and return the new flag's integer - value. :func:`register_optionflag` can be used when subclassing - :class:`OutputChecker` or :class:`DocTestRunner` to create new options that are - supported by your subclasses. :func:`register_optionflag` should always be - called using the following idiom:: - - MY_FLAG = register_optionflag('MY_FLAG') - - .. versionadded:: 2.4 + Support for doctest directives was added. .. _doctest-warnings: @@ -1060,6 +1076,16 @@ from text files and modules with doctests: .. versionchanged:: 2.5 The parameter *encoding* was added. + .. note:: + Unlike :func:`testmod` and :class:`DocTestFinder`, this function raises + a :exc:`ValueError` if *module* contains no docstrings. You can prevent + this error by passing a :class:`DocTestFinder` instance as the + *test_finder* argument with its *exclude_empty* keyword argument set + to ``False``:: + + >>> finder = doctest.DocTestFinder(exclude_empty=False) + >>> suite = doctest.DocTestSuite(test_finder=finder) + .. function:: DocTestSuite([module][, globs][, extraglobs][, test_finder][, setUp][, tearDown][, checker]) diff --git a/Doc/library/docxmlrpcserver.rst b/Doc/library/docxmlrpcserver.rst index 67cb3b9..08e4e4b 100644 --- a/Doc/library/docxmlrpcserver.rst +++ b/Doc/library/docxmlrpcserver.rst @@ -8,8 +8,8 @@ .. note:: The :mod:`DocXMLRPCServer` module has been merged into :mod:`xmlrpc.server` - in Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + in Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. .. versionadded:: 2.3 diff --git a/Doc/library/dumbdbm.rst b/Doc/library/dumbdbm.rst index a511855..1a9a647 100644 --- a/Doc/library/dumbdbm.rst +++ b/Doc/library/dumbdbm.rst @@ -5,9 +5,9 @@ :synopsis: Portable implementation of the simple DBM interface. .. note:: - The :mod:`dumbdbm` module has been renamed to :mod:`dbm.dumb` in Python 3.0. + The :mod:`dumbdbm` module has been renamed to :mod:`dbm.dumb` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. .. index:: single: databases @@ -49,6 +49,14 @@ The module defines the following: .. versionchanged:: 2.2 The *mode* argument was ignored in earlier versions. +In addition to the dictionary-like methods, ``dumbdm`` objects +provide the following method: + + +.. function:: close() + + Close the ``dumbdm`` database. + .. seealso:: diff --git a/Doc/library/dummy_thread.rst b/Doc/library/dummy_thread.rst index a4dba86..a1d977d 100644 --- a/Doc/library/dummy_thread.rst +++ b/Doc/library/dummy_thread.rst @@ -6,8 +6,8 @@ .. note:: The :mod:`dummy_thread` module has been renamed to :mod:`_dummy_thread` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0; however, you should consider using the + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3; however, you should consider using the high-lever :mod:`dummy_threading` module instead. **Source code:** :source:`Lib/dummy_thread.py` diff --git a/Doc/library/email.charset.rst b/Doc/library/email.charset.rst index b008386..0ed6f0a 100644 --- a/Doc/library/email.charset.rst +++ b/Doc/library/email.charset.rst @@ -1,5 +1,5 @@ -:mod:`email`: Representing character sets ------------------------------------------ +:mod:`email.charset`: Representing character sets +------------------------------------------------- .. module:: email.charset :synopsis: Character Sets @@ -249,5 +249,5 @@ new entries to the global character set, alias, and codec registries: *charset* is the canonical name of a character set. *codecname* is the name of a Python codec, as appropriate for the second argument to the :func:`unicode` - built-in, or to the :meth:`encode` method of a Unicode string. + built-in, or to the :meth:`~unicode.encode` method of a Unicode string. diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst index 5421b9f..916ba5d 100644 --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -1,5 +1,5 @@ -:mod:`email`: Encoders ----------------------- +:mod:`email.encoders`: Encoders +------------------------------- .. module:: email.encoders :synopsis: Encoders for email message payloads. @@ -18,6 +18,10 @@ exactly one argument, the message object to encode. They usually extract the payload, encode it, and reset the payload to this newly encoded value. They should also set the :mailheader:`Content-Transfer-Encoding` header as appropriate. +Note that these functions are not meaningful for a multipart message. They +must be applied to individual subparts instead, and will raise a +:exc:`TypeError` if passed a message whose type is multipart. + Here are the encoding functions provided: diff --git a/Doc/library/email.errors.rst b/Doc/library/email.errors.rst index 06598d5..499d754 100644 --- a/Doc/library/email.errors.rst +++ b/Doc/library/email.errors.rst @@ -1,5 +1,5 @@ -:mod:`email`: Exception and Defect classes ------------------------------------------- +:mod:`email.errors`: Exception and Defect classes +------------------------------------------------- .. module:: email.errors :synopsis: The exception classes used by the email package. @@ -25,7 +25,8 @@ The following exception classes are defined in the :mod:`email.errors` module: Raised under some error conditions when parsing the :rfc:`2822` headers of a message, this class is derived from :exc:`MessageParseError`. It can be raised - from the :meth:`Parser.parse` or :meth:`Parser.parsestr` methods. + from the :meth:`Parser.parse <email.parser.Parser.parse>` or + :meth:`Parser.parsestr <email.parser.Parser.parsestr>` methods. Situations where it can be raised include finding an envelope header after the first :rfc:`2822` header of the message, finding a continuation line before the @@ -37,7 +38,8 @@ The following exception classes are defined in the :mod:`email.errors` module: Raised under some error conditions when parsing the :rfc:`2822` headers of a message, this class is derived from :exc:`MessageParseError`. It can be raised - from the :meth:`Parser.parse` or :meth:`Parser.parsestr` methods. + from the :meth:`Parser.parse <email.parser.Parser.parse>` or + :meth:`Parser.parsestr <email.parser.Parser.parsestr>` methods. Situations where it can be raised include not being able to find the starting or terminating boundary in a :mimetype:`multipart/\*` message when strict parsing @@ -46,19 +48,20 @@ The following exception classes are defined in the :mod:`email.errors` module: .. exception:: MultipartConversionError() - Raised when a payload is added to a :class:`Message` object using - :meth:`add_payload`, but the payload is already a scalar and the message's - :mailheader:`Content-Type` main type is not either :mimetype:`multipart` or - missing. :exc:`MultipartConversionError` multiply inherits from - :exc:`MessageError` and the built-in :exc:`TypeError`. + Raised when a payload is added to a :class:`~email.message.Message` object + using :meth:`add_payload`, but the payload is already a scalar and the + message's :mailheader:`Content-Type` main type is not either + :mimetype:`multipart` or missing. :exc:`MultipartConversionError` multiply + inherits from :exc:`MessageError` and the built-in :exc:`TypeError`. - Since :meth:`Message.add_payload` is deprecated, this exception is rarely raised - in practice. However the exception may also be raised if the :meth:`attach` + Since :meth:`Message.add_payload` is deprecated, this exception is rarely + raised in practice. However the exception may also be raised if the + :meth:`~email.message.Message.attach` method is called on an instance of a class derived from :class:`~email.mime.nonmultipart.MIMENonMultipart` (e.g. :class:`~email.mime.image.MIMEImage`). -Here's the list of the defects that the :class:`~email.mime.parser.FeedParser` +Here's the list of the defects that the :class:`~email.parser.FeedParser` can find while parsing messages. Note that the defects are added to the message where the problem was found, so for example, if a message nested inside a :mimetype:`multipart/alternative` had a malformed header, that nested message @@ -86,7 +89,7 @@ this class is *not* an exception! or was otherwise malformed. * :class:`MultipartInvariantViolationDefect` -- A message claimed to be a - :mimetype:`multipart`, but no subparts were found. Note that when a message has - this defect, its :meth:`is_multipart` method may return false even though its - content type claims to be :mimetype:`multipart`. + :mimetype:`multipart`, but no subparts were found. Note that when a message + has this defect, its :meth:`~email.message.Message.is_multipart` method may + return false even though its content type claims to be :mimetype:`multipart`. diff --git a/Doc/library/email.generator.rst b/Doc/library/email.generator.rst index f02e7d8..4ea7e6a 100644 --- a/Doc/library/email.generator.rst +++ b/Doc/library/email.generator.rst @@ -1,5 +1,5 @@ -:mod:`email`: Generating MIME documents ---------------------------------------- +:mod:`email.generator`: Generating MIME documents +------------------------------------------------- .. module:: email.generator :synopsis: Generate flat text email messages from a message structure. @@ -17,10 +17,10 @@ yourself. However the bundled generator knows how to generate most email in a standards-compliant way, should handle MIME and non-MIME email messages just fine, and is designed so that the transformation from flat text, to a message structure via the :class:`~email.parser.Parser` class, and back to flat text, -is idempotent (the input is identical to the output). On the other hand, using -the Generator on a :class:`~email.message.Message` constructed by program may -result in changes to the :class:`~email.message.Message` object as defaults are -filled in. +is idempotent (the input is identical to the output) [#]_. On the other hand, +using the Generator on a :class:`~email.message.Message` constructed by program +may result in changes to the :class:`~email.message.Message` object as defaults +are filled in. Here are the public methods of the :class:`Generator` class, imported from the :mod:`email.generator` module: @@ -125,3 +125,11 @@ representing the part. .. versionchanged:: 2.5 The previously deprecated method :meth:`__call__` was removed. + +.. rubric:: Footnotes + +.. [#] This statement assumes that you use the appropriate setting for the + ``unixfrom`` argument, and that you set maxheaderlen=0 (which will + preserve whatever the input line lengths were). It is also not strictly + true, since in many cases runs of whitespace in headers are collapsed + into single blanks. The latter is a bug that will eventually be fixed. diff --git a/Doc/library/email.header.rst b/Doc/library/email.header.rst index fe09de5..4e585fc 100644 --- a/Doc/library/email.header.rst +++ b/Doc/library/email.header.rst @@ -1,5 +1,5 @@ -:mod:`email`: Internationalized headers ---------------------------------------- +:mod:`email.header`: Internationalized headers +---------------------------------------------- .. module:: email.header :synopsis: Representing non-ASCII headers @@ -103,7 +103,7 @@ Here is the :class:`Header` class description: not provoke a :exc:`UnicodeError` is used. Optional *errors* is passed through to any :func:`unicode` or - :func:`ustr.encode` call, and defaults to "strict". + :meth:`unicode.encode` call, and defaults to "strict". .. method:: encode([splitchars]) diff --git a/Doc/library/email.iterators.rst b/Doc/library/email.iterators.rst index aa70141..6621d51 100644 --- a/Doc/library/email.iterators.rst +++ b/Doc/library/email.iterators.rst @@ -1,13 +1,14 @@ -:mod:`email`: Iterators ------------------------ +:mod:`email.iterators`: Iterators +--------------------------------- .. module:: email.iterators :synopsis: Iterate over a message object tree. Iterating over a message object tree is fairly easy with the -:meth:`Message.walk` method. The :mod:`email.iterators` module provides some -useful higher level iterations over message object trees. +:meth:`Message.walk <email.message.Message.walk>` method. The +:mod:`email.iterators` module provides some useful higher level iterations over +message object trees. .. function:: body_line_iterator(msg[, decode]) @@ -16,9 +17,11 @@ useful higher level iterations over message object trees. string payloads line-by-line. It skips over all the subpart headers, and it skips over any subpart with a payload that isn't a Python string. This is somewhat equivalent to reading the flat text representation of the message from - a file using :meth:`readline`, skipping over all the intervening headers. + a file using :meth:`~io.TextIOBase.readline`, skipping over all the + intervening headers. - Optional *decode* is passed through to :meth:`Message.get_payload`. + Optional *decode* is passed through to :meth:`Message.get_payload + <email.message.Message.get_payload>`. .. function:: typed_subpart_iterator(msg[, maintype[, subtype]]) diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst index 77ce99e..838bcf5 100644 --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -1,5 +1,5 @@ -:mod:`email`: Representing an email message -------------------------------------------- +:mod:`email.message`: Representing an email message +--------------------------------------------------- .. module:: email.message :synopsis: The base class representing email messages. @@ -48,8 +48,8 @@ Here are the methods of the :class:`Message` class: Note that this method is provided as a convenience and may not always format the message the way you want. For example, by default it mangles lines that begin with ``From``. For more flexibility, instantiate a - :class:`~email.generator.Generator` instance and use its :meth:`flatten` - method directly. For example:: + :class:`~email.generator.Generator` instance and use its + :meth:`~email.generator.Generator.flatten` method directly. For example:: from cStringIO import StringIO from email.generator import Generator @@ -68,7 +68,7 @@ Here are the methods of the :class:`Message` class: Return ``True`` if the message's payload is a list of sub-\ :class:`Message` objects, otherwise return ``False``. When - :meth:`is_multipart` returns False, the payload should be a string object. + :meth:`is_multipart` returns ``False``, the payload should be a string object. .. method:: set_unixfrom(unixfrom) @@ -494,8 +494,8 @@ Here are the methods of the :class:`Message` class: Set the ``boundary`` parameter of the :mailheader:`Content-Type` header to *boundary*. :meth:`set_boundary` will always quote *boundary* if - necessary. A :exc:`HeaderParseError` is raised if the message object has - no :mailheader:`Content-Type` header. + necessary. A :exc:`~email.errors.HeaderParseError` is raised if the + message object has no :mailheader:`Content-Type` header. Note that using this method is subtly different than deleting the old :mailheader:`Content-Type` header and adding a new one with the new @@ -589,7 +589,8 @@ Here are the methods of the :class:`Message` class: .. versionchanged:: 2.5 You do not need to set the epilogue to the empty string in order for the - :class:`Generator` to print a newline at the end of the file. + :class:`~email.generator.Generator` to print a newline at the end of the + file. .. attribute:: defects diff --git a/Doc/library/email.mime.rst b/Doc/library/email.mime.rst index 78fdc76..dcf7b59 100644 --- a/Doc/library/email.mime.rst +++ b/Doc/library/email.mime.rst @@ -1,5 +1,5 @@ -:mod:`email`: Creating email and MIME objects from scratch ----------------------------------------------------------- +:mod:`email.mime`: Creating email and MIME objects from scratch +--------------------------------------------------------------- .. module:: email.mime :synopsis: Build MIME messages. @@ -35,7 +35,8 @@ Here are the classes: *_maintype* is the :mailheader:`Content-Type` major type (e.g. :mimetype:`text` or :mimetype:`image`), and *_subtype* is the :mailheader:`Content-Type` minor type (e.g. :mimetype:`plain` or :mimetype:`gif`). *_params* is a parameter - key/value dictionary and is passed directly to :meth:`Message.add_header`. + key/value dictionary and is passed directly to :meth:`Message.add_header + <email.message.Message.add_header>`. The :class:`MIMEBase` class always adds a :mailheader:`Content-Type` header (based on *_maintype*, *_subtype*, and *_params*), and a @@ -50,8 +51,9 @@ Here are the classes: A subclass of :class:`~email.mime.base.MIMEBase`, this is an intermediate base class for MIME messages that are not :mimetype:`multipart`. The primary - purpose of this class is to prevent the use of the :meth:`attach` method, - which only makes sense for :mimetype:`multipart` messages. If :meth:`attach` + purpose of this class is to prevent the use of the + :meth:`~email.message.Message.attach` method, which only makes sense for + :mimetype:`multipart` messages. If :meth:`~email.message.Message.attach` is called, a :exc:`~email.errors.MultipartConversionError` exception is raised. .. versionadded:: 2.2.2 @@ -76,7 +78,8 @@ Here are the classes: *_subparts* is a sequence of initial subparts for the payload. It must be possible to convert this sequence to a list. You can always attach new subparts - to the message by using the :meth:`Message.attach` method. + to the message by using the :meth:`Message.attach + <email.message.Message.attach>` method. Additional parameters for the :mailheader:`Content-Type` header are taken from the keyword arguments, or passed into the *_params* argument, which is a keyword @@ -99,8 +102,10 @@ Here are the classes: Optional *_encoder* is a callable (i.e. function) which will perform the actual encoding of the data for transport. This callable takes one argument, which is - the :class:`MIMEApplication` instance. It should use :meth:`get_payload` and - :meth:`set_payload` to change the payload to encoded form. It should also add + the :class:`MIMEApplication` instance. It should use + :meth:`~email.message.Message.get_payload` and + :meth:`~email.message.Message.set_payload` to change the payload to encoded + form. It should also add any :mailheader:`Content-Transfer-Encoding` or other headers to the message object as necessary. The default encoding is base64. See the :mod:`email.encoders` module for a list of the built-in encoders. @@ -127,8 +132,10 @@ Here are the classes: Optional *_encoder* is a callable (i.e. function) which will perform the actual encoding of the audio data for transport. This callable takes one argument, - which is the :class:`MIMEAudio` instance. It should use :meth:`get_payload` and - :meth:`set_payload` to change the payload to encoded form. It should also add + which is the :class:`MIMEAudio` instance. It should use + :meth:`~email.message.Message.get_payload` and + :meth:`~email.message.Message.set_payload` to change the payload to encoded + form. It should also add any :mailheader:`Content-Transfer-Encoding` or other headers to the message object as necessary. The default encoding is base64. See the :mod:`email.encoders` module for a list of the built-in encoders. @@ -153,8 +160,10 @@ Here are the classes: Optional *_encoder* is a callable (i.e. function) which will perform the actual encoding of the image data for transport. This callable takes one argument, - which is the :class:`MIMEImage` instance. It should use :meth:`get_payload` and - :meth:`set_payload` to change the payload to encoded form. It should also add + which is the :class:`MIMEImage` instance. It should use + :meth:`~email.message.Message.get_payload` and + :meth:`~email.message.Message.set_payload` to change the payload to encoded + form. It should also add any :mailheader:`Content-Transfer-Encoding` or other headers to the message object as necessary. The default encoding is base64. See the :mod:`email.encoders` module for a list of the built-in encoders. @@ -199,3 +208,12 @@ Here are the classes: Transfer Encoding now happens implicitly based on the *_charset* argument. + Unless the ``_charset`` parameter is explicitly set to ``None``, the + MIMEText object created will have both a :mailheader:`Content-Type` header + with a ``charset`` parameter, and a :mailheader:`Content-Transfer-Endcoding` + header. This means that a subsequent ``set_payload`` call will not result + in an encoded payload, even if a charset is passed in the ``set_payload`` + command. You can "reset" this behavior by deleting the + ``Content-Transfer-Encoding`` header, after which a ``set_payload`` call + will automatically encode the new payload (and add a new + :mailheader:`Content-Transfer-Encoding` header). diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst index b2f6b05..0f99a2f 100644 --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -1,5 +1,5 @@ -:mod:`email`: Parsing email messages ------------------------------------- +:mod:`email.parser`: Parsing email messages +------------------------------------------- .. module:: email.parser :synopsis: Parse flat text email messages to produce a message object structure. @@ -7,7 +7,8 @@ Message object structures can be created in one of two ways: they can be created from whole cloth by instantiating :class:`~email.message.Message` objects and -stringing them together via :meth:`attach` and :meth:`set_payload` calls, or they +stringing them together via :meth:`~email.message.Message.attach` and +:meth:`~email.message.Message.set_payload` calls, or they can be created by parsing a flat text representation of the email message. The :mod:`email` package provides a standard parser that understands most email @@ -16,8 +17,9 @@ or a file object, and the parser will return to you the root :class:`~email.message.Message` instance of the object structure. For simple, non-MIME messages the payload of this root object will likely be a string containing the text of the message. For MIME messages, the root object will -return ``True`` from its :meth:`is_multipart` method, and the subparts can be -accessed via the :meth:`get_payload` and :meth:`walk` methods. +return ``True`` from its :meth:`~email.message.Message.is_multipart` method, and +the subparts can be accessed via the :meth:`~email.message.Message.get_payload` +and :meth:`~email.message.Message.walk` methods. There are actually two parser interfaces available for use, the classic :class:`Parser` API and the incremental :class:`FeedParser` API. The classic @@ -127,7 +129,8 @@ class. Read all the data from the file-like object *fp*, parse the resulting text, and return the root message object. *fp* must support both the - :meth:`readline` and the :meth:`read` methods on file-like objects. + :meth:`~io.TextIOBase.readline` and the :meth:`~io.TextIOBase.read` + methods on file-like objects. The text contained in *fp* must be formatted as a block of :rfc:`2822` style headers and header continuation lines, optionally preceded by a @@ -147,7 +150,7 @@ class. Similar to the :meth:`parse` method, except it takes a string object instead of a file-like object. Calling this method on a string is exactly - equivalent to wrapping *text* in a :class:`StringIO` instance first and + equivalent to wrapping *text* in a :class:`~StringIO.StringIO` instance first and calling :meth:`parse`. Optional *headersonly* is as with the :meth:`parse` method. @@ -165,7 +168,7 @@ in the top-level :mod:`email` package namespace. Return a message object structure from a string. This is exactly equivalent to ``Parser().parsestr(s)``. Optional *_class* and *strict* are interpreted as - with the :class:`Parser` class constructor. + with the :class:`~email.parser.Parser` class constructor. .. versionchanged:: 2.2.2 The *strict* flag was added. @@ -175,7 +178,7 @@ in the top-level :mod:`email` package namespace. Return a message object structure tree from an open file object. This is exactly equivalent to ``Parser().parse(fp)``. Optional *_class* and *strict* - are interpreted as with the :class:`Parser` class constructor. + are interpreted as with the :class:`~email.parser.Parser` class constructor. .. versionchanged:: 2.2.2 The *strict* flag was added. @@ -193,32 +196,35 @@ Here are some notes on the parsing semantics: * Most non-\ :mimetype:`multipart` type messages are parsed as a single message object with a string payload. These objects will return ``False`` for - :meth:`is_multipart`. Their :meth:`get_payload` method will return a string - object. + :meth:`~email.message.Message.is_multipart`. Their + :meth:`~email.message.Message.get_payload` method will return a string object. * All :mimetype:`multipart` type messages will be parsed as a container message object with a list of sub-message objects for their payload. The outer - container message will return ``True`` for :meth:`is_multipart` and their - :meth:`get_payload` method will return the list of :class:`~email.message.Message` - subparts. + container message will return ``True`` for + :meth:`~email.message.Message.is_multipart` and their + :meth:`~email.message.Message.get_payload` method will return the list of + :class:`~email.message.Message` subparts. * Most messages with a content type of :mimetype:`message/\*` (e.g. :mimetype:`message/delivery-status` and :mimetype:`message/rfc822`) will also be parsed as container object containing a list payload of length 1. Their - :meth:`is_multipart` method will return ``True``. The single element in the - list payload will be a sub-message object. + :meth:`~email.message.Message.is_multipart` method will return ``True``. + The single element in the list payload will be a sub-message object. * Some non-standards compliant messages may not be internally consistent about their :mimetype:`multipart`\ -edness. Such messages may have a :mailheader:`Content-Type` header of type :mimetype:`multipart`, but their - :meth:`is_multipart` method may return ``False``. If such messages were parsed - with the :class:`FeedParser`, they will have an instance of the - :class:`MultipartInvariantViolationDefect` class in their *defects* attribute - list. See :mod:`email.errors` for details. + :meth:`~email.message.Message.is_multipart` method may return ``False``. + If such messages were parsed with the :class:`~email.parser.FeedParser`, + they will have an instance of the + :class:`~email.errors.MultipartInvariantViolationDefect` class in their + *defects* attribute list. See :mod:`email.errors` for details. .. rubric:: Footnotes .. [#] As of email package version 3.0, introduced in Python 2.4, the classic - :class:`Parser` was re-implemented in terms of the :class:`FeedParser`, so the - semantics and results are identical between the two parsers. + :class:`~email.parser.Parser` was re-implemented in terms of the + :class:`~email.parser.FeedParser`, so the semantics and results are + identical between the two parsers. diff --git a/Doc/library/email.rst b/Doc/library/email.rst index aaff153..4e90ce4 100644 --- a/Doc/library/email.rst +++ b/Doc/library/email.rst @@ -112,14 +112,15 @@ Here are the major differences between :mod:`email` version 4 and version 3: *Note that the version 3 names will continue to work until Python 2.6*. * The :mod:`email.mime.application` module was added, which contains the - :class:`MIMEApplication` class. + :class:`~email.mime.application.MIMEApplication` class. * Methods that were deprecated in version 3 have been removed. These include :meth:`Generator.__call__`, :meth:`Message.get_type`, :meth:`Message.get_main_type`, :meth:`Message.get_subtype`. * Fixes have been added for :rfc:`2231` support which can change some of the - return types for :func:`Message.get_param` and friends. Under some + return types for :func:`Message.get_param <email.message.Message.get_param>` + and friends. Under some circumstances, values which used to return a 3-tuple now return simple strings (specifically, if all extended parameter segments were unencoded, there is no language and charset designation expected, so the return type is now a simple @@ -128,23 +129,24 @@ Here are the major differences between :mod:`email` version 4 and version 3: Here are the major differences between :mod:`email` version 3 and version 2: -* The :class:`FeedParser` class was introduced, and the :class:`Parser` class - was implemented in terms of the :class:`FeedParser`. All parsing therefore is +* The :class:`~email.parser.FeedParser` class was introduced, and the + :class:`~email.parser.Parser` class was implemented in terms of the + :class:`~email.parser.FeedParser`. All parsing therefore is non-strict, and parsing will make a best effort never to raise an exception. Problems found while parsing messages are stored in the message's *defect* attribute. * All aspects of the API which raised :exc:`DeprecationWarning`\ s in version 2 have been removed. These include the *_encoder* argument to the - :class:`MIMEText` constructor, the :meth:`Message.add_payload` method, the - :func:`Utils.dump_address_pair` function, and the functions :func:`Utils.decode` - and :func:`Utils.encode`. + :class:`~email.mime.text.MIMEText` constructor, the + :meth:`Message.add_payload` method, the :func:`Utils.dump_address_pair` + function, and the functions :func:`Utils.decode` and :func:`Utils.encode`. * New :exc:`DeprecationWarning`\ s have been added to: :meth:`Generator.__call__`, :meth:`Message.get_type`, :meth:`Message.get_main_type`, :meth:`Message.get_subtype`, and the *strict* - argument to the :class:`Parser` class. These are expected to be removed in - future versions. + argument to the :class:`~email.parser.Parser` class. These are expected to + be removed in future versions. * Support for Pythons earlier than 2.3 has been removed. @@ -152,53 +154,61 @@ Here are the differences between :mod:`email` version 2 and version 1: * The :mod:`email.Header` and :mod:`email.Charset` modules have been added. -* The pickle format for :class:`Message` instances has changed. Since this was - never (and still isn't) formally defined, this isn't considered a backward - incompatibility. However if your application pickles and unpickles - :class:`Message` instances, be aware that in :mod:`email` version 2, - :class:`Message` instances now have private variables *_charset* and - *_default_type*. +* The pickle format for :class:`~email.message.Message` instances has changed. + Since this was never (and still isn't) formally defined, this isn't + considered a backward incompatibility. However if your application pickles + and unpickles :class:`~email.message.Message` instances, be aware that in + :mod:`email` version 2, :class:`~email.message.Message` instances now have + private variables *_charset* and *_default_type*. -* Several methods in the :class:`Message` class have been deprecated, or their - signatures changed. Also, many new methods have been added. See the - documentation for the :class:`Message` class for details. The changes should be - completely backward compatible. +* Several methods in the :class:`~email.message.Message` class have been + deprecated, or their signatures changed. Also, many new methods have been + added. See the documentation for the :class:`~email.message.Message` class + for details. The changes should be completely backward compatible. * The object structure has changed in the face of :mimetype:`message/rfc822` - content types. In :mod:`email` version 1, such a type would be represented by a - scalar payload, i.e. the container message's :meth:`is_multipart` returned - false, :meth:`get_payload` was not a list object, but a single :class:`Message` - instance. + content types. In :mod:`email` version 1, such a type would be represented + by a scalar payload, i.e. the container message's + :meth:`~email.message.Message.is_multipart` returned false, + :meth:`~email.message.Message.get_payload` was not a list object, but a + single :class:`~email.message.Message` instance. This structure was inconsistent with the rest of the package, so the object representation for :mimetype:`message/rfc822` content types was changed. In :mod:`email` version 2, the container *does* return ``True`` from - :meth:`is_multipart`, and :meth:`get_payload` returns a list containing a single - :class:`Message` item. - - Note that this is one place that backward compatibility could not be completely - maintained. However, if you're already testing the return type of - :meth:`get_payload`, you should be fine. You just need to make sure your code - doesn't do a :meth:`set_payload` with a :class:`Message` instance on a container - with a content type of :mimetype:`message/rfc822`. - -* The :class:`Parser` constructor's *strict* argument was added, and its - :meth:`parse` and :meth:`parsestr` methods grew a *headersonly* argument. The - *strict* flag was also added to functions :func:`email.message_from_file` and - :func:`email.message_from_string`. - -* :meth:`Generator.__call__` is deprecated; use :meth:`Generator.flatten` - instead. The :class:`Generator` class has also grown the :meth:`clone` method. - -* The :class:`DecodedGenerator` class in the :mod:`email.Generator` module was - added. - -* The intermediate base classes :class:`MIMENonMultipart` and - :class:`MIMEMultipart` have been added, and interposed in the class hierarchy - for most of the other MIME-related derived classes. - -* The *_encoder* argument to the :class:`MIMEText` constructor has been - deprecated. Encoding now happens implicitly based on the *_charset* argument. + :meth:`~email.message.Message.is_multipart`, and + :meth:`~email.message.Message.get_payload` returns a list containing a single + :class:`~email.message.Message` item. + + Note that this is one place that backward compatibility could not be + completely maintained. However, if you're already testing the return type of + :meth:`~email.message.Message.get_payload`, you should be fine. You just need + to make sure your code doesn't do a :meth:`~email.message.Message.set_payload` + with a :class:`~email.message.Message` instance on a container with a content + type of :mimetype:`message/rfc822`. + +* The :class:`~email.parser.Parser` constructor's *strict* argument was added, + and its :meth:`~email.parser.Parser.parse` and + :meth:`~email.parser.Parser.parsestr` methods grew a *headersonly* argument. + The *strict* flag was also added to functions :func:`email.message_from_file` + and :func:`email.message_from_string`. + +* :meth:`Generator.__call__` is deprecated; use :meth:`Generator.flatten + <email.generator.Generator.flatten>` instead. The + :class:`~email.generator.Generator` class has also grown the + :meth:`~email.generator.Generator.clone` method. + +* The :class:`~email.generator.DecodedGenerator` class in the + :mod:`email.generator` module was added. + +* The intermediate base classes + :class:`~email.mime.nonmultipart.MIMENonMultipart` and + :class:`~email.mime.multipart.MIMEMultipart` have been added, and interposed + in the class hierarchy for most of the other MIME-related derived classes. + +* The *_encoder* argument to the :class:`~email.mime.text.MIMEText` constructor + has been deprecated. Encoding now happens implicitly based on the + *_charset* argument. * The following functions in the :mod:`email.Utils` module have been deprecated: :func:`dump_address_pairs`, :func:`decode`, and :func:`encode`. The following @@ -231,17 +241,22 @@ package has the following differences: * :func:`messageFromFile` has been renamed to :func:`message_from_file`. -The :class:`Message` class has the following differences: +The :class:`~email.message.Message` class has the following differences: -* The method :meth:`asString` was renamed to :meth:`as_string`. +* The method :meth:`asString` was renamed to + :meth:`~email.message.Message.as_string`. -* The method :meth:`ismultipart` was renamed to :meth:`is_multipart`. +* The method :meth:`ismultipart` was renamed to + :meth:`~email.message.Message.is_multipart`. -* The :meth:`get_payload` method has grown a *decode* optional argument. +* The :meth:`~email.message.Message.get_payload` method has grown a *decode* + optional argument. -* The method :meth:`getall` was renamed to :meth:`get_all`. +* The method :meth:`getall` was renamed to + :meth:`~email.message.Message.get_all`. -* The method :meth:`addheader` was renamed to :meth:`add_header`. +* The method :meth:`addheader` was renamed to + :meth:`~email.message.Message.add_header`. * The method :meth:`gettype` was renamed to :meth:`get_type`. @@ -249,48 +264,57 @@ The :class:`Message` class has the following differences: * The method :meth:`getsubtype` was renamed to :meth:`get_subtype`. -* The method :meth:`getparams` was renamed to :meth:`get_params`. Also, whereas - :meth:`getparams` returned a list of strings, :meth:`get_params` returns a list - of 2-tuples, effectively the key/value pairs of the parameters, split on the - ``'='`` sign. +* The method :meth:`getparams` was renamed to + :meth:`~email.message.Message.get_params`. Also, whereas :meth:`getparams` + returned a list of strings, :meth:`~email.message.Message.get_params` returns + a list of 2-tuples, effectively the key/value pairs of the parameters, split + on the ``'='`` sign. -* The method :meth:`getparam` was renamed to :meth:`get_param`. +* The method :meth:`getparam` was renamed to + :meth:`~email.message.Message.get_param`. -* The method :meth:`getcharsets` was renamed to :meth:`get_charsets`. +* The method :meth:`getcharsets` was renamed to + :meth:`~email.message.Message.get_charsets`. -* The method :meth:`getfilename` was renamed to :meth:`get_filename`. +* The method :meth:`getfilename` was renamed to + :meth:`~email.message.Message.get_filename`. -* The method :meth:`getboundary` was renamed to :meth:`get_boundary`. +* The method :meth:`getboundary` was renamed to + :meth:`~email.message.Message.get_boundary`. -* The method :meth:`setboundary` was renamed to :meth:`set_boundary`. +* The method :meth:`setboundary` was renamed to + :meth:`~email.message.Message.set_boundary`. * The method :meth:`getdecodedpayload` was removed. To get similar - functionality, pass the value 1 to the *decode* flag of the get_payload() - method. + functionality, pass the value 1 to the *decode* flag of the + :meth:`~email.message.Message.get_payload` method. * The method :meth:`getpayloadastext` was removed. Similar functionality is - supported by the :class:`DecodedGenerator` class in the :mod:`email.generator` - module. + supported by the :class:`~email.generator.DecodedGenerator` class in the + :mod:`email.generator` module. * The method :meth:`getbodyastext` was removed. You can get similar - functionality by creating an iterator with :func:`typed_subpart_iterator` in the - :mod:`email.iterators` module. + functionality by creating an iterator with + :func:`~email.iterators.typed_subpart_iterator` in the :mod:`email.iterators` + module. -The :class:`Parser` class has no differences in its public interface. It does -have some additional smarts to recognize :mimetype:`message/delivery-status` -type messages, which it represents as a :class:`Message` instance containing -separate :class:`Message` subparts for each header block in the delivery status -notification [#]_. +The :class:`~email.parser.Parser` class has no differences in its public +interface. It does have some additional smarts to recognize +:mimetype:`message/delivery-status` type messages, which it represents as a +:class:`~email.message.Message` instance containing separate +:class:`~email.message.Message` subparts for each header block in the delivery +status notification [#]_. -The :class:`Generator` class has no differences in its public interface. There -is a new class in the :mod:`email.generator` module though, called -:class:`DecodedGenerator` which provides most of the functionality previously -available in the :meth:`Message.getpayloadastext` method. +The :class:`~email.generator.Generator` class has no differences in its public +interface. There is a new class in the :mod:`email.generator` module though, +called :class:`~email.generator.DecodedGenerator` which provides most of the +functionality previously available in the :meth:`Message.getpayloadastext` +method. The following modules and classes have been changed: -* The :class:`MIMEBase` class constructor arguments *_major* and *_minor* have - changed to *_maintype* and *_subtype* respectively. +* The :class:`~email.mime.base.MIMEBase` class constructor arguments *_major* + and *_minor* have changed to *_maintype* and *_subtype* respectively. * The ``Image`` class/module has been renamed to ``MIMEImage``. The *_minor* argument has been renamed to *_subtype*. @@ -303,7 +327,8 @@ The following modules and classes have been changed: but that clashed with the Python standard library module :mod:`rfc822` on some case-insensitive file systems. - Also, the :class:`MIMEMessage` class now represents any kind of MIME message + Also, the :class:`~email.mime.message.MIMEMessage` class now represents any + kind of MIME message with main type :mimetype:`message`. It takes an optional argument *_subtype* which is used to set the MIME subtype. *_subtype* defaults to :mimetype:`rfc822`. @@ -313,8 +338,8 @@ The following modules and classes have been changed: :mod:`email.utils` module. The ``MsgReader`` class/module has been removed. Its functionality is most -closely supported in the :func:`body_line_iterator` function in the -:mod:`email.iterators` module. +closely supported in the :func:`~email.iterators.body_line_iterator` function +in the :mod:`email.iterators` module. .. rubric:: Footnotes diff --git a/Doc/library/email.util.rst b/Doc/library/email.util.rst index 153ba78..9b583c0 100644 --- a/Doc/library/email.util.rst +++ b/Doc/library/email.util.rst @@ -1,5 +1,5 @@ -:mod:`email`: Miscellaneous utilities -------------------------------------- +:mod:`email.utils`: Miscellaneous utilities +------------------------------------------- .. module:: email.utils :synopsis: Miscellaneous email package utilities. @@ -41,8 +41,8 @@ There are several useful utilities provided in the :mod:`email.utils` module: This method returns a list of 2-tuples of the form returned by ``parseaddr()``. *fieldvalues* is a sequence of header field values as might be returned by - :meth:`Message.get_all`. Here's a simple example that gets all the recipients - of a message:: + :meth:`Message.get_all <email.message.Message.get_all>`. Here's a simple + example that gets all the recipients of a message:: from email.utils import getaddresses @@ -76,12 +76,9 @@ There are several useful utilities provided in the :mod:`email.utils` module: .. function:: mktime_tz(tuple) - Turn a 10-tuple as returned by :func:`parsedate_tz` into a UTC timestamp. It - the timezone item in the tuple is ``None``, assume local time. Minor - deficiency: :func:`mktime_tz` interprets the first 8 elements of *tuple* as a - local time and then compensates for the timezone difference. This may yield a - slight error around changes in daylight savings time, though not worth worrying - about for common use. + Turn a 10-tuple as returned by :func:`parsedate_tz` into a UTC + timestamp (seconds since the Epoch). If the timezone item in the + tuple is ``None``, assume local time. .. function:: formatdate([timeval[, localtime][, usegmt]]) @@ -130,7 +127,8 @@ There are several useful utilities provided in the :mod:`email.utils` module: .. function:: collapse_rfc2231_value(value[, errors[, fallback_charset]]) When a header parameter is encoded in :rfc:`2231` format, - :meth:`Message.get_param` may return a 3-tuple containing the character set, + :meth:`Message.get_param <email.message.Message.get_param>` may return a + 3-tuple containing the character set, language, and value. :func:`collapse_rfc2231_value` turns this into a unicode string. Optional *errors* is passed to the *errors* argument of the built-in :func:`unicode` function; it defaults to ``replace``. Optional @@ -152,15 +150,15 @@ There are several useful utilities provided in the :mod:`email.utils` module: .. versionchanged:: 2.4 The :func:`decode` function has been removed; use the - :meth:`Header.decode_header` method instead. + :meth:`Header.decode_header <email.header.Header.decode_header>` method + instead. .. versionchanged:: 2.4 - The :func:`encode` function has been removed; use the :meth:`Header.encode` - method instead. + The :func:`encode` function has been removed; use the :meth:`Header.encode + <email.header.Header.encode>` method instead. .. rubric:: Footnotes .. [#] Note that the sign of the timezone offset is the opposite of the sign of the ``time.timezone`` variable for the same timezone; the latter variable follows the POSIX standard while this module follows :rfc:`2822`. - diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index 6c80df7..9291c18 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -38,10 +38,10 @@ handler or to report an error condition "just like" the situation in which the interpreter raises the same exception; but beware that there is nothing to prevent user code from raising an inappropriate error. -The built-in exception classes can be sub-classed to define new exceptions; -programmers are encouraged to at least derive new exceptions from the -:exc:`Exception` class and not :exc:`BaseException`. More information on -defining exceptions is available in the Python Tutorial under +The built-in exception classes can be subclassed to define new exceptions; +programmers are encouraged to derive new exceptions from the :exc:`Exception` +class or one of its subclasses, and not from :exc:`BaseException`. More +information on defining exceptions is available in the Python Tutorial under :ref:`tut-userexceptions`. The following exceptions are only used as base classes for other exceptions. @@ -158,9 +158,9 @@ The following exceptions are the exceptions that are actually raised. .. exception:: GeneratorExit - Raise when a :term:`generator`\'s :meth:`close` method is called. It - directly inherits from :exc:`BaseException` instead of :exc:`StandardError` since - it is technically not an error. + Raised when a :term:`generator`\'s :meth:`close` method is called. It + directly inherits from :exc:`BaseException` instead of :exc:`StandardError` + since it is technically not an error. .. versionadded:: 2.5 @@ -286,8 +286,7 @@ The following exceptions are the exceptions that are actually raised. Raised when an error is detected that doesn't fall in any of the other categories. The associated value is a string indicating what precisely went - wrong. (This exception is mostly a relic from a previous version of the - interpreter; it is not used very much any more.) + wrong. .. exception:: StopIteration @@ -346,7 +345,7 @@ The following exceptions are the exceptions that are actually raised. it has another type (such as a string), the object's value is printed and the exit status is one. - Instances have an attribute :attr:`code` which is set to the proposed exit + Instances have an attribute :attr:`!code` which is set to the proposed exit status or error message (defaulting to ``None``). Also, this exception derives directly from :exc:`BaseException` and not :exc:`StandardError`, since it is not technically an error. @@ -356,7 +355,7 @@ The following exceptions are the exceptions that are actually raised. executed, and so that a debugger can execute a script without running the risk of losing control. The :func:`os._exit` function can be used if it is absolutely positively necessary to exit immediately (for example, in the child - process after a call to :func:`fork`). + process after a call to :func:`os.fork`). The exception inherits from :exc:`BaseException` instead of :exc:`StandardError` or :exc:`Exception` so that it is not accidentally caught by code that catches @@ -387,6 +386,30 @@ The following exceptions are the exceptions that are actually raised. Raised when a Unicode-related encoding or decoding error occurs. It is a subclass of :exc:`ValueError`. + :exc:`UnicodeError` has attributes that describe the encoding or decoding + error. For example, ``err.object[err.start:err.end]`` gives the particular + invalid input that the codec failed on. + + .. attribute:: encoding + + The name of the encoding that raised the error. + + .. attribute:: reason + + A string describing the specific codec error. + + .. attribute:: object + + The object the codec was attempting to encode or decode. + + .. attribute:: start + + The first index of invalid data in :attr:`object`. + + .. attribute:: end + + The index after the last invalid data in :attr:`object`. + .. versionadded:: 2.0 diff --git a/Doc/library/fcntl.rst b/Doc/library/fcntl.rst index 40ae08b..562e78f 100644 --- a/Doc/library/fcntl.rst +++ b/Doc/library/fcntl.rst @@ -1,6 +1,5 @@ - -:mod:`fcntl` --- The :func:`fcntl` and :func:`ioctl` system calls -================================================================= +:mod:`fcntl` --- The ``fcntl`` and ``ioctl`` system calls +========================================================= .. module:: fcntl :platform: Unix @@ -18,17 +17,18 @@ interface to the :c:func:`fcntl` and :c:func:`ioctl` Unix routines. All functions in this module take a file descriptor *fd* as their first argument. This can be an integer file descriptor, such as returned by ``sys.stdin.fileno()``, or a file object, such as ``sys.stdin`` itself, which -provides a :meth:`fileno` which returns a genuine file descriptor. +provides a :meth:`~io.IOBase.fileno` which returns a genuine file descriptor. The module defines the following functions: .. function:: fcntl(fd, op[, arg]) - Perform the requested operation on file descriptor *fd* (file objects providing - a :meth:`fileno` method are accepted as well). The operation is defined by *op* - and is operating system dependent. These codes are also found in the - :mod:`fcntl` module. The argument *arg* is optional, and defaults to the integer + Perform the operation *op* on file descriptor *fd* (file objects providing + a :meth:`~io.IOBase.fileno` method are accepted as well). The values used + for for *op* are operating system dependent, and are available as constants + in the :mod:`fcntl` module, using the same names as used in the relevant C + header files. The argument *arg* is optional, and defaults to the integer value ``0``. When present, it can either be an integer value, or a string. With the argument missing or an integer value, the return value of this function is the integer return value of the C :c:func:`fcntl` call. When the argument is @@ -46,17 +46,21 @@ The module defines the following functions: .. function:: ioctl(fd, op[, arg[, mutate_flag]]) - This function is identical to the :func:`fcntl` function, except that the + This function is identical to the :func:`~fcntl.fcntl` function, except that the operations are typically defined in the library module :mod:`termios` and the argument handling is even more complicated. The op parameter is limited to values that can fit in 32-bits. + Additional constants of interest for use as the *op* argument can be + found in the :mod:`termios` module, under the same names as used in + the relevant C header files. The parameter *arg* can be one of an integer, absent (treated identically to the integer ``0``), an object supporting the read-only buffer interface (most likely a plain Python string) or an object supporting the read-write buffer interface. - In all but the last case, behaviour is as for the :func:`fcntl` function. + In all but the last case, behaviour is as for the :func:`~fcntl.fcntl` + function. If a mutable buffer is passed, then the behaviour is determined by the value of the *mutate_flag* parameter. @@ -95,16 +99,16 @@ The module defines the following functions: .. function:: flock(fd, op) Perform the lock operation *op* on file descriptor *fd* (file objects providing - a :meth:`fileno` method are accepted as well). See the Unix manual + a :meth:`~io.IOBase.fileno` method are accepted as well). See the Unix manual :manpage:`flock(2)` for details. (On some systems, this function is emulated using :c:func:`fcntl`.) .. function:: lockf(fd, operation, [length, [start, [whence]]]) - This is essentially a wrapper around the :func:`fcntl` locking calls. *fd* is - the file descriptor of the file to lock or unlock, and *operation* is one of the - following values: + This is essentially a wrapper around the :func:`~fcntl.fcntl` locking calls. + *fd* is the file descriptor of the file to lock or unlock, and *operation* + is one of the following values: * :const:`LOCK_UN` -- unlock * :const:`LOCK_SH` -- acquire a shared lock @@ -119,13 +123,13 @@ The module defines the following functions: systems, :const:`LOCK_EX` can only be used if the file descriptor refers to a file opened for writing. - *length* is the number of bytes to lock, *start* is the byte offset at which the - lock starts, relative to *whence*, and *whence* is as with :func:`fileobj.seek`, - specifically: + *length* is the number of bytes to lock, *start* is the byte offset at + which the lock starts, relative to *whence*, and *whence* is as with + :func:`io.IOBase.seek`, specifically: - * :const:`0` -- relative to the start of the file (:const:`SEEK_SET`) - * :const:`1` -- relative to the current buffer position (:const:`SEEK_CUR`) - * :const:`2` -- relative to the end of the file (:const:`SEEK_END`) + * :const:`0` -- relative to the start of the file (:data:`os.SEEK_SET`) + * :const:`1` -- relative to the current buffer position (:data:`os.SEEK_CUR`) + * :const:`2` -- relative to the end of the file (:data:`os.SEEK_END`) The default for *start* is 0, which means to start at the beginning of the file. The default for *length* is 0 which means to lock to the end of the file. The @@ -150,7 +154,8 @@ lay-out for the *lockdata* variable is system dependent --- therefore using the .. seealso:: Module :mod:`os` - If the locking flags :const:`O_SHLOCK` and :const:`O_EXLOCK` are present - in the :mod:`os` module (on BSD only), the :func:`os.open` function - provides an alternative to the :func:`lockf` and :func:`flock` functions. + If the locking flags :data:`~os.O_SHLOCK` and :data:`~os.O_EXLOCK` are + present in the :mod:`os` module (on BSD only), the :func:`os.open` + function provides an alternative to the :func:`lockf` and :func:`flock` + functions. diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst index 25d0701..f9c5fb1 100644 --- a/Doc/library/filecmp.rst +++ b/Doc/library/filecmp.rst @@ -54,9 +54,9 @@ The :mod:`filecmp` module defines the following functions: Example:: >>> import filecmp - >>> filecmp.cmp('undoc.rst', 'undoc.rst') + >>> filecmp.cmp('undoc.rst', 'undoc.rst') # doctest: +SKIP True - >>> filecmp.cmp('undoc.rst', 'index.rst') + >>> filecmp.cmp('undoc.rst', 'index.rst') # doctest: +SKIP False @@ -75,6 +75,9 @@ The :class:`dircmp` class 'tags']``. *hide* is a list of names to hide, and defaults to ``[os.curdir, os.pardir]``. + The :class:`dircmp` class compares files by doing *shallow* comparisons + as described for :func:`filecmp.cmp`. + The :class:`dircmp` class provides the following methods: @@ -94,7 +97,7 @@ The :class:`dircmp` class Print a comparison between *a* and *b* and common subdirectories (recursively). - The :class:`dircmp` offers a number of interesting attributes that may be + The :class:`dircmp` class offers a number of interesting attributes that may be used to get various bits of information about the directory trees being compared. @@ -103,6 +106,16 @@ The :class:`dircmp` class to compute are used. + .. attribute:: left + + The directory *a*. + + + .. attribute:: right + + The directory *b*. + + .. attribute:: left_list Files and subdirectories in *a*, filtered by *hide* and *ignore*. @@ -146,12 +159,14 @@ The :class:`dircmp` class .. attribute:: same_files - Files which are identical in both *a* and *b*. + Files which are identical in both *a* and *b*, using the class's + file comparison operator. .. attribute:: diff_files - Files which are in both *a* and *b*, whose contents differ. + Files which are in both *a* and *b*, whose contents differ according + to the class's file comparison operator. .. attribute:: funny_files @@ -163,3 +178,18 @@ The :class:`dircmp` class A dictionary mapping names in :attr:`common_dirs` to :class:`dircmp` objects. + +Here is a simplified example of using the ``subdirs`` attribute to search +recursively through two directories to show common different files:: + + >>> from filecmp import dircmp + >>> def print_diff_files(dcmp): + ... for name in dcmp.diff_files: + ... print "diff_file %s found in %s and %s" % (name, dcmp.left, + ... dcmp.right) + ... for sub_dcmp in dcmp.subdirs.values(): + ... print_diff_files(sub_dcmp) + ... + >>> dcmp = dircmp('dir1', 'dir2') # doctest: +SKIP + >>> print_diff_files(dcmp) # doctest: +SKIP + diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst index 172a643..710bef3 100644 --- a/Doc/library/fileinput.rst +++ b/Doc/library/fileinput.rst @@ -50,7 +50,7 @@ provided by this module. The following function is the primary interface of this module: -.. function:: input([files[, inplace[, backup[, mode[, openhook]]]]]) +.. function:: input([files[, inplace[, backup[, bufsize[, mode[, openhook]]]]]]) Create an instance of the :class:`FileInput` class. The instance will be used as global state for the functions of this module, and is also returned to use @@ -122,15 +122,16 @@ The class which implements the sequence behavior provided by the module is available for subclassing as well: -.. class:: FileInput([files[, inplace[, backup[, mode[, openhook]]]]]) +.. class:: FileInput([files[, inplace[, backup[,bufsize[, mode[, openhook]]]]]]) Class :class:`FileInput` is the implementation; its methods :meth:`filename`, :meth:`fileno`, :meth:`lineno`, :meth:`filelineno`, :meth:`isfirstline`, - :meth:`isstdin`, :meth:`nextfile` and :meth:`close` correspond to the functions - of the same name in the module. In addition it has a :meth:`readline` method - which returns the next input line, and a :meth:`__getitem__` method which - implements the sequence behavior. The sequence must be accessed in strictly - sequential order; random access and :meth:`readline` cannot be mixed. + :meth:`isstdin`, :meth:`nextfile` and :meth:`close` correspond to the + functions of the same name in the module. In addition it has a + :meth:`~file.readline` method which returns the next input line, + and a :meth:`__getitem__` method which implements the sequence behavior. + The sequence must be accessed in strictly sequential order; random access + and :meth:`~file.readline` cannot be mixed. With *mode* you can specify which file mode will be passed to :func:`open`. It must be one of ``'r'``, ``'rU'``, ``'U'`` and ``'rb'``. diff --git a/Doc/library/fl.rst b/Doc/library/fl.rst index 540cac9..c689372 100644 --- a/Doc/library/fl.rst +++ b/Doc/library/fl.rst @@ -9,7 +9,7 @@ .. deprecated:: 2.6 - The :mod:`fl` module has been deprecated for removal in Python 3.0. + The :mod:`fl` module has been removed in Python 3. .. index:: @@ -487,7 +487,7 @@ FORMS objects have these data attributes; see the FORMS documentation: .. deprecated:: 2.6 - The :mod:`FL` module has been deprecated for removal in Python 3.0. + The :mod:`FL` module has been removed in Python 3. This module defines symbolic constants needed to use the built-in module @@ -509,7 +509,7 @@ source for a complete list of the defined names. Suggested use:: .. deprecated:: 2.6 - The :mod:`flp` module has been deprecated for removal in Python 3.0. + The :mod:`flp` module has been removed in Python 3. This module defines functions that can read form definitions created by the diff --git a/Doc/library/fm.rst b/Doc/library/fm.rst index 6bf6808..c7eb4f3 100644 --- a/Doc/library/fm.rst +++ b/Doc/library/fm.rst @@ -8,7 +8,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`fm` module has been deprecated for removal in Python 3.0. + The :mod:`fm` module has been removed in Python 3. diff --git a/Doc/library/fnmatch.rst b/Doc/library/fnmatch.rst index 4911980..b14c551 100644 --- a/Doc/library/fnmatch.rst +++ b/Doc/library/fnmatch.rst @@ -29,6 +29,9 @@ special characters used in shell-style wildcards are: | ``[!seq]`` | matches any character not in *seq* | +------------+------------------------------------+ +For a literal match, wrap the meta-characters in brackets. +For example, ``'[?]'`` matches the character ``'?'``. + .. index:: module: glob Note that the filename separator (``'/'`` on Unix) is *not* special to this @@ -76,8 +79,6 @@ patterns. Return the shell-style *pattern* converted to a regular expression. - Be aware there is no way to quote meta-characters. - Example: >>> import fnmatch, re diff --git a/Doc/library/formatter.rst b/Doc/library/formatter.rst index ba09b8e..e696fec 100644 --- a/Doc/library/formatter.rst +++ b/Doc/library/formatter.rst @@ -341,10 +341,10 @@ this module. Most applications will need to derive new writer classes from the output. -.. class:: DumbWriter([file[, maxcol=72]]) +.. class:: DumbWriter(file=None, maxcol=72) Simple writer class which writes output on the file object passed in as *file* - or, if *file* is omitted, on standard output. The output is simply word-wrapped + or, if *file* is None, on standard output. The output is simply word-wrapped to the number of columns specified by *maxcol*. This class is suitable for reflowing a sequence of paragraphs. diff --git a/Doc/library/fpectl.rst b/Doc/library/fpectl.rst index ef030f0..8ca671b 100644 --- a/Doc/library/fpectl.rst +++ b/Doc/library/fpectl.rst @@ -113,8 +113,8 @@ The :mod:`fpectl` module is not thread-safe. .. seealso:: Some files in the source distribution may be interesting in learning more about - how this module operates. The include file :file:`Include/pyfpe.h` discusses the - implementation of this module at some length. :file:`Modules/fpetestmodule.c` + how this module operates. The include file :source:`Include/pyfpe.h` discusses the + implementation of this module at some length. :source:`Modules/fpetestmodule.c` gives several examples of use. Many additional examples can be found in - :file:`Objects/floatobject.c`. + :source:`Objects/floatobject.c`. diff --git a/Doc/library/fpformat.rst b/Doc/library/fpformat.rst index 3448585..1713301 100644 --- a/Doc/library/fpformat.rst +++ b/Doc/library/fpformat.rst @@ -7,7 +7,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`fpformat` module has been removed in Python 3.0. + The :mod:`fpformat` module has been removed in Python 3. .. sectionauthor:: Moshe Zadka <moshez@zadka.site.co.il> diff --git a/Doc/library/fractions.rst b/Doc/library/fractions.rst index 4d10cbd..81b419e 100644 --- a/Doc/library/fractions.rst +++ b/Doc/library/fractions.rst @@ -57,7 +57,6 @@ another rational number, or from a string. Fraction(0, 1) >>> Fraction('3/7') Fraction(3, 7) - [40794 refs] >>> Fraction(' -3/7 ') Fraction(-3, 7) >>> Fraction('1.414213 \t\n') diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index b8f5b4e..b42cf64 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -23,16 +23,17 @@ see Internet :rfc:`959`. Here's a sample session using the :mod:`ftplib` module:: >>> from ftplib import FTP - >>> ftp = FTP('ftp.cwi.nl') # connect to host, default port - >>> ftp.login() # user anonymous, passwd anonymous@ - >>> ftp.retrlines('LIST') # list directory contents - total 24418 - drwxrwsr-x 5 ftp-usr pdmaint 1536 Mar 20 09:48 . - dr-xr-srwt 105 ftp-usr pdmaint 1536 Mar 21 14:32 .. - -rw-r--r-- 1 ftp-usr pdmaint 5305 Mar 20 09:48 INDEX - . - . - . + >>> ftp = FTP('ftp.debian.org') # connect to host, default port + >>> ftp.login() # user anonymous, passwd anonymous@ + '230 Login successful.' + >>> ftp.cwd('debian') # change into "debian" directory + >>> ftp.retrlines('LIST') # list directory contents + -rw-rw-r-- 1 1176 1176 1063 Jun 15 10:18 README + ... + drwxr-sr-x 5 1176 1176 4096 Dec 19 2000 pool + drwxr-sr-x 4 1176 1176 4096 Nov 17 2008 project + drwxr-xr-x 3 1176 1176 4096 Oct 10 2012 tools + '226 Directory send OK.' >>> ftp.retrbinary('RETR README', open('README', 'wb').write) '226 Transfer complete.' >>> ftp.quit() @@ -264,8 +265,8 @@ followed by ``lines`` for the text version or ``binary`` for the binary version. Store a file in ASCII transfer mode. *command* should be an appropriate ``STOR`` command (see :meth:`storbinary`). Lines are read until EOF from the - open file object *file* using its :meth:`readline` method to provide the data to - be stored. *callback* is an optional single parameter callable + open file object *file* using its :meth:`~file.readline` method to provide + the data to be stored. *callback* is an optional single parameter callable that is called on each line after it is sent. .. versionchanged:: 2.6 @@ -370,10 +371,10 @@ followed by ``lines`` for the text version or ``binary`` for the binary version. .. method:: FTP.close() Close the connection unilaterally. This should not be applied to an already - closed connection such as after a successful call to :meth:`quit`. After this - call the :class:`FTP` instance should not be used any more (after a call to - :meth:`close` or :meth:`quit` you cannot reopen the connection by issuing - another :meth:`login` method). + closed connection such as after a successful call to :meth:`~FTP.quit`. + After this call the :class:`FTP` instance should not be used any more (after + a call to :meth:`close` or :meth:`~FTP.quit` you cannot reopen the + connection by issuing another :meth:`login` method). FTP_TLS Objects @@ -387,7 +388,8 @@ FTP_TLS Objects .. method:: FTP_TLS.auth() - Set up secure control connection by using TLS or SSL, depending on what specified in :meth:`ssl_version` attribute. + Set up secure control connection by using TLS or SSL, depending on what + specified in :meth:`ssl_version` attribute. .. method:: FTP_TLS.prot_p() @@ -396,5 +398,3 @@ FTP_TLS Objects .. method:: FTP_TLS.prot_c() Set up clear text data connection. - - diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 5f7bf4d..80ce681 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -18,16 +18,26 @@ available. They are listed here in alphabetical order. :func:`bool` :func:`filter` :func:`len` :func:`range` :func:`type` :func:`bytearray` :func:`float` :func:`list` :func:`raw_input` :func:`unichr` :func:`callable` :func:`format` :func:`locals` :func:`reduce` :func:`unicode` -:func:`chr` :func:`frozenset` :func:`long` :func:`reload` :func:`vars` -:func:`classmethod` :func:`getattr` :func:`map` :func:`repr` :func:`xrange` +:func:`chr` |func-frozenset|_ :func:`long` :func:`reload` :func:`vars` +:func:`classmethod` :func:`getattr` :func:`map` |func-repr|_ :func:`xrange` :func:`cmp` :func:`globals` :func:`max` :func:`reversed` :func:`zip` -:func:`compile` :func:`hasattr` :func:`memoryview` :func:`round` :func:`__import__` -:func:`complex` :func:`hash` :func:`min` :func:`set` :func:`apply` +:func:`compile` :func:`hasattr` |func-memoryview|_ :func:`round` :func:`__import__` +:func:`complex` :func:`hash` :func:`min` |func-set|_ :func:`apply` :func:`delattr` :func:`help` :func:`next` :func:`setattr` :func:`buffer` -:func:`dict` :func:`hex` :func:`object` :func:`slice` :func:`coerce` +|func-dict|_ :func:`hex` :func:`object` :func:`slice` :func:`coerce` :func:`dir` :func:`id` :func:`oct` :func:`sorted` :func:`intern` =================== ================= ================== ================= ==================== +.. using :func:`dict` would create a link to another page, so local targets are + used, with replacement texts to make the output in the table consistent + +.. |func-dict| replace:: ``dict()`` +.. |func-frozenset| replace:: ``frozenset()`` +.. |func-memoryview| replace:: ``memoryview()`` +.. |func-repr| replace:: ``repr()`` +.. |func-set| replace:: ``set()`` + + .. function:: abs(x) Return the absolute value of a number. The argument may be a plain or long @@ -37,7 +47,7 @@ available. They are listed here in alphabetical order. .. function:: all(iterable) - Return True if all elements of the *iterable* are true (or if the iterable + Return ``True`` if all elements of the *iterable* are true (or if the iterable is empty). Equivalent to:: def all(iterable): @@ -51,8 +61,8 @@ available. They are listed here in alphabetical order. .. function:: any(iterable) - Return True if any element of the *iterable* is true. If the iterable - is empty, return False. Equivalent to:: + Return ``True`` if any element of the *iterable* is true. If the iterable + is empty, return ``False``. Equivalent to:: def any(iterable): for element in iterable: @@ -153,9 +163,10 @@ available. They are listed here in alphabetical order. instance method receives the instance. To declare a class method, use this idiom:: - class C: + class C(object): @classmethod - def f(cls, arg1, arg2, ...): ... + def f(cls, arg1, arg2, ...): + ... The ``@classmethod`` form is a function :term:`decorator` -- see the description of function definitions in :ref:`function` for details. @@ -188,8 +199,10 @@ available. They are listed here in alphabetical order. Compile the *source* into a code or AST object. Code objects can be executed by an :keyword:`exec` statement or evaluated by a call to :func:`eval`. - *source* can either be a string or an AST object. Refer to the :mod:`ast` - module documentation for information on how to work with AST objects. + *source* can either be a Unicode string, a *Latin-1* encoded string or an + AST object. + Refer to the :mod:`ast` module documentation for information on how to work + with AST objects. The *filename* argument should give the file from which the code was read; pass some recognizable value if it wasn't read from a file (``'<string>'`` is @@ -213,8 +226,8 @@ available. They are listed here in alphabetical order. Future statements are specified by bits which can be bitwise ORed together to specify multiple statements. The bitfield required to specify a given feature - can be found as the :attr:`compiler_flag` attribute on the :class:`_Feature` - instance in the :mod:`__future__` module. + can be found as the :attr:`~__future__._Feature.compiler_flag` attribute on + the :class:`~__future__._Feature` instance in the :mod:`__future__` module. This function raises :exc:`SyntaxError` if the compiled source is invalid, and :exc:`TypeError` if the source contains null bytes. @@ -247,6 +260,13 @@ available. They are listed here in alphabetical order. the function serves as a numeric conversion function like :func:`int`, :func:`long` and :func:`float`. If both arguments are omitted, returns ``0j``. + .. note:: + + When converting from a string, the string must not contain whitespace + around the central ``+`` or ``-`` operator. For example, + ``complex('1+2j')`` is fine, but ``complex('1 + 2j')`` raises + :exc:`ValueError`. + The complex type is described in :ref:`typesnumeric`. @@ -258,14 +278,18 @@ available. They are listed here in alphabetical order. example, ``delattr(x, 'foobar')`` is equivalent to ``del x.foobar``. -.. function:: dict([arg]) +.. _func-dict: +.. function:: dict(**kwarg) + dict(mapping, **kwarg) + dict(iterable, **kwarg) :noindex: - Create a new data dictionary, optionally with items taken from *arg*. - The dictionary type is described in :ref:`typesmapping`. + Create a new dictionary. The :class:`dict` object is the dictionary class. + See :class:`dict` and :ref:`typesmapping` for documentation about this + class. - For other containers see the built in :class:`list`, :class:`set`, and - :class:`tuple` classes, and the :mod:`collections` module. + For other containers see the built-in :class:`list`, :class:`set`, and + :class:`tuple` classes, as well as the :mod:`collections` module. .. function:: dir([object]) @@ -337,7 +361,7 @@ available. They are listed here in alphabetical order. Using :func:`divmod` with complex numbers is deprecated. -.. function:: enumerate(sequence[, start=0]) +.. function:: enumerate(sequence, start=0) Return an enumerate object. *sequence* must be a sequence, an :term:`iterator`, or some other object which supports iteration. The @@ -366,9 +390,9 @@ available. They are listed here in alphabetical order. .. function:: eval(expression[, globals[, locals]]) - The arguments are a string and optional globals and locals. If provided, - *globals* must be a dictionary. If provided, *locals* can be any mapping - object. + The arguments are a Unicode or *Latin-1* encoded string and optional + globals and locals. If provided, *globals* must be a dictionary. + If provided, *locals* can be any mapping object. .. versionchanged:: 2.4 formerly *locals* was required to be a dictionary. @@ -413,7 +437,10 @@ available. They are listed here in alphabetical order. The arguments are a file name and two optional dictionaries. The file is parsed and evaluated as a sequence of Python statements (similarly to a module) using the *globals* and *locals* dictionaries as global and local namespace. If - provided, *locals* can be any mapping object. + provided, *locals* can be any mapping object. Remember that at module level, + globals and locals are the same dictionary. If two separate objects are + passed as *globals* and *locals*, the code will be executed as if it were + embedded in a class definition. .. versionchanged:: 2.4 formerly *locals* was required to be a dictionary. @@ -431,7 +458,7 @@ available. They are listed here in alphabetical order. used reliably to modify a function's locals. -.. function:: file(filename[, mode[, bufsize]]) +.. function:: file(name[, mode[, buffering]]) Constructor function for the :class:`file` type, described further in section :ref:`bltin-file-objects`. The constructor's arguments are the same as those @@ -506,14 +533,17 @@ available. They are listed here in alphabetical order. .. versionadded:: 2.6 +.. _func-frozenset: .. function:: frozenset([iterable]) :noindex: - Return a frozenset object, optionally with elements taken from *iterable*. - The frozenset type is described in :ref:`types-set`. + Return a new :class:`frozenset` object, optionally with elements taken from + *iterable*. ``frozenset`` is a built-in class. See :class:`frozenset` and + :ref:`types-set` for documentation about this class. - For other containers see the built in :class:`dict`, :class:`list`, and - :class:`tuple` classes, and the :mod:`collections` module. + For other containers see the built-in :class:`set`, :class:`list`, + :class:`tuple`, and :class:`dict` classes, as well as the :mod:`collections` + module. .. versionadded:: 2.4 @@ -566,8 +596,21 @@ available. They are listed here in alphabetical order. .. function:: hex(x) - Convert an integer number (of any size) to a hexadecimal string. The result is a - valid Python expression. + Convert an integer number (of any size) to a lowercase hexadecimal string + prefixed with "0x", for example: + + >>> hex(255) + '0xff' + >>> hex(-42) + '-0x2a' + >>> hex(1L) + '0x1L' + + If x is not a Python :class:`int` or :class:`long` object, it has to + define an __index__() method that returns an integer. + + See also :func:`int` for converting a hexadecimal string to an + integer using a base of 16. .. note:: @@ -602,20 +645,26 @@ available. They are listed here in alphabetical order. Consider using the :func:`raw_input` function for general input from users. -.. function:: int([x[, base]]) +.. function:: int(x=0) + int(x, base=10) + + Convert a number or string *x* to an integer, or return ``0`` if no + arguments are given. If *x* is a number, it can be a plain integer, a long + integer, or a floating point number. If *x* is floating point, the conversion + truncates towards zero. If the argument is outside the integer range, the + function returns a long object instead. - Convert a string or number to a plain integer. If the argument is a string, - it must contain a possibly signed decimal number representable as a Python - integer, possibly embedded in whitespace. The *base* parameter gives the - base for the conversion (which is 10 by default) and may be any integer in - the range [2, 36], or zero. If *base* is zero, the proper radix is - determined based on the contents of string; the interpretation is the same as - for integer literals. (See :ref:`numbers`.) If *base* is specified and *x* - is not a string, :exc:`TypeError` is raised. Otherwise, the argument may be a - plain or long integer or a floating point number. Conversion of floating - point numbers to integers truncates (towards zero). If the argument is - outside the integer range a long object will be returned instead. If no - arguments are given, returns ``0``. + If *x* is not a number or if *base* is given, then *x* must be a string or + Unicode object representing an :ref:`integer literal <integers>` in radix + *base*. Optionally, the literal can be + preceded by ``+`` or ``-`` (with no space in between) and surrounded by + whitespace. A base-n literal consists of the digits 0 to n-1, with ``a`` + to ``z`` (or ``A`` to ``Z``) having + values 10 to 35. The default *base* is 10. The allowed values are 0 and 2-36. + Base-2, -8, and -16 literals can be optionally prefixed with ``0b``/``0B``, + ``0o``/``0O``/``0``, or ``0x``/``0X``, as with integer literals in code. + Base 0 means to interpret the string exactly as an integer literal, so that + the actual base is 2, 8, 10, or 16. The integer type is described in :ref:`typesnumeric`. @@ -665,7 +714,7 @@ available. They are listed here in alphabetical order. One useful application of the second form of :func:`iter` is to read lines of a file until a certain line is reached. The following example reads a file - until the :meth:`readline` method returns an empty string:: + until the :meth:`~io.TextIOBase.readline` method returns an empty string:: with open('mydata.txt') as fp: for line in iter(fp.readline, ''): @@ -677,7 +726,8 @@ available. They are listed here in alphabetical order. .. function:: len(s) Return the length (the number of items) of an object. The argument may be a - sequence (string, tuple or list) or a mapping (dictionary). + sequence (such as a string, bytes, tuple, list, or range) or a collection + (such as a dictionary, set, or frozen set). .. function:: list([iterable]) @@ -706,7 +756,8 @@ available. They are listed here in alphabetical order. affect the values of local and free variables used by the interpreter. -.. function:: long([x[, base]]) +.. function:: long(x=0) + long(x, base=10) Convert a string or number to a long integer. If the argument is a string, it must contain a possibly signed number of arbitrary size, possibly embedded in @@ -732,11 +783,16 @@ available. They are listed here in alphabetical order. the result is always a list. -.. function:: max(iterable[, args...][key]) +.. function:: max(iterable[, key]) + max(arg1, arg2, *args[, key]) - With a single argument *iterable*, return the largest item of a non-empty - iterable (such as a string, tuple or list). With more than one argument, return - the largest of the arguments. + Return the largest item in an iterable or the largest of two or more + arguments. + + If one positional argument is provided, *iterable* must be a non-empty + iterable (such as a non-empty string, tuple or list). The largest item + in the iterable is returned. If two or more positional arguments are + provided, the largest of the positional arguments is returned. The optional *key* argument specifies a one-argument ordering function like that used for :meth:`list.sort`. The *key* argument, if supplied, must be in keyword @@ -745,7 +801,7 @@ available. They are listed here in alphabetical order. .. versionchanged:: 2.5 Added support for the optional *key* argument. - +.. _func-memoryview: .. function:: memoryview(obj) :noindex: @@ -753,11 +809,16 @@ available. They are listed here in alphabetical order. :ref:`typememoryview` for more information. -.. function:: min(iterable[, args...][key]) +.. function:: min(iterable[, key]) + min(arg1, arg2, *args[, key]) + + Return the smallest item in an iterable or the smallest of two or more + arguments. - With a single argument *iterable*, return the smallest item of a non-empty - iterable (such as a string, tuple or list). With more than one argument, return - the smallest of the arguments. + If one positional argument is provided, *iterable* must be a non-empty + iterable (such as a non-empty string, tuple or list). The smallest item + in the iterable is returned. If two or more positional arguments are + provided, the smallest of the positional arguments is returned. The optional *key* argument specifies a one-argument ordering function like that used for :meth:`list.sort`. The *key* argument, if supplied, must be in keyword @@ -829,26 +890,29 @@ available. They are listed here in alphabetical order. The optional *buffering* argument specifies the file's desired buffer size: 0 means unbuffered, 1 means line buffered, any other positive value means use a - buffer of (approximately) that size. A negative *buffering* means to use the - system default, which is usually line buffered for tty devices and fully - buffered for other files. If omitted, the system default is used. [#]_ + buffer of (approximately) that size (in bytes). A negative *buffering* means + to use the system default, which is usually line buffered for tty devices and + fully buffered for other files. If omitted, the system default is used. [#]_ - Modes ``'r+'``, ``'w+'`` and ``'a+'`` open the file for updating (note that - ``'w+'`` truncates the file). Append ``'b'`` to the mode to open the file in + Modes ``'r+'``, ``'w+'`` and ``'a+'`` open the file for updating (reading and writing); + note that ``'w+'`` truncates the file. Append ``'b'`` to the mode to open the file in binary mode, on systems that differentiate between binary and text files; on systems that don't have this distinction, adding the ``'b'`` has no effect. + .. index:: + single: universal newlines; open() built-in function + In addition to the standard :c:func:`fopen` values *mode* may be ``'U'`` or - ``'rU'``. Python is usually built with universal newline support; supplying - ``'U'`` opens the file as a text file, but lines may be terminated by any of the - following: the Unix end-of-line convention ``'\n'``, the Macintosh convention - ``'\r'``, or the Windows convention ``'\r\n'``. All of these external - representations are seen as ``'\n'`` by the Python program. If Python is built - without universal newline support a *mode* with ``'U'`` is the same as normal - text mode. Note that file objects so opened also have an attribute called - :attr:`newlines` which has a value of ``None`` (if no newlines have yet been - seen), ``'\n'``, ``'\r'``, ``'\r\n'``, or a tuple containing all the newline - types seen. + ``'rU'``. Python is usually built with :term:`universal newlines` support; + supplying ``'U'`` opens the file as a text file, but lines may be terminated + by any of the following: the Unix end-of-line convention ``'\n'``, the + Macintosh convention ``'\r'``, or the Windows convention ``'\r\n'``. All of + these external representations are seen as ``'\n'`` by the Python program. + If Python is built without universal newlines support a *mode* with ``'U'`` + is the same as normal text mode. Note that file objects so opened also have + an attribute called :attr:`newlines` which has a value of ``None`` (if no + newlines have yet been seen), ``'\n'``, ``'\r'``, ``'\r\n'``, or a tuple + containing all the newline types seen. Python enforces that the mode, after stripping ``'U'``, begins with ``'r'``, ``'w'`` or ``'a'``. @@ -894,16 +958,16 @@ available. They are listed here in alphabetical order. accidents.) -.. function:: print([object, ...][, sep=' '][, end='\\n'][, file=sys.stdout]) +.. function:: print(*objects, sep=' ', end='\\n', file=sys.stdout) - Print *object*\(s) to the stream *file*, separated by *sep* and followed by + Print *objects* to the stream *file*, separated by *sep* and followed by *end*. *sep*, *end* and *file*, if present, must be given as keyword arguments. All non-keyword arguments are converted to strings like :func:`str` does and written to the stream, separated by *sep* and followed by *end*. Both *sep* and *end* must be strings; they can also be ``None``, which means to use the - default values. If no *object* is given, :func:`print` will just write + default values. If no *objects* are given, :func:`print` will just write *end*. The *file* argument must be an object with a ``write(string)`` method; if it @@ -963,10 +1027,10 @@ available. They are listed here in alphabetical order. turns the :meth:`voltage` method into a "getter" for a read-only attribute with the same name. - A property object has :attr:`getter`, :attr:`setter`, and :attr:`deleter` - methods usable as decorators that create a copy of the property with the - corresponding accessor function set to the decorated function. This is - best explained with an example:: + A property object has :attr:`~property.getter`, :attr:`~property.setter`, + and :attr:`~property.deleter` methods usable as decorators that create a + copy of the property with the corresponding accessor function set to the + decorated function. This is best explained with an example:: class C(object): def __init__(self): @@ -1001,7 +1065,8 @@ available. They are listed here in alphabetical order. The ``getter``, ``setter``, and ``deleter`` attributes were added. -.. function:: range([start,] stop[, step]) +.. function:: range(stop) + range(start, stop[, step]) This is a versatile function to create lists containing arithmetic progressions. It is most often used in :keyword:`for` loops. The arguments must be plain @@ -1065,7 +1130,7 @@ available. They are listed here in alphabetical order. except StopIteration: raise TypeError('reduce() of empty sequence with no initial value') accum_value = initializer - for x in iterable: + for x in it: accum_value = function(accum_value, x) return accum_value @@ -1131,6 +1196,7 @@ available. They are listed here in alphabetical order. continue to use the old class definition. The same is true for derived classes. +.. _func-repr: .. function:: repr(object) Return a string containing a printable representation of an object. This is @@ -1157,13 +1223,14 @@ available. They are listed here in alphabetical order. Added the possibility to write a custom :meth:`__reversed__` method. -.. function:: round(x[, n]) +.. function:: round(number[, ndigits]) - Return the floating point value *x* rounded to *n* digits after the decimal - point. If *n* is omitted, it defaults to zero. The result is a floating point - number. Values are rounded to the closest multiple of 10 to the power minus - *n*; if two multiples are equally close, rounding is done away from 0 (so. for - example, ``round(0.5)`` is ``1.0`` and ``round(-0.5)`` is ``-1.0``). + Return the floating point value *number* rounded to *ndigits* digits after + the decimal point. If *ndigits* is omitted, it defaults to zero. The result + is a floating point number. Values are rounded to the closest multiple of + 10 to the power minus *ndigits*; if two multiples are equally close, + rounding is done away from 0 (so. for example, ``round(0.5)`` is ``1.0`` and + ``round(-0.5)`` is ``-1.0``). .. note:: @@ -1174,14 +1241,18 @@ available. They are listed here in alphabetical order. can't be represented exactly as a float. See :ref:`tut-fp-issues` for more information. + +.. _func-set: .. function:: set([iterable]) :noindex: - Return a new set, optionally with elements taken from *iterable*. - The set type is described in :ref:`types-set`. + Return a new :class:`set` object, optionally with elements taken from + *iterable*. ``set`` is a built-in class. See :class:`set` and + :ref:`types-set` for documentation about this class. - For other containers see the built in :class:`dict`, :class:`list`, and - :class:`tuple` classes, and the :mod:`collections` module. + For other containers see the built-in :class:`frozenset`, :class:`list`, + :class:`tuple`, and :class:`dict` classes, as well as the :mod:`collections` + module. .. versionadded:: 2.4 @@ -1195,19 +1266,20 @@ available. They are listed here in alphabetical order. ``x.foobar = 123``. -.. function:: slice([start,] stop[, step]) +.. function:: slice(stop) + slice(start, stop[, step]) .. index:: single: Numerical Python Return a :term:`slice` object representing the set of indices specified by ``range(start, stop, step)``. The *start* and *step* arguments default to - ``None``. Slice objects have read-only data attributes :attr:`start`, - :attr:`stop` and :attr:`step` which merely return the argument values (or their - default). They have no other explicit functionality; however they are used by - Numerical Python and other third party extensions. Slice objects are also - generated when extended indexing syntax is used. For example: - ``a[start:stop:step]`` or ``a[start:stop, i]``. See :func:`itertools.islice` - for an alternate version that returns an iterator. + ``None``. Slice objects have read-only data attributes :attr:`~slice.start`, + :attr:`~slice.stop` and :attr:`~slice.step` which merely return the argument + values (or their default). They have no other explicit functionality; + however they are used by Numerical Python and other third party extensions. + Slice objects are also generated when extended indexing syntax is used. For + example: ``a[start:stop:step]`` or ``a[start:stop, i]``. See + :func:`itertools.islice` for an alternate version that returns an iterator. .. function:: sorted(iterable[, cmp[, key[, reverse]]]) @@ -1250,9 +1322,10 @@ available. They are listed here in alphabetical order. A static method does not receive an implicit first argument. To declare a static method, use this idiom:: - class C: + class C(object): @staticmethod - def f(arg1, arg2, ...): ... + def f(arg1, arg2, ...): + ... The ``@staticmethod`` form is a function :term:`decorator` -- see the description of function definitions in :ref:`function` for details. @@ -1273,7 +1346,7 @@ available. They are listed here in alphabetical order. Function decorator syntax added. -.. function:: str([object]) +.. function:: str(object='') Return a string containing a nicely printable representation of an object. For strings, this returns the string itself. The difference with ``repr(object)`` @@ -1311,9 +1384,10 @@ available. They are listed here in alphabetical order. been overridden in a class. The search order is same as that used by :func:`getattr` except that the *type* itself is skipped. - The :attr:`__mro__` attribute of the *type* lists the method resolution - search order used by both :func:`getattr` and :func:`super`. The attribute - is dynamic and can change whenever the inheritance hierarchy is updated. + The :attr:`~class.__mro__` attribute of the *type* lists the method + resolution search order used by both :func:`getattr` and :func:`super`. The + attribute is dynamic and can change whenever the inheritance hierarchy is + updated. If the second argument is omitted, the super object returned is unbound. If the second argument is an object, ``isinstance(obj, type)`` must be true. If @@ -1377,26 +1451,21 @@ available. They are listed here in alphabetical order. .. function:: type(object) + type(name, bases, dict) .. index:: object: type - Return the type of an *object*. The return value is a type object. The - :func:`isinstance` built-in function is recommended for testing the type of an - object. - - With three arguments, :func:`type` functions as a constructor as detailed below. - - -.. function:: type(name, bases, dict) - :noindex: + With one argument, return the type of an *object*. The return value is a + type object. The :func:`isinstance` built-in function is recommended for + testing the type of an object. - Return a new type object. This is essentially a dynamic form of the - :keyword:`class` statement. The *name* string is the class name and becomes the - :attr:`__name__` attribute; the *bases* tuple itemizes the base classes and - becomes the :attr:`__bases__` attribute; and the *dict* dictionary is the - namespace containing definitions for class body and becomes the :attr:`__dict__` - attribute. For example, the following two statements create identical - :class:`type` objects: + With three arguments, return a new type object. This is essentially a + dynamic form of the :keyword:`class` statement. The *name* string is the + class name and becomes the :attr:`~class.__name__` attribute; the *bases* tuple + itemizes the base classes and becomes the :attr:`~class.__bases__` attribute; + and the *dict* dictionary is the namespace containing definitions for class + body and becomes the :attr:`~object.__dict__` attribute. For example, the + following two statements create identical :class:`type` objects: >>> class X(object): ... a = 1 @@ -1418,7 +1487,8 @@ available. They are listed here in alphabetical order. .. versionadded:: 2.0 -.. function:: unicode([object[, encoding [, errors]]]) +.. function:: unicode(object='') + unicode(object[, encoding [, errors]]) Return the Unicode string version of *object* using one of the following modes: @@ -1458,7 +1528,7 @@ available. They are listed here in alphabetical order. .. function:: vars([object]) - Return the :attr:`__dict__` attribute for a module, class, instance, + Return the :attr:`~object.__dict__` attribute for a module, class, instance, or any other object with a :attr:`__dict__` attribute. Objects such as modules and instances have an updateable :attr:`__dict__` @@ -1471,16 +1541,19 @@ available. They are listed here in alphabetical order. dictionary are ignored. -.. function:: xrange([start,] stop[, step]) +.. function:: xrange(stop) + xrange(start, stop[, step]) - This function is very similar to :func:`range`, but returns an "xrange object" + This function is very similar to :func:`range`, but returns an :ref:`xrange + object <typesseq-xrange>` instead of a list. This is an opaque sequence type which yields the same values as the corresponding list, without actually storing them all simultaneously. The advantage of :func:`xrange` over :func:`range` is minimal (since :func:`xrange` still has to create the values when asked for them) except when a very large range is used on a memory-starved machine or when all of the range's elements are never used (such as when the loop is usually terminated with - :keyword:`break`). + :keyword:`break`). For more information on xrange objects, see + :ref:`typesseq-xrange` and :ref:`typesseq`. .. impl-detail:: @@ -1535,7 +1608,7 @@ available. They are listed here in alphabetical order. .. note:: This is an advanced function that is not needed in everyday Python - programming. + programming, unlike :func:`importlib.import_module`. This function is invoked by the :keyword:`import` statement. It can be replaced (by importing the :mod:`__builtin__` module and assigning to @@ -1586,15 +1659,8 @@ available. They are listed here in alphabetical order. names. If you simply want to import a module (potentially within a package) by name, - you can call :func:`__import__` and then look it up in :data:`sys.modules`:: + use :func:`importlib.import_module`. - >>> import sys - >>> name = 'foo.bar.baz' - >>> __import__(name) - <module 'foo' from ...> - >>> baz = sys.modules[name] - >>> baz - <module 'foo.bar.baz' from ...> .. versionchanged:: 2.5 The level parameter was added. @@ -1631,7 +1697,8 @@ bypass these functions without concerns about missing something important. ``function(*args, **keywords)``. .. deprecated:: 2.3 - Use the extended call syntax with ``*args`` and ``**keywords`` instead. + Use ``function(*args, **keywords)`` instead of + ``apply(function, args, keywords)`` (see :ref:`tut-unpacking-arguments`). .. function:: buffer(object[, offset[, size]]) diff --git a/Doc/library/future_builtins.rst b/Doc/library/future_builtins.rst index 04f2052..62392eb 100644 --- a/Doc/library/future_builtins.rst +++ b/Doc/library/future_builtins.rst @@ -50,6 +50,11 @@ Available builtins are: Works like :func:`itertools.imap`. + .. note:: + + In Python 3, :func:`map` does not accept ``None`` for the + function argument. + .. function:: oct(object) Works like the built-in :func:`oct`, but instead of :meth:`__oct__` it will diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst index 80a2d92..be0063d 100644 --- a/Doc/library/gc.rst +++ b/Doc/library/gc.rst @@ -132,8 +132,8 @@ The :mod:`gc` module provides the following functions: Return a list of objects directly referred to by any of the arguments. The referents returned are those objects visited by the arguments' C-level - :attr:`tp_traverse` methods (if any), and may not be all objects actually - directly reachable. :attr:`tp_traverse` methods are supported only by objects + :c:member:`~PyTypeObject.tp_traverse` methods (if any), and may not be all objects actually + directly reachable. :c:member:`~PyTypeObject.tp_traverse` methods are supported only by objects that support garbage collection, and are only required to visit objects that may be involved in a cycle. So, for example, if an integer is directly reachable from an argument, that integer object may or may not appear in the result list. @@ -142,8 +142,8 @@ The :mod:`gc` module provides the following functions: .. function:: is_tracked(obj) - Returns True if the object is currently tracked by the garbage collector, - False otherwise. As a general rule, instances of atomic types aren't + Returns ``True`` if the object is currently tracked by the garbage collector, + ``False`` otherwise. As a general rule, instances of atomic types aren't tracked and instances of non-atomic types (containers, user-defined objects...) are. However, some type-specific optimizations can be present in order to suppress the garbage collector footprint of simple instances diff --git a/Doc/library/gdbm.rst b/Doc/library/gdbm.rst index aec23e6..f36bb28 100644 --- a/Doc/library/gdbm.rst +++ b/Doc/library/gdbm.rst @@ -6,9 +6,9 @@ :synopsis: GNU's reinterpretation of dbm. .. note:: - The :mod:`gdbm` module has been renamed to :mod:`dbm.gnu` in Python 3.0. The + The :mod:`gdbm` module has been renamed to :mod:`dbm.gnu` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. .. index:: module: dbm @@ -116,6 +116,11 @@ methods: unwritten data to be written to the disk. +.. function:: close() + + Close the ``gdbm`` database. + + .. seealso:: Module :mod:`anydbm` diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst index b3ba614..2dfb102 100644 --- a/Doc/library/getopt.rst +++ b/Doc/library/getopt.rst @@ -10,6 +10,7 @@ -------------- .. note:: + The :mod:`getopt` module is a parser for command line options whose API is designed to be familiar to users of the C :c:func:`getopt` function. Users who are unfamiliar with the C :c:func:`getopt` function or who would like to write @@ -126,7 +127,7 @@ In a script, typical usage is something like this:: def main(): try: opts, args = getopt.getopt(sys.argv[1:], "ho:v", ["help", "output="]) - except getopt.GetoptError, err: + except getopt.GetoptError as err: # print help information and exit: print str(err) # will print something like "option -a not recognized" usage() diff --git a/Doc/library/gl.rst b/Doc/library/gl.rst index 0d189dc..7ea9cf3 100644 --- a/Doc/library/gl.rst +++ b/Doc/library/gl.rst @@ -8,7 +8,7 @@ .. deprecated:: 2.6 - The :mod:`gl` module has been deprecated for removal in Python 3.0. + The :mod:`gl` module has been removed in Python 3. This module provides access to the Silicon Graphics *Graphics Library*. It is @@ -168,7 +168,7 @@ Here is a tiny but complete example GL program in Python:: .. deprecated:: 2.6 - The :mod:`DEVICE` module has been deprecated for removal in Python 3.0. + The :mod:`DEVICE` module has been removed in Python 3. This modules defines the constants used by the Silicon Graphics *Graphics @@ -186,7 +186,7 @@ module source file for details. .. deprecated:: 2.6 - The :mod:`GL` module has been deprecated for removal in Python 3.0. + The :mod:`GL` module has been removed in Python 3. This module contains constants used by the Silicon Graphics *Graphics Library* from the C header file ``<gl/gl.h>``. Read the module source file for details. diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst index 68cc9f0..75f67b9 100644 --- a/Doc/library/glob.rst +++ b/Doc/library/glob.rst @@ -16,8 +16,13 @@ according to the rules used by the Unix shell. No tilde expansion is done, but ``*``, ``?``, and character ranges expressed with ``[]`` will be correctly matched. This is done by using the :func:`os.listdir` and :func:`fnmatch.fnmatch` functions in concert, and not by actually invoking a -subshell. (For tilde and shell variable expansion, use -:func:`os.path.expanduser` and :func:`os.path.expandvars`.) +subshell. Note that unlike :func:`fnmatch.fnmatch`, :mod:`glob` treats +filenames beginning with a dot (``.``) as special cases. (For tilde and shell +variable expansion, use :func:`os.path.expanduser` and +:func:`os.path.expandvars`.) + +For a literal match, wrap the meta-characters in brackets. +For example, ``'[?]'`` matches the character ``'?'``. .. function:: glob(pathname) @@ -49,6 +54,15 @@ preserved. :: >>> glob.glob('?.gif') ['1.gif'] +If the directory contains files starting with ``.`` they won't be matched by +default. For example, consider a directory containing :file:`card.gif` and +:file:`.card.gif`:: + + >>> import glob + >>> glob.glob('*.gif') + ['card.gif'] + >>> glob.glob('.c*') + ['.card.gif'] .. seealso:: diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index e074bfc..e26fe28 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -22,9 +22,6 @@ Note that additional file formats which can be decompressed by the :program:`gzip` and :program:`gunzip` programs, such as those produced by :program:`compress` and :program:`pack`, are not supported by this module. -For other archive formats, see the :mod:`bz2`, :mod:`zipfile`, and -:mod:`tarfile` modules. - The module defines the following items: @@ -36,12 +33,12 @@ The module defines the following items: given a non-trivial value. The new class instance is based on *fileobj*, which can be a regular file, a - :class:`StringIO` object, or any other object which simulates a file. It + :class:`~StringIO.StringIO` object, or any other object which simulates a file. It defaults to ``None``, in which case *filename* is opened to provide a file object. When *fileobj* is not ``None``, the *filename* argument is only used to be - included in the :program:`gzip` file header, which may includes the original + included in the :program:`gzip` file header, which may include the original filename of the uncompressed file. It defaults to the filename of *fileobj*, if discernible; otherwise, it defaults to the empty string, and in this case the original filename is not included in the header. @@ -52,9 +49,10 @@ The module defines the following items: not given, the 'b' flag will be added to the mode to ensure the file is opened in binary mode for cross-platform portability. - The *compresslevel* argument is an integer from ``1`` to ``9`` controlling the - level of compression; ``1`` is fastest and produces the least compression, and - ``9`` is slowest and produces the most compression. The default is ``9``. + The *compresslevel* argument is an integer from ``0`` to ``9`` controlling + the level of compression; ``1`` is fastest and produces the least + compression, and ``9`` is slowest and produces the most compression. ``0`` + is no compression. The default is ``9``. The *mtime* argument is an optional numeric timestamp to be written to the stream when compressing. All :program:`gzip` compressed streams are @@ -67,9 +65,9 @@ The module defines the following items: Calling a :class:`GzipFile` object's :meth:`close` method does not close *fileobj*, since you might wish to append more material after the compressed - data. This also allows you to pass a :class:`StringIO` object opened for + data. This also allows you to pass a :class:`~StringIO.StringIO` object opened for writing as *fileobj*, and retrieve the resulting memory buffer using the - :class:`StringIO` object's :meth:`getvalue` method. + :class:`StringIO` object's :meth:`~StringIO.StringIO.getvalue` method. :class:`GzipFile` supports iteration and the :keyword:`with` statement. @@ -79,6 +77,9 @@ The module defines the following items: .. versionchanged:: 2.7 Support for zero-padded files was added. + .. versionadded:: 2.7 + The *mtime* argument. + .. function:: open(filename[, mode[, compresslevel]]) @@ -95,7 +96,7 @@ Examples of usage Example of how to read a compressed file:: import gzip - f = gzip.open('/home/joe/file.txt.gz', 'rb') + f = gzip.open('file.txt.gz', 'rb') file_content = f.read() f.close() @@ -103,15 +104,15 @@ Example of how to create a compressed GZIP file:: import gzip content = "Lots of content here" - f = gzip.open('/home/joe/file.txt.gz', 'wb') + f = gzip.open('file.txt.gz', 'wb') f.write(content) f.close() Example of how to GZIP compress an existing file:: import gzip - f_in = open('/home/joe/file.txt', 'rb') - f_out = gzip.open('/home/joe/file.txt.gz', 'wb') + f_in = open('file.txt', 'rb') + f_out = gzip.open('file.txt.gz', 'wb') f_out.writelines(f_in) f_out.close() f_in.close() diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst index 063ad59..e3b5ebb 100644 --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -25,12 +25,14 @@ digest are interchangeable. Older algorithms were called message digests. The modern term is secure hash. .. note:: - If you want the adler32 or crc32 hash functions they are available in + + If you want the adler32 or crc32 hash functions, they are available in the :mod:`zlib` module. .. warning:: - Some algorithms have known hash collision weaknesses, see the FAQ at the end. + Some algorithms have known hash collision weaknesses, refer to the "See + also" section at the end. There is one constructor method named for each type of :dfn:`hash`. All return a hash object with the same simple interface. For example: use :func:`sha1` to @@ -108,7 +110,6 @@ A hash object has the following methods: m.update(b)`` is equivalent to ``m.update(a+b)``. .. versionchanged:: 2.7 - The Python GIL is released to allow other threads to run while hash updates on data larger than 2048 bytes is taking place when using hash algorithms supplied by OpenSSL. @@ -134,6 +135,46 @@ A hash object has the following methods: compute the digests of strings that share a common initial substring. +Key Derivation Function +----------------------- + +Key derivation and key stretching algorithms are designed for secure password +hashing. Naive algorithms such as ``sha1(password)`` are not resistant against +brute-force attacks. A good password hashing function must be tunable, slow, and +include a `salt <https://en.wikipedia.org/wiki/Salt_%28cryptography%29>`_. + + +.. function:: pbkdf2_hmac(name, password, salt, rounds, dklen=None) + + The function provides PKCS#5 password-based key derivation function 2. It + uses HMAC as pseudorandom function. + + The string *name* is the desired name of the hash digest algorithm for + HMAC, e.g. 'sha1' or 'sha256'. *password* and *salt* are interpreted as + buffers of bytes. Applications and libraries should limit *password* to + a sensible value (e.g. 1024). *salt* should be about 16 or more bytes from + a proper source, e.g. :func:`os.urandom`. + + The number of *rounds* should be chosen based on the hash algorithm and + computing power. As of 2013, at least 100,000 rounds of SHA-256 is suggested. + + *dklen* is the length of the derived key. If *dklen* is ``None`` then the + digest size of the hash algorithm *name* is used, e.g. 64 for SHA-512. + + >>> import hashlib, binascii + >>> dk = hashlib.pbkdf2_hmac('sha256', b'password', b'salt', 100000) + >>> binascii.hexlify(dk) + b'0394a2ede332c9a13eb82e9b24631604c31df978b4e2f0fbd2c549944f9d79a5' + + .. versionadded:: 2.7.8 + + .. note:: + + A fast implementation of *pbkdf2_hmac* is available with OpenSSL. The + Python implementation uses an inline version of :mod:`hmac`. It is about + three times slower and doesn't release the GIL. + + .. seealso:: Module :mod:`hmac` diff --git a/Doc/library/heapq.rst b/Doc/library/heapq.rst index f0723b7..e8acd6c 100644 --- a/Doc/library/heapq.rst +++ b/Doc/library/heapq.rst @@ -260,7 +260,7 @@ A nice feature of this sort is that you can efficiently insert new items while the sort is going on, provided that the inserted items are not "better" than the last 0'th element you extracted. This is especially useful in simulation contexts, where the tree holds all incoming events, and the "win" condition -means the smallest scheduled time. When an event schedule other events for +means the smallest scheduled time. When an event schedules other events for execution, they are scheduled into the future, so they can easily go into the heap. So, a heap is a good structure for implementing schedulers (this is what I used for my MIDI sequencer :-). diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst index e962ff0..09e819a 100644 --- a/Doc/library/hmac.rst +++ b/Doc/library/hmac.rst @@ -20,7 +20,7 @@ This module implements the HMAC algorithm as described by :rfc:`2104`. Return a new hmac object. If *msg* is present, the method call ``update(msg)`` is made. *digestmod* is the digest constructor or module for the HMAC object to - use. It defaults to the :func:`hashlib.md5` constructor. + use. It defaults to the :data:`hashlib.md5` constructor. An HMAC object has the following methods: @@ -38,6 +38,13 @@ An HMAC object has the following methods: This string will be the same length as the *digest_size* of the digest given to the constructor. It may contain non-ASCII characters, including NUL bytes. + .. warning:: + + When comparing the output of :meth:`digest` to an externally-supplied + digest during a verification routine, it is recommended to use the + :func:`compare_digest` function instead of the ``==`` operator + to reduce the vulnerability to timing attacks. + .. method:: HMAC.hexdigest() @@ -45,6 +52,13 @@ An HMAC object has the following methods: containing only hexadecimal digits. This may be used to exchange the value safely in email or other non-binary environments. + .. warning:: + + When comparing the output of :meth:`hexdigest` to an externally-supplied + digest during a verification routine, it is recommended to use the + :func:`compare_digest` function instead of the ``==`` operator + to reduce the vulnerability to timing attacks. + .. method:: HMAC.copy() @@ -52,6 +66,25 @@ An HMAC object has the following methods: compute the digests of strings that share a common initial substring. +This module also provides the following helper function: + +.. function:: compare_digest(a, b) + + Return ``a == b``. This function uses an approach designed to prevent + timing analysis by avoiding content-based short circuiting behaviour, + making it appropriate for cryptography. *a* and *b* must both be of the + same type: either :class:`unicode` or a :term:`bytes-like object`. + + .. note:: + + If *a* and *b* are of different lengths, or if an error occurs, + a timing attack could theoretically reveal information about the + types and lengths of *a* and *b*--but not their values. + + + .. versionadded:: 2.7.7 + + .. seealso:: Module :mod:`hashlib` diff --git a/Doc/library/htmllib.rst b/Doc/library/htmllib.rst index f253d12..9e68f45 100644 --- a/Doc/library/htmllib.rst +++ b/Doc/library/htmllib.rst @@ -6,7 +6,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`htmllib` module has been removed in Python 3.0. + The :mod:`htmllib` module has been removed in Python 3. .. index:: @@ -162,8 +162,8 @@ additional methods and instance variables for use within tag methods. .. note:: The :mod:`htmlentitydefs` module has been renamed to :mod:`html.entities` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. **Source code:** :source:`Lib/htmlentitydefs.py` diff --git a/Doc/library/httplib.rst b/Doc/library/httplib.rst index f26fba6..fcdfbc0 100644 --- a/Doc/library/httplib.rst +++ b/Doc/library/httplib.rst @@ -6,8 +6,8 @@ .. note:: The :mod:`httplib` module has been renamed to :mod:`http.client` in Python - 3.0. The :term:`2to3` tool will automatically adapt imports when converting - your sources to 3.0. + 3. The :term:`2to3` tool will automatically adapt imports when converting + your sources to Python 3. .. index:: @@ -89,7 +89,7 @@ The module provides the following classes: *source_address* was added. -.. class:: HTTPResponse(sock[, debuglevel=0][, strict=0]) +.. class:: HTTPResponse(sock, debuglevel=0, strict=0) Class whose instances are returned upon successful connection. Not instantiated directly by user. @@ -612,3 +612,20 @@ Here is an example session that shows how to ``POST`` requests:: 'Redirecting to <a href="http://bugs.python.org/issue12524">http://bugs.python.org/issue12524</a>' >>> conn.close() +Client side ``HTTP PUT`` requests are very similar to ``POST`` requests. The +difference lies only the server side where HTTP server will allow resources to +be created via ``PUT`` request. Here is an example session that shows how to do +``PUT`` request using httplib:: + + >>> # This creates an HTTP message + >>> # with the content of BODY as the enclosed representation + >>> # for the resource http://localhost:8080/foobar + ... + >>> import httplib + >>> BODY = "***filecontents***" + >>> conn = httplib.HTTPConnection("localhost", 8080) + >>> conn.request("PUT", "/file", BODY) + >>> response = conn.getresponse() + >>> print response.status, response.reason + 200, OK + diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 6bd1898..36d78b0 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -33,8 +33,8 @@ Menus File menu ^^^^^^^^^ -New window - create a new editing window +New file + create a new file editing window Open... open an existing file @@ -154,27 +154,77 @@ The rest of this menu lists the names of all open windows; select one to bring it to the foreground (deiconifying it if necessary). -Debug menu (in the Python Shell window only) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Debug menu +^^^^^^^^^^ + +* in the Python Shell window only Go to file/line - look around the insert point for a filename and linenumber, open the file, and - show the line. + Look around the insert point for a filename and line number, open the file, + and show the line. Useful to view the source lines referenced in an + exception traceback. -Open stack viewer - show the stack traceback of the last exception +Debugger + Run commands in the shell under the debugger. -Debugger toggle - Run commands in the shell under the debugger +Stack viewer + Show the stack traceback of the last exception. -JIT Stack viewer toggle - Open stack viewer on traceback +Auto-open Stack Viewer + Open stack viewer on traceback. .. index:: single: stack viewer single: debugger +Edit context menu +^^^^^^^^^^^^^^^^^ + +* Right-click in Edit window (Control-click on OS X) + +Cut + Copy selection into system-wide clipboard; then delete selection + +Copy + Copy selection into system-wide clipboard + +Paste + Insert system-wide clipboard into window + +Set Breakpoint + Sets a breakpoint. Breakpoints are only enabled when the debugger is open. + +Clear Breakpoint + Clears the breakpoint on that line. + +.. index:: + single: Cut + single: Copy + single: Paste + single: Set Breakpoint + single: Clear Breakpoint + single: breakpoints + + +Shell context menu +^^^^^^^^^^^^^^^^^^ + +* Right-click in Python Shell window (Control-click on OS X) + +Cut + Copy selection into system-wide clipboard; then delete selection + +Copy + Copy selection into system-wide clipboard + +Paste + Insert system-wide clipboard into window + +Go to file/line + Same as in Debug menu. + + Basic editing and navigation ---------------------------- diff --git a/Doc/library/imageop.rst b/Doc/library/imageop.rst index ceef0c7..e6cb669 100644 --- a/Doc/library/imageop.rst +++ b/Doc/library/imageop.rst @@ -7,7 +7,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`imageop` module has been removed in Python 3.0. + The :mod:`imageop` module has been removed in Python 3. The :mod:`imageop` module contains some useful operations on images. It operates on images consisting of 8 or 32 bit pixels stored in Python strings. This is diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 9fcbaaa..ca18a9c 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -313,8 +313,9 @@ An :class:`IMAP4` instance has the following methods: Opens socket to *port* at *host*. This method is implicitly called by the :class:`IMAP4` constructor. The connection objects established by this - method will be used in the ``read``, ``readline``, ``send``, and ``shutdown`` - methods. You may override this method. + method will be used in the :meth:`IMAP4.read`, :meth:`IMAP4.readline`, + :meth:`IMAP4.send`, and :meth:`IMAP4.shutdown` methods. You may override + this method. .. method:: IMAP4.partial(message_num, message_part, start, length) diff --git a/Doc/library/imgfile.rst b/Doc/library/imgfile.rst index 84ede95..f4c670f 100644 --- a/Doc/library/imgfile.rst +++ b/Doc/library/imgfile.rst @@ -8,7 +8,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`imgfile` module has been deprecated for removal in Python 3.0. + The :mod:`imgfile` module has been removed in Python 3. diff --git a/Doc/library/imghdr.rst b/Doc/library/imghdr.rst index 20f789f..24ab571 100644 --- a/Doc/library/imghdr.rst +++ b/Doc/library/imghdr.rst @@ -68,6 +68,6 @@ to this variable: Example:: >>> import imghdr - >>> imghdr.what('/tmp/bass.gif') + >>> imghdr.what('bass.gif') 'gif' diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst index 607dd14..8f98d65 100644 --- a/Doc/library/imp.rst +++ b/Doc/library/imp.rst @@ -65,7 +65,7 @@ This module provides an interface to the mechanisms used to implement the path and the last item in the *description* tuple is :const:`PKG_DIRECTORY`. This function does not handle hierarchical module names (names containing - dots). In order to find *P*.*M*, that is, submodule *M* of package *P*, use + dots). In order to find *P.M*, that is, submodule *M* of package *P*, use :func:`find_module` and :func:`load_module` to find and load package *P*, and then use :func:`find_module` with the *path* argument set to ``P.__path__``. When *P* itself has a dotted name, apply this recipe recursively. @@ -237,6 +237,17 @@ around for backward compatibility: using shared libraries is highly system dependent, and not all systems support it.) + .. impl-detail:: + + The import internals identify extension modules by filename, so doing + ``foo = load_dynamic("foo", "mod.so")`` and + ``bar = load_dynamic("bar", "mod.so")`` will result in both foo and bar + referring to the same module, regardless of whether or not + ``mod.so`` exports an ``initbar`` function. On systems which + support them, symlinks can be used to import multiple modules from + the same shared library, as each reference to the module will use + a different file name. + .. function:: load_source(name, pathname[, file]) diff --git a/Doc/library/imputil.rst b/Doc/library/imputil.rst index 94194e2..14d7041 100644 --- a/Doc/library/imputil.rst +++ b/Doc/library/imputil.rst @@ -7,7 +7,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`imputil` module has been removed in Python 3.0. + The :mod:`imputil` module has been removed in Python 3. .. index:: statement: import diff --git a/Doc/library/index.rst b/Doc/library/index.rst index 5d860f8..71ba916 100644 --- a/Doc/library/index.rst +++ b/Doc/library/index.rst @@ -4,9 +4,6 @@ The Python Standard Library ############################### -:Release: |version| -:Date: |today| - While :ref:`reference-index` describes the exact syntax and semantics of the Python language, this library reference manual describes the standard library that is distributed with Python. It also diff --git a/Doc/library/io.rst b/Doc/library/io.rst index 1d1f490..7b40085 100644 --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -92,7 +92,7 @@ Module Interface ``'b'`` binary mode ``'t'`` text mode (default) ``'+'`` open a disk file for updating (reading and writing) - ``'U'`` universal newline mode (for backwards compatibility; should + ``'U'`` universal newlines mode (for backwards compatibility; should not be used in new code) ========= =============================================================== @@ -141,14 +141,17 @@ Module Interface used. Any other error handling name that has been registered with :func:`codecs.register_error` is also valid. - *newline* controls how universal newlines works (it only applies to text - mode). It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It - works as follows: + .. index:: + single: universal newlines; open() (in module io) + + *newline* controls how :term:`universal newlines` works (it only applies to + text mode). It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. + It works as follows: * On input, if *newline* is ``None``, universal newlines mode is enabled. Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, and these are translated into ``'\n'`` before being returned to the caller. If it is - ``''``, universal newline mode is enabled, but line endings are returned to + ``''``, universal newlines mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated. @@ -275,7 +278,7 @@ I/O Base Classes .. method:: readable() - Return ``True`` if the stream can be read from. If False, :meth:`read` + Return ``True`` if the stream can be read from. If ``False``, :meth:`read` will raise :exc:`IOError`. .. method:: readline(limit=-1) @@ -293,6 +296,9 @@ I/O Base Classes to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds *hint*. + Note that it's already possible to iterate on file objects using ``for + line in file: ...`` without calling ``file.readlines()``. + .. method:: seek(offset, whence=SEEK_SET) Change the stream position to the given byte *offset*. *offset* is @@ -340,6 +346,12 @@ I/O Base Classes is usual for each of the lines provided to have a line separator at the end. + .. method:: __del__() + + Prepare for object destruction. :class:`IOBase` provides a default + implementation of this method that calls the instance's + :meth:`~IOBase.close` method. + .. class:: RawIOBase @@ -638,6 +650,7 @@ than raw I/O does. :exc:`UnsupportedOperation`. .. warning:: + :class:`BufferedRWPair` does not attempt to synchronize accesses to its underlying raw streams. You should not pass it the same object as reader and writer; use :class:`BufferedRandom` instead. @@ -696,11 +709,13 @@ Text I/O Read and return at most *n* characters from the stream as a single :class:`unicode`. If *n* is negative or ``None``, reads until EOF. - .. method:: readline() + .. method:: readline(limit=-1) Read until newline or EOF and return a single ``unicode``. If the stream is already at EOF, an empty string is returned. + If *limit* is specified, at most *limit* characters will be read. + .. method:: seek(offset, whence=SEEK_SET) Change the stream position to the given *offset*. Behaviour depends @@ -752,14 +767,25 @@ Text I/O sequences) can be used. Any other error handling name that has been registered with :func:`codecs.register_error` is also valid. - *newline* can be ``None``, ``''``, ``'\n'``, ``'\r'``, or ``'\r\n'``. It - controls the handling of line endings. If it is ``None``, universal newlines - is enabled. With this enabled, on input, the lines endings ``'\n'``, - ``'\r'``, or ``'\r\n'`` are translated to ``'\n'`` before being returned to - the caller. Conversely, on output, ``'\n'`` is translated to the system - default line separator, :data:`os.linesep`. If *newline* is any other of its - legal values, that newline becomes the newline when the file is read and it - is returned untranslated. On output, ``'\n'`` is converted to the *newline*. + .. index:: + single: universal newlines; io.TextIOWrapper class + + *newline* controls how line endings are handled. It can be ``None``, + ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It works as follows: + + * On input, if *newline* is ``None``, :term:`universal newlines` mode is + enabled. Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, + and these are translated into ``'\n'`` before being returned to the + caller. If it is ``''``, universal newlines mode is enabled, but line + endings are returned to the caller untranslated. If it has any of the + other legal values, input lines are only terminated by the given string, + and the line ending is returned to the caller untranslated. + + * On output, if *newline* is ``None``, any ``'\n'`` characters written are + translated to the system default line separator, :data:`os.linesep`. If + *newline* is ``''``, no translation takes place. If *newline* is any of + the other legal values, any ``'\n'`` characters written are translated to + the given string. If *line_buffering* is ``True``, :meth:`flush` is implied when a call to write contains a newline character. @@ -772,14 +798,14 @@ Text I/O Whether line buffering is enabled. -.. class:: StringIO(initial_value=u'', newline=None) +.. class:: StringIO(initial_value=u'', newline=u'\\n') An in-memory stream for unicode text. It inherits :class:`TextIOWrapper`. The initial value of the buffer (an empty unicode string by default) can be set by providing *initial_value*. The *newline* argument works like - that of :class:`TextIOWrapper`. The default is to do no newline - translation. + that of :class:`TextIOWrapper`. The default is to consider only ``\n`` + characters as end of lines and to do no newline translation. :class:`StringIO` provides this method in addition to those from :class:`TextIOWrapper` and its parents: @@ -807,10 +833,13 @@ Text I/O output.close() +.. index:: + single: universal newlines; io.IncrementalNewlineDecoder class + .. class:: IncrementalNewlineDecoder - A helper codec that decodes newlines for universal newlines mode. It - inherits :class:`codecs.IncrementalDecoder`. + A helper codec that decodes newlines for :term:`universal newlines` mode. + It inherits :class:`codecs.IncrementalDecoder`. Advanced topics diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index a553d09..ea279b0 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -52,12 +52,12 @@ Iterator Arguments Results :func:`compress` data, selectors (d[0] if s[0]), (d[1] if s[1]), ... ``compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F`` :func:`dropwhile` pred, seq seq[n], seq[n+1], starting when pred fails ``dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1`` :func:`groupby` iterable[, keyfunc] sub-iterators grouped by value of keyfunc(v) -:func:`ifilter` pred, seq elements of seq where pred(elem) is True ``ifilter(lambda x: x%2, range(10)) --> 1 3 5 7 9`` -:func:`ifilterfalse` pred, seq elements of seq where pred(elem) is False ``ifilterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8`` +:func:`ifilter` pred, seq elements of seq where pred(elem) is true ``ifilter(lambda x: x%2, range(10)) --> 1 3 5 7 9`` +:func:`ifilterfalse` pred, seq elements of seq where pred(elem) is false ``ifilterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8`` :func:`islice` seq, [start,] stop [, step] elements from seq[start:stop:step] ``islice('ABCDEFG', 2, None) --> C D E F G`` :func:`imap` func, p, q, ... func(p0, q0), func(p1, q1), ... ``imap(pow, (2,3,10), (5,2,3)) --> 32 9 1000`` :func:`starmap` func, seq func(\*seq[0]), func(\*seq[1]), ... ``starmap(pow, [(2,5), (3,2), (10,3)]) --> 32 9 1000`` -:func:`tee` it, n it1, it2 , ... itn splits one iterator into n +:func:`tee` it, n it1, it2, ... itn splits one iterator into n :func:`takewhile` pred, seq seq[0], seq[1], until pred fails ``takewhile(lambda x: x<5, [1,4,6,4,1]) --> 1 4`` :func:`izip` p, q, ... (p[0], q[0]), (p[1], q[1]), ... ``izip('ABCD', 'xy') --> Ax By`` :func:`izip_longest` p, q, ... (p[0], q[0]), (p[1], q[1]), ... ``izip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-`` @@ -106,9 +106,8 @@ loops that truncate the stream. .. classmethod:: chain.from_iterable(iterable) Alternate constructor for :func:`chain`. Gets chained inputs from a - single iterable argument that is evaluated lazily. Equivalent to:: + single iterable argument that is evaluated lazily. Roughly equivalent to:: - @classmethod def from_iterable(iterables): # chain.from_iterable(['ABC', 'DEF']) --> A B C D E F for it in iterables: @@ -393,7 +392,8 @@ loops that truncate the stream. yield function(*args) -.. function:: islice(iterable, [start,] stop [, step]) +.. function:: islice(iterable, stop) + islice(iterable, start, stop[, step]) Make an iterator that returns selected elements from the iterable. If *start* is non-zero, then elements from the iterable are skipped until start is reached. @@ -732,8 +732,9 @@ which incur interpreter overhead. next(b, None) return izip(a, b) - def grouper(n, iterable, fillvalue=None): - "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx" + def grouper(iterable, n, fillvalue=None): + "Collect data into fixed-length chunks or blocks" + # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx args = [iter(iterable)] * n return izip_longest(fillvalue=fillvalue, *args) @@ -827,6 +828,18 @@ which incur interpreter overhead. indices = sorted(random.randrange(n) for i in xrange(r)) return tuple(pool[i] for i in indices) + def tee_lookahead(t, i): + """Inspect the i-th upcomping value from a tee object + while leaving the tee object at its current position. + + Raise an IndexError if the underlying iterator doesn't + have enough values. + + """ + for value in islice(t.__copy__(), i, None): + return value + raise IndexError(i) + Note, many of the above recipes can be optimized by replacing global lookups with local variables defined as default values. For example, the *dotproduct* recipe can be written as:: diff --git a/Doc/library/jpeg.rst b/Doc/library/jpeg.rst index 98497ad..2a8e4e8 100644 --- a/Doc/library/jpeg.rst +++ b/Doc/library/jpeg.rst @@ -8,7 +8,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`jpeg` module has been deprecated for removal in Python 3.0. + The :mod:`jpeg` module has been removed in Python 3. diff --git a/Doc/library/json.rst b/Doc/library/json.rst index 546a09d..caee953 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -7,8 +7,10 @@ .. sectionauthor:: Bob Ippolito <bob@redivi.com> .. versionadded:: 2.6 -`JSON (JavaScript Object Notation) <http://json.org>`_ is a subset of JavaScript -syntax (ECMA-262 3rd edition) used as a lightweight data interchange format. +`JSON (JavaScript Object Notation) <http://json.org>`_, specified by +:rfc:`4627`, is a lightweight data interchange format based on a subset of +`JavaScript <http://en.wikipedia.org/wiki/JavaScript>`_ syntax (`ECMA-262 3rd +edition <http://www.ecma-international.org/publications/files/ECMA-ST-ARCH/ECMA-262,%203rd%20edition,%20December%201999.pdf>`_). :mod:`json` exposes an API familiar to users of the standard library :mod:`marshal` and :mod:`pickle` modules. @@ -41,7 +43,8 @@ Compact encoding:: Pretty printing:: >>> import json - >>> print json.dumps({'4': 5, '6': 7}, sort_keys=True, indent=4) + >>> print json.dumps({'4': 5, '6': 7}, sort_keys=True, + ... indent=4, separators=(',', ': ')) { "4": 5, "6": 7 @@ -81,6 +84,7 @@ Extending :class:`JSONEncoder`:: ... def default(self, obj): ... if isinstance(obj, complex): ... return [obj.real, obj.imag] + ... # Let the base class default method raise the TypeError ... return json.JSONEncoder.default(self, obj) ... >>> dumps(2 + 1j, cls=ComplexEncoder) @@ -99,35 +103,44 @@ Using json.tool from the shell to validate and pretty-print:: { "json": "obj" } - $ echo '{ 1.2:3.4}' | python -mjson.tool - Expecting property name: line 1 column 2 (char 2) + $ echo '{1.2:3.4}' | python -mjson.tool + Expecting property name enclosed in double quotes: line 1 column 2 (char 1) .. highlight:: python .. note:: - The JSON produced by this module's default settings is a subset of - YAML, so it may be used as a serializer for that as well. + JSON is a subset of `YAML <http://yaml.org/>`_ 1.2. The JSON produced by + this module's default settings (in particular, the default *separators* + value) is also a subset of YAML 1.0 and 1.1. This module can thus also be + used as a YAML serializer. Basic Usage ----------- -.. function:: dump(obj, fp[, skipkeys[, ensure_ascii[, check_circular[, allow_nan[, cls[, indent[, separators[, encoding[, default[, **kw]]]]]]]]]]) +.. function:: dump(obj, fp, skipkeys=False, ensure_ascii=True, \ + check_circular=True, allow_nan=True, cls=None, \ + indent=None, separators=None, encoding="utf-8", \ + default=None, sort_keys=False, **kw) Serialize *obj* as a JSON formatted stream to *fp* (a ``.write()``-supporting - file-like object). + :term:`file-like object`) using this :ref:`conversion table + <py-to-json-table>`. If *skipkeys* is ``True`` (default: ``False``), then dict keys that are not of a basic type (:class:`str`, :class:`unicode`, :class:`int`, :class:`long`, :class:`float`, :class:`bool`, ``None``) will be skipped instead of raising a :exc:`TypeError`. - If *ensure_ascii* is ``False`` (default: ``True``), then some chunks written - to *fp* may be :class:`unicode` instances, subject to normal Python - :class:`str` to :class:`unicode` coercion rules. Unless ``fp.write()`` - explicitly understands :class:`unicode` (as in :func:`codecs.getwriter`) this - is likely to cause an error. + If *ensure_ascii* is ``True`` (the default), all non-ASCII characters in the + output are escaped with ``\uXXXX`` sequences, and the result is a + :class:`str` instance consisting of ASCII characters only. If + *ensure_ascii* is ``False``, some chunks written to *fp* may be + :class:`unicode` instances. This usually happens because the input contains + unicode strings or the *encoding* parameter is used. Unless ``fp.write()`` + explicitly understands :class:`unicode` (as in :func:`codecs.getwriter`) + this is likely to cause an error. If *check_circular* is ``False`` (default: ``True``), then the circular reference check for container types will be skipped and a circular reference @@ -143,6 +156,12 @@ Basic Usage or negative, will only insert newlines. ``None`` (the default) selects the most compact representation. + .. note:: + + Since the default item separator is ``', '``, the output might include + trailing whitespace when *indent* is specified. You can use + ``separators=(',', ': ')`` to avoid this. + If *separators* is an ``(item_separator, dict_separator)`` tuple, then it will be used instead of the default ``(', ', ': ')`` separators. ``(',', ':')`` is the most compact JSON representation. @@ -152,6 +171,9 @@ Basic Usage *default(obj)* is a function that should return a serializable version of *obj* or raise :exc:`TypeError`. The default simply raises :exc:`TypeError`. + If *sort_keys* is ``True`` (default: ``False``), then the output of + dictionaries will be sorted by key. + To use a custom :class:`JSONEncoder` subclass (e.g. one that overrides the :meth:`default` method to serialize additional types), specify it with the *cls* kwarg; otherwise :class:`JSONEncoder` is used. @@ -162,19 +184,32 @@ Basic Usage trying to serialize more objects with repeated calls to :func:`dump` and the same *fp* will result in an invalid JSON file. -.. function:: dumps(obj[, skipkeys[, ensure_ascii[, check_circular[, allow_nan[, cls[, indent[, separators[, encoding[, default[, **kw]]]]]]]]]]) +.. function:: dumps(obj, skipkeys=False, ensure_ascii=True, \ + check_circular=True, allow_nan=True, cls=None, \ + indent=None, separators=None, encoding="utf-8", \ + default=None, sort_keys=False, **kw) + + Serialize *obj* to a JSON formatted :class:`str` using this :ref:`conversion + table <py-to-json-table>`. If *ensure_ascii* is ``False``, the result may + contain non-ASCII characters and the return value may be a :class:`unicode` + instance. - Serialize *obj* to a JSON formatted :class:`str`. + The arguments have the same meaning as in :func:`dump`. - If *ensure_ascii* is ``False``, then the return value will be a - :class:`unicode` instance. The other arguments have the same meaning as in - :func:`dump`. + .. note:: + Keys in key/value pairs of JSON are always of the type :class:`str`. When + a dictionary is converted into JSON, all the keys of the dictionary are + coerced to strings. As a result of this, if a dictionary is converted + into JSON and then back into a dictionary, the dictionary may not equal + the original one. That is, ``loads(dumps(x)) != x`` if x has non-string + keys. .. function:: load(fp[, encoding[, cls[, object_hook[, parse_float[, parse_int[, parse_constant[, object_pairs_hook[, **kw]]]]]]]]) - Deserialize *fp* (a ``.read()``-supporting file-like object containing a JSON - document) to a Python object. + Deserialize *fp* (a ``.read()``-supporting :term:`file-like object` + containing a JSON document) to a Python object using this :ref:`conversion + table <json-to-py-table>`. If the contents of *fp* are encoded with an ASCII based encoding other than UTF-8 (e.g. latin-1), then an appropriate *encoding* name must be specified. @@ -185,7 +220,8 @@ Basic Usage *object_hook* is an optional function that will be called with the result of any object literal decoded (a :class:`dict`). The return value of *object_hook* will be used instead of the :class:`dict`. This feature can be used - to implement custom decoders (e.g. JSON-RPC class hinting). + to implement custom decoders (e.g. `JSON-RPC <http://www.jsonrpc.org>`_ + class hinting). *object_pairs_hook* is an optional function that will be called with the result of any object literal decoded with an ordered list of pairs. The @@ -209,10 +245,13 @@ Basic Usage (e.g. :class:`float`). *parse_constant*, if specified, will be called with one of the following - strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``, ``'null'``, ``'true'``, - ``'false'``. This can be used to raise an exception if invalid JSON numbers + strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``. + This can be used to raise an exception if invalid JSON numbers are encountered. + .. versionchanged:: 2.7 + *parse_constant* doesn't get called on 'null', 'true', 'false' anymore. + To use a custom :class:`JSONDecoder` subclass, specify it with the ``cls`` kwarg; otherwise :class:`JSONDecoder` is used. Additional keyword arguments will be passed to the constructor of the class. @@ -221,7 +260,8 @@ Basic Usage .. function:: loads(s[, encoding[, cls[, object_hook[, parse_float[, parse_int[, parse_constant[, object_pairs_hook[, **kw]]]]]]]]) Deserialize *s* (a :class:`str` or :class:`unicode` instance containing a JSON - document) to a Python object. + document) to a Python object using this :ref:`conversion table + <json-to-py-table>`. If *s* is a :class:`str` instance and is encoded with an ASCII based encoding other than UTF-8 (e.g. latin-1), then an appropriate *encoding* name must be @@ -231,7 +271,7 @@ Basic Usage The other arguments have the same meaning as in :func:`load`. -Encoders and decoders +Encoders and Decoders --------------------- .. class:: JSONDecoder([encoding[, object_hook[, parse_float[, parse_int[, parse_constant[, strict[, object_pairs_hook]]]]]]]) @@ -240,6 +280,8 @@ Encoders and decoders Performs the following translations in decoding by default: + .. _json-to-py-table: + +---------------+-------------------+ | JSON | Python | +===============+===================+ @@ -306,6 +348,8 @@ Encoders and decoders those with character codes in the 0-31 range, including ``'\t'`` (tab), ``'\n'``, ``'\r'`` and ``'\0'``. + If the data being deserialized is not a valid JSON document, a + :exc:`ValueError` will be raised. .. method:: decode(s) @@ -328,6 +372,8 @@ Encoders and decoders Supports the following objects and types by default: + .. _py-to-json-table: + +-------------------+---------------+ | Python | JSON | +===================+===============+ @@ -355,9 +401,12 @@ Encoders and decoders attempt encoding of keys that are not str, int, long, float or None. If *skipkeys* is ``True``, such items are simply skipped. - If *ensure_ascii* is ``True`` (the default), the output is guaranteed to be - :class:`str` objects with all incoming unicode characters escaped. If - *ensure_ascii* is ``False``, the output will be a unicode object. + If *ensure_ascii* is ``True`` (the default), all non-ASCII characters in the + output are escaped with ``\uXXXX`` sequences, and the results are + :class:`str` instances consisting of ASCII characters only. If + *ensure_ascii* is ``False``, a result may be a :class:`unicode` + instance. This usually happens if the input contains unicode strings or the + *encoding* parameter is used. If *check_circular* is ``True`` (the default), then lists, dicts, and custom encoded objects will be checked for circular references during encoding to @@ -379,6 +428,12 @@ Encoders and decoders level. An indent level of 0 will only insert newlines. ``None`` is the most compact representation. + .. note:: + + Since the default item separator is ``', '``, the output might include + trailing whitespace when *indent* is specified. You can use + ``separators=(',', ': ')`` to avoid this. + If specified, *separators* should be an ``(item_separator, key_separator)`` tuple. The default is ``(', ', ': ')``. To get the most compact JSON representation, you should specify ``(',', ':')`` to eliminate whitespace. @@ -408,6 +463,7 @@ Encoders and decoders pass else: return list(iterable) + # Let the base class default method raise the TypeError return JSONEncoder.default(self, o) @@ -427,3 +483,108 @@ Encoders and decoders for chunk in JSONEncoder().iterencode(bigobject): mysocket.write(chunk) + + +Standard Compliance +------------------- + +The JSON format is specified by :rfc:`4627`. This section details this +module's level of compliance with the RFC. For simplicity, +:class:`JSONEncoder` and :class:`JSONDecoder` subclasses, and parameters other +than those explicitly mentioned, are not considered. + +This module does not comply with the RFC in a strict fashion, implementing some +extensions that are valid JavaScript but not valid JSON. In particular: + +- Top-level non-object, non-array values are accepted and output; +- Infinite and NaN number values are accepted and output; +- Repeated names within an object are accepted, and only the value of the last + name-value pair is used. + +Since the RFC permits RFC-compliant parsers to accept input texts that are not +RFC-compliant, this module's deserializer is technically RFC-compliant under +default settings. + +Character Encodings +^^^^^^^^^^^^^^^^^^^ + +The RFC recommends that JSON be represented using either UTF-8, UTF-16, or +UTF-32, with UTF-8 being the default. Accordingly, this module uses UTF-8 as +the default for its *encoding* parameter. + +This module's deserializer only directly works with ASCII-compatible encodings; +UTF-16, UTF-32, and other ASCII-incompatible encodings require the use of +workarounds described in the documentation for the deserializer's *encoding* +parameter. + +The RFC also non-normatively describes a limited encoding detection technique +for JSON texts; this module's deserializer does not implement this or any other +kind of encoding detection. + +As permitted, though not required, by the RFC, this module's serializer sets +*ensure_ascii=True* by default, thus escaping the output so that the resulting +strings only contain ASCII characters. + + +Top-level Non-Object, Non-Array Values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The RFC specifies that the top-level value of a JSON text must be either a +JSON object or array (Python :class:`dict` or :class:`list`). This module's +deserializer also accepts input texts consisting solely of a +JSON null, boolean, number, or string value:: + + >>> just_a_json_string = '"spam and eggs"' # Not by itself a valid JSON text + >>> json.loads(just_a_json_string) + u'spam and eggs' + +This module itself does not include a way to request that such input texts be +regarded as illegal. Likewise, this module's serializer also accepts single +Python :data:`None`, :class:`bool`, numeric, and :class:`str` +values as input and will generate output texts consisting solely of a top-level +JSON null, boolean, number, or string value without raising an exception:: + + >>> neither_a_list_nor_a_dict = u"spam and eggs" + >>> json.dumps(neither_a_list_nor_a_dict) # The result is not a valid JSON text + '"spam and eggs"' + +This module's serializer does not itself include a way to enforce the +aforementioned constraint. + + +Infinite and NaN Number Values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The RFC does not permit the representation of infinite or NaN number values. +Despite that, by default, this module accepts and outputs ``Infinity``, +``-Infinity``, and ``NaN`` as if they were valid JSON number literal values:: + + >>> # Neither of these calls raises an exception, but the results are not valid JSON + >>> json.dumps(float('-inf')) + '-Infinity' + >>> json.dumps(float('nan')) + 'NaN' + >>> # Same when deserializing + >>> json.loads('-Infinity') + -inf + >>> json.loads('NaN') + nan + +In the serializer, the *allow_nan* parameter can be used to alter this +behavior. In the deserializer, the *parse_constant* parameter can be used to +alter this behavior. + + +Repeated Names Within an Object +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The RFC specifies that the names within a JSON object should be unique, but +does not specify how repeated names in JSON objects should be handled. By +default, this module does not raise an exception; instead, it ignores all but +the last name-value pair for a given name:: + + >>> weird_json = '{"x": 1, "x": 2, "x": 3}' + >>> json.loads(weird_json) + {u'x': 3} + +The *object_pairs_hook* parameter can be used to alter this behavior. diff --git a/Doc/library/locale.rst b/Doc/library/locale.rst index 36fbde8..5590514 100644 --- a/Doc/library/locale.rst +++ b/Doc/library/locale.rst @@ -59,6 +59,8 @@ The :mod:`locale` module defines the following exception and functions: Returns the database of the local conventions as a dictionary. This dictionary has the following strings as keys: + .. tabularcolumns:: |l|l|L| + +----------------------+-------------------------------------+--------------------------------+ | Category | Key | Meaning | +======================+=====================================+================================+ @@ -164,22 +166,22 @@ The :mod:`locale` module defines the following exception and functions: .. data:: D_T_FMT - Get a string that can be used as a format string for :func:`strftime` to + Get a string that can be used as a format string for :func:`time.strftime` to represent date and time in a locale-specific way. .. data:: D_FMT - Get a string that can be used as a format string for :func:`strftime` to + Get a string that can be used as a format string for :func:`time.strftime` to represent a date in a locale-specific way. .. data:: T_FMT - Get a string that can be used as a format string for :func:`strftime` to + Get a string that can be used as a format string for :func:`time.strftime` to represent a time in a locale-specific way. .. data:: T_FMT_AMPM - Get a format string for :func:`strftime` to represent time in the am/pm + Get a format string for :func:`time.strftime` to represent time in the am/pm format. .. data:: DAY_1 ... DAY_7 @@ -243,24 +245,24 @@ The :mod:`locale` module defines the following exception and functions: then-emperor's reign. Normally it should not be necessary to use this value directly. Specifying - the ``E`` modifier in their format strings causes the :func:`strftime` + the ``E`` modifier in their format strings causes the :func:`time.strftime` function to use this information. The format of the returned string is not specified, and therefore you should not assume knowledge of it on different systems. .. data:: ERA_D_T_FMT - Get a format string for :func:`strftime` to represent date and time in a + Get a format string for :func:`time.strftime` to represent date and time in a locale-specific era-based way. .. data:: ERA_D_FMT - Get a format string for :func:`strftime` to represent a date in a + Get a format string for :func:`time.strftime` to represent a date in a locale-specific era-based way. .. data:: ERA_T_FMT - Get a format string for :func:`strftime` to represent a time in a + Get a format string for :func:`time.strftime` to represent a time in a locale-specific era-based way. .. data:: ALT_DIGITS diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst index 8b39be4..c618aa8 100644 --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -17,6 +17,10 @@ * :ref:`Advanced Tutorial <logging-advanced-tutorial>` * :ref:`Logging Cookbook <logging-cookbook>` +**Source code:** :source:`Lib/logging/config.py` + +-------------- + This section describes the API for configuring the logging module. .. _logging-config-api: @@ -77,8 +81,9 @@ in :mod:`logging` itself) and defining handlers which are declared either in .. function:: fileConfig(fname, defaults=None, disable_existing_loggers=True) Reads the logging configuration from a :mod:`configparser`\-format file - named *fname*. This function can be called several times from an - application, allowing an end user to select from various pre-canned + named *fname*. The format of the file should be as described in + :ref:`logging-config-fileformat`. This function can be called several times + from an application, allowing an end user to select from various pre-canned configurations (if the developer provides a mechanism to present the choices and load the chosen configuration). @@ -104,14 +109,30 @@ in :mod:`logging` itself) and defining handlers which are declared either in configurations. If no port is specified, the module's default :const:`DEFAULT_LOGGING_CONFIG_PORT` is used. Logging configurations will be sent as a file suitable for processing by :func:`fileConfig`. Returns a - :class:`Thread` instance on which you can call :meth:`start` to start the - server, and which you can :meth:`join` when appropriate. To stop the server, + :class:`~threading.Thread` instance on which you can call + :meth:`~threading.Thread.start` to start the server, and which you can + :meth:`~threading.Thread.join` when appropriate. To stop the server, call :func:`stopListening`. To send a configuration to the socket, read in the configuration file and send it to the socket as a string of bytes preceded by a four-byte length string packed in binary using ``struct.pack('>L', n)``. + .. note:: + + Because portions of the configuration are passed through + :func:`eval`, use of this function may open its users to a security risk. + While the function only binds to a socket on ``localhost``, and so does + not accept connections from remote machines, there are scenarios where + untrusted code could be run under the account of the process which calls + :func:`listen`. Specifically, if the process calling :func:`listen` runs + on a multi-user machine where users cannot trust each other, then a + malicious user could arrange to run essentially arbitrary code in a + victim user's process, simply by connecting to the victim's + :func:`listen` socket and sending a configuration which runs whatever + code the attacker wants to have executed in the victim's process. This is + especially easy to do if the default port is used, but not hard even if a + different port is used). .. function:: stopListening() @@ -156,11 +177,11 @@ otherwise, the context is used to determine what to instantiate. * *formatters* - the corresponding value will be a dict in which each key is a formatter id and each value is a dict describing how to - configure the corresponding Formatter instance. + configure the corresponding :class:`~logging.Formatter` instance. The configuring dict is searched for keys ``format`` and ``datefmt`` (with defaults of ``None``) and these are used to construct a - :class:`logging.Formatter` instance. + :class:`~logging.Formatter` instance. * *filters* - the corresponding value will be a dict in which each key is a filter id and each value is a dict describing how to configure @@ -698,8 +719,17 @@ format string, with a comma separator. An example time in ISO8601 format is The ``class`` entry is optional. It indicates the name of the formatter's class (as a dotted module and class name.) This option is useful for instantiating a -:class:`Formatter` subclass. Subclasses of :class:`Formatter` can present -exception tracebacks in an expanded or condensed format. +:class:`~logging.Formatter` subclass. Subclasses of +:class:`~logging.Formatter` can present exception tracebacks in an expanded or +condensed format. + +.. note:: + + Due to the use of :func:`eval` as described above, there are + potential security risks which result from using the :func:`listen` to send + and receive configurations via sockets. The risks are limited to where + multiple users with no mutual trust run code on the same machine; see the + :func:`listen` documentation for more information. .. seealso:: diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index c944454..d0f9be8 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -17,6 +17,10 @@ * :ref:`Advanced Tutorial <logging-advanced-tutorial>` * :ref:`Logging Cookbook <logging-cookbook>` +**Source code:** :source:`Lib/logging/handlers.py` + +-------------- + .. currentmodule:: logging The following useful handlers are provided in the package. Note that three of @@ -53,8 +57,8 @@ and :meth:`flush` methods). .. method:: flush() Flushes the stream by calling its :meth:`flush` method. Note that the - :meth:`close` method is inherited from :class:`Handler` and so does - no output, so an explicit :meth:`flush` call may be needed at times. + :meth:`close` method is inherited from :class:`~logging.Handler` and so + does no output, so an explicit :meth:`flush` call may be needed at times. .. _file-handler: @@ -142,8 +146,8 @@ new stream. This handler is not appropriate for use under Windows, because under Windows open log files cannot be moved or renamed - logging opens the files with exclusive locks - and so there is no need for such a handler. Furthermore, -*ST_INO* is not supported under Windows; :func:`stat` always returns zero for -this value. +*ST_INO* is not supported under Windows; :func:`~os.stat` always returns zero +for this value. .. class:: WatchedFileHandler(filename[,mode[, encoding[, delay]]]) @@ -236,11 +240,15 @@ timed intervals. +----------------+-----------------------+ | ``'D'`` | Days | +----------------+-----------------------+ - | ``'W'`` | Week day (0=Monday) | + | ``'W0'-'W6'`` | Weekday (0=Monday) | +----------------+-----------------------+ | ``'midnight'`` | Roll over at midnight | +----------------+-----------------------+ + When using weekday-based rotation, specify 'W0' for Monday, 'W1' for + Tuesday, and so on up to 'W6' for Sunday. In this case, the value passed for + *interval* isn't used. + The system will save old log files by appending extensions to the filename. The extensions are date-and-time based, using the strftime format ``%Y-%m-%d_%H-%M-%S`` or a leading portion thereof, depending on the @@ -301,7 +309,8 @@ sends logging output to a network socket. The base class uses a TCP socket. binary format. If there is an error with the socket, silently drops the packet. If the connection was previously lost, re-establishes the connection. To unpickle the record at the receiving end into a - :class:`LogRecord`, use the :func:`makeLogRecord` function. + :class:`~logging.LogRecord`, use the :func:`~logging.makeLogRecord` + function. .. method:: handleError() @@ -379,7 +388,8 @@ over UDP sockets. Pickles the record's attribute dictionary and writes it to the socket in binary format. If there is an error with the socket, silently drops the packet. To unpickle the record at the receiving end into a - :class:`LogRecord`, use the :func:`makeLogRecord` function. + :class:`~logging.LogRecord`, use the :func:`~logging.makeLogRecord` + function. .. method:: makeSocket() @@ -650,7 +660,7 @@ event of a certain severity or greater is seen. :class:`BufferingHandler`, which is an abstract class. This buffers logging records in memory. Whenever each record is added to the buffer, a check is made by calling :meth:`shouldFlush` to see if the buffer should be flushed. If it -should, then :meth:`flush` is expected to do the needful. +should, then :meth:`flush` is expected to do the flushing. .. class:: BufferingHandler(capacity) @@ -698,9 +708,6 @@ should, then :meth:`flush` is expected to do the needful. .. method:: setTarget(target) - .. versionchanged:: 2.6 - *credentials* was added. - Sets the target handler for this handler. @@ -722,15 +729,29 @@ supports sending logging messages to a Web server, using either ``GET`` or .. class:: HTTPHandler(host, url, method='GET') - Returns a new instance of the :class:`HTTPHandler` class. The *host* can be + Returns a new instance of the :class:`HTTPHandler` class. The ``host`` can be of the form ``host:port``, should you need to use a specific port number. - If no *method* is specified, ``GET`` is used. + .. method:: mapLogRecord(record) + + Provides a dictionary, based on ``record``, which is to be URL-encoded + and sent to the web server. The default implementation just returns + ``record.__dict__``. This method can be overridden if e.g. only a + subset of :class:`~logging.LogRecord` is to be sent to the web server, or + if more specific customization of what's sent to the server is required. .. method:: emit(record) - Sends the record to the Web server as a percent-encoded dictionary. + Sends the record to the Web server as an URL-encoded dictionary. The + :meth:`mapLogRecord` method is used to convert the record to the + dictionary to be sent. + .. note:: Since preparing a record for sending it to a Web server is not + the same as a generic formatting operation, using :meth:`setFormatter` + to specify a :class:`Formatter` for a :class:`HTTPHandler` has no effect. + Instead of calling :meth:`format`, this handler calls :meth:`mapLogRecord` + and then :func:`urllib.urlencode` to encode the dictionary in a form + suitable for sending to a Web server. .. seealso:: diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index ea69b9e..0c50b2f 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -20,6 +20,9 @@ * :ref:`Advanced Tutorial <logging-advanced-tutorial>` * :ref:`Logging Cookbook <logging-cookbook>` +**Source code:** :source:`Lib/logging/__init__.py` + +-------------- .. versionadded:: 2.3 @@ -51,24 +54,46 @@ listed below. Logger Objects -------------- -Loggers have the following attributes and methods. Note that Loggers are never +Loggers have the following attributes and methods. Note that Loggers are never instantiated directly, but always through the module-level function -``logging.getLogger(name)``. +``logging.getLogger(name)``. Multiple calls to :func:`getLogger` with the same +name will always return a reference to the same Logger object. + +The ``name`` is potentially a period-separated hierarchical value, like +``foo.bar.baz`` (though it could also be just plain ``foo``, for example). +Loggers that are further down in the hierarchical list are children of loggers +higher up in the list. For example, given a logger with a name of ``foo``, +loggers with names of ``foo.bar``, ``foo.bar.baz``, and ``foo.bam`` are all +descendants of ``foo``. The logger name hierarchy is analogous to the Python +package hierarchy, and identical to it if you organise your loggers on a +per-module basis using the recommended construction +``logging.getLogger(__name__)``. That's because in a module, ``__name__`` +is the module's name in the Python package namespace. + .. class:: Logger .. attribute:: Logger.propagate - If this evaluates to true, logging messages are passed by this logger and by - its child loggers to the handlers of higher level (ancestor) loggers. - Messages are passed directly to the ancestor loggers' handlers - neither the - level nor filters of the ancestor loggers in question are considered. + If this evaluates to true, events logged to this logger will be passed to the + handlers of higher level (ancestor) loggers, in addition to any handlers + attached to this logger. Messages are passed directly to the ancestor + loggers' handlers - neither the level nor filters of the ancestor loggers in + question are considered. If this evaluates to false, logging messages are not passed to the handlers of ancestor loggers. The constructor sets this attribute to ``True``. + .. note:: If you attach a handler to a logger *and* one or more of its + ancestors, it may emit the same record multiple times. In general, you + should not need to attach a handler to more than one logger - if you just + attach it to the appropriate logger which is highest in the logger + hierarchy, then it will see all events logged by all descendant loggers, + provided that their propagate setting is left set to ``True``. A common + scenario is to attach handlers only to the root logger, and to let + propagation take care of the rest. .. method:: Logger.setLevel(lvl) @@ -89,6 +114,8 @@ instantiated directly, but always through the module-level function If the root is reached, and it has a level of NOTSET, then all messages will be processed. Otherwise, the root's level will be used as the effective level. + See :ref:`levels` for a list of levels. + .. method:: Logger.isEnabledFor(lvl) @@ -138,7 +165,7 @@ instantiated directly, but always through the module-level function FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s' logging.basicConfig(format=FORMAT) - d = { 'clientip' : '192.168.0.1', 'user' : 'fbloggs' } + d = {'clientip': '192.168.0.1', 'user': 'fbloggs'} logger = logging.getLogger('tcpserver') logger.warning('Protocol problem: %s', 'connection reset', extra=d) @@ -195,7 +222,7 @@ instantiated directly, but always through the module-level function interpreted as for :meth:`debug`. -.. method:: Logger.exception(msg, *args) +.. method:: Logger.exception(msg, *args, **kwargs) Logs a message with level :const:`ERROR` on this logger. The arguments are interpreted as for :meth:`debug`. Exception info is added to the logging @@ -215,7 +242,10 @@ instantiated directly, but always through the module-level function .. method:: Logger.filter(record) Applies this logger's filters to the record and returns a true value if the - record is to be processed. + record is to be processed. The filters are consulted in turn, until one of + them returns a false value. If none of them return a false value, the record + will be processed (passed to handlers). If one returns a false value, no + further processing of the record occurs. .. method:: Logger.addHandler(hdlr) @@ -253,6 +283,35 @@ instantiated directly, but always through the module-level function .. versionchanged:: 2.5 *func* and *extra* were added. + +.. _levels: + +Logging Levels +-------------- + +The numeric values of logging levels are given in the following table. These are +primarily of interest if you want to define your own levels, and need them to +have specific values relative to the predefined levels. If you define a level +with the same numeric value, it overwrites the predefined value; the predefined +name is lost. + ++--------------+---------------+ +| Level | Numeric value | ++==============+===============+ +| ``CRITICAL`` | 50 | ++--------------+---------------+ +| ``ERROR`` | 40 | ++--------------+---------------+ +| ``WARNING`` | 30 | ++--------------+---------------+ +| ``INFO`` | 20 | ++--------------+---------------+ +| ``DEBUG`` | 10 | ++--------------+---------------+ +| ``NOTSET`` | 0 | ++--------------+---------------+ + + .. _handler: Handler Objects @@ -293,6 +352,7 @@ subclasses. However, the :meth:`__init__` method in subclasses needs to call severe than *lvl* will be ignored. When a handler is created, the level is set to :const:`NOTSET` (which causes all messages to be processed). + See :ref:`levels` for a list of levels. .. method:: Handler.setFormatter(form) @@ -312,7 +372,10 @@ subclasses. However, the :meth:`__init__` method in subclasses needs to call .. method:: Handler.filter(record) Applies this handler's filters to the record and returns a true value if the - record is to be processed. + record is to be processed. The filters are consulted in turn, until one of + them returns a false value. If none of them return a false value, the record + will be emitted. If one returns a false value, the handler will not emit the + record. .. method:: Handler.flush() @@ -465,12 +528,12 @@ empty string, all events are passed. yes. If deemed appropriate, the record may be modified in-place by this method. -Note that filters attached to handlers are consulted whenever an event is +Note that filters attached to handlers are consulted before an event is emitted by the handler, whereas filters attached to loggers are consulted -whenever an event is logged to the handler (using :meth:`debug`, :meth:`info`, -etc.) This means that events which have been generated by descendant loggers -will not be filtered by a logger's filter setting, unless the filter has also -been applied to those descendant loggers. +whenever an event is logged (using :meth:`debug`, :meth:`info`, +etc.), before sending an event to handlers. This means that events which have +been generated by descendant loggers will not be filtered by a logger's filter +setting, unless the filter has also been applied to those descendant loggers. You don't actually need to subclass ``Filter``: you can pass any instance which has a ``filter`` method with the same semantics. @@ -504,7 +567,9 @@ wire). record. :param name: The name of the logger used to log the event represented by - this LogRecord. + this LogRecord. Note that this name will always have this + value, even though it may be emitted by a handler attached to + a different (ancestor) logger. :param level: The numeric level of the logging event (one of DEBUG, INFO etc.) Note that this is converted to *two* attributes of the LogRecord: ``levelno`` for the numeric value and ``levelname`` for the @@ -617,13 +682,16 @@ format string. .. versionchanged:: 2.5 *funcName* was added. +.. versionchanged:: 2.6 + *processName* was added. + .. _logger-adapter: LoggerAdapter Objects --------------------- :class:`LoggerAdapter` instances are used to conveniently pass contextual -information into logging calls. For a usage example , see the section on +information into logging calls. For a usage example, see the section on :ref:`adding contextual information to your logging output <context-info>`. .. versionadded:: 2.6 @@ -643,16 +711,15 @@ information into logging calls. For a usage example , see the section on (possibly modified) versions of the arguments passed in. In addition to the above, :class:`LoggerAdapter` supports the following -methods of :class:`Logger`, i.e. :meth:`debug`, :meth:`info`, :meth:`warning`, -:meth:`error`, :meth:`exception`, :meth:`critical`, :meth:`log`, -:meth:`isEnabledFor`, :meth:`getEffectiveLevel`, :meth:`setLevel`, -:meth:`hasHandlers`. These methods have the same signatures as their -counterparts in :class:`Logger`, so you can use the two types of instances -interchangeably. +methods of :class:`Logger`: :meth:`~Logger.debug`, :meth:`~Logger.info`, +:meth:`~Logger.warning`, :meth:`~Logger.error`, :meth:`~Logger.exception`, +:meth:`~Logger.critical`, :meth:`~Logger.log` and :meth:`~Logger.isEnabledFor`. +These methods have the same signatures as their counterparts in :class:`Logger`, +so you can use the two types of instances interchangeably for these calls. .. versionchanged:: 2.7 - The :meth:`isEnabledFor` method was added to :class:`LoggerAdapter`. This - method delegates to the underlying logger. + The :meth:`~Logger.isEnabledFor` method was added to :class:`LoggerAdapter`. + This method delegates to the underlying logger. Thread Safety @@ -692,8 +759,8 @@ functions. Return either the standard :class:`Logger` class, or the last class passed to :func:`setLoggerClass`. This function may be called from within a new class - definition, to ensure that installing a customised :class:`Logger` class will - not undo customisations already applied by other code. For example:: + definition, to ensure that installing a customized :class:`Logger` class will + not undo customizations already applied by other code. For example:: class MyLogger(logging.getLoggerClass()): # ... override behaviour here @@ -773,7 +840,7 @@ functions. are interpreted as for :func:`debug`. -.. function:: exception(msg[, *args]) +.. function:: exception(msg[, *args[, **kwargs]]) Logs a message with level :const:`ERROR` on the root logger. The arguments are interpreted as for :func:`debug`. Exception info is added to the logging @@ -785,14 +852,15 @@ functions. Logs a message with level *level* on the root logger. The other arguments are interpreted as for :func:`debug`. - PLEASE NOTE: The above module-level functions which delegate to the root - logger should *not* be used in threads, in versions of Python earlier than - 2.7.1 and 3.2, unless at least one handler has been added to the root - logger *before* the threads are started. These convenience functions call - :func:`basicConfig` to ensure that at least one handler is available; in - earlier versions of Python, this can (under rare circumstances) lead to - handlers being added multiple times to the root logger, which can in turn - lead to multiple messages for the same event. + .. note:: The above module-level convenience functions, which delegate to the + root logger, call :func:`basicConfig` to ensure that at least one handler + is available. Because of this, they should *not* be used in threads, + in versions of Python earlier than 2.7.1 and 3.2, unless at least one + handler has been added to the root logger *before* the threads are + started. In earlier versions of Python, due to a thread safety shortcoming + in :func:`basicConfig`, this can (under rare circumstances) lead to + handlers being added multiple times to the root logger, which can in turn + lead to multiple messages for the same event. .. function:: disable(lvl) @@ -802,7 +870,10 @@ functions. effect is to disable all logging calls of severity *lvl* and below, so that if you call it with a value of INFO, then all INFO and DEBUG events would be discarded, whereas those of severity WARNING and above would be processed - according to the logger's effective level. + according to the logger's effective level. If + ``logging.disable(logging.NOTSET)`` is called, it effectively removes this + overriding level, so that logging output again depends on the effective + levels of individual loggers. .. function:: addLevelName(lvl, levelName) @@ -814,8 +885,8 @@ functions. registered using this function, levels should be positive integers and they should increase in increasing order of severity. - NOTE: If you are thinking of defining your own levels, please see the section - on :ref:`custom-levels`. + .. note:: If you are thinking of defining your own levels, please see the + section on :ref:`custom-levels`. .. function:: getLevelName(lvl) @@ -850,15 +921,17 @@ functions. .. versionchanged:: 2.4 Formerly, :func:`basicConfig` did not take any keyword arguments. - PLEASE NOTE: This function should be called from the main thread - before other threads are started. In versions of Python prior to - 2.7.1 and 3.2, if this function is called from multiple threads, - it is possible (in rare circumstances) that a handler will be added - to the root logger more than once, leading to unexpected results - such as messages being duplicated in the log. + .. note:: This function should be called from the main thread before other + threads are started. In versions of Python prior to 2.7.1 and 3.2, if + this function is called from multiple threads, it is possible (in rare + circumstances) that a handler will be added to the root logger more than + once, leading to unexpected results such as messages being duplicated in + the log. The following keyword arguments are supported. + .. tabularcolumns:: |l|L| + +--------------+---------------------------------------------+ | Format | Description | +==============+=============================================+ @@ -915,12 +988,11 @@ with the :mod:`warnings` module. If *capture* is ``True``, warnings issued by the :mod:`warnings` module will be redirected to the logging system. Specifically, a warning will be formatted using :func:`warnings.formatwarning` and the resulting string - logged to a logger named 'py.warnings' with a severity of `WARNING`. + logged to a logger named ``'py.warnings'`` with a severity of :const:`WARNING`. If *capture* is ``False``, the redirection of warnings to the logging system will stop, and warnings will be redirected to their original destinations - (i.e. those in effect before `captureWarnings(True)` was called). - + (i.e. those in effect before ``captureWarnings(True)`` was called). .. seealso:: diff --git a/Doc/library/mac.rst b/Doc/library/mac.rst index 7ac1ca2..d66931c 100644 --- a/Doc/library/mac.rst +++ b/Doc/library/mac.rst @@ -12,7 +12,10 @@ Mac-specific Python programming. .. note:: - These modules are deprecated and have been removed in Python 3.x. + Most of the OS X APIs that these modules use are deprecated or removed + in recent versions of OS X. Many are not available when Python is + executing in 64-bit mode. These modules have been removed in + Python 3. You should avoid using them in Python 2. .. toctree:: diff --git a/Doc/library/macosa.rst b/Doc/library/macosa.rst index 54e62f2..f6b3b48 100644 --- a/Doc/library/macosa.rst +++ b/Doc/library/macosa.rst @@ -9,8 +9,7 @@ This chapter describes the current implementation of the Open Scripting Architecture (OSA, also commonly referred to as AppleScript) for Python, allowing you to control scriptable applications from your Python program, and with a fairly pythonic interface. Development on this set of modules has -stopped. For more up-to-date implementation of AppleScript support for Python, -see the third-party py-appscript project: <http://pypi.python.org/pypi/appscript/>. +stopped. For a description of the various components of AppleScript and OSA, and to get an understanding of the architecture and terminology, you should read Apple's diff --git a/Doc/library/macostools.rst b/Doc/library/macostools.rst index f2a2643..7924669 100644 --- a/Doc/library/macostools.rst +++ b/Doc/library/macostools.rst @@ -15,7 +15,7 @@ files, so it should not be used on UFS partitions. .. note:: - This module has been removed in Python 3.0. + This module has been removed in Python 3. diff --git a/Doc/library/mailbox.rst b/Doc/library/mailbox.rst index 7e6f44a..94fde14 100644 --- a/Doc/library/mailbox.rst +++ b/Doc/library/mailbox.rst @@ -11,8 +11,9 @@ This module defines two classes, :class:`Mailbox` and :class:`Message`, for accessing and manipulating on-disk mailboxes and the messages they contain. :class:`Mailbox` offers a dictionary-like mapping from keys to messages. -:class:`Message` extends the :mod:`email.Message` module's :class:`Message` -class with format-specific state and behavior. Supported mailbox formats are +:class:`Message` extends the :mod:`email.message` module's +:class:`~email.message.Message` class with format-specific state and behavior. +Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. @@ -83,7 +84,7 @@ Maildir, mbox, MH, Babyl, and MMDF. it. Parameter *message* may be a :class:`Message` instance, an - :class:`email.Message.Message` instance, a string, or a file-like object + :class:`email.message.Message` instance, a string, or a file-like object (which should be open in text mode). If *message* is an instance of the appropriate format-specific :class:`Message` subclass (e.g., if it's an :class:`mboxMessage` instance and this is an :class:`mbox` instance), its @@ -110,7 +111,7 @@ Maildir, mbox, MH, Babyl, and MMDF. :exc:`KeyError` exception if no message already corresponds to *key*. As with :meth:`add`, parameter *message* may be a :class:`Message` - instance, an :class:`email.Message.Message` instance, a string, or a + instance, an :class:`email.message.Message` instance, a string, or a file-like object (which should be open in text mode). If *message* is an instance of the appropriate format-specific :class:`Message` subclass (e.g., if it's an :class:`mboxMessage` instance and this is an @@ -154,7 +155,7 @@ Maildir, mbox, MH, Babyl, and MMDF. when the :class:`Mailbox` instance was initialized. - .. method:: get(key[, default=None]) + .. method:: get(key, default=None) __getitem__(key) Return a representation of the message corresponding to *key*. If no such @@ -278,7 +279,7 @@ Maildir, mbox, MH, Babyl, and MMDF. ^^^^^^^^^^^^^^^^ -.. class:: Maildir(dirname[, factory=rfc822.Message[, create=True]]) +.. class:: Maildir(dirname, factory=rfc822.Message, create=True) A subclass of :class:`Mailbox` for mailboxes in Maildir format. Parameter *factory* is a callable object that accepts a file-like message representation @@ -423,7 +424,7 @@ Maildir, mbox, MH, Babyl, and MMDF. ^^^^^^^^^^^^^ -.. class:: mbox(path[, factory=None[, create=True]]) +.. class:: mbox(path, factory=None, create=True) A subclass of :class:`Mailbox` for mailboxes in mbox format. Parameter *factory* is a callable object that accepts a file-like message representation (which @@ -483,7 +484,7 @@ Maildir, mbox, MH, Babyl, and MMDF. ^^^^^^^^^^^ -.. class:: MH(path[, factory=None[, create=True]]) +.. class:: MH(path, factory=None, create=True) A subclass of :class:`Mailbox` for mailboxes in MH format. Parameter *factory* is a callable object that accepts a file-like message representation (which @@ -613,7 +614,7 @@ Maildir, mbox, MH, Babyl, and MMDF. ^^^^^^^^^^^^^^ -.. class:: Babyl(path[, factory=None[, create=True]]) +.. class:: Babyl(path, factory=None, create=True) A subclass of :class:`Mailbox` for mailboxes in Babyl format. Parameter *factory* is a callable object that accepts a file-like message representation @@ -660,7 +661,7 @@ Maildir, mbox, MH, Babyl, and MMDF. In Babyl mailboxes, the headers of a message are not stored contiguously with the body of the message. To generate a file-like representation, the - headers and body are copied together into a :class:`StringIO` instance + headers and body are copied together into a :class:`~StringIO.StringIO` instance (from the :mod:`StringIO` module), which has an API identical to that of a file. As a result, the file-like object is truly independent of the underlying mailbox but does not save memory compared to a string @@ -689,7 +690,7 @@ Maildir, mbox, MH, Babyl, and MMDF. ^^^^^^^^^^^^^ -.. class:: MMDF(path[, factory=None[, create=True]]) +.. class:: MMDF(path, factory=None, create=True) A subclass of :class:`Mailbox` for mailboxes in MMDF format. Parameter *factory* is a callable object that accepts a file-like message representation (which @@ -743,11 +744,12 @@ Maildir, mbox, MH, Babyl, and MMDF. .. class:: Message([message]) - A subclass of the :mod:`email.Message` module's :class:`Message`. Subclasses of - :class:`mailbox.Message` add mailbox-format-specific state and behavior. + A subclass of the :mod:`email.message` module's + :class:`~email.message.Message`. Subclasses of :class:`mailbox.Message` add + mailbox-format-specific state and behavior. If *message* is omitted, the new instance is created in a default, empty state. - If *message* is an :class:`email.Message.Message` instance, its contents are + If *message* is an :class:`email.message.Message` instance, its contents are copied; furthermore, any format-specific information is converted insofar as possible if *message* is a :class:`Message` instance. If *message* is a string or a file, it should contain an :rfc:`2822`\ -compliant message, which is read @@ -987,12 +989,12 @@ When a :class:`MaildirMessage` instance is created based upon a are excluded. - .. method:: set_from(from_[, time_=None]) + .. method:: set_from(from_, time_=None) Set the "From " line to *from_*, which should be specified without a leading "From " or trailing newline. For convenience, *time_* may be specified and will be formatted appropriately and appended to *from_*. If - *time_* is specified, it should be a :class:`struct_time` instance, a + *time_* is specified, it should be a :class:`time.struct_time` instance, a tuple suitable for passing to :meth:`time.strftime`, or ``True`` (to use :meth:`time.gmtime`). @@ -1251,7 +1253,7 @@ When an :class:`MHMessage` instance is created based upon a Set the message's visible headers to be the same as the headers in *message*. Parameter *visible* should be a :class:`Message` instance, an - :class:`email.Message.Message` instance, a string, or a file-like object + :class:`email.message.Message` instance, a string, or a file-like object (which should be open in text mode). @@ -1358,12 +1360,12 @@ When a :class:`BabylMessage` instance is created based upon an are excluded. - .. method:: set_from(from_[, time_=None]) + .. method:: set_from(from_, time_=None) Set the "From " line to *from_*, which should be specified without a leading "From " or trailing newline. For convenience, *time_* may be specified and will be formatted appropriately and appended to *from_*. If - *time_* is specified, it should be a :class:`struct_time` instance, a + *time_* is specified, it should be a :class:`time.struct_time` instance, a tuple suitable for passing to :meth:`time.strftime`, or ``True`` (to use :meth:`time.gmtime`). @@ -1513,7 +1515,7 @@ Older versions of the :mod:`mailbox` module do not support modification of mailboxes, such as adding or removing message, and do not provide classes to represent format-specific message properties. For backward compatibility, the older mailbox classes are still available, but the newer classes should be used -in preference to them. The old classes will be removed in Python 3.0. +in preference to them. The old classes have been removed in Python 3. Older mailbox objects support only iteration and provide a single public method: @@ -1662,7 +1664,7 @@ programs, mail loss due to interruption of the program, or premature termination due to malformed messages in the mailbox:: import mailbox - import email.Errors + import email.errors list_names = ('python-list', 'python-dev', 'python-bugs') @@ -1672,7 +1674,7 @@ due to malformed messages in the mailbox:: for key in inbox.iterkeys(): try: message = inbox[key] - except email.Errors.MessageParseError: + except email.errors.MessageParseError: continue # The message is malformed. Just leave it. for name in list_names: diff --git a/Doc/library/mailcap.rst b/Doc/library/mailcap.rst index 5507211..b359509 100644 --- a/Doc/library/mailcap.rst +++ b/Doc/library/mailcap.rst @@ -71,6 +71,6 @@ An example usage:: >>> import mailcap >>> d=mailcap.getcaps() - >>> mailcap.findmatch(d, 'video/mpeg', filename='/tmp/tmp1223') - ('xmpeg /tmp/tmp1223', {'view': 'xmpeg %s'}) + >>> mailcap.findmatch(d, 'video/mpeg', filename='tmp1223') + ('xmpeg tmp1223', {'view': 'xmpeg %s'}) diff --git a/Doc/library/markup.rst b/Doc/library/markup.rst index 8508a1f..0d05ef1 100644 --- a/Doc/library/markup.rst +++ b/Doc/library/markup.rst @@ -1,4 +1,3 @@ - .. _markup: ********************************** @@ -26,7 +25,8 @@ definition of the Python bindings for the DOM and SAX interfaces. htmlparser.rst sgmllib.rst htmllib.rst - pyexpat.rst + xml.rst + xml.etree.elementtree.rst xml.dom.rst xml.dom.minidom.rst xml.dom.pulldom.rst @@ -34,4 +34,4 @@ definition of the Python bindings for the DOM and SAX interfaces. xml.sax.handler.rst xml.sax.utils.rst xml.sax.reader.rst - xml.etree.elementtree.rst + pyexpat.rst diff --git a/Doc/library/marshal.rst b/Doc/library/marshal.rst index f463a7a..f4dfc1f 100644 --- a/Doc/library/marshal.rst +++ b/Doc/library/marshal.rst @@ -66,8 +66,9 @@ The module defines these functions: .. function:: dump(value, file[, version]) Write the value on the open file. The value must be a supported type. The - file must be an open file object such as ``sys.stdout`` or returned by - :func:`open` or :func:`os.popen`. It must be opened in binary mode (``'wb'`` + file must be a open file object such as ``sys.stdout`` or returned by + :func:`open` or :func:`os.popen`. It may not be a wrapper such as + TemporaryFile on Windows. It must be opened in binary mode (``'wb'`` or ``'w+b'``). If the value has (or contains an object that has) an unsupported type, a diff --git a/Doc/library/math.rst b/Doc/library/math.rst index e5ffba0..562388e 100644 --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -5,6 +5,9 @@ .. module:: math :synopsis: Mathematical functions (sin() etc.). +.. testsetup:: + + from math import fsum This module is always available. It provides access to the mathematical functions defined by the C standard. @@ -133,8 +136,9 @@ Number-theoretic and representation functions .. function:: trunc(x) - Return the :class:`Real` value *x* truncated to an :class:`Integral` (usually - a long integer). Uses the ``__trunc__`` method. + Return the :class:`~numbers.Real` value *x* truncated to an + :class:`~numbers.Integral` (usually a long integer). Uses the + ``__trunc__`` method. .. versionadded:: 2.6 @@ -209,6 +213,10 @@ Power and logarithmic functions ``x`` is negative, and ``y`` is not an integer then ``pow(x, y)`` is undefined, and raises :exc:`ValueError`. + Unlike the built-in ``**`` operator, :func:`math.pow` converts both + its arguments to type :class:`float`. Use ``**`` or the built-in + :func:`pow` function for computing exact integer powers. + .. versionchanged:: 2.6 The outcome of ``1**nan`` and ``nan**0`` was undefined. diff --git a/Doc/library/mhlib.rst b/Doc/library/mhlib.rst index 2aab1dc..939bdc7 100644 --- a/Doc/library/mhlib.rst +++ b/Doc/library/mhlib.rst @@ -6,7 +6,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`mhlib` module has been removed in Python 3.0. Use the + The :mod:`mhlib` module has been removed in Python 3. Use the :mod:`mailbox` instead. .. sectionauthor:: Skip Montanaro <skip@pobox.com> diff --git a/Doc/library/mimetypes.rst b/Doc/library/mimetypes.rst index ccda1e9..8891e7a 100644 --- a/Doc/library/mimetypes.rst +++ b/Doc/library/mimetypes.rst @@ -85,6 +85,9 @@ behavior of the module. :const:`knownfiles` takes precedence over those named before it. Calling :func:`init` repeatedly is allowed. + Specifying an empty list for *files* will prevent the system defaults from + being applied: only the well-known values will be present from a built-in list. + .. versionchanged:: 2.7 Previously, Windows registry settings were ignored. diff --git a/Doc/library/mimewriter.rst b/Doc/library/mimewriter.rst index 2070ff6..a30caef 100644 --- a/Doc/library/mimewriter.rst +++ b/Doc/library/mimewriter.rst @@ -24,7 +24,7 @@ to rearrange their order. Return a new instance of the :class:`MimeWriter` class. The only argument passed, *fp*, is a file object to be used for writing. Note that a - :class:`StringIO` object could also be used. + :class:`~StringIO.StringIO` object could also be used. .. _mimewriter-objects: diff --git a/Doc/library/mmap.rst b/Doc/library/mmap.rst index 55861f9..ac1963f 100644 --- a/Doc/library/mmap.rst +++ b/Doc/library/mmap.rst @@ -114,19 +114,19 @@ memory but does not update the underlying file. with open("hello.txt", "r+b") as f: # memory-map the file, size 0 means whole file - map = mmap.mmap(f.fileno(), 0) + mm = mmap.mmap(f.fileno(), 0) # read content via standard file methods - print map.readline() # prints "Hello Python!" + print mm.readline() # prints "Hello Python!" # read content via slice notation - print map[:5] # prints "Hello" + print mm[:5] # prints "Hello" # update content using slice notation; # note that new content must have same size - map[6:] = " world!\n" + mm[6:] = " world!\n" # ... and read again using standard file methods - map.seek(0) - print map.readline() # prints "Hello world!" + mm.seek(0) + print mm.readline() # prints "Hello world!" # close the map - map.close() + mm.close() The next example demonstrates how to create an anonymous map and exchange @@ -135,16 +135,16 @@ memory but does not update the underlying file. import mmap import os - map = mmap.mmap(-1, 13) - map.write("Hello world!") + mm = mmap.mmap(-1, 13) + mm.write("Hello world!") pid = os.fork() if pid == 0: # In a child process - map.seek(0) - print map.readline() + mm.seek(0) + print mm.readline() - map.close() + mm.close() Memory-mapped file objects support the following methods: @@ -152,8 +152,9 @@ memory but does not update the underlying file. .. method:: close() - Close the file. Subsequent calls to other methods of the object will - result in an exception being raised. + Closes the mmap. Subsequent calls to other methods of the object will + result in a ValueError exception being raised. This will not close + the open file. .. method:: find(string[, start[, end]]) diff --git a/Doc/library/msilib.rst b/Doc/library/msilib.rst index 59e9cf9..17d71ab 100644 --- a/Doc/library/msilib.rst +++ b/Doc/library/msilib.rst @@ -432,8 +432,9 @@ GUI classes ----------- :mod:`msilib` provides several classes that wrap the GUI tables in an MSI -database. However, no standard user interface is provided; use :mod:`bdist_msi` -to create MSI files with a user-interface for installing Python packages. +database. However, no standard user interface is provided; use +:mod:`~distutils.command.bdist_msi` to create MSI files with a user-interface +for installing Python packages. .. class:: Control(dlg, name) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 7d9aaf3..886b173 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -81,7 +81,8 @@ To show the individual process IDs involved, here is an expanded example:: def info(title): print title print 'module name:', __name__ - print 'parent process:', os.getppid() + if hasattr(os, 'getppid'): # only available on Unix + print 'parent process:', os.getppid() print 'process id:', os.getpid() def f(name): @@ -107,7 +108,7 @@ processes: **Queues** - The :class:`Queue` class is a near clone of :class:`Queue.Queue`. For + The :class:`~multiprocessing.Queue` class is a near clone of :class:`Queue.Queue`. For example:: from multiprocessing import Process, Queue @@ -231,7 +232,7 @@ However, if you really do need to use some shared data then A manager returned by :func:`Manager` will support types :class:`list`, :class:`dict`, :class:`Namespace`, :class:`Lock`, :class:`RLock`, :class:`Semaphore`, :class:`BoundedSemaphore`, :class:`Condition`, - :class:`Event`, :class:`Queue`, :class:`Value` and :class:`Array`. For + :class:`Event`, :class:`~multiprocessing.Queue`, :class:`Value` and :class:`Array`. For example, :: from multiprocessing import Process, Manager @@ -286,6 +287,9 @@ For example:: print result.get(timeout=1) # prints "100" unless your computer is *very* slow print pool.map(f, range(10)) # prints "[0, 1, 4,..., 81]" +Note that the methods of a pool should only ever be used by the +process which created it. + Reference --------- @@ -297,7 +301,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the :class:`Process` and exceptions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. class:: Process([group[, target[, name[, args[, kwargs]]]]]) +.. class:: Process(group=None, target=None, name=None, args=(), kwargs={}) Process objects represent activity that is run in a separate process. The :class:`Process` class has equivalents of all the methods of @@ -378,7 +382,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the Unix daemons or services, they are normal processes that will be terminated (and not joined) if non-daemonic processes have exited. - In addition to the :class:`Threading.Thread` API, :class:`Process` objects + In addition to the :class:`threading.Thread` API, :class:`Process` objects also support the following attributes and methods: .. attribute:: pid @@ -397,7 +401,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the The process's authentication key (a byte string). When :mod:`multiprocessing` is initialized the main process is assigned a - random string using :func:`os.random`. + random string using :func:`os.urandom`. When a :class:`Process` object is created, it will inherit the authentication key of its parent process, although this may be changed by @@ -422,9 +426,9 @@ The :mod:`multiprocessing` package mostly replicates the API of the acquired a lock or semaphore etc. then terminating it is liable to cause other processes to deadlock. - Note that the :meth:`start`, :meth:`join`, :meth:`is_alive` and - :attr:`exit_code` methods should only be called by the process that created - the process object. + Note that the :meth:`start`, :meth:`join`, :meth:`is_alive`, + :meth:`terminate` and :attr:`exitcode` methods should only be called by + the process that created the process object. Example usage of some of the methods of :class:`Process`: @@ -464,9 +468,9 @@ primitives like locks. For passing messages one can use :func:`Pipe` (for a connection between two processes) or a queue (which allows multiple producers and consumers). -The :class:`Queue`, :class:`multiprocessing.queues.SimpleQueue` and :class:`JoinableQueue` types are multi-producer, +The :class:`~multiprocessing.Queue`, :class:`multiprocessing.queues.SimpleQueue` and :class:`JoinableQueue` types are multi-producer, multi-consumer FIFO queues modelled on the :class:`Queue.Queue` class in the -standard library. They differ in that :class:`Queue` lacks the +standard library. They differ in that :class:`~multiprocessing.Queue` lacks the :meth:`~Queue.Queue.task_done` and :meth:`~Queue.Queue.join` methods introduced into Python 2.5's :class:`Queue.Queue` class. @@ -485,18 +489,37 @@ Note that one can also create a shared queue by using a manager object -- see the :mod:`multiprocessing` namespace so you need to import them from :mod:`Queue`. +.. note:: + + When an object is put on a queue, the object is pickled and a + background thread later flushes the pickled data to an underlying + pipe. This has some consequences which are a little surprising, + but should not cause any practical difficulties -- if they really + bother you then you can instead use a queue created with a + :ref:`manager <multiprocessing-managers>`. + + (1) After putting an object on an empty queue there may be an + infinitesimal delay before the queue's :meth:`~Queue.empty` + method returns :const:`False` and :meth:`~Queue.get_nowait` can + return without raising :exc:`Queue.Empty`. + + (2) If multiple processes are enqueuing objects, it is possible for + the objects to be received at the other end out-of-order. + However, objects enqueued by the same process will always be in + the expected order with respect to each other. .. warning:: If a process is killed using :meth:`Process.terminate` or :func:`os.kill` - while it is trying to use a :class:`Queue`, then the data in the queue is + while it is trying to use a :class:`~multiprocessing.Queue`, then the data in the queue is likely to become corrupted. This may cause any other process to get an exception when it tries to use the queue later on. .. warning:: As mentioned above, if a child process has put items on a queue (and it has - not used :meth:`JoinableQueue.cancel_join_thread`), then that process will + not used :meth:`JoinableQueue.cancel_join_thread + <multiprocessing.Queue.cancel_join_thread>`), then that process will not terminate until all buffered items have been flushed to the pipe. This means that if you try joining that process you may get a deadlock unless @@ -531,7 +554,7 @@ For an example of the usage of queues for interprocess communication see The usual :exc:`Queue.Empty` and :exc:`Queue.Full` exceptions from the standard library's :mod:`Queue` module are raised to signal timeouts. - :class:`Queue` implements all the methods of :class:`Queue.Queue` except for + :class:`~multiprocessing.Queue` implements all the methods of :class:`Queue.Queue` except for :meth:`~Queue.Queue.task_done` and :meth:`~Queue.Queue.join`. .. method:: qsize() @@ -578,11 +601,10 @@ For an example of the usage of queues for interprocess communication see :exc:`Queue.Empty` exception (*timeout* is ignored in that case). .. method:: get_nowait() - get_no_wait() Equivalent to ``get(False)``. - :class:`multiprocessing.Queue` has a few additional methods not found in + :class:`~multiprocessing.Queue` has a few additional methods not found in :class:`Queue.Queue`. These methods are usually unnecessary for most code: @@ -609,10 +631,17 @@ For an example of the usage of queues for interprocess communication see the background thread from being joined automatically when the process exits -- see :meth:`join_thread`. + A better name for this method might be + ``allow_exit_without_flush()``. It is likely to cause enqueued + data to lost, and you almost certainly will not need to use it. + It is really only there if you need the current process to exit + immediately without waiting to flush enqueued data to the + underlying pipe, and you don't care about lost data. + .. class:: multiprocessing.queues.SimpleQueue() - It is a simplified :class:`Queue` type, very close to a locked :class:`Pipe`. + It is a simplified :class:`~multiprocessing.Queue` type, very close to a locked :class:`Pipe`. .. method:: empty() @@ -629,7 +658,7 @@ For an example of the usage of queues for interprocess communication see .. class:: JoinableQueue([maxsize]) - :class:`JoinableQueue`, a :class:`Queue` subclass, is a queue which + :class:`JoinableQueue`, a :class:`~multiprocessing.Queue` subclass, is a queue which additionally has :meth:`task_done` and :meth:`join` methods. .. method:: task_done() @@ -639,7 +668,7 @@ For an example of the usage of queues for interprocess communication see call to :meth:`task_done` tells the queue that the processing on the task is complete. - If a :meth:`~Queue.join` is currently blocking, it will resume when all + If a :meth:`~Queue.Queue.join` is currently blocking, it will resume when all items have been processed (meaning that a :meth:`task_done` call was received for every item that had been :meth:`~Queue.put` into the queue). @@ -655,7 +684,7 @@ For an example of the usage of queues for interprocess communication see queue. The count goes down whenever a consumer thread calls :meth:`task_done` to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, - :meth:`~Queue.join` unblocks. + :meth:`~Queue.Queue.join` unblocks. Miscellaneous @@ -931,12 +960,24 @@ inherited by child processes. ctypes type or a one character typecode of the kind used by the :mod:`array` module. *\*args* is passed on to the constructor for the type. - If *lock* is ``True`` (the default) then a new lock object is created to - synchronize access to the value. If *lock* is a :class:`Lock` or - :class:`RLock` object then that will be used to synchronize access to the - value. If *lock* is ``False`` then access to the returned object will not be - automatically protected by a lock, so it will not necessarily be - "process-safe". + If *lock* is ``True`` (the default) then a new recursive lock + object is created to synchronize access to the value. If *lock* is + a :class:`Lock` or :class:`RLock` object then that will be used to + synchronize access to the value. If *lock* is ``False`` then + access to the returned object will not be automatically protected + by a lock, so it will not necessarily be "process-safe". + + Operations like ``+=`` which involve a read and write are not + atomic. So if, for instance, you want to atomically increment a + shared value it is insufficient to just do :: + + counter.value += 1 + + Assuming the associated lock is recursive (which it is by default) + you can instead do :: + + with counter.get_lock(): + counter.value += 1 Note that *lock* is a keyword-only argument. @@ -1021,8 +1062,9 @@ processes. array. If *lock* is ``True`` (the default) then a new lock object is created to - synchronize access to the value. If *lock* is a :class:`Lock` or - :class:`RLock` object then that will be used to synchronize access to the + synchronize access to the value. If *lock* is a + :class:`~multiprocessing.Lock` or :class:`~multiprocessing.RLock` object + then that will be used to synchronize access to the value. If *lock* is ``False`` then access to the returned object will not be automatically protected by a lock, so it will not necessarily be "process-safe". @@ -1036,8 +1078,8 @@ processes. object. If *lock* is ``True`` (the default) then a new lock object is created to - synchronize access to the value. If *lock* is a :class:`Lock` or - :class:`RLock` object then that will be used to synchronize access to the + synchronize access to the value. If *lock* is a :class:`~multiprocessing.Lock` or + :class:`~multiprocessing.RLock` object then that will be used to synchronize access to the value. If *lock* is ``False`` then access to the returned object will not be automatically protected by a lock, so it will not necessarily be "process-safe". @@ -1215,12 +1257,12 @@ their parent process exits. The manager classes are defined in the *exposed* is used to specify a sequence of method names which proxies for this typeid should be allowed to access using - :meth:`BaseProxy._callMethod`. (If *exposed* is ``None`` then + :meth:`BaseProxy._callmethod`. (If *exposed* is ``None`` then :attr:`proxytype._exposed_` is used instead if it exists.) In the case where no exposed list is specified, all "public methods" of the shared object will be accessible. (Here a "public method" means any attribute - which has a :meth:`__call__` method and whose name does not begin with - ``'_'``.) + which has a :meth:`~object.__call__` method and whose name does not begin + with ``'_'``.) *method_to_typeid* is a mapping used to specify the return type of those exposed methods which should return a proxy. It maps method names to @@ -1581,6 +1623,9 @@ with the :class:`Pool` class. *initializer* is not ``None`` then each worker process will call ``initializer(*initargs)`` when it starts. + Note that the methods of the pool object should only be called by + the process which created the pool. + .. versionadded:: 2.7 *maxtasksperchild* is the number of tasks a worker process can complete before it will exit and be replaced with a fresh worker process, to enable @@ -1727,7 +1772,8 @@ Listeners and Clients :synopsis: API for dealing with sockets. Usually message passing between processes is done using queues or by using -:class:`Connection` objects returned by :func:`Pipe`. +:class:`~multiprocessing.Connection` objects returned by +:func:`~multiprocessing.Pipe`. However, the :mod:`multiprocessing.connection` module allows some extra flexibility. It basically gives a high level message oriented API for dealing @@ -1744,7 +1790,7 @@ authentication* using the :mod:`hmac` module. then a welcome message is sent to the other end of the connection. Otherwise :exc:`AuthenticationError` is raised. -.. function:: answerChallenge(connection, authkey) +.. function:: answer_challenge(connection, authkey) Receive a message, calculate the digest of the message using *authkey* as the key, and then send the digest back. @@ -1793,7 +1839,8 @@ authentication* using the :mod:`hmac` module. private temporary directory created using :func:`tempfile.mkstemp`. If the listener object uses a socket then *backlog* (1 by default) is passed - to the :meth:`listen` method of the socket once it has been bound. + to the :meth:`~socket.socket.listen` method of the socket once it has been + bound. If *authenticate* is ``True`` (``False`` by default) or *authkey* is not ``None`` then digest authentication is used. @@ -1810,8 +1857,9 @@ authentication* using the :mod:`hmac` module. .. method:: accept() Accept a connection on the bound socket or named pipe of the listener - object and return a :class:`Connection` object. If authentication is - attempted and fails, then :exc:`AuthenticationError` is raised. + object and return a :class:`~multiprocessing.Connection` object. If + authentication is attempted and fails, then + :exc:`~multiprocessing.AuthenticationError` is raised. .. method:: close() @@ -1907,7 +1955,8 @@ an ``'AF_PIPE'`` address rather than an ``'AF_UNIX'`` address. Authentication keys ~~~~~~~~~~~~~~~~~~~ -When one uses :meth:`Connection.recv`, the data received is automatically +When one uses :meth:`Connection.recv <multiprocessing.Connection.recv>`, the +data received is automatically unpickled. Unfortunately unpickling data from an untrusted source is a security risk. Therefore :class:`Listener` and :func:`Client` use the :mod:`hmac` module to provide digest authentication. @@ -2056,9 +2105,10 @@ Joining zombie processes On Unix when a process finishes but has not been joined it becomes a zombie. There should never be very many because each time a new process starts (or - :func:`active_children` is called) all completed processes which have not - yet been joined will be joined. Also calling a finished process's - :meth:`Process.is_alive` will join the process. Even so it is probably good + :func:`~multiprocessing.active_children` is called) all completed processes + which have not yet been joined will be joined. Also calling a finished + process's :meth:`Process.is_alive <multiprocessing.Process.is_alive>` will + join the process. Even so it is probably good practice to explicitly join all the processes that you start. Better to inherit than pickle/unpickle @@ -2071,20 +2121,22 @@ Better to inherit than pickle/unpickle Avoid terminating processes - Using the :meth:`Process.terminate` method to stop a process is liable to + Using the :meth:`Process.terminate <multiprocessing.Process.terminate>` + method to stop a process is liable to cause any shared resources (such as locks, semaphores, pipes and queues) currently being used by the process to become broken or unavailable to other processes. Therefore it is probably best to only consider using - :meth:`Process.terminate` on processes which never use any shared resources. + :meth:`Process.terminate <multiprocessing.Process.terminate>` on processes + which never use any shared resources. Joining processes that use queues Bear in mind that a process that has put items in a queue will wait before terminating until all the buffered items are fed by the "feeder" thread to the underlying pipe. (The child process can call the - :meth:`Queue.cancel_join_thread` method of the queue to avoid this behaviour.) + :meth:`~multiprocessing.Queue.cancel_join_thread` method of the queue to avoid this behaviour.) This means that whenever you use a queue you need to make sure that all items which have been put on the queue will eventually be removed before the @@ -2161,7 +2213,7 @@ Beware of replacing :data:`sys.stdin` with a "file like object" resulting in a bad file descriptor error, but introduces a potential danger to applications which replace :func:`sys.stdin` with a "file-like object" with output buffering. This danger is that if multiple processes call - :func:`close()` on this file-like object, it could result in the same + :meth:`~io.IOBase.close()` on this file-like object, it could result in the same data being flushed to the object multiple times, resulting in corruption. If you write a file-like object and implement your own caching, you can @@ -2190,14 +2242,16 @@ More picklability as the ``target`` argument on Windows --- just define a function and use that instead. - Also, if you subclass :class:`Process` then make sure that instances will be - picklable when the :meth:`Process.start` method is called. + Also, if you subclass :class:`~multiprocessing.Process` then make sure that + instances will be picklable when the :meth:`Process.start + <multiprocessing.Process.start>` method is called. Global variables Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value - in the parent process at the time that :meth:`Process.start` was called. + in the parent process at the time that :meth:`Process.start + <multiprocessing.Process.start>` was called. However, global variables which are just module level constants cause no problems. @@ -2252,7 +2306,7 @@ Demonstration of how to create and use customized managers and proxies: .. literalinclude:: ../includes/mp_newtype.py -Using :class:`Pool`: +Using :class:`~multiprocessing.pool.Pool`: .. literalinclude:: ../includes/mp_pool.py diff --git a/Doc/library/mutex.rst b/Doc/library/mutex.rst index 2d41350..57c3971 100644 --- a/Doc/library/mutex.rst +++ b/Doc/library/mutex.rst @@ -7,7 +7,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`mutex` module has been removed in Python 3.0. + The :mod:`mutex` module has been removed in Python 3. .. sectionauthor:: Moshe Zadka <moshez@zadka.site.co.il> diff --git a/Doc/library/netrc.rst b/Doc/library/netrc.rst index 323fd69..713c8df 100644 --- a/Doc/library/netrc.rst +++ b/Doc/library/netrc.rst @@ -25,6 +25,14 @@ the Unix :program:`ftp` program and other FTP clients. no argument is given, the file :file:`.netrc` in the user's home directory will be read. Parse errors will raise :exc:`NetrcParseError` with diagnostic information including the file name, line number, and terminating token. + If no argument is specified on a POSIX system, the presence of passwords in + the :file:`.netrc` file will raise a :exc:`NetrcParseError` if the file + ownership or permissions are insecure (owned by a user other than the user + running the process, or accessible for read or write by any other user). + This implements security behavior equivalent to that of ftp and other + programs that use :file:`.netrc`. + + .. versionchanged:: 2.7.6 Added the POSIX permissions check. .. exception:: NetrcParseError diff --git a/Doc/library/new.rst b/Doc/library/new.rst index 8dd965e..667e586 100644 --- a/Doc/library/new.rst +++ b/Doc/library/new.rst @@ -6,7 +6,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`new` module has been removed in Python 3.0. Use the :mod:`types` + The :mod:`new` module has been removed in Python 3. Use the :mod:`types` module's classes instead. .. sectionauthor:: Moshe Zadka <moshez@zadka.site.co.il> diff --git a/Doc/library/nntplib.rst b/Doc/library/nntplib.rst index acbb7a5..92180d6 100644 --- a/Doc/library/nntplib.rst +++ b/Doc/library/nntplib.rst @@ -46,7 +46,7 @@ To post an article from a file (this assumes that the article has valid headers, and that you have right to post on the particular newsgroup):: >>> s = NNTP('news.gmane.org') - >>> f = open('/tmp/article') + >>> f = open('articlefile') >>> s.post(f) '240 Article posted successfully.' >>> s.quit() @@ -234,25 +234,25 @@ indicates an error, the method raises one of the above exceptions. .. method:: NNTP.next() - Send a ``NEXT`` command. Return as for :meth:`stat`. + Send a ``NEXT`` command. Return as for :meth:`.stat`. .. method:: NNTP.last() - Send a ``LAST`` command. Return as for :meth:`stat`. + Send a ``LAST`` command. Return as for :meth:`.stat`. .. method:: NNTP.head(id) - Send a ``HEAD`` command, where *id* has the same meaning as for :meth:`stat`. + Send a ``HEAD`` command, where *id* has the same meaning as for :meth:`.stat`. Return a tuple ``(response, number, id, list)`` where the first three are the - same as for :meth:`stat`, and *list* is a list of the article's headers (an + same as for :meth:`.stat`, and *list* is a list of the article's headers (an uninterpreted list of lines, without trailing newlines). .. method:: NNTP.body(id,[file]) - Send a ``BODY`` command, where *id* has the same meaning as for :meth:`stat`. + Send a ``BODY`` command, where *id* has the same meaning as for :meth:`.stat`. If the *file* parameter is supplied, then the body is stored in a file. If *file* is a string, then the method will open a file object with that name, write to it then close it. If *file* is a file object, then it will start @@ -263,7 +263,7 @@ indicates an error, the method raises one of the above exceptions. .. method:: NNTP.article(id) Send an ``ARTICLE`` command, where *id* has the same meaning as for - :meth:`stat`. Return as for :meth:`head`. + :meth:`.stat`. Return as for :meth:`head`. .. method:: NNTP.slave() @@ -290,7 +290,7 @@ indicates an error, the method raises one of the above exceptions. .. method:: NNTP.post(file) Post an article using the ``POST`` command. The *file* argument is an open file - object which is read until EOF using its :meth:`readline` method. It should be + object which is read until EOF using its :meth:`~file.readline` method. It should be a well-formed news article, including the required headers. The :meth:`post` method automatically escapes lines beginning with ``.``. diff --git a/Doc/library/numbers.rst b/Doc/library/numbers.rst index f46e8ac..8811b5d 100644 --- a/Doc/library/numbers.rst +++ b/Doc/library/numbers.rst @@ -73,10 +73,10 @@ The numeric tower .. class:: Integral - Subtypes :class:`Rational` and adds a conversion to :class:`int`. - Provides defaults for :func:`float`, :attr:`~Rational.numerator`, and - :attr:`~Rational.denominator`, and bit-string operations: ``<<``, - ``>>``, ``&``, ``^``, ``|``, ``~``. + Subtypes :class:`Rational` and adds a conversion to :class:`int`. Provides + defaults for :func:`float`, :attr:`~Rational.numerator`, and + :attr:`~Rational.denominator`. Adds abstract methods for ``**`` and + bit-string operations: ``<<``, ``>>``, ``&``, ``^``, ``|``, ``~``. Notes for type implementors diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst index 53d45b2..0b46504 100644 --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -490,13 +490,22 @@ lookups. These are useful for making fast field extractors as arguments for expect a function argument. -.. function:: attrgetter(attr[, args...]) +.. function:: attrgetter(attr) + attrgetter(*attrs) - Return a callable object that fetches *attr* from its operand. If more than one - attribute is requested, returns a tuple of attributes. After, - ``f = attrgetter('name')``, the call ``f(b)`` returns ``b.name``. After, - ``f = attrgetter('name', 'date')``, the call ``f(b)`` returns ``(b.name, - b.date)``. Equivalent to:: + Return a callable object that fetches *attr* from its operand. + If more than one attribute is requested, returns a tuple of attributes. + The attribute names can also contain dots. For example: + + * After ``f = attrgetter('name')``, the call ``f(b)`` returns ``b.name``. + + * After ``f = attrgetter('name', 'date')``, the call ``f(b)`` returns + ``(b.name, b.date)``. + + * After ``f = attrgetter('name.first', 'name.last')``, the call ``f(b)`` + returns ``(b.name.first, b.name.last)``. + + Equivalent to:: def attrgetter(*items): if len(items) == 1: @@ -505,7 +514,7 @@ expect a function argument. return resolve_attr(obj, attr) else: def g(obj): - return tuple(resolve_att(obj, attr) for attr in items) + return tuple(resolve_attr(obj, attr) for attr in items) return g def resolve_attr(obj, attr): @@ -514,9 +523,6 @@ expect a function argument. return obj - The attribute names can also contain dots; after ``f = attrgetter('date.month')``, - the call ``f(b)`` returns ``b.date.month``. - .. versionadded:: 2.4 .. versionchanged:: 2.5 @@ -526,11 +532,19 @@ expect a function argument. Added support for dotted attributes. -.. function:: itemgetter(item[, args...]) +.. function:: itemgetter(item) + itemgetter(*items) Return a callable object that fetches *item* from its operand using the operand's :meth:`__getitem__` method. If multiple items are specified, - returns a tuple of lookup values. Equivalent to:: + returns a tuple of lookup values. For example: + + * After ``f = itemgetter(2)``, the call ``f(r)`` returns ``r[2]``. + + * After ``g = itemgetter(2, 5, 3)``, the call ``g(r)`` returns + ``(r[2], r[5], r[3])``. + + Equivalent to:: def itemgetter(*items): if len(items) == 1: @@ -573,9 +587,14 @@ expect a function argument. Return a callable object that calls the method *name* on its operand. If additional arguments and/or keyword arguments are given, they will be given - to the method as well. After ``f = methodcaller('name')``, the call ``f(b)`` - returns ``b.name()``. After ``f = methodcaller('name', 'foo', bar=1)``, the - call ``f(b)`` returns ``b.name('foo', bar=1)``. Equivalent to:: + to the method as well. For example: + + * After ``f = methodcaller('name')``, the call ``f(b)`` returns ``b.name()``. + + * After ``f = methodcaller('name', 'foo', bar=1)``, the call ``f(b)`` + returns ``b.name('foo', bar=1)``. + + Equivalent to:: def methodcaller(name, *args, **kwargs): def caller(obj): diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index d0783e7..417b3bb 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -173,10 +173,10 @@ required option For example, consider this hypothetical command-line:: - prog -v --report /tmp/report.txt foo bar + prog -v --report report.txt foo bar ``-v`` and ``--report`` are both options. Assuming that ``--report`` -takes one argument, ``/tmp/report.txt`` is an option argument. ``foo`` and +takes one argument, ``report.txt`` is an option argument. ``foo`` and ``bar`` are positional arguments. @@ -275,7 +275,8 @@ You're free to define as many short option strings and as many long option strings as you like (including zero), as long as there is at least one option string overall. -The option strings passed to :meth:`add_option` are effectively labels for the +The option strings passed to :meth:`OptionParser.add_option` are effectively +labels for the option defined by that call. For brevity, we will frequently refer to *encountering an option* on the command line; in reality, :mod:`optparse` encounters *option strings* and looks up options from them. @@ -895,7 +896,8 @@ long option strings, but you must specify at least one overall option string. The canonical way to create an :class:`Option` instance is with the :meth:`add_option` method of :class:`OptionParser`. -.. method:: OptionParser.add_option(opt_str[, ...], attr=value, ...) +.. method:: OptionParser.add_option(option) + OptionParser.add_option(*opt_str, attr=value, ...) To define an option with only a short option string:: @@ -1168,6 +1170,17 @@ must specify for any option using that action. options.tracks.append(int("4")) + The ``append`` action calls the ``append`` method on the current value of the + option. This means that any default value specified must have an ``append`` + method. It also means that if the default value is non-empty, the default + elements will be present in the parsed value for the option, with any values + from the command line appended after those default values:: + + >>> parser.add_option("--files", action="append", default=['~/.mypkg/defaults']) + >>> opts, args = parser.parse_args(['--files', 'overrides.mypkg']) + >>> opts.files + ['~/.mypkg/defaults', 'overrides.mypkg'] + * ``"append_const"`` [required: :attr:`~Option.const`; relevant: :attr:`~Option.dest`] diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst index 62bbdff..ed3aaf4 100644 --- a/Doc/library/os.path.rst +++ b/Doc/library/os.path.rst @@ -16,6 +16,11 @@ write files see :func:`open`, and for accessing the filesystem see the :func:`splitunc` and :func:`ismount` do handle them correctly. +Unlike a unix shell, Python does not do any *automatic* path expansions. +Functions such as :func:`expanduser` and :func:`expandvars` can be invoked +explicitly when an application desires shell-like path expansion. (See also +the :mod:`glob` module.) + .. note:: Since different operating systems have different path name conventions, there @@ -35,15 +40,17 @@ write files see :func:`open`, and for accessing the filesystem see the .. function:: abspath(path) Return a normalized absolutized version of the pathname *path*. On most - platforms, this is equivalent to ``normpath(join(os.getcwd(), path))``. + platforms, this is equivalent to calling the function :func:`normpath` as + follows: ``normpath(join(os.getcwd(), path))``. .. versionadded:: 1.5.2 .. function:: basename(path) - Return the base name of pathname *path*. This is the second half of the pair - returned by ``split(path)``. Note that the result of this function is different + Return the base name of pathname *path*. This is the second element of the + pair returned by passing *path* to the function :func:`split`. Note that + the result of this function is different from the Unix :program:`basename` program; where :program:`basename` for ``'/foo/bar/'`` returns ``'bar'``, the :func:`basename` function returns an empty string (``''``). @@ -58,8 +65,8 @@ write files see :func:`open`, and for accessing the filesystem see the .. function:: dirname(path) - Return the directory name of pathname *path*. This is the first half of the - pair returned by ``split(path)``. + Return the directory name of pathname *path*. This is the first element of + the pair returned by passing *path* to the function :func:`split`. .. function:: exists(path) @@ -120,7 +127,7 @@ write files see :func:`open`, and for accessing the filesystem see the .. versionadded:: 1.5.2 .. versionchanged:: 2.3 - If :func:`os.stat_float_times` returns True, the result is a floating point + If :func:`os.stat_float_times` returns ``True``, the result is a floating point number. @@ -133,14 +140,14 @@ write files see :func:`open`, and for accessing the filesystem see the .. versionadded:: 1.5.2 .. versionchanged:: 2.3 - If :func:`os.stat_float_times` returns True, the result is a floating point + If :func:`os.stat_float_times` returns ``True``, the result is a floating point number. .. function:: getctime(path) Return the system's ctime which, on some systems (like Unix) is the time of the - last change, and, on others (like Windows), is the creation time for *path*. + last metadata change, and, on others (like Windows), is the creation time for *path*. The return value is a number giving the number of seconds since the epoch (see the :mod:`time` module). Raise :exc:`os.error` if the file does not exist or is inaccessible. @@ -178,7 +185,7 @@ write files see :func:`open`, and for accessing the filesystem see the .. function:: islink(path) Return ``True`` if *path* refers to a directory entry that is a symbolic link. - Always ``False`` if symbolic links are not supported. + Always ``False`` if symbolic links are not supported by the python runtime. .. function:: ismount(path) @@ -212,13 +219,11 @@ write files see :func:`open`, and for accessing the filesystem see the .. function:: normpath(path) - Normalize a pathname. This collapses redundant separators and up-level - references so that ``A//B``, ``A/B/``, ``A/./B`` and ``A/foo/../B`` all become - ``A/B``. - - It does not normalize the case (use :func:`normcase` for that). On Windows, it - converts forward slashes to backward slashes. It should be understood that this - may change the meaning of the path if it contains symbolic links! + Normalize a pathname by collapsing redundant separators and up-level + references so that ``A//B``, ``A/B/``, ``A/./B`` and ``A/foo/../B`` all + become ``A/B``. This string manipulation may change the meaning of a path + that contains symbolic links. On Windows, it converts forward slashes to + backward slashes. To normalize case, use :func:`normcase`. .. function:: realpath(path) @@ -231,8 +236,10 @@ write files see :func:`open`, and for accessing the filesystem see the .. function:: relpath(path[, start]) - Return a relative filepath to *path* either from the current directory or from - an optional *start* point. + Return a relative filepath to *path* either from the current directory or + from an optional *start* directory. This is a path computation: the + filesystem is not accessed to confirm the existence or nature of *path* or + *start*. *start* defaults to :attr:`os.curdir`. @@ -260,9 +267,9 @@ write files see :func:`open`, and for accessing the filesystem see the .. function:: samestat(stat1, stat2) Return ``True`` if the stat tuples *stat1* and *stat2* refer to the same file. - These structures may have been returned by :func:`fstat`, :func:`lstat`, or - :func:`stat`. This function implements the underlying comparison used by - :func:`samefile` and :func:`sameopenfile`. + These structures may have been returned by :func:`os.fstat`, + :func:`os.lstat`, or :func:`os.stat`. This function implements the + underlying comparison used by :func:`samefile` and :func:`sameopenfile`. Availability: Unix. @@ -276,7 +283,8 @@ write files see :func:`open`, and for accessing the filesystem see the *path* is empty, both *head* and *tail* are empty. Trailing slashes are stripped from *head* unless it is the root (one or more slashes only). In all cases, ``join(head, tail)`` returns a path to the same location as *path* - (but the strings may differ). + (but the strings may differ). Also see the functions :func:`dirname` and + :func:`basename`. .. function:: splitdrive(path) @@ -331,13 +339,13 @@ write files see :func:`open`, and for accessing the filesystem see the .. note:: - This function is deprecated and has been removed in 3.0 in favor of + This function is deprecated and has been removed in Python 3 in favor of :func:`os.walk`. .. data:: supports_unicode_filenames - True if arbitrary Unicode strings can be used as file names (within limitations + ``True`` if arbitrary Unicode strings can be used as file names (within limitations imposed by the file system). .. versionadded:: 2.3 diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 8c63444..32051c0 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -72,7 +72,7 @@ process and user. .. data:: environ - A mapping object representing the string environment. For example, + A :term:`mapping` object representing the string environment. For example, ``environ['HOME']`` is the pathname of your home directory (on some platforms), and is equivalent to ``getenv("HOME")`` in C. @@ -157,6 +157,22 @@ process and user. Availability: Unix. + .. note:: + + On Mac OS X, :func:`getgroups` behavior differs somewhat from + other Unix platforms. If the Python interpreter was built with a + deployment target of :const:`10.5` or earlier, :func:`getgroups` returns + the list of effective group ids associated with the current user process; + this list is limited to a system-defined number of entries, typically 16, + and may be modified by calls to :func:`setgroups` if suitably privileged. + If built with a deployment target greater than :const:`10.5`, + :func:`getgroups` returns the current group access list for the user + associated with the effective user id of the process; the group access + list may change over the lifetime of the process, it is not affected by + calls to :func:`setgroups`, and its length is not limited to 16. The + deployment target value, :const:`MACOSX_DEPLOYMENT_TARGET`, can be + obtained with :func:`sysconfig.get_config_var`. + .. function:: initgroups(username, gid) @@ -241,7 +257,7 @@ process and user. .. index:: single: user; id - Return the current process's user id. + Return the current process's real user id. Availability: Unix. @@ -306,6 +322,10 @@ process and user. .. versionadded:: 2.2 + .. note:: On Mac OS X, the length of *groups* may not exceed the + system-defined maximum number of effective group ids, typically 16. + See the documentation for :func:`getgroups` for cases where it may not + return the same group list set by calling setgroups(). .. function:: setpgrp() @@ -443,8 +463,9 @@ These functions create new file objects. (See also :func:`open`.) .. index:: single: I/O control; buffering Return an open file object connected to the file descriptor *fd*. The *mode* - and *bufsize* arguments have the same meaning as the corresponding arguments to - the built-in :func:`open` function. + and *bufsize* arguments have the same meaning as the corresponding arguments + to the built-in :func:`open` function. If :func:`fdopen` raises an + exception, it leaves *fd* untouched (unclosed). Availability: Unix, Windows. @@ -597,7 +618,7 @@ as internal buffering of data. This function is intended for low-level I/O and must be applied to a file descriptor as returned by :func:`os.open` or :func:`pipe`. To close a "file object" returned by the built-in function :func:`open` or by :func:`popen` or - :func:`fdopen`, use its :meth:`~file.close` method. + :func:`fdopen`, use its :meth:`~io.IOBase.close` method. .. function:: closerange(fd_low, fd_high) @@ -719,16 +740,14 @@ as internal buffering of data. Return ``True`` if the file descriptor *fd* is open and connected to a tty(-like) device, else ``False``. - Availability: Unix. - .. function:: lseek(fd, pos, how) Set the current position of file descriptor *fd* to position *pos*, modified by *how*: :const:`SEEK_SET` or ``0`` to set the position relative to the beginning of the file; :const:`SEEK_CUR` or ``1`` to set it relative to the - current position; :const:`os.SEEK_END` or ``2`` to set it relative to the end of - the file. + current position; :const:`SEEK_END` or ``2`` to set it relative to the end of + the file. Return the new cursor position in bytes, starting from the beginning. Availability: Unix, Windows. @@ -1163,7 +1182,7 @@ Files and Directories doesn't open the FIFO --- it just creates the rendezvous point. -.. function:: mknod(filename[, mode=0600, device]) +.. function:: mknod(filename[, mode=0600[, device=0]]) Create a filesystem node (file, device special file or named pipe) named *filename*. *mode* specifies both the permissions to use and the type of node to @@ -1363,15 +1382,14 @@ Files and Directories .. versionchanged:: 2.3 If :func:`stat_float_times` returns ``True``, the time values are floats, measuring - seconds. Fractions of a second may be reported if the system supports that. On - Mac OS, the times are always floats. See :func:`stat_float_times` for further - discussion. + seconds. Fractions of a second may be reported if the system supports that. + See :func:`stat_float_times` for further discussion. On some Unix systems (such as Linux), the following attributes may also be available: - * :attr:`st_blocks` - number of blocks allocated for file - * :attr:`st_blksize` - filesystem blocksize + * :attr:`st_blocks` - number of 512-byte blocks allocated for file + * :attr:`st_blksize` - filesystem blocksize for efficient file system I/O * :attr:`st_rdev` - type of device if an inode device * :attr:`st_flags` - user defined flags for file @@ -1381,12 +1399,6 @@ Files and Directories * :attr:`st_gen` - file generation number * :attr:`st_birthtime` - time of file creation - On Mac OS systems, the following attributes may also be available: - - * :attr:`st_rsize` - * :attr:`st_creator` - * :attr:`st_type` - On RISCOS systems, the following attributes are also available: * :attr:`st_ftype` (file type) @@ -1565,7 +1577,7 @@ Files and Directories Availability: Unix, Windows. -.. function:: walk(top[, topdown=True [, onerror=None[, followlinks=False]]]) +.. function:: walk(top, topdown=True, onerror=None, followlinks=False) .. index:: single: directory; walking @@ -1585,9 +1597,11 @@ Files and Directories If optional argument *topdown* is ``True`` or not specified, the triple for a directory is generated before the triples for any of its subdirectories - (directories are generated top-down). If *topdown* is ``False``, the triple for a - directory is generated after the triples for all of its subdirectories - (directories are generated bottom-up). + (directories are generated top-down). If *topdown* is ``False``, the triple + for a directory is generated after the triples for all of its subdirectories + (directories are generated bottom-up). No matter the value of *topdown*, the + list of subdirectories is retrieved before the tuples for the directory and + its subdirectories are generated. When *topdown* is ``True``, the caller can modify the *dirnames* list in-place (perhaps using :keyword:`del` or slice assignment), and :func:`walk` will only @@ -1660,7 +1674,7 @@ Process Management These functions may be used to create and manage processes. -The various :func:`exec\*` functions take a list of arguments for the new +The various :func:`exec\* <execl>` functions take a list of arguments for the new program loaded into the process. In each case, the first of these arguments is passed to the new program as its own name rather than as an argument a user may have typed on a command line. For the C programmer, this is the ``argv[0]`` @@ -1698,9 +1712,9 @@ to be ignored. descriptors are not flushed, so if there may be data buffered on these open files, you should flush them using :func:`sys.stdout.flush` or :func:`os.fsync` before calling an - :func:`exec\*` function. + :func:`exec\* <execl>` function. - The "l" and "v" variants of the :func:`exec\*` functions differ in how + The "l" and "v" variants of the :func:`exec\* <execl>` functions differ in how command-line arguments are passed. The "l" variants are perhaps the easiest to work with if the number of parameters is fixed when the code is written; the individual parameters simply become additional parameters to the :func:`execl\*` @@ -1712,7 +1726,7 @@ to be ignored. The variants which include a "p" near the end (:func:`execlp`, :func:`execlpe`, :func:`execvp`, and :func:`execvpe`) will use the :envvar:`PATH` environment variable to locate the program *file*. When the - environment is being replaced (using one of the :func:`exec\*e` variants, + environment is being replaced (using one of the :func:`exec\*e <execl>` variants, discussed in the next paragraph), the new environment is used as the source of the :envvar:`PATH` variable. The other variants, :func:`execl`, :func:`execle`, :func:`execv`, and :func:`execve`, will not use the :envvar:`PATH` variable to @@ -1920,6 +1934,10 @@ written in Python, such as a mail server's external command delivery program. Note that some platforms including FreeBSD <= 6.3, Cygwin and OS/2 EMX have known issues when using fork() from a thread. + .. warning:: + + See :mod:`ssl` for applications that use the SSL module with fork(). + Availability: Unix. @@ -2014,7 +2032,7 @@ written in Python, such as a mail server's external command delivery program. process. On Windows, the process id will actually be the process handle, so can be used with the :func:`waitpid` function. - The "l" and "v" variants of the :func:`spawn\*` functions differ in how + The "l" and "v" variants of the :func:`spawn\* <spawnl>` functions differ in how command-line arguments are passed. The "l" variants are perhaps the easiest to work with if the number of parameters is fixed when the code is written; the individual parameters simply become additional parameters to the @@ -2026,7 +2044,7 @@ written in Python, such as a mail server's external command delivery program. The variants which include a second "p" near the end (:func:`spawnlp`, :func:`spawnlpe`, :func:`spawnvp`, and :func:`spawnvpe`) will use the :envvar:`PATH` environment variable to locate the program *file*. When the - environment is being replaced (using one of the :func:`spawn\*e` variants, + environment is being replaced (using one of the :func:`spawn\*e <spawnl>` variants, discussed in the next paragraph), the new environment is used as the source of the :envvar:`PATH` variable. The other variants, :func:`spawnl`, :func:`spawnle`, :func:`spawnv`, and :func:`spawnve`, will not use the @@ -2062,7 +2080,7 @@ written in Python, such as a mail server's external command delivery program. .. data:: P_NOWAIT P_NOWAITO - Possible values for the *mode* parameter to the :func:`spawn\*` family of + Possible values for the *mode* parameter to the :func:`spawn\* <spawnl>` family of functions. If either of these values is given, the :func:`spawn\*` functions will return as soon as the new process has been created, with the process id as the return value. @@ -2074,7 +2092,7 @@ written in Python, such as a mail server's external command delivery program. .. data:: P_WAIT - Possible value for the *mode* parameter to the :func:`spawn\*` family of + Possible value for the *mode* parameter to the :func:`spawn\* <spawnl>` family of functions. If this is given as *mode*, the :func:`spawn\*` functions will not return until the new process has run to completion and will return the exit code of the process the run is successful, or ``-signal`` if a signal kills the @@ -2088,7 +2106,7 @@ written in Python, such as a mail server's external command delivery program. .. data:: P_DETACH P_OVERLAY - Possible values for the *mode* parameter to the :func:`spawn\*` family of + Possible values for the *mode* parameter to the :func:`spawn\* <spawnl>` family of functions. These are less portable than those listed above. :const:`P_DETACH` is similar to :const:`P_NOWAIT`, but the new process is detached from the console of the calling process. If :const:`P_OVERLAY` is used, the current @@ -2204,17 +2222,18 @@ written in Python, such as a mail server's external command delivery program. (shifting makes cross-platform use of the function easier). A *pid* less than or equal to ``0`` has no special meaning on Windows, and raises an exception. The value of integer *options* has no effect. *pid* can refer to any process whose - id is known, not necessarily a child process. The :func:`spawn` functions called - with :const:`P_NOWAIT` return suitable process handles. + id is known, not necessarily a child process. The :func:`spawn\* <spawnl>` + functions called with :const:`P_NOWAIT` return suitable process handles. -.. function:: wait3([options]) +.. function:: wait3(options) Similar to :func:`waitpid`, except no process id argument is given and a 3-element tuple containing the child's process id, exit status indication, and resource usage information is returned. Refer to :mod:`resource`.\ - :func:`getrusage` for details on resource usage information. The option - argument is the same as that provided to :func:`waitpid` and :func:`wait4`. + :func:`~resource.getrusage` for details on resource usage information. The + option argument is the same as that provided to :func:`waitpid` and + :func:`wait4`. Availability: Unix. @@ -2225,9 +2244,9 @@ written in Python, such as a mail server's external command delivery program. Similar to :func:`waitpid`, except a 3-element tuple, containing the child's process id, exit status indication, and resource usage information is returned. - Refer to :mod:`resource`.\ :func:`getrusage` for details on resource usage - information. The arguments to :func:`wait4` are the same as those provided to - :func:`waitpid`. + Refer to :mod:`resource`.\ :func:`~resource.getrusage` for details on + resource usage information. The arguments to :func:`wait4` are the same as + those provided to :func:`waitpid`. Availability: Unix. @@ -2451,8 +2470,9 @@ Higher-level operations on pathnames are defined in the :mod:`os.path` module. .. data:: defpath - The default search path used by :func:`exec\*p\*` and :func:`spawn\*p\*` if the - environment doesn't have a ``'PATH'`` key. Also available via :mod:`os.path`. + The default search path used by :func:`exec\*p\* <execl>` and + :func:`spawn\*p\* <spawnl>` if the environment doesn't have a ``'PATH'`` + key. Also available via :mod:`os.path`. .. data:: linesep @@ -2485,8 +2505,11 @@ Miscellaneous Functions This function returns random bytes from an OS-specific randomness source. The returned data should be unpredictable enough for cryptographic applications, though its exact quality depends on the OS implementation. On a UNIX-like - system this will query /dev/urandom, and on Windows it will use CryptGenRandom. - If a randomness source is not found, :exc:`NotImplementedError` will be raised. + system this will query ``/dev/urandom``, and on Windows it will use + ``CryptGenRandom()``. If a randomness source is not found, + :exc:`NotImplementedError` will be raised. - .. versionadded:: 2.4 + For an easy-to-use interface to the random number generator + provided by your platform, please see :class:`random.SystemRandom`. + .. versionadded:: 2.4 diff --git a/Doc/library/ossaudiodev.rst b/Doc/library/ossaudiodev.rst index 00c113b..79c5ea5 100644 --- a/Doc/library/ossaudiodev.rst +++ b/Doc/library/ossaudiodev.rst @@ -48,7 +48,7 @@ the standard audio interface for Linux and recent versions of FreeBSD. the official documentation for the OSS C API The module defines a large number of constants supplied by the OSS device - driver; see ``<sys/soundcard.h>`` on either Linux or FreeBSD for a listing . + driver; see ``<sys/soundcard.h>`` on either Linux or FreeBSD for a listing. :mod:`ossaudiodev` defines the following variables and functions: @@ -66,7 +66,8 @@ the standard audio interface for Linux and recent versions of FreeBSD. ``ossaudiodev.error``.) -.. function:: open([device, ]mode) +.. function:: open(mode) + open(device, mode) Open an audio device and return an OSS audio device object. This object supports many file-like methods, such as :meth:`read`, :meth:`write`, and @@ -162,11 +163,11 @@ and (read-only) attributes: is only useful in non-blocking mode. Has no return value, since the amount of data written is always equal to the amount of data supplied. -The following methods each map to exactly one :func:`ioctl` system call. The +The following methods each map to exactly one :c:func:`ioctl` system call. The correspondence is obvious: for example, :meth:`setfmt` corresponds to the ``SNDCTL_DSP_SETFMT`` ioctl, and :meth:`sync` to ``SNDCTL_DSP_SYNC`` (this can be useful when consulting the OSS documentation). If the underlying -:func:`ioctl` fails, they all raise :exc:`IOError`. +:c:func:`ioctl` fails, they all raise :exc:`IOError`. .. method:: oss_audio_device.nonblock() @@ -275,7 +276,7 @@ The following convenience methods combine several ioctls, or one ioctl and some simple calculations. -.. method:: oss_audio_device.setparameters(format, nchannels, samplerate [, strict=False]) +.. method:: oss_audio_device.setparameters(format, nchannels, samplerate[, strict=False]) Set the key audio sampling parameters---sample format, number of channels, and sampling rate---in one method call. *format*, *nchannels*, and *samplerate* @@ -295,7 +296,7 @@ simple calculations. fmt = dsp.setfmt(fmt) channels = dsp.channels(channels) - rate = dsp.rate(channels) + rate = dsp.rate(rate) .. method:: oss_audio_device.bufsize() diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst index 69df9df..1ee5c5b 100644 --- a/Doc/library/othergui.rst +++ b/Doc/library/othergui.rst @@ -13,8 +13,7 @@ available for Python: provides an object oriented interface that is slightly higher level than the C one. It comes with many more widgets than Tkinter provides, and has good Python-specific reference documentation. There are also bindings to - `GNOME <http://www.gnome.org>`_. One well known PyGTK application is - `PythonCAD <http://www.pythoncad.org/>`_. An online `tutorial + `GNOME <http://www.gnome.org>`_. An online `tutorial <http://www.pygtk.org/pygtk2tutorial/index.html>`_ is available. `PyQt <http://www.riverbankcomputing.co.uk/software/pyqt/>`_ diff --git a/Doc/library/parser.rst b/Doc/library/parser.rst index c46aeae..acda372 100644 --- a/Doc/library/parser.rst +++ b/Doc/library/parser.rst @@ -34,7 +34,7 @@ the code forming the application. It is also faster. replaced by "ast"; this is a legacy from the time when there was no other AST and has nothing to do with the AST found in Python 2.5. This is also the reason for the functions' keyword arguments being called *ast*, not *st*. - The "ast" functions will be removed in Python 3.0. + The "ast" functions have been removed in Python 3. There are a few things to note about this module which are important to making use of the data structures created. This is not a tutorial on editing the parse @@ -200,7 +200,7 @@ numbering information. information is omitted if the flag is false or omitted. -.. function:: compilest(ast[, filename='<syntax-tree>']) +.. function:: compilest(ast, filename='<syntax-tree>') .. index:: builtin: eval diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst index 4d417d2..a42eabc 100644 --- a/Doc/library/pickle.rst +++ b/Doc/library/pickle.rst @@ -352,8 +352,9 @@ The following types can be pickled: * classes that are defined at the top level of a module -* instances of such classes whose :attr:`__dict__` or :meth:`__setstate__` is - picklable (see section :ref:`pickle-protocol` for details) +* instances of such classes whose :attr:`~object.__dict__` or the result of + calling :meth:`__getstate__` is picklable (see section :ref:`pickle-protocol` + for details). Attempts to pickle unpicklable objects will raise the :exc:`PicklingError` exception; when this happens, an unspecified number of bytes may have already @@ -364,8 +365,8 @@ raised in this case. You can carefully raise this limit with Note that functions (built-in and user-defined) are pickled by "fully qualified" name reference, not by value. This means that only the function name is -pickled, along with the name of the module the function is defined in. Neither the -function's code, nor any of its function attributes are pickled. Thus the +pickled, along with the name of the module the function is defined in. Neither +the function's code, nor any of its function attributes are pickled. Thus the defining module must be importable in the unpickling environment, and the module must contain the named object, otherwise an exception will be raised. [#]_ @@ -442,7 +443,7 @@ Pickling and unpickling normal class instances defines the method :meth:`__getstate__`, it is called and the return state is pickled as the contents for the instance, instead of the contents of the instance's dictionary. If there is no :meth:`__getstate__` method, the - instance's :attr:`__dict__` is pickled. + instance's :attr:`~object.__dict__` is pickled. .. method:: object.__setstate__(state) @@ -510,7 +511,8 @@ Pickling and unpickling extension types * Optionally, the object's state, which will be passed to the object's :meth:`__setstate__` method as described in section :ref:`pickle-inst`. If the object has no :meth:`__setstate__` method, then, as above, the value - must be a dictionary and it will be added to the object's :attr:`__dict__`. + must be a dictionary and it will be added to the object's + :attr:`~object.__dict__`. * Optionally, an iterator (and not a sequence) yielding successive list items. These list items will be pickled, and appended to the object using @@ -568,19 +570,20 @@ the :mod:`pickle` module; it will delegate this resolution to user defined functions on the pickler and unpickler. [#]_ To define external persistent id resolution, you need to set the -:attr:`persistent_id` attribute of the pickler object and the -:attr:`persistent_load` attribute of the unpickler object. +:attr:`~Pickler.persistent_id` attribute of the pickler object and the +:attr:`~Unpickler.persistent_load` attribute of the unpickler object. To pickle objects that have an external persistent id, the pickler must have a -custom :func:`persistent_id` method that takes an object as an argument and -returns either ``None`` or the persistent id for that object. When ``None`` is -returned, the pickler simply pickles the object as normal. When a persistent id -string is returned, the pickler will pickle that string, along with a marker so -that the unpickler will recognize the string as a persistent id. +custom :func:`~Pickler.persistent_id` method that takes an object as an +argument and returns either ``None`` or the persistent id for that object. +When ``None`` is returned, the pickler simply pickles the object as normal. +When a persistent id string is returned, the pickler will pickle that string, +along with a marker so that the unpickler will recognize the string as a +persistent id. To unpickle external objects, the unpickler must have a custom -:func:`persistent_load` function that takes a persistent id string and returns -the referenced object. +:func:`~Unpickler.persistent_load` function that takes a persistent id string +and returns the referenced object. Here's a silly example that *might* shed more light:: @@ -630,13 +633,14 @@ Here's a silly example that *might* shed more light:: j = up.load() print j -In the :mod:`cPickle` module, the unpickler's :attr:`persistent_load` attribute -can also be set to a Python list, in which case, when the unpickler reaches a -persistent id, the persistent id string will simply be appended to this list. -This functionality exists so that a pickle data stream can be "sniffed" for -object references without actually instantiating all the objects in a pickle. -[#]_ Setting :attr:`persistent_load` to a list is usually used in conjunction -with the :meth:`noload` method on the Unpickler. +In the :mod:`cPickle` module, the unpickler's :attr:`~Unpickler.persistent_load` +attribute can also be set to a Python list, in which case, when the unpickler +reaches a persistent id, the persistent id string will simply be appended to +this list. This functionality exists so that a pickle data stream can be +"sniffed" for object references without actually instantiating all the objects +in a pickle. +[#]_ Setting :attr:`~Unpickler.persistent_load` to a list is usually used in +conjunction with the :meth:`~Unpickler.noload` method on the Unpickler. .. BAW: Both pickle and cPickle support something called inst_persistent_id() which appears to give unknown types a second shot at producing a persistent @@ -674,13 +678,13 @@ want to disallow all unpickling of instances. If this sounds like a hack, you're right. Refer to the source code to make this work. Things are a little cleaner with :mod:`cPickle`, but not by much. To control -what gets unpickled, you can set the unpickler's :attr:`find_global` attribute -to a function or ``None``. If it is ``None`` then any attempts to unpickle -instances will raise an :exc:`UnpicklingError`. If it is a function, then it -should accept a module name and a class name, and return the corresponding class -object. It is responsible for looking up the class and performing any necessary -imports, and it may raise an error to prevent instances of the class from being -unpickled. +what gets unpickled, you can set the unpickler's :attr:`~Unpickler.find_global` +attribute to a function or ``None``. If it is ``None`` then any attempts to +unpickle instances will raise an :exc:`UnpicklingError`. If it is a function, +then it should accept a module name and a class name, and return the +corresponding class object. It is responsible for looking up the class and +performing any necessary imports, and it may raise an error to prevent +instances of the class from being unpickled. The moral of the story is that you should be really careful about the source of the strings your application unpickles. @@ -731,7 +735,7 @@ can't be sure if the ASCII or binary format was used. :: Here's a larger example that shows how to modify pickling behavior for a class. The :class:`TextReader` class opens a text file, and returns the line number and -line contents each time its :meth:`readline` method is called. If a +line contents each time its :meth:`!readline` method is called. If a :class:`TextReader` instance is pickled, all attributes *except* the file object member are saved. When the instance is unpickled, the file is reopened, and reading resumes from the last location. The :meth:`__setstate__` and diff --git a/Doc/library/pickletools.rst b/Doc/library/pickletools.rst index ce47c97..ebb30ab 100644 --- a/Doc/library/pickletools.rst +++ b/Doc/library/pickletools.rst @@ -20,7 +20,7 @@ useful for Python core developers who are working on the :mod:`pickle` and probably won't find the :mod:`pickletools` module relevant. -.. function:: dis(pickle[, out=None, memo=None, indentlevel=4]) +.. function:: dis(pickle, out=None, memo=None, indentlevel=4) Outputs a symbolic disassembly of the pickle to the file-like object *out*, defaulting to ``sys.stdout``. *pickle* can be a string or a file-like object. diff --git a/Doc/library/pipes.rst b/Doc/library/pipes.rst index 016a720..415d5c7 100644 --- a/Doc/library/pipes.rst +++ b/Doc/library/pipes.rst @@ -16,8 +16,6 @@ The :mod:`pipes` module defines a class to abstract the concept of a *pipeline* Because the module uses :program:`/bin/sh` command lines, a POSIX or compatible shell for :func:`os.system` and :func:`os.popen` is required. -The :mod:`pipes` module defines the following class: - .. class:: Template() @@ -26,15 +24,52 @@ The :mod:`pipes` module defines the following class: Example:: >>> import pipes - >>> t=pipes.Template() + >>> t = pipes.Template() >>> t.append('tr a-z A-Z', '--') - >>> f=t.open('/tmp/1', 'w') + >>> f = t.open('pipefile', 'w') >>> f.write('hello world') >>> f.close() - >>> open('/tmp/1').read() + >>> open('pipefile').read() 'HELLO WORLD' +.. function:: quote(s) + + .. deprecated:: 2.7 + Prior to Python 2.7, this function was not publicly documented. It is + finally exposed publicly in Python 3.3 as the + :func:`quote <shlex.quote>` function in the :mod:`shlex` module. + + Return a shell-escaped version of the string *s*. The returned value is a + string that can safely be used as one token in a shell command line, for + cases where you cannot use a list. + + This idiom would be unsafe:: + + >>> filename = 'somefile; rm -rf ~' + >>> command = 'ls -l {}'.format(filename) + >>> print command # executed by a shell: boom! + ls -l somefile; rm -rf ~ + + :func:`quote` lets you plug the security hole:: + + >>> command = 'ls -l {}'.format(quote(filename)) + >>> print command + ls -l 'somefile; rm -rf ~' + >>> remote_command = 'ssh home {}'.format(quote(command)) + >>> print remote_command + ssh home 'ls -l '"'"'somefile; rm -rf ~'"'"'' + + The quoting is compatible with UNIX shells and with :func:`shlex.split`: + + >>> remote_command = shlex.split(remote_command) + >>> remote_command + ['ssh', 'home', "ls -l 'somefile; rm -rf ~'"] + >>> command = shlex.split(remote_command[-1]) + >>> command + ['ls', '-l', 'somefile; rm -rf ~'] + + .. _template-objects: Template Objects diff --git a/Doc/library/platform.rst b/Doc/library/platform.rst index 26f587e..cb7144a 100644 --- a/Doc/library/platform.rst +++ b/Doc/library/platform.rst @@ -197,8 +197,8 @@ Windows Platform .. function:: win32_ver(release='', version='', csd='', ptype='') Get additional version information from the Windows Registry and return a tuple - ``(version, csd, ptype)`` referring to version number, CSD level - (service pack) and OS type (multi/single processor). + ``(release, version, csd, ptype)`` referring to OS release, version number, + CSD level (service pack) and OS type (multi/single processor). As a hint: *ptype* is ``'Uniprocessor Free'`` on single processor NT machines and ``'Multiprocessor Free'`` on multi processor machines. The *'Free'* refers diff --git a/Doc/library/plistlib.rst b/Doc/library/plistlib.rst index 11268c2..c6930c6 100644 --- a/Doc/library/plistlib.rst +++ b/Doc/library/plistlib.rst @@ -74,7 +74,7 @@ This module defines the following functions: -.. function:: readPlistFromResource(path[, restype='plst'[, resid=0]]) +.. function:: readPlistFromResource(path, restype='plst', resid=0) Read a plist from the resource with type *restype* from the resource fork of *path*. Availability: Mac OS X. @@ -84,7 +84,7 @@ This module defines the following functions: In Python 3.x, this function has been removed. -.. function:: writePlistToResource(rootObject, path[, restype='plst'[, resid=0]]) +.. function:: writePlistToResource(rootObject, path, restype='plst', resid=0) Write *rootObject* as a resource with type *restype* to the resource fork of *path*. Availability: Mac OS X. diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index 8456304..07c243f 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -24,7 +24,7 @@ quality of POP3 servers varies widely, and too many are quite poor. If your mailserver supports IMAP, you would be better off using the :class:`imaplib.IMAP4` class, as IMAP servers tend to be better implemented. -A single class is provided by the :mod:`poplib` module: +The :mod:`poplib` module provides two classes: .. class:: POP3(host[, port[, timeout]]) @@ -102,7 +102,7 @@ An :class:`POP3` instance has the following methods: .. method:: POP3.pass_(password) Send password, response includes message count and mailbox size. Note: the - mailbox on the server is locked until :meth:`quit` is called. + mailbox on the server is locked until :meth:`~poplib.quit` is called. .. method:: POP3.apop(user, secret) diff --git a/Doc/library/posix.rst b/Doc/library/posix.rst index 8b5cac0..d28ed2d 100644 --- a/Doc/library/posix.rst +++ b/Doc/library/posix.rst @@ -19,7 +19,7 @@ systems the :mod:`posix` module is not available, but a subset is always available through the :mod:`os` interface. Once :mod:`os` is imported, there is *no* performance penalty in using it instead of :mod:`posix`. In addition, :mod:`os` provides some additional functionality, such as automatically calling -:func:`putenv` when an entry in ``os.environ`` is changed. +:func:`~os.putenv` when an entry in ``os.environ`` is changed. Errors are reported as exceptions; the usual exceptions are given for type errors, while errors reported by the system calls raise :exc:`OSError`. @@ -74,9 +74,10 @@ In addition to many functions described in the :mod:`os` module documentation, directory, equivalent to ``getenv("HOME")`` in C. Modifying this dictionary does not affect the string environment passed on by - :func:`execv`, :func:`popen` or :func:`system`; if you need to change the - environment, pass ``environ`` to :func:`execve` or add variable assignments and - export statements to the command string for :func:`system` or :func:`popen`. + :func:`~os.execv`, :func:`~os.popen` or :func:`~os.system`; if you need to + change the environment, pass ``environ`` to :func:`~os.execve` or add + variable assignments and export statements to the command string for + :func:`~os.system` or :func:`~os.popen`. .. note:: diff --git a/Doc/library/posixfile.rst b/Doc/library/posixfile.rst index c27e412..97ef800 100644 --- a/Doc/library/posixfile.rst +++ b/Doc/library/posixfile.rst @@ -181,7 +181,7 @@ Examples:: import posixfile - file = posixfile.open('/tmp/test', 'w') + file = posixfile.open('testfile', 'w') file.lock('w|') ... file.lock('u') diff --git a/Doc/library/pprint.rst b/Doc/library/pprint.rst index a0a7200..8e7baf8 100644 --- a/Doc/library/pprint.rst +++ b/Doc/library/pprint.rst @@ -36,7 +36,7 @@ The :mod:`pprint` module defines one class: .. First the implementation class: -.. class:: PrettyPrinter(...) +.. class:: PrettyPrinter(indent=1, width=80, depth=None, stream=None) Construct a :class:`PrettyPrinter` instance. This constructor understands several keyword parameters. An output stream may be set using the *stream* @@ -73,9 +73,7 @@ The :mod:`pprint` module defines one class: The :class:`PrettyPrinter` class supports several derivative functions: -.. Now the derivative functions: - -.. function:: pformat(object[, indent[, width[, depth]]]) +.. function:: pformat(object, indent=1, width=80, depth=None) Return the formatted representation of *object* as a string. *indent*, *width* and *depth* will be passed to the :class:`PrettyPrinter` constructor as @@ -85,10 +83,10 @@ The :class:`PrettyPrinter` class supports several derivative functions: The parameters *indent*, *width* and *depth* were added. -.. function:: pprint(object[, stream[, indent[, width[, depth]]]]) +.. function:: pprint(object, stream=None, indent=1, width=80, depth=None) Prints the formatted representation of *object* on *stream*, followed by a - newline. If *stream* is omitted, ``sys.stdout`` is used. This may be used in + newline. If *stream* is ``None``, ``sys.stdout`` is used. This may be used in the interactive interpreter instead of a :keyword:`print` statement for inspecting values. *indent*, *width* and *depth* will be passed to the :class:`PrettyPrinter` constructor as formatting parameters. @@ -206,7 +204,8 @@ are converted to strings. The default implementation uses the internals of the pprint Example -------------- -This example demonstrates several uses of the :func:`pprint` function and its parameters. +This example demonstrates several uses of the :func:`pprint` function and its +parameters. >>> import pprint >>> tup = ('spam', ('eggs', ('lumberjack', ('knights', ('ni', ('dead', diff --git a/Doc/library/profile.rst b/Doc/library/profile.rst index 236324d..0fb1489 100644 --- a/Doc/library/profile.rst +++ b/Doc/library/profile.rst @@ -4,11 +4,6 @@ The Python Profilers ******************** -.. sectionauthor:: James Roskind - -.. module:: profile - :synopsis: Python source profiler. - **Source code:** :source:`Lib/profile.py` and :source:`Lib/pstats.py` -------------- @@ -22,33 +17,31 @@ Introduction to the profilers single: deterministic profiling single: profiling, deterministic -A :dfn:`profiler` is a program that describes the run time performance -of a program, providing a variety of statistics. This documentation -describes the profiler functionality provided in the modules -:mod:`cProfile`, :mod:`profile` and :mod:`pstats`. This profiler -provides :dfn:`deterministic profiling` of Python programs. It also -provides a series of report generation tools to allow users to rapidly -examine the results of a profile operation. +:mod:`cProfile` and :mod:`profile` provide :dfn:`deterministic profiling` of +Python programs. A :dfn:`profile` is a set of statistics that describes how +often and for how long various parts of the program executed. These statistics +can be formatted into reports via the :mod:`pstats` module. -The Python standard library provides three different profilers: +The Python standard library provides three different implementations of the same +profiling interface: -#. :mod:`cProfile` is recommended for most users; it's a C extension - with reasonable overhead - that makes it suitable for profiling long-running programs. - Based on :mod:`lsprof`, - contributed by Brett Rosen and Ted Czotter. +1. :mod:`cProfile` is recommended for most users; it's a C extension with + reasonable overhead that makes it suitable for profiling long-running + programs. Based on :mod:`lsprof`, contributed by Brett Rosen and Ted + Czotter. .. versionadded:: 2.5 -#. :mod:`profile`, a pure Python module whose interface is imitated by - :mod:`cProfile`. Adds significant overhead to profiled programs. - If you're trying to extend - the profiler in some way, the task might be easier with this module. +2. :mod:`profile`, a pure Python module whose interface is imitated by + :mod:`cProfile`, but which adds significant overhead to profiled programs. + If you're trying to extend the profiler in some way, the task might be easier + with this module. .. versionchanged:: 2.4 - Now also reports the time spent in calls to built-in functions and methods. + Now also reports the time spent in calls to built-in functions + and methods. -#. :mod:`hotshot` was an experimental C module that focused on minimizing +3. :mod:`hotshot` was an experimental C module that focused on minimizing the overhead of profiling, at the expense of longer data post-processing times. It is no longer maintained and may be dropped in a future version of Python. @@ -65,6 +58,15 @@ is newer and might not be available on all systems. :mod:`_lsprof` module. The :mod:`hotshot` module is reserved for specialized usage. +.. note:: + + The profiler modules are designed to provide an execution profile for a given + program, not for benchmarking purposes (for that, there is :mod:`timeit` for + reasonably accurate results). This particularly applies to benchmarking + Python code against C code: the profilers introduce overhead for Python code, + but not for C-level functions, and so the C code would seem faster than any + Python one. + .. _profile-instant: @@ -75,57 +77,94 @@ This section is provided for users that "don't want to read the manual." It provides a very brief overview, and allows a user to rapidly perform profiling on an existing application. -To profile an application with a main entry point of :func:`foo`, you would add -the following to your module:: +To profile a function that takes a single argument, you can do:: import cProfile - cProfile.run('foo()') + import re + cProfile.run('re.compile("foo|bar")') (Use :mod:`profile` instead of :mod:`cProfile` if the latter is not available on your system.) -The above action would cause :func:`foo` to be run, and a series of informative -lines (the profile) to be printed. The above approach is most useful when -working with the interpreter. If you would like to save the results of a -profile into a file for later examination, you can supply a file name as the -second argument to the :func:`run` function:: +The above action would run :func:`re.compile` and print profile results like +the following:: - import cProfile - cProfile.run('foo()', 'fooprof') + 197 function calls (192 primitive calls) in 0.002 seconds -The file :file:`cProfile.py` can also be invoked as a script to profile another -script. For example:: + Ordered by: standard name - python -m cProfile myscript.py + ncalls tottime percall cumtime percall filename:lineno(function) + 1 0.000 0.000 0.001 0.001 <string>:1(<module>) + 1 0.000 0.000 0.001 0.001 re.py:212(compile) + 1 0.000 0.000 0.001 0.001 re.py:268(_compile) + 1 0.000 0.000 0.000 0.000 sre_compile.py:172(_compile_charset) + 1 0.000 0.000 0.000 0.000 sre_compile.py:201(_optimize_charset) + 4 0.000 0.000 0.000 0.000 sre_compile.py:25(_identityfunction) + 3/1 0.000 0.000 0.000 0.000 sre_compile.py:33(_compile) -:file:`cProfile.py` accepts two optional arguments on the command line:: +The first line indicates that 197 calls were monitored. Of those calls, 192 +were :dfn:`primitive`, meaning that the call was not induced via recursion. The +next line: ``Ordered by: standard name``, indicates that the text string in the +far right column was used to sort the output. The column headings include: - cProfile.py [-o output_file] [-s sort_order] +ncalls + for the number of calls, -``-s`` only applies to standard output (``-o`` is not supplied). -Look in the :class:`Stats` documentation for valid sort values. +tottime + for the total time spent in the given function (and excluding time made in + calls to sub-functions) -When you wish to review the profile, you should use the methods in the -:mod:`pstats` module. Typically you would load the statistics data as follows:: +percall + is the quotient of ``tottime`` divided by ``ncalls`` - import pstats - p = pstats.Stats('fooprof') +cumtime + is the cumulative time spent in this and all subfunctions (from invocation + till exit). This figure is accurate *even* for recursive functions. -The class :class:`Stats` (the above code just created an instance of this class) -has a variety of methods for manipulating and printing the data that was just -read into ``p``. When you ran :func:`cProfile.run` above, what was printed was -the result of three method calls:: +percall + is the quotient of ``cumtime`` divided by primitive calls - p.strip_dirs().sort_stats(-1).print_stats() +filename:lineno(function) + provides the respective data of each function + +When there are two numbers in the first column (for example ``3/1``), it means +that the function recursed. The second value is the number of primitive calls +and the former is the total number of calls. Note that when the function does +not recurse, these two values are the same, and only the single figure is +printed. -The first method removed the extraneous path from all the module names. The -second method sorted all the entries according to the standard module/line/name -string that is printed. The third method printed out all the statistics. You -might try the following sort calls: +Instead of printing the output at the end of the profile run, you can save the +results to a file by specifying a filename to the :func:`run` function:: -.. (this is to comply with the semantics of the old profiler). + import cProfile + import re + cProfile.run('re.compile("foo|bar")', 'restats') + +The :class:`pstats.Stats` class reads profile results from a file and formats +them in various ways. + +The file :mod:`cProfile` can also be invoked as a script to profile another +script. For example:: -:: + python -m cProfile [-o output_file] [-s sort_order] myscript.py + +``-o`` writes the profile results to a file instead of to stdout + +``-s`` specifies one of the :func:`~pstats.Stats.sort_stats` sort values to sort +the output by. This only applies when ``-o`` is not supplied. + +The :mod:`pstats` module's :class:`~pstats.Stats` class has a variety of methods +for manipulating and printing the data saved into a profile results file:: + + import pstats + p = pstats.Stats('restats') + p.strip_dirs().sort_stats(-1).print_stats() + +The :meth:`~pstats.Stats.strip_dirs` method removed the extraneous path from all +the module names. The :meth:`~pstats.Stats.sort_stats` method sorted all the +entries according to the standard module/line/name string that is printed. The +:meth:`~pstats.Stats.print_stats` method printed out all the statistics. You +might try the following sort calls:: p.sort_stats('name') p.print_stats() @@ -174,315 +213,340 @@ If you want more functionality, you're going to have to read the manual, or guess what the following functions do:: p.print_callees() - p.add('fooprof') + p.add('restats') Invoked as a script, the :mod:`pstats` module is a statistics browser for reading and examining profile dumps. It has a simple line-oriented interface (implemented using :mod:`cmd`) and interactive help. +:mod:`profile` and :mod:`cProfile` Module Reference +======================================================= -.. _deterministic-profiling: +.. module:: cProfile +.. module:: profile + :synopsis: Python source profiler. -What Is Deterministic Profiling? -================================ +Both the :mod:`profile` and :mod:`cProfile` modules provide the following +functions: -:dfn:`Deterministic profiling` is meant to reflect the fact that all *function -call*, *function return*, and *exception* events are monitored, and precise -timings are made for the intervals between these events (during which time the -user's code is executing). In contrast, :dfn:`statistical profiling` (which is -not done by this module) randomly samples the effective instruction pointer, and -deduces where time is being spent. The latter technique traditionally involves -less overhead (as the code does not need to be instrumented), but provides only -relative indications of where time is being spent. +.. function:: run(command, filename=None, sort=-1) -In Python, since there is an interpreter active during execution, the presence -of instrumented code is not required to do deterministic profiling. Python -automatically provides a :dfn:`hook` (optional callback) for each event. In -addition, the interpreted nature of Python tends to add so much overhead to -execution, that deterministic profiling tends to only add small processing -overhead in typical applications. The result is that deterministic profiling is -not that expensive, yet provides extensive run time statistics about the -execution of a Python program. + This function takes a single argument that can be passed to the :func:`exec` + function, and an optional file name. In all cases this routine executes:: -Call count statistics can be used to identify bugs in code (surprising counts), -and to identify possible inline-expansion points (high call counts). Internal -time statistics can be used to identify "hot loops" that should be carefully -optimized. Cumulative time statistics should be used to identify high level -errors in the selection of algorithms. Note that the unusual handling of -cumulative times in this profiler allows statistics for recursive -implementations of algorithms to be directly compared to iterative -implementations. + exec(command, __main__.__dict__, __main__.__dict__) + and gathers profiling statistics from the execution. If no file name is + present, then this function automatically creates a :class:`~pstats.Stats` + instance and prints a simple profiling report. If the sort value is specified + it is passed to this :class:`~pstats.Stats` instance to control how the + results are sorted. -Reference Manual -- :mod:`profile` and :mod:`cProfile` -====================================================== +.. function:: runctx(command, globals, locals, filename=None) -.. module:: cProfile - :synopsis: Python profiler + This function is similar to :func:`run`, with added arguments to supply the + globals and locals dictionaries for the *command* string. This routine + executes:: + exec(command, globals, locals) -The primary entry point for the profiler is the global function -:func:`profile.run` (resp. :func:`cProfile.run`). It is typically used to create -any profile information. The reports are formatted and printed using methods of -the class :class:`pstats.Stats`. The following is a description of all of these -standard entry points and functions. For a more in-depth view of some of the -code, consider reading the later section on Profiler Extensions, which includes -discussion of how to derive "better" profilers from the classes presented, or -reading the source code for these modules. + and gathers profiling statistics as in the :func:`run` function above. +.. class:: Profile(timer=None, timeunit=0.0, subcalls=True, builtins=True) -.. function:: run(command[, filename]) + This class is normally only used if more precise control over profiling is + needed than what the :func:`cProfile.run` function provides. - This function takes a single argument that can be passed to the - :keyword:`exec` statement, and an optional file name. In all cases this - routine attempts to :keyword:`exec` its first argument, and gather profiling - statistics from the execution. If no file name is present, then this function - automatically prints a simple profiling report, sorted by the standard name - string (file/line/function-name) that is presented in each line. The - following is a typical output from such a call:: + A custom timer can be supplied for measuring how long code takes to run via + the *timer* argument. This must be a function that returns a single number + representing the current time. If the number is an integer, the *timeunit* + specifies a multiplier that specifies the duration of each unit of time. For + example, if the timer returns times measured in thousands of seconds, the + time unit would be ``.001``. - 2706 function calls (2004 primitive calls) in 4.504 CPU seconds + Directly using the :class:`Profile` class allows formatting profile results + without writing the profile data to a file:: - Ordered by: standard name + import cProfile, pstats, StringIO + pr = cProfile.Profile() + pr.enable() + # ... do something ... + pr.disable() + s = StringIO.StringIO() + sortby = 'cumulative' + ps = pstats.Stats(pr, stream=s).sort_stats(sortby) + ps.print_stats() + print s.getvalue() - ncalls tottime percall cumtime percall filename:lineno(function) - 2 0.006 0.003 0.953 0.477 pobject.py:75(save_objects) - 43/3 0.533 0.012 0.749 0.250 pobject.py:99(evaluate) - ... + .. method:: enable() - The first line indicates that 2706 calls were monitored. Of those calls, 2004 - were :dfn:`primitive`. We define :dfn:`primitive` to mean that the call was not - induced via recursion. The next line: ``Ordered by: standard name``, indicates - that the text string in the far right column was used to sort the output. The - column headings include: + Start collecting profiling data. - ncalls - for the number of calls, + .. method:: disable() - tottime - for the total time spent in the given function (and excluding time made in calls - to sub-functions), + Stop collecting profiling data. - percall - is the quotient of ``tottime`` divided by ``ncalls`` + .. method:: create_stats() - cumtime - is the total time spent in this and all subfunctions (from invocation till - exit). This figure is accurate *even* for recursive functions. + Stop collecting profiling data and record the results internally + as the current profile. - percall - is the quotient of ``cumtime`` divided by primitive calls + .. method:: print_stats(sort=-1) - filename:lineno(function) - provides the respective data of each function + Create a :class:`~pstats.Stats` object based on the current + profile and print the results to stdout. - When there are two numbers in the first column (for example, ``43/3``), then the - latter is the number of primitive calls, and the former is the actual number of - calls. Note that when the function does not recurse, these two values are the - same, and only the single figure is printed. + .. method:: dump_stats(filename) + Write the results of the current profile to *filename*. -.. function:: runctx(command, globals, locals[, filename]) + .. method:: run(cmd) - This function is similar to :func:`run`, with added arguments to supply the - globals and locals dictionaries for the *command* string. + Profile the cmd via :func:`exec`. -Analysis of the profiler data is done using the :class:`Stats` class. + .. method:: runctx(cmd, globals, locals) -.. note:: + Profile the cmd via :func:`exec` with the specified global and + local environment. + + .. method:: runcall(func, *args, **kwargs) - The :class:`Stats` class is defined in the :mod:`pstats` module. + Profile ``func(*args, **kwargs)`` +.. _profile-stats: + +The :class:`Stats` Class +======================== + +Analysis of the profiler data is done using the :class:`~pstats.Stats` class. .. module:: pstats :synopsis: Statistics object for use with the profiler. - -.. class:: Stats(filename[, stream=sys.stdout[, ...]]) +.. class:: Stats(*filenames or profile, stream=sys.stdout) This class constructor creates an instance of a "statistics object" from a - *filename* (or set of filenames). :class:`Stats` objects are manipulated by - methods, in order to print useful reports. You may specify an alternate output - stream by giving the keyword argument, ``stream``. + *filename* (or list of filenames) or from a :class:`Profile` instance. Output + will be printed to the stream specified by *stream*. The file selected by the above constructor must have been created by the corresponding version of :mod:`profile` or :mod:`cProfile`. To be specific, there is *no* file compatibility guaranteed with future versions of this - profiler, and there is no compatibility with files produced by other profilers. - If several files are provided, all the statistics for identical functions will - be coalesced, so that an overall view of several processes can be considered in - a single report. If additional files need to be combined with data in an - existing :class:`Stats` object, the :meth:`add` method can be used. - - .. (such as the old system profiler). - - .. versionchanged:: 2.5 - The *stream* parameter was added. - + profiler, and there is no compatibility with files produced by other + profilers. If several files are provided, all the statistics for identical + functions will be coalesced, so that an overall view of several processes can + be considered in a single report. If additional files need to be combined + with data in an existing :class:`~pstats.Stats` object, the + :meth:`~pstats.Stats.add` method can be used. -.. _profile-stats: + Instead of reading the profile data from a file, a :class:`cProfile.Profile` + or :class:`profile.Profile` object can be used as the profile data source. -The :class:`Stats` Class ------------------------- + :class:`Stats` objects have the following methods: -:class:`Stats` objects have the following methods: + .. method:: strip_dirs() + This method for the :class:`Stats` class removes all leading path + information from file names. It is very useful in reducing the size of + the printout to fit within (close to) 80 columns. This method modifies + the object, and the stripped information is lost. After performing a + strip operation, the object is considered to have its entries in a + "random" order, as it was just after object initialization and loading. + If :meth:`~pstats.Stats.strip_dirs` causes two function names to be + indistinguishable (they are on the same line of the same filename, and + have the same function name), then the statistics for these two entries + are accumulated into a single entry. -.. method:: Stats.strip_dirs() - This method for the :class:`Stats` class removes all leading path information - from file names. It is very useful in reducing the size of the printout to fit - within (close to) 80 columns. This method modifies the object, and the stripped - information is lost. After performing a strip operation, the object is - considered to have its entries in a "random" order, as it was just after object - initialization and loading. If :meth:`strip_dirs` causes two function names to - be indistinguishable (they are on the same line of the same filename, and have - the same function name), then the statistics for these two entries are - accumulated into a single entry. + .. method:: add(*filenames) + This method of the :class:`Stats` class accumulates additional profiling + information into the current profiling object. Its arguments should refer + to filenames created by the corresponding version of :func:`profile.run` + or :func:`cProfile.run`. Statistics for identically named (re: file, line, + name) functions are automatically accumulated into single function + statistics. -.. method:: Stats.add(filename[, ...]) - This method of the :class:`Stats` class accumulates additional profiling - information into the current profiling object. Its arguments should refer to - filenames created by the corresponding version of :func:`profile.run` or - :func:`cProfile.run`. Statistics for identically named (re: file, line, name) - functions are automatically accumulated into single function statistics. + .. method:: dump_stats(filename) - -.. method:: Stats.dump_stats(filename) - - Save the data loaded into the :class:`Stats` object to a file named *filename*. - The file is created if it does not exist, and is overwritten if it already - exists. This is equivalent to the method of the same name on the - :class:`profile.Profile` and :class:`cProfile.Profile` classes. + Save the data loaded into the :class:`Stats` object to a file named + *filename*. The file is created if it does not exist, and is overwritten + if it already exists. This is equivalent to the method of the same name + on the :class:`profile.Profile` and :class:`cProfile.Profile` classes. .. versionadded:: 2.3 -.. method:: Stats.sort_stats(key[, ...]) - - This method modifies the :class:`Stats` object by sorting it according to the - supplied criteria. The argument is typically a string identifying the basis of - a sort (example: ``'time'`` or ``'name'``). - - When more than one key is provided, then additional keys are used as secondary - criteria when there is equality in all keys selected before them. For example, - ``sort_stats('name', 'file')`` will sort all the entries according to their - function name, and resolve all ties (identical function names) by sorting by - file name. - - Abbreviations can be used for any key names, as long as the abbreviation is - unambiguous. The following are the keys currently defined: - - +------------------+----------------------+ - | Valid Arg | Meaning | - +==================+======================+ - | ``'calls'`` | call count | - +------------------+----------------------+ - | ``'cumulative'`` | cumulative time | - +------------------+----------------------+ - | ``'file'`` | file name | - +------------------+----------------------+ - | ``'module'`` | file name | - +------------------+----------------------+ - | ``'pcalls'`` | primitive call count | - +------------------+----------------------+ - | ``'line'`` | line number | - +------------------+----------------------+ - | ``'name'`` | function name | - +------------------+----------------------+ - | ``'nfl'`` | name/file/line | - +------------------+----------------------+ - | ``'stdname'`` | standard name | - +------------------+----------------------+ - | ``'time'`` | internal time | - +------------------+----------------------+ - - Note that all sorts on statistics are in descending order (placing most time - consuming items first), where as name, file, and line number searches are in - ascending order (alphabetical). The subtle distinction between ``'nfl'`` and - ``'stdname'`` is that the standard name is a sort of the name as printed, which - means that the embedded line numbers get compared in an odd way. For example, - lines 3, 20, and 40 would (if the file names were the same) appear in the string - order 20, 3 and 40. In contrast, ``'nfl'`` does a numeric compare of the line - numbers. In fact, ``sort_stats('nfl')`` is the same as ``sort_stats('name', - 'file', 'line')``. - - For backward-compatibility reasons, the numeric arguments ``-1``, ``0``, ``1``, - and ``2`` are permitted. They are interpreted as ``'stdname'``, ``'calls'``, - ``'time'``, and ``'cumulative'`` respectively. If this old style format - (numeric) is used, only one sort key (the numeric key) will be used, and - additional arguments will be silently ignored. - - .. For compatibility with the old profiler, - + .. method:: sort_stats(*keys) + + This method modifies the :class:`Stats` object by sorting it according to + the supplied criteria. The argument is typically a string identifying the + basis of a sort (example: ``'time'`` or ``'name'``). + + When more than one key is provided, then additional keys are used as + secondary criteria when there is equality in all keys selected before + them. For example, ``sort_stats('name', 'file')`` will sort all the + entries according to their function name, and resolve all ties (identical + function names) by sorting by file name. + + Abbreviations can be used for any key names, as long as the abbreviation + is unambiguous. The following are the keys currently defined: + + +------------------+----------------------+ + | Valid Arg | Meaning | + +==================+======================+ + | ``'calls'`` | call count | + +------------------+----------------------+ + | ``'cumulative'`` | cumulative time | + +------------------+----------------------+ + | ``'cumtime'`` | cumulative time | + +------------------+----------------------+ + | ``'file'`` | file name | + +------------------+----------------------+ + | ``'filename'`` | file name | + +------------------+----------------------+ + | ``'module'`` | file name | + +------------------+----------------------+ + | ``'ncalls'`` | call count | + +------------------+----------------------+ + | ``'pcalls'`` | primitive call count | + +------------------+----------------------+ + | ``'line'`` | line number | + +------------------+----------------------+ + | ``'name'`` | function name | + +------------------+----------------------+ + | ``'nfl'`` | name/file/line | + +------------------+----------------------+ + | ``'stdname'`` | standard name | + +------------------+----------------------+ + | ``'time'`` | internal time | + +------------------+----------------------+ + | ``'tottime'`` | internal time | + +------------------+----------------------+ + + Note that all sorts on statistics are in descending order (placing most + time consuming items first), where as name, file, and line number searches + are in ascending order (alphabetical). The subtle distinction between + ``'nfl'`` and ``'stdname'`` is that the standard name is a sort of the + name as printed, which means that the embedded line numbers get compared + in an odd way. For example, lines 3, 20, and 40 would (if the file names + were the same) appear in the string order 20, 3 and 40. In contrast, + ``'nfl'`` does a numeric compare of the line numbers. In fact, + ``sort_stats('nfl')`` is the same as ``sort_stats('name', 'file', + 'line')``. + + For backward-compatibility reasons, the numeric arguments ``-1``, ``0``, + ``1``, and ``2`` are permitted. They are interpreted as ``'stdname'``, + ``'calls'``, ``'time'``, and ``'cumulative'`` respectively. If this old + style format (numeric) is used, only one sort key (the numeric key) will + be used, and additional arguments will be silently ignored. + + .. For compatibility with the old profiler. + + + .. method:: reverse_order() + + This method for the :class:`Stats` class reverses the ordering of the + basic list within the object. Note that by default ascending vs + descending order is properly selected based on the sort key of choice. + + .. This method is provided primarily for compatibility with the old + profiler. + + + .. method:: print_stats(*restrictions) + + This method for the :class:`Stats` class prints out a report as described + in the :func:`profile.run` definition. + + The order of the printing is based on the last + :meth:`~pstats.Stats.sort_stats` operation done on the object (subject to + caveats in :meth:`~pstats.Stats.add` and + :meth:`~pstats.Stats.strip_dirs`). + + The arguments provided (if any) can be used to limit the list down to the + significant entries. Initially, the list is taken to be the complete set + of profiled functions. Each restriction is either an integer (to select a + count of lines), or a decimal fraction between 0.0 and 1.0 inclusive (to + select a percentage of lines), or a regular expression (to pattern match + the standard name that is printed. If several restrictions are provided, + then they are applied sequentially. For example:: + + print_stats(.1, 'foo:') + + would first limit the printing to first 10% of list, and then only print + functions that were part of filename :file:`.\*foo:`. In contrast, the + command:: + + print_stats('foo:', .1) + + would limit the list to all functions having file names :file:`.\*foo:`, + and then proceed to only print the first 10% of them. + + + .. method:: print_callers(*restrictions) + + This method for the :class:`Stats` class prints a list of all functions + that called each function in the profiled database. The ordering is + identical to that provided by :meth:`~pstats.Stats.print_stats`, and the + definition of the restricting argument is also identical. Each caller is + reported on its own line. The format differs slightly depending on the + profiler that produced the stats: + + * With :mod:`profile`, a number is shown in parentheses after each caller + to show how many times this specific call was made. For convenience, a + second non-parenthesized number repeats the cumulative time spent in the + function at the right. + + * With :mod:`cProfile`, each caller is preceded by three numbers: the + number of times this specific call was made, and the total and + cumulative times spent in the current function while it was invoked by + this specific caller. + + + .. method:: print_callees(*restrictions) -.. method:: Stats.reverse_order() + This method for the :class:`Stats` class prints a list of all function + that were called by the indicated function. Aside from this reversal of + direction of calls (re: called vs was called by), the arguments and + ordering are identical to the :meth:`~pstats.Stats.print_callers` method. - This method for the :class:`Stats` class reverses the ordering of the basic list - within the object. Note that by default ascending vs descending order is - properly selected based on the sort key of choice. - .. This method is provided primarily for compatibility with the old profiler. - - -.. method:: Stats.print_stats([restriction, ...]) - - This method for the :class:`Stats` class prints out a report as described in the - :func:`profile.run` definition. - - The order of the printing is based on the last :meth:`sort_stats` operation done - on the object (subject to caveats in :meth:`add` and :meth:`strip_dirs`). - - The arguments provided (if any) can be used to limit the list down to the - significant entries. Initially, the list is taken to be the complete set of - profiled functions. Each restriction is either an integer (to select a count of - lines), or a decimal fraction between 0.0 and 1.0 inclusive (to select a - percentage of lines), or a regular expression (to pattern match the standard - name that is printed; as of Python 1.5b1, this uses the Perl-style regular - expression syntax defined by the :mod:`re` module). If several restrictions are - provided, then they are applied sequentially. For example:: - - print_stats(.1, 'foo:') - - would first limit the printing to first 10% of list, and then only print - functions that were part of filename :file:`.\*foo:`. In contrast, the - command:: - - print_stats('foo:', .1) - - would limit the list to all functions having file names :file:`.\*foo:`, and - then proceed to only print the first 10% of them. - - -.. method:: Stats.print_callers([restriction, ...]) - - This method for the :class:`Stats` class prints a list of all functions that - called each function in the profiled database. The ordering is identical to - that provided by :meth:`print_stats`, and the definition of the restricting - argument is also identical. Each caller is reported on its own line. The - format differs slightly depending on the profiler that produced the stats: - - * With :mod:`profile`, a number is shown in parentheses after each caller to - show how many times this specific call was made. For convenience, a second - non-parenthesized number repeats the cumulative time spent in the function - at the right. +.. _deterministic-profiling: - * With :mod:`cProfile`, each caller is preceded by three numbers: the number of - times this specific call was made, and the total and cumulative times spent in - the current function while it was invoked by this specific caller. +What Is Deterministic Profiling? +================================ +:dfn:`Deterministic profiling` is meant to reflect the fact that all *function +call*, *function return*, and *exception* events are monitored, and precise +timings are made for the intervals between these events (during which time the +user's code is executing). In contrast, :dfn:`statistical profiling` (which is +not done by this module) randomly samples the effective instruction pointer, and +deduces where time is being spent. The latter technique traditionally involves +less overhead (as the code does not need to be instrumented), but provides only +relative indications of where time is being spent. -.. method:: Stats.print_callees([restriction, ...]) +In Python, since there is an interpreter active during execution, the presence +of instrumented code is not required to do deterministic profiling. Python +automatically provides a :dfn:`hook` (optional callback) for each event. In +addition, the interpreted nature of Python tends to add so much overhead to +execution, that deterministic profiling tends to only add small processing +overhead in typical applications. The result is that deterministic profiling is +not that expensive, yet provides extensive run time statistics about the +execution of a Python program. - This method for the :class:`Stats` class prints a list of all function that were - called by the indicated function. Aside from this reversal of direction of - calls (re: called vs was called by), the arguments and ordering are identical to - the :meth:`print_callers` method. +Call count statistics can be used to identify bugs in code (surprising counts), +and to identify possible inline-expansion points (high call counts). Internal +time statistics can be used to identify "hot loops" that should be carefully +optimized. Cumulative time statistics should be used to identify high level +errors in the selection of algorithms. Note that the unusual handling of +cumulative times in this profiler allows statistics for recursive +implementations of algorithms to be directly compared to iterative +implementations. -.. _profile-limits: +.. _profile-limitations: Limitations =========== @@ -525,7 +589,7 @@ The profiler of the :mod:`profile` module subtracts a constant from each event handling time to compensate for the overhead of calling the time function, and socking away the results. By default, the constant is 0. The following procedure can be used to obtain a better constant for a given platform (see -discussion in section Limitations above). :: +:ref:`profile-limitations`). :: import profile pr = profile.Profile() @@ -535,8 +599,8 @@ discussion in section Limitations above). :: The method executes the number of Python calls given by the argument, directly and again under the profiler, measuring the time for both. It then computes the hidden overhead per profiler event, and returns that as a float. For example, -on an 800 MHz Pentium running Windows 2000, and using Python's time.clock() as -the timer, the magical number is about 12.5e-6. +on a 1.8Ghz Intel Core i5 running Mac OS X, and using Python's time.clock() as +the timer, the magical number is about 4.04e-6. The object of this exercise is to get a fairly consistent result. If your computer is *very* fast, or your timer function has poor resolution, you might @@ -559,61 +623,54 @@ When you have a consistent answer, there are three ways you can use it: [#]_ :: If you have a choice, you are better off choosing a smaller constant, and then your results will "less often" show up as negative in profile statistics. +.. _profile-timers: -.. _profiler-extensions: - -Extensions --- Deriving Better Profilers -======================================== - -The :class:`Profile` class of both modules, :mod:`profile` and :mod:`cProfile`, -were written so that derived classes could be developed to extend the profiler. -The details are not described here, as doing this successfully requires an -expert understanding of how the :class:`Profile` class works internally. Study -the source code of the module carefully if you want to pursue this. +Using a custom timer +==================== -If all you want to do is change how current time is determined (for example, to -force use of wall-clock time or elapsed process time), pass the timing function -you want to the :class:`Profile` class constructor:: +If you want to change how current time is determined (for example, to force use +of wall-clock time or elapsed process time), pass the timing function you want +to the :class:`Profile` class constructor:: - pr = profile.Profile(your_time_func) + pr = profile.Profile(your_time_func) -The resulting profiler will then call :func:`your_time_func`. +The resulting profiler will then call ``your_time_func``. Depending on whether +you are using :class:`profile.Profile` or :class:`cProfile.Profile`, +``your_time_func``'s return value will be interpreted differently: :class:`profile.Profile` - :func:`your_time_func` should return a single number, or a list of numbers whose - sum is the current time (like what :func:`os.times` returns). If the function - returns a single time number, or the list of returned numbers has length 2, then - you will get an especially fast version of the dispatch routine. + ``your_time_func`` should return a single number, or a list of numbers whose + sum is the current time (like what :func:`os.times` returns). If the + function returns a single time number, or the list of returned numbers has + length 2, then you will get an especially fast version of the dispatch + routine. Be warned that you should calibrate the profiler class for the timer function - that you choose. For most machines, a timer that returns a lone integer value - will provide the best results in terms of low overhead during profiling. - (:func:`os.times` is *pretty* bad, as it returns a tuple of floating point - values). If you want to substitute a better timer in the cleanest fashion, - derive a class and hardwire a replacement dispatch method that best handles your - timer call, along with the appropriate calibration constant. + that you choose (see :ref:`profile-calibration`). For most machines, a timer + that returns a lone integer value will provide the best results in terms of + low overhead during profiling. (:func:`os.times` is *pretty* bad, as it + returns a tuple of floating point values). If you want to substitute a + better timer in the cleanest fashion, derive a class and hardwire a + replacement dispatch method that best handles your timer call, along with the + appropriate calibration constant. :class:`cProfile.Profile` - :func:`your_time_func` should return a single number. If it returns plain - integers, you can also invoke the class constructor with a second argument - specifying the real duration of one unit of time. For example, if - :func:`your_integer_time_func` returns times measured in thousands of seconds, + ``your_time_func`` should return a single number. If it returns integers, + you can also invoke the class constructor with a second argument specifying + the real duration of one unit of time. For example, if + ``your_integer_time_func`` returns times measured in thousands of seconds, you would construct the :class:`Profile` instance as follows:: - pr = profile.Profile(your_integer_time_func, 0.001) + pr = cProfile.Profile(your_integer_time_func, 0.001) As the :mod:`cProfile.Profile` class cannot be calibrated, custom timer - functions should be used with care and should be as fast as possible. For the - best results with a custom timer, it might be necessary to hard-code it in the C - source of the internal :mod:`_lsprof` module. + functions should be used with care and should be as fast as possible. For + the best results with a custom timer, it might be necessary to hard-code it + in the C source of the internal :mod:`_lsprof` module. -.. rubric:: Footnotes -.. [#] Updated and converted to LaTeX by Guido van Rossum. Further updated by Armin - Rigo to integrate the documentation for the new :mod:`cProfile` module of Python - 2.5. +.. rubric:: Footnotes -.. [#] Prior to Python 2.2, it was necessary to edit the profiler source code to embed - the bias as a literal number. You still can, but that method is no longer +.. [#] Prior to Python 2.2, it was necessary to edit the profiler source code to + embed the bias as a literal number. You still can, but that method is no longer described, because no longer needed. - diff --git a/Doc/library/pyclbr.rst b/Doc/library/pyclbr.rst index 2f81451..13eaabf 100644 --- a/Doc/library/pyclbr.rst +++ b/Doc/library/pyclbr.rst @@ -19,7 +19,7 @@ not implemented in Python, including all standard and optional extension modules. -.. function:: readmodule(module[, path=None]) +.. function:: readmodule(module, path=None) Read a module and return a dictionary mapping class names to class descriptor objects. The parameter *module* should be the name of a @@ -28,7 +28,7 @@ modules. of ``sys.path``, which is used to locate module source code. -.. function:: readmodule_ex(module[, path=None]) +.. function:: readmodule_ex(module, path=None) Like :func:`readmodule`, but the returned dictionary, in addition to mapping class names to class descriptor objects, also maps top-level diff --git a/Doc/library/pyexpat.rst b/Doc/library/pyexpat.rst index 8299739..20ca3bc 100644 --- a/Doc/library/pyexpat.rst +++ b/Doc/library/pyexpat.rst @@ -14,6 +14,14 @@ directive. Since they are attributes which are set by client code, in-text references to these attributes should be marked using the :member: role. + +.. warning:: + + The :mod:`pyexpat` module is not secure against maliciously + constructed data. If you need to parse untrusted or unauthenticated data see + :ref:`xml-vulnerabilities`. + + .. versionadded:: 2.0 .. index:: single: Expat @@ -95,6 +103,10 @@ The :mod:`xml.parsers.expat` module contains two functions: http://www.python.org/ns/ elem1 elem2 + Due to limitations in the ``Expat`` library used by :mod:`pyexpat`, + the :class:`xmlparser` instance returned can only be used to parse a single + XML document. Call ``ParserCreate`` for each document to provide unique + parser instances. .. seealso:: @@ -114,7 +126,9 @@ XMLParser Objects Parses the contents of the string *data*, calling the appropriate handler functions to process the parsed data. *isfinal* must be true on the final call - to this method. *data* can be the empty string at any time. + to this method; it allows the parsing of a single file in fragments, + not the submission of multiple files. + *data* can be the empty string at any time. .. method:: xmlparser.ParseFile(file) @@ -437,7 +451,7 @@ otherwise stated. .. method:: xmlparser.CommentHandler(data) Called for comments. *data* is the text of the comment, excluding the leading - '``<!-``\ ``-``' and trailing '``-``\ ``->``'. + ``'<!-``\ ``-'`` and trailing ``'-``\ ``->'``. .. method:: xmlparser.StartCdataSectionHandler() @@ -898,5 +912,5 @@ The ``errors`` object has the following attributes: .. [#] The encoding string included in XML output should conform to the appropriate standards. For example, "UTF-8" is valid, but "UTF8" is not. See http://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl - and http://www.iana.org/assignments/character-sets . + and http://www.iana.org/assignments/character-sets\ . diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index 36ff346..b525705 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -5,9 +5,9 @@ :synopsis: A synchronized queue class. .. note:: - The :mod:`Queue` module has been renamed to :mod:`queue` in Python 3.0. The + The :mod:`Queue` module has been renamed to :mod:`queue` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. **Source code:** :source:`Lib/Queue.py` @@ -20,8 +20,8 @@ module implements all the required locking semantics. It depends on the availability of thread support in Python; see the :mod:`threading` module. -Implements three types of queue whose only difference is the order that -the entries are retrieved. In a FIFO queue, the first tasks added are +The module implements three types of queue, which differ only in the order in +which the entries are retrieved. In a FIFO queue, the first tasks added are the first retrieved. In a LIFO queue, the most recently added entry is the first retrieved (operating like a stack). With a priority queue, the entries are kept sorted (using the :mod:`heapq` module) and the @@ -60,13 +60,15 @@ The :mod:`Queue` module defines the following classes and exceptions: .. exception:: Empty - Exception raised when non-blocking :meth:`get` (or :meth:`get_nowait`) is called + Exception raised when non-blocking :meth:`~Queue.get` (or + :meth:`~Queue.get_nowait`) is called on a :class:`Queue` object which is empty. .. exception:: Full - Exception raised when non-blocking :meth:`put` (or :meth:`put_nowait`) is called + Exception raised when non-blocking :meth:`~Queue.put` (or + :meth:`~Queue.put_nowait`) is called on a :class:`Queue` object which is full. .. seealso:: diff --git a/Doc/library/random.rst b/Doc/library/random.rst index de98c04..1bc9989 100644 --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -60,6 +60,13 @@ The :mod:`random` module also provides the :class:`SystemRandom` class which uses the system function :func:`os.urandom` to generate random numbers from sources provided by the operating system. +.. warning:: + + The pseudo-random generators of this module should not be used for + security purposes. Use :func:`os.urandom` or :class:`SystemRandom` if + you require a cryptographically secure pseudo-random number generator. + + Bookkeeping functions: @@ -90,7 +97,7 @@ Bookkeeping functions: *state* should have been obtained from a previous call to :func:`getstate`, and :func:`setstate` restores the internal state of the generator to what it was at - the time :func:`setstate` was called. + the time :func:`getstate` was called. .. versionadded:: 2.1 @@ -124,7 +131,8 @@ Bookkeeping functions: Functions for integers: -.. function:: randrange([start,] stop[, step]) +.. function:: randrange(stop) + randrange(start, stop[, step]) Return a randomly selected element from ``range(start, stop, step)``. This is equivalent to ``choice(range(start, stop, step))``, but doesn't actually build a diff --git a/Doc/library/re.rst b/Doc/library/re.rst index df3f9e5..e5bbd03 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -237,21 +237,32 @@ The special characters are: ``(?P<name>...)`` Similar to regular parentheses, but the substring matched by the group is - accessible within the rest of the regular expression via the symbolic group - name *name*. Group names must be valid Python identifiers, and each group - name must be defined only once within a regular expression. A symbolic group - is also a numbered group, just as if the group were not named. So the group - named ``id`` in the example below can also be referenced as the numbered group - ``1``. - - For example, if the pattern is ``(?P<id>[a-zA-Z_]\w*)``, the group can be - referenced by its name in arguments to methods of match objects, such as - ``m.group('id')`` or ``m.end('id')``, and also by name in the regular - expression itself (using ``(?P=id)``) and replacement text given to - ``.sub()`` (using ``\g<id>``). + accessible via the symbolic group name *name*. Group names must be valid + Python identifiers, and each group name must be defined only once within a + regular expression. A symbolic group is also a numbered group, just as if + the group were not named. + + Named groups can be referenced in three contexts. If the pattern is + ``(?P<quote>['"]).*?(?P=quote)`` (i.e. matching a string quoted with either + single or double quotes): + + +---------------------------------------+----------------------------------+ + | Context of reference to group "quote" | Ways to reference it | + +=======================================+==================================+ + | in the same pattern itself | * ``(?P=quote)`` (as shown) | + | | * ``\1`` | + +---------------------------------------+----------------------------------+ + | when processing match object ``m`` | * ``m.group('quote')`` | + | | * ``m.end('quote')`` (etc.) | + +---------------------------------------+----------------------------------+ + | in a string passed to the ``repl`` | * ``\g<quote>`` | + | argument of ``re.sub()`` | * ``\g<1>`` | + | | * ``\1`` | + +---------------------------------------+----------------------------------+ ``(?P=name)`` - Matches whatever text was matched by the earlier group named *name*. + A backreference to a named group; it matches whatever text was matched by the + earlier group named *name*. ``(?#...)`` A comment; the contents of the parentheses are simply ignored. @@ -273,7 +284,7 @@ The special characters are: lookbehind will back up 3 characters and check if the contained pattern matches. The contained pattern must only match strings of some fixed length, meaning that ``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that - patterns which start with positive lookbehind assertions will never match at the + patterns which start with positive lookbehind assertions will not match at the beginning of the string being searched; you will most likely want to use the :func:`search` function rather than the :func:`match` function: @@ -311,7 +322,7 @@ the second character. For example, ``\$`` matches the character ``'$'``. ``\number`` Matches the contents of the group of the same number. Groups are numbered starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``, - but not ``'the end'`` (note the space after the group). This special sequence + but not ``'thethe'`` (note the space after the group). This special sequence can only be used to match one of the first 99 groups. If the first digit of *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as a group match, but as the character with octal value *number*. Inside the @@ -325,14 +336,20 @@ the second character. For example, ``\$`` matches the character ``'$'``. Matches the empty string, but only at the beginning or end of a word. A word is defined as a sequence of alphanumeric or underscore characters, so the end of a word is indicated by whitespace or a non-alphanumeric, non-underscore character. - Note that ``\b`` is defined as the boundary between ``\w`` and ``\W``, so the - precise set of characters deemed to be alphanumeric depends on the values of the - ``UNICODE`` and ``LOCALE`` flags. Inside a character range, ``\b`` represents - the backspace character, for compatibility with Python's string literals. + Note that formally, ``\b`` is defined as the boundary between a ``\w`` and + a ``\W`` character (or vice versa), or between ``\w`` and the beginning/end + of the string, so the precise set of characters deemed to be alphanumeric + depends on the values of the ``UNICODE`` and ``LOCALE`` flags. + For example, ``r'\bfoo\b'`` matches ``'foo'``, ``'foo.'``, ``'(foo)'``, + ``'bar foo baz'`` but not ``'foobar'`` or ``'foo3'``. + Inside a character range, ``\b`` represents the backspace character, for + compatibility with Python's string literals. ``\B`` Matches the empty string, but only when it is *not* at the beginning or end of a - word. This is just the opposite of ``\b``, so is also subject to the settings + word. This means that ``r'py\B'`` matches ``'python'``, ``'py3'``, ``'py2'``, + but not ``'py'``, ``'py.'``, or ``'py!'``. + ``\B`` is just the opposite of ``\b``, so is also subject to the settings of ``LOCALE`` and ``UNICODE``. ``\d`` @@ -348,20 +365,20 @@ the second character. For example, ``\$`` matches the character ``'$'``. character properties database. ``\s`` - When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches - any whitespace character; this is equivalent to the set ``[ \t\n\r\f\v]``. With - :const:`LOCALE`, it will match this set plus whatever characters are defined as - space for the current locale. If :const:`UNICODE` is set, this will match the - characters ``[ \t\n\r\f\v]`` plus whatever is classified as space in the Unicode - character properties database. + When the :const:`UNICODE` flag is not specified, it matches any whitespace + character, this is equivalent to the set ``[ \t\n\r\f\v]``. The + :const:`LOCALE` flag has no extra effect on matching of the space. + If :const:`UNICODE` is set, this will match the characters ``[ \t\n\r\f\v]`` + plus whatever is classified as space in the Unicode character properties + database. ``\S`` - When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches - any non-whitespace character; this is equivalent to the set ``[^ \t\n\r\f\v]`` - With :const:`LOCALE`, it will match any character not in this set, and not - defined as space in the current locale. If :const:`UNICODE` is set, this will - match anything other than ``[ \t\n\r\f\v]`` and characters marked as space in - the Unicode character properties database. + When the :const:`UNICODE` flags is not specified, matches any non-whitespace + character; this is equivalent to the set ``[^ \t\n\r\f\v]`` The + :const:`LOCALE` flag has no extra effect on non-whitespace match. If + :const:`UNICODE` is set, then any character not marked as space in the + Unicode character properties database is matched. + ``\w`` When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches @@ -376,12 +393,16 @@ the second character. For example, ``\$`` matches the character ``'$'``. any non-alphanumeric character; this is equivalent to the set ``[^a-zA-Z0-9_]``. With :const:`LOCALE`, it will match any character not in the set ``[0-9_]``, and not defined as alphanumeric for the current locale. If :const:`UNICODE` is set, - this will match anything other than ``[0-9_]`` and characters marked as - alphanumeric in the Unicode character properties database. + this will match anything other than ``[0-9_]`` plus characters classied as + not alphanumeric in the Unicode character properties database. ``\Z`` Matches only at the end of the string. +If both :const:`LOCALE` and :const:`UNICODE` flags are included for a +particular sequence, then :const:`LOCALE` flag takes effect first followed by +the :const:`UNICODE`. + Most of the standard escapes supported by Python string literals are also accepted by the regular expression parser:: @@ -389,37 +410,15 @@ accepted by the regular expression parser:: \r \t \v \x \\ +(Note that ``\b`` is used to represent word boundaries, and means "backspace" +only inside character classes.) + Octal escapes are included in a limited form: If the first digit is a 0, or if there are three octal digits, it is considered an octal escape. Otherwise, it is a group reference. As for string literals, octal escapes are always at most three digits in length. -.. _matching-searching: - -Matching vs Searching ---------------------- - -.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> - - -Python offers two different primitive operations based on regular expressions: -**match** checks for a match only at the beginning of the string, while -**search** checks for a match anywhere in the string (this is what Perl does -by default). - -Note that match may differ from search even when using a regular expression -beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in -:const:`MULTILINE` mode also immediately following a newline. The "match" -operation succeeds only if the pattern matches at the start of the string -regardless of mode, or at the starting position given by the optional *pos* -argument regardless of whether a newline precedes it. - - >>> re.match("c", "abcdef") # No match - >>> re.search("c", "abcdef") # Match - <_sre.SRE_Match object at ...> - - .. _contents-of-module-re: Module Contents @@ -434,8 +433,8 @@ form. .. function:: compile(pattern, flags=0) Compile a regular expression pattern into a regular expression object, which - can be used for matching using its :func:`match` and :func:`search` methods, - described below. + can be used for matching using its :func:`~RegexObject.match` and + :func:`~RegexObject.search` methods, described below. The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the following variables, combined using bitwise OR (the @@ -528,7 +527,7 @@ form. .. function:: search(pattern, string, flags=0) - Scan through *string* looking for a location where the regular expression + Scan through *string* looking for the first location where the regular expression *pattern* produces a match, and return a corresponding :class:`MatchObject` instance. Return ``None`` if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the @@ -542,10 +541,11 @@ form. Return ``None`` if the string does not match the pattern; note that this is different from a zero-length match. - .. note:: + Note that even in :const:`MULTILINE` mode, :func:`re.match` will only match + at the beginning of the string and not at the beginning of each line. - If you want to locate a match anywhere in *string*, use :func:`search` - instead. + If you want to locate a match anywhere in *string*, use :func:`search` + instead (see also :ref:`search-vs-match`). .. function:: split(pattern, string, maxsplit=0, flags=0) @@ -654,7 +654,8 @@ form. when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns ``'-a-b-c-'``. - In addition to character escapes and backreferences as described above, + In string-type *repl* arguments, in addition to the character escapes and + backreferences described above, ``\g<name>`` will use the substring matched by the group named ``name``, as defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous @@ -741,16 +742,14 @@ Regular Expression Objects The optional *pos* and *endpos* parameters have the same meaning as for the :meth:`~RegexObject.search` method. - .. note:: - - If you want to locate a match anywhere in *string*, use - :meth:`~RegexObject.search` instead. - >>> pattern = re.compile("o") >>> pattern.match("dog") # No match as "o" is not at the start of "dog". >>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog". <_sre.SRE_Match object at ...> + If you want to locate a match anywhere in *string*, use + :meth:`~RegexObject.search` instead (see also :ref:`search-vs-match`). + .. method:: RegexObject.split(string, maxsplit=0) @@ -783,8 +782,8 @@ Regular Expression Objects .. attribute:: RegexObject.flags - The flags argument used when the RE object was compiled, or ``0`` if no flags - were provided. + The regex matching flags. This is a combination of the flags given to + :func:`.compile` and any ``(?...)`` inline flags in the pattern. .. attribute:: RegexObject.groups @@ -811,9 +810,16 @@ Match Objects .. class:: MatchObject - Match Objects always have a boolean value of :const:`True`, so that you can test - whether e.g. :func:`match` resulted in a match with a simple if statement. They - support the following methods and attributes: + Match objects always have a boolean value of ``True``. + Since :meth:`~regex.match` and :meth:`~regex.search` return ``None`` + when there is no match, you can test whether there was a match with a simple + ``if`` statement:: + + match = re.search(pattern, string) + if match: + process(match) + + Match objects support the following methods and attributes: .. method:: MatchObject.expand(template) @@ -1072,13 +1078,13 @@ expressions. +--------------------------------+---------------------------------------------+ | ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` | +--------------------------------+---------------------------------------------+ -| ``%o`` | ``0[0-7]*`` | +| ``%o`` | ``[-+]?[0-7]+`` | +--------------------------------+---------------------------------------------+ | ``%s`` | ``\S+`` | +--------------------------------+---------------------------------------------+ | ``%u`` | ``\d+`` | +--------------------------------+---------------------------------------------+ -| ``%x``, ``%X`` | ``0[xX][\dA-Fa-f]+`` | +| ``%x``, ``%X`` | ``[-+]?(0[xX])?[\dA-Fa-f]+`` | +--------------------------------+---------------------------------------------+ To extract the filename and numbers from a string like :: @@ -1094,59 +1100,39 @@ The equivalent regular expression would be :: (\S+) - (\d+) errors, (\d+) warnings -Avoiding recursion -^^^^^^^^^^^^^^^^^^ - -If you create regular expressions that require the engine to perform a lot of -recursion, you may encounter a :exc:`RuntimeError` exception with the message -``maximum recursion limit`` exceeded. For example, :: - - >>> s = 'Begin ' + 1000*'a very long string ' + 'end' - >>> re.match('Begin (\w| )*? end', s).end() - Traceback (most recent call last): - File "<stdin>", line 1, in ? - File "/usr/local/lib/python2.5/re.py", line 132, in match - return _compile(pattern, flags).match(string) - RuntimeError: maximum recursion limit exceeded - -You can often restructure your regular expression to avoid recursion. - -Starting with Python 2.3, simple uses of the ``*?`` pattern are special-cased to -avoid recursion. Thus, the above regular expression can avoid recursion by -being recast as ``Begin [a-zA-Z0-9_ ]*?end``. As a further benefit, such -regular expressions will run faster than their recursive equivalents. - +.. _search-vs-match: search() vs. match() ^^^^^^^^^^^^^^^^^^^^ -In a nutshell, :func:`match` only attempts to match a pattern at the beginning -of a string where :func:`search` will match a pattern anywhere in a string. -For example: +.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> - >>> re.match("o", "dog") # No match as "o" is not the first letter of "dog". - >>> re.search("o", "dog") # Match as search() looks everywhere in the string. - <_sre.SRE_Match object at ...> +Python offers two different primitive operations based on regular expressions: +:func:`re.match` checks for a match only at the beginning of the string, while +:func:`re.search` checks for a match anywhere in the string (this is what Perl +does by default). -.. note:: +For example:: - The following applies only to regular expression objects like those created - with ``re.compile("pattern")``, not the primitives ``re.match(pattern, - string)`` or ``re.search(pattern, string)``. + >>> re.match("c", "abcdef") # No match + >>> re.search("c", "abcdef") # Match + <_sre.SRE_Match object at ...> -:func:`match` has an optional second parameter that gives an index in the string -where the search is to start:: +Regular expressions beginning with ``'^'`` can be used with :func:`search` to +restrict the match at the beginning of the string:: - >>> pattern = re.compile("o") - >>> pattern.match("dog") # No match as "o" is not at the start of "dog." + >>> re.match("c", "abcdef") # No match + >>> re.search("^c", "abcdef") # No match + >>> re.search("^a", "abcdef") # Match + <_sre.SRE_Match object at ...> - # Equivalent to the above expression as 0 is the default starting index: - >>> pattern.match("dog", 0) +Note however that in :const:`MULTILINE` mode :func:`match` only matches at the +beginning of the string, whereas using :func:`search` with a regular expression +beginning with ``'^'`` will match at the beginning of each line. - # Match as "o" is the 2nd character of "dog" (index 0 is the first): - >>> pattern.match("dog", 1) + >>> re.match('X', 'A\nB\nX', re.MULTILINE) # No match + >>> re.search('^X', 'A\nB\nX', re.MULTILINE) # Match <_sre.SRE_Match object at ...> - >>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog." Making a Phonebook @@ -1160,7 +1146,7 @@ creates a phonebook. First, here is the input. Normally it may come from a file, here we are using triple-quoted string syntax: - >>> input = """Ross McFluff: 834.345.1254 155 Elm Street + >>> text = """Ross McFluff: 834.345.1254 155 Elm Street ... ... Ronald Heathmore: 892.345.3428 436 Finley Avenue ... Frank Burger: 925.541.7625 662 South Dogwood Way @@ -1174,7 +1160,7 @@ into a list with each nonempty line having its own entry: .. doctest:: :options: +NORMALIZE_WHITESPACE - >>> entries = re.split("\n+", input) + >>> entries = re.split("\n+", text) >>> entries ['Ross McFluff: 834.345.1254 155 Elm Street', 'Ronald Heathmore: 892.345.3428 436 Finley Avenue', diff --git a/Doc/library/repr.rst b/Doc/library/repr.rst index 11e6ae2..b604186 100644 --- a/Doc/library/repr.rst +++ b/Doc/library/repr.rst @@ -6,9 +6,9 @@ .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> .. note:: - The :mod:`repr` module has been renamed to :mod:`reprlib` in Python 3.0. The + The :mod:`repr` module has been renamed to :mod:`reprlib` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. **Source code:** :source:`Lib/repr.py` @@ -24,8 +24,9 @@ This module provides a class, an instance, and a function: .. class:: Repr() Class which provides formatting services useful in implementing functions - similar to the built-in :func:`repr`; size limits for different object types - are added to avoid the generation of representations which are excessively long. + similar to the built-in :ref:`repr() <func-repr>`; size limits for different + object types are added to avoid the generation of representations which are + excessively long. .. data:: aRepr @@ -96,8 +97,8 @@ which format specific object types. .. method:: Repr.repr(obj) - The equivalent to the built-in :func:`repr` that uses the formatting imposed by - the instance. + The equivalent to the built-in :ref:`repr() <func-repr>` that uses the + formatting imposed by the instance. .. method:: Repr.repr1(obj, level) diff --git a/Doc/library/resource.rst b/Doc/library/resource.rst index 834dace..7ca4534 100644 --- a/Doc/library/resource.rst +++ b/Doc/library/resource.rst @@ -42,6 +42,11 @@ which cannot be checked or controlled by the operating system are not defined in this module for those platforms. +.. data:: RLIM_INFINITY + + Constant used to represent the the limit for an unlimited resource. + + .. function:: getrlimit(resource) Returns a tuple ``(soft, hard)`` with the current soft and hard limits of @@ -53,12 +58,20 @@ this module for those platforms. Sets new limits of consumption of *resource*. The *limits* argument must be a tuple ``(soft, hard)`` of two integers describing the new limits. A value of - ``-1`` can be used to specify the maximum possible upper limit. + :data:`~resource.RLIM_INFINITY` can be used to request a limit that is + unlimited. Raises :exc:`ValueError` if an invalid resource is specified, if the new soft - limit exceeds the hard limit, or if a process tries to raise its hard limit - (unless the process has an effective UID of super-user). Can also raise - :exc:`error` if the underlying system call fails. + limit exceeds the hard limit, or if a process tries to raise its hard limit. + Specifying a limit of :data:`~resource.RLIM_INFINITY` when the hard or + system limit for that resource is not unlimited will result in a + :exc:`ValueError`. A process with the effective UID of super-user can + request any valid limit value, including unlimited, but :exc:`ValueError` + will still be raised if the requested limit exceeds the system imposed + limit. + + ``setrlimit`` may also raise :exc:`error` if the underlying system call + fails. These symbols define resources whose consumption can be controlled using the :func:`setrlimit` and :func:`getrlimit` functions described below. The values of diff --git a/Doc/library/rexec.rst b/Doc/library/rexec.rst index 2ce612a..12f6faa 100644 --- a/Doc/library/rexec.rst +++ b/Doc/library/rexec.rst @@ -6,7 +6,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`rexec` module has been removed in Python 3.0. + The :mod:`rexec` module has been removed in Python 3. .. versionchanged:: 2.3 Disabled module. @@ -270,7 +270,7 @@ Let us say that we want a slightly more relaxed policy than the standard if mode in ('r', 'rb'): pass elif mode in ('w', 'wb', 'a', 'ab'): - # check filename : must begin with /tmp/ + # check filename: must begin with /tmp/ if file[:5]!='/tmp/': raise IOError("can't write outside /tmp") elif (string.find(file, '/../') >= 0 or diff --git a/Doc/library/rfc822.rst b/Doc/library/rfc822.rst index 8e563dd..33aa851 100644 --- a/Doc/library/rfc822.rst +++ b/Doc/library/rfc822.rst @@ -10,7 +10,7 @@ .. deprecated:: 2.3 The :mod:`email` package should be used in preference to the :mod:`rfc822` module. This module is present only to maintain backward compatibility, and - has been removed in 3.0. + has been removed in Python 3. This module defines a class, :class:`Message`, which represents an "email message" as defined by the Internet standard :rfc:`2822`. [#]_ Such messages diff --git a/Doc/library/robotparser.rst b/Doc/library/robotparser.rst index ba7e557..d95b629 100644 --- a/Doc/library/robotparser.rst +++ b/Doc/library/robotparser.rst @@ -16,9 +16,9 @@ .. note:: The :mod:`robotparser` module has been renamed :mod:`urllib.robotparser` in - Python 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when converting - your sources to 3.0. + your sources to Python 3. This module provides a single class, :class:`RobotFileParser`, which answers questions about whether or not a particular user agent can fetch a URL on the @@ -26,10 +26,10 @@ Web site that published the :file:`robots.txt` file. For more details on the structure of :file:`robots.txt` files, see http://www.robotstxt.org/orig.html. -.. class:: RobotFileParser() +.. class:: RobotFileParser(url='') - This class provides a set of methods to read, parse and answer questions - about a single :file:`robots.txt` file. + This class provides methods to read, parse and answer questions about the + :file:`robots.txt` file at *url*. .. method:: set_url(url) diff --git a/Doc/library/runpy.rst b/Doc/library/runpy.rst index 31562fe..fc9e7c1 100644 --- a/Doc/library/runpy.rst +++ b/Doc/library/runpy.rst @@ -22,6 +22,9 @@ The :mod:`runpy` module provides two functions: .. function:: run_module(mod_name, init_globals=None, run_name=None, alter_sys=False) + .. index:: + module: __main__ + Execute the code of the specified module and return the resulting module globals dictionary. The module's code is first located using the standard import mechanism (refer to :pep:`302` for details) and then executed in a @@ -77,6 +80,9 @@ The :mod:`runpy` module provides two functions: .. function:: run_path(file_path, init_globals=None, run_name=None) + .. index:: + module: __main__ + Execute the code at the named filesystem location and return the resulting module globals dictionary. As with a script name supplied to the CPython command line, the supplied path may refer to a Python source file, a diff --git a/Doc/library/scrolledtext.rst b/Doc/library/scrolledtext.rst index 5c666c3..6af59dc 100644 --- a/Doc/library/scrolledtext.rst +++ b/Doc/library/scrolledtext.rst @@ -16,8 +16,8 @@ as that of the :class:`Tkinter.Text` class. .. note:: :mod:`ScrolledText` has been renamed to :mod:`tkinter.scrolledtext` in Python - 3.0. The :term:`2to3` tool will automatically adapt imports when converting - your sources to 3.0. + 3. The :term:`2to3` tool will automatically adapt imports when converting + your sources to Python 3. The text widget and scrollbar are packed together in a :class:`Frame`, and the methods of the :class:`Grid` and :class:`Pack` geometry managers are acquired diff --git a/Doc/library/select.rst b/Doc/library/select.rst index d131cb9..24cb756 100644 --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -63,7 +63,7 @@ The module defines the following: This is a straightforward interface to the Unix :c:func:`select` system call. The first three arguments are sequences of 'waitable objects': either integers representing file descriptors or objects with a parameterless method - named :meth:`fileno` returning such an integer: + named :meth:`~io.IOBase.fileno` returning such an integer: * *rlist*: wait until ready for reading * *wlist*: wait until ready for writing @@ -88,8 +88,8 @@ The module defines the following: Among the acceptable object types in the sequences are Python file objects (e.g. ``sys.stdin``, or objects returned by :func:`open` or :func:`os.popen`), socket objects returned by :func:`socket.socket`. You may also define a :dfn:`wrapper` - class yourself, as long as it has an appropriate :meth:`fileno` method (that - really returns a file descriptor, not just a random integer). + class yourself, as long as it has an appropriate :meth:`~io.IOBase.fileno` + method (that really returns a file descriptor, not just a random integer). .. note:: @@ -207,10 +207,10 @@ linearly scanned again. :c:func:`select` is O(highest file descriptor), while .. method:: poll.register(fd[, eventmask]) Register a file descriptor with the polling object. Future calls to the - :meth:`poll` method will then check whether the file descriptor has any pending - I/O events. *fd* can be either an integer, or an object with a :meth:`fileno` - method that returns an integer. File objects implement :meth:`fileno`, so they - can also be used as the argument. + :meth:`poll` method will then check whether the file descriptor has any + pending I/O events. *fd* can be either an integer, or an object with a + :meth:`~io.IOBase.fileno` method that returns an integer. File objects + implement :meth:`!fileno`, so they can also be used as the argument. *eventmask* is an optional bitmask describing the type of events you want to check for, and can be a combination of the constants :const:`POLLIN`, @@ -251,7 +251,7 @@ linearly scanned again. :c:func:`select` is O(highest file descriptor), while Remove a file descriptor being tracked by a polling object. Just like the :meth:`register` method, *fd* can be an integer or an object with a - :meth:`fileno` method that returns an integer. + :meth:`~io.IOBase.fileno` method that returns an integer. Attempting to remove a file descriptor that was never registered causes a :exc:`KeyError` exception to be raised. diff --git a/Doc/library/sgmllib.rst b/Doc/library/sgmllib.rst index f50b02c..1da19cf 100644 --- a/Doc/library/sgmllib.rst +++ b/Doc/library/sgmllib.rst @@ -6,7 +6,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`sgmllib` module has been removed in Python 3.0. + The :mod:`sgmllib` module has been removed in Python 3. .. index:: single: SGML diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst index de12420..b02f763 100644 --- a/Doc/library/shelve.rst +++ b/Doc/library/shelve.rst @@ -18,7 +18,7 @@ This includes most class instances, recursive data types, and objects containing lots of shared sub-objects. The keys are ordinary strings. -.. function:: open(filename[, flag='c'[, protocol=None[, writeback=False]]]) +.. function:: open(filename, flag='c', protocol=None, writeback=False) Open a persistent dictionary. The filename specified is the base filename for the underlying database. As a side-effect, an extension may be added to the @@ -47,9 +47,11 @@ lots of shared sub-objects. The keys are ordinary strings. Like file objects, shelve objects should be closed explicitly to ensure that the persistent data is flushed to disk. - Since the :mod:`shelve` module stores objects using :mod:`pickle`, the same - security precautions apply. Accordingly, you should avoid loading a shelf - from an untrusted source. +.. warning:: + + Because the :mod:`shelve` module is backed by :mod:`pickle`, it is insecure + to load a shelf from an untrusted source. Like with pickle, loading a shelf + can execute arbitrary code. Shelf objects support all methods supported by dictionaries. This eases the transition from dictionary based scripts to those requiring persistent storage. @@ -100,7 +102,7 @@ Restrictions implementation used. -.. class:: Shelf(dict[, protocol=None[, writeback=False]]) +.. class:: Shelf(dict, protocol=None, writeback=False) A subclass of :class:`UserDict.DictMixin` which stores pickled values in the *dict* object. @@ -118,7 +120,7 @@ Restrictions memory and make sync and close take a long time. -.. class:: BsdDbShelf(dict[, protocol=None[, writeback=False]]) +.. class:: BsdDbShelf(dict, protocol=None, writeback=False) A subclass of :class:`Shelf` which exposes :meth:`first`, :meth:`!next`, :meth:`previous`, :meth:`last` and :meth:`set_location` which are available in @@ -129,7 +131,7 @@ Restrictions the same interpretation as for the :class:`Shelf` class. -.. class:: DbfilenameShelf(filename[, flag='c'[, protocol=None[, writeback=False]]]) +.. class:: DbfilenameShelf(filename, flag='c', protocol=None, writeback=False) A subclass of :class:`Shelf` which accepts a *filename* instead of a dict-like object. The underlying file will be opened using :func:`anydbm.open`. By diff --git a/Doc/library/shlex.rst b/Doc/library/shlex.rst index bb05c7d..be08e01 100644 --- a/Doc/library/shlex.rst +++ b/Doc/library/shlex.rst @@ -16,9 +16,9 @@ -------------- -The :class:`shlex` class makes it easy to write lexical analyzers for simple -syntaxes resembling that of the Unix shell. This will often be useful for -writing minilanguages, (for example, in run control files for Python +The :class:`~shlex.shlex` class makes it easy to write lexical analyzers for +simple syntaxes resembling that of the Unix shell. This will often be useful +for writing minilanguages, (for example, in run control files for Python applications) or for parsing quoted strings. Prior to Python 2.7.3, this module did not support Unicode input. @@ -30,9 +30,10 @@ The :mod:`shlex` module defines the following functions: Split the string *s* using shell-like syntax. If *comments* is :const:`False` (the default), the parsing of comments in the given string will be disabled - (setting the :attr:`commenters` attribute of the :class:`shlex` instance to - the empty string). This function operates in POSIX mode by default, but uses - non-POSIX mode if the *posix* argument is false. + (setting the :attr:`~shlex.commenters` attribute of the + :class:`~shlex.shlex` instance to the empty string). This function operates + in POSIX mode by default, but uses non-POSIX mode if the *posix* argument is + false. .. versionadded:: 2.3 @@ -41,26 +42,28 @@ The :mod:`shlex` module defines the following functions: .. note:: - Since the :func:`split` function instantiates a :class:`shlex` instance, passing - ``None`` for *s* will read the string to split from standard input. + Since the :func:`split` function instantiates a :class:`~shlex.shlex` + instance, passing ``None`` for *s* will read the string to split from + standard input. The :mod:`shlex` module defines the following class: .. class:: shlex([instream[, infile[, posix]]]) - A :class:`shlex` instance or subclass instance is a lexical analyzer object. - The initialization argument, if present, specifies where to read characters - from. It must be a file-/stream-like object with :meth:`read` and - :meth:`readline` methods, or a string (strings are accepted since Python 2.3). - If no argument is given, input will be taken from ``sys.stdin``. The second - optional argument is a filename string, which sets the initial value of the - :attr:`infile` attribute. If the *instream* argument is omitted or equal to - ``sys.stdin``, this second argument defaults to "stdin". The *posix* argument - was introduced in Python 2.3, and defines the operational mode. When *posix* is - not true (default), the :class:`shlex` instance will operate in compatibility - mode. When operating in POSIX mode, :class:`shlex` will try to be as close as - possible to the POSIX shell parsing rules. + A :class:`~shlex.shlex` instance or subclass instance is a lexical analyzer + object. The initialization argument, if present, specifies where to read + characters from. It must be a file-/stream-like object with + :meth:`~io.TextIOBase.read` and :meth:`~io.TextIOBase.readline` methods, or + a string (strings are accepted since Python 2.3). If no argument is given, + input will be taken from ``sys.stdin``. The second optional argument is a + filename string, which sets the initial value of the :attr:`~shlex.infile` + attribute. If the *instream* argument is omitted or equal to ``sys.stdin``, + this second argument defaults to "stdin". The *posix* argument was + introduced in Python 2.3, and defines the operational mode. When *posix* is + not true (default), the :class:`~shlex.shlex` instance will operate in + compatibility mode. When operating in POSIX mode, :class:`~shlex.shlex` + will try to be as close as possible to the POSIX shell parsing rules. .. seealso:: @@ -74,14 +77,14 @@ The :mod:`shlex` module defines the following class: shlex Objects ------------- -A :class:`shlex` instance has the following methods: +A :class:`~shlex.shlex` instance has the following methods: .. method:: shlex.get_token() Return a token. If tokens have been stacked using :meth:`push_token`, pop a token off the stack. Otherwise, read one from the input stream. If reading - encounters an immediate end-of-file, :attr:`self.eof` is returned (the empty + encounters an immediate end-of-file, :attr:`eof` is returned (the empty string (``''``) in non-POSIX mode, and ``None`` in POSIX mode). @@ -99,9 +102,9 @@ A :class:`shlex` instance has the following methods: .. method:: shlex.sourcehook(filename) - When :class:`shlex` detects a source request (see :attr:`source` below) this - method is given the following token as argument, and expected to return a tuple - consisting of a filename and an open file-like object. + When :class:`~shlex.shlex` detects a source request (see :attr:`source` + below) this method is given the following token as argument, and expected + to return a tuple consisting of a filename and an open file-like object. Normally, this method first strips any quotes off the argument. If the result is an absolute pathname, or there was no previous source request in effect, or @@ -118,8 +121,9 @@ A :class:`shlex` instance has the following methods: This hook is exposed so that you can use it to implement directory search paths, addition of file extensions, and other namespace hacks. There is no - corresponding 'close' hook, but a shlex instance will call the :meth:`close` - method of the sourced input stream when it returns EOF. + corresponding 'close' hook, but a shlex instance will call the + :meth:`~io.IOBase.close` method of the sourced input stream when it returns + EOF. For more explicit control of source stacking, use the :meth:`push_source` and :meth:`pop_source` methods. @@ -153,8 +157,8 @@ A :class:`shlex` instance has the following methods: messages in the standard, parseable format understood by Emacs and other Unix tools. -Instances of :class:`shlex` subclasses have some public instance variables which -either control lexical analysis or can be used for debugging: +Instances of :class:`~shlex.shlex` subclasses have some public instance +variables which either control lexical analysis or can be used for debugging: .. attribute:: shlex.commenters @@ -203,8 +207,8 @@ either control lexical analysis or can be used for debugging: .. attribute:: shlex.whitespace_split If ``True``, tokens will only be split in whitespaces. This is useful, for - example, for parsing command lines with :class:`shlex`, getting tokens in a - similar way to shell arguments. + example, for parsing command lines with :class:`~shlex.shlex`, getting + tokens in a similar way to shell arguments. .. versionadded:: 2.3 @@ -218,7 +222,8 @@ either control lexical analysis or can be used for debugging: .. attribute:: shlex.instream - The input stream from which this :class:`shlex` instance is reading characters. + The input stream from which this :class:`~shlex.shlex` instance is reading + characters. .. attribute:: shlex.source @@ -227,16 +232,16 @@ either control lexical analysis or can be used for debugging: string will be recognized as a lexical-level inclusion request similar to the ``source`` keyword in various shells. That is, the immediately following token will opened as a filename and input taken from that stream until EOF, at which - point the :meth:`close` method of that stream will be called and the input - source will again become the original input stream. Source requests may be - stacked any number of levels deep. + point the :meth:`~io.IOBase.close` method of that stream will be called and + the input source will again become the original input stream. Source + requests may be stacked any number of levels deep. .. attribute:: shlex.debug - If this attribute is numeric and ``1`` or more, a :class:`shlex` instance will - print verbose progress output on its behavior. If you need to use this, you can - read the module source code to learn the details. + If this attribute is numeric and ``1`` or more, a :class:`~shlex.shlex` + instance will print verbose progress output on its behavior. If you need + to use this, you can read the module source code to learn the details. .. attribute:: shlex.lineno @@ -262,7 +267,7 @@ either control lexical analysis or can be used for debugging: Parsing Rules ------------- -When operating in non-POSIX mode, :class:`shlex` will try to obey to the +When operating in non-POSIX mode, :class:`~shlex.shlex` will try to obey to the following rules. * Quote characters are not recognized within words (``Do"Not"Separate`` is @@ -276,16 +281,17 @@ following rules. * Closing quotes separate words (``"Do"Separate`` is parsed as ``"Do"`` and ``Separate``); -* If :attr:`whitespace_split` is ``False``, any character not declared to be a - word character, whitespace, or a quote will be returned as a single-character - token. If it is ``True``, :class:`shlex` will only split words in whitespaces; +* If :attr:`~shlex.whitespace_split` is ``False``, any character not + declared to be a word character, whitespace, or a quote will be returned as + a single-character token. If it is ``True``, :class:`~shlex.shlex` will only + split words in whitespaces; * EOF is signaled with an empty string (``''``); * It's not possible to parse empty strings, even if quoted. -When operating in POSIX mode, :class:`shlex` will try to obey to the following -parsing rules. +When operating in POSIX mode, :class:`~shlex.shlex` will try to obey to the +following parsing rules. * Quotes are stripped out, and do not separate words (``"Do"Not"Separate"`` is parsed as the single word ``DoNotSeparate``); @@ -293,14 +299,16 @@ parsing rules. * Non-quoted escape characters (e.g. ``'\'``) preserve the literal value of the next character that follows; -* Enclosing characters in quotes which are not part of :attr:`escapedquotes` - (e.g. ``"'"``) preserve the literal value of all characters within the quotes; +* Enclosing characters in quotes which are not part of + :attr:`~shlex.escapedquotes` (e.g. ``"'"``) preserve the literal value + of all characters within the quotes; -* Enclosing characters in quotes which are part of :attr:`escapedquotes` (e.g. - ``'"'``) preserves the literal value of all characters within the quotes, with - the exception of the characters mentioned in :attr:`escape`. The escape - characters retain its special meaning only when followed by the quote in use, or - the escape character itself. Otherwise the escape character will be considered a +* Enclosing characters in quotes which are part of + :attr:`~shlex.escapedquotes` (e.g. ``'"'``) preserves the literal value + of all characters within the quotes, with the exception of the characters + mentioned in :attr:`~shlex.escape`. The escape characters retain its + special meaning only when followed by the quote in use, or the escape + character itself. Otherwise the escape character will be considered a normal character. * EOF is signaled with a :const:`None` value; diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 78c38e5..e897483 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -31,6 +31,8 @@ copying and removal. For operations on individual files, see also the are not copied. +.. _file-operations: + Directory and files operations ------------------------------ @@ -94,7 +96,7 @@ Directory and files operations .. versionadded:: 2.6 -.. function:: copytree(src, dst[, symlinks=False[, ignore=None]]) +.. function:: copytree(src, dst, symlinks=False, ignore=None) Recursively copy an entire directory tree rooted at *src*. The destination directory, named by *dst*, must not already exist; it will be created as @@ -185,7 +187,7 @@ Directory and files operations .. versionadded:: 2.3 -.. _shutil-example: +.. _copytree-example: copytree example :::::::::::::::: @@ -217,18 +219,18 @@ provided by this module. :: else: copy2(srcname, dstname) # XXX What about devices, sockets etc.? - except (IOError, os.error), why: + except (IOError, os.error) as why: errors.append((srcname, dstname, str(why))) # catch the Error from the recursive copytree so that we can # continue with other files - except Error, err: + except Error as err: errors.extend(err.args[0]) try: copystat(src, dst) except WindowsError: # can't copy file access times on Windows pass - except OSError, why: + except OSError as why: errors.extend((src, dst, str(why))) if errors: raise Error(errors) @@ -254,8 +256,13 @@ Another example that uses the *ignore* argument to add a logging call:: copytree(source, destination, ignore=_logpath) -Archives operations -------------------- +.. _archiving-operations: + +Archiving operations +-------------------- + +High-level utilities to create and read compressed and archived files are also +provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules. .. function:: make_archive(base_name, format, [root_dir, [base_dir, [verbose, [dry_run, [owner, [group, [logger]]]]]]]) @@ -278,7 +285,8 @@ Archives operations *owner* and *group* are used when creating a tar archive. By default, uses the current owner and group. - *logger* is an instance of :class:`logging.Logger`. + *logger* must be an object compatible with :pep:`282`, usually an instance of + :class:`logging.Logger`. .. versionadded:: 2.7 @@ -322,6 +330,8 @@ Archives operations .. versionadded:: 2.7 +.. _archiving-example: + Archiving example ::::::::::::::::: @@ -346,5 +356,3 @@ The resulting archive contains:: -rw------- tarek/staff 1675 2008-06-09 13:26:54 ./id_rsa -rw-r--r-- tarek/staff 397 2008-06-09 13:26:54 ./id_rsa.pub -rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts - - diff --git a/Doc/library/simplehttpserver.rst b/Doc/library/simplehttpserver.rst index a92c7c9..2e7e97a 100644 --- a/Doc/library/simplehttpserver.rst +++ b/Doc/library/simplehttpserver.rst @@ -8,8 +8,8 @@ .. note:: The :mod:`SimpleHTTPServer` module has been merged into :mod:`http.server` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. The :mod:`SimpleHTTPServer` module defines a single class, diff --git a/Doc/library/simplexmlrpcserver.rst b/Doc/library/simplexmlrpcserver.rst index 3618728..8f805e9 100644 --- a/Doc/library/simplexmlrpcserver.rst +++ b/Doc/library/simplexmlrpcserver.rst @@ -8,8 +8,8 @@ .. note:: The :mod:`SimpleXMLRPCServer` module has been merged into - :mod:`xmlrpc.server` in Python 3.0. The :term:`2to3` tool will automatically - adapt imports when converting your sources to 3.0. + :mod:`xmlrpc.server` in Python 3. The :term:`2to3` tool will automatically + adapt imports when converting your sources to Python 3. .. versionadded:: 2.2 @@ -197,6 +197,38 @@ server:: # Print list of available methods print s.system.listMethods() +The following :class:`SimpleXMLRPCServer` example is included in the module +`Lib/SimpleXMLRPCServer.py`:: + + server = SimpleXMLRPCServer(("localhost", 8000)) + server.register_function(pow) + server.register_function(lambda x,y: x+y, 'add') + server.register_multicall_functions() + server.serve_forever() + +This demo server can be run from the command line as:: + + python -m SimpleXMLRPCServer + +Example client code which talks to the above server is included with +`Lib/xmlrpclib.py`:: + + server = ServerProxy("http://localhost:8000") + print server + multi = MultiCall(server) + multi.pow(2, 9) + multi.add(5, 1) + multi.add(24, 11) + try: + for response in multi(): + print response + except Error, v: + print "ERROR", v + +And the client can be invoked directly using the following command:: + + python -m xmlrpclib + CGIXMLRPCRequestHandler ----------------------- @@ -247,7 +279,7 @@ requests sent to Python CGI scripts. Example:: class MyFuncs: - def div(self, x, y) : return x // y + def div(self, x, y): return x // y handler = CGIXMLRPCRequestHandler() diff --git a/Doc/library/site.rst b/Doc/library/site.rst index c0fafed..ff7195d 100644 --- a/Doc/library/site.rst +++ b/Doc/library/site.rst @@ -26,7 +26,7 @@ It starts by constructing up to four directories from a head and a tail part. For the head part, it uses ``sys.prefix`` and ``sys.exec_prefix``; empty heads are skipped. For the tail part, it uses the empty string and then :file:`lib/site-packages` (on Windows) or -:file:`lib/python|version|/site-packages` and then :file:`lib/site-python` (on +:file:`lib/python{X.Y}/site-packages` and then :file:`lib/site-python` (on Unix and Macintosh). For each of the distinct head-tail combinations, it sees if it refers to an existing directory, and if so, adds it to ``sys.path`` and also inspects the newly added path for configuration files. @@ -83,7 +83,11 @@ After these path manipulations, an attempt is made to import a module named :mod:`sitecustomize`, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory. If this import fails with an :exc:`ImportError` exception, it is -silently ignored. +silently ignored. If Python is started without output streams available, as +with :file:`pythonw.exe` on Windows (which is used by default to start IDLE), +attempted output from :mod:`sitecustomize` is ignored. Any exception other +than :exc:`ImportError` causes a silent and perhaps mysterious failure of the +process. .. index:: module: usercustomize @@ -181,8 +185,8 @@ command line: .. code-block:: sh - $ python3 -m site --user-site - /home/user/.local/lib/python3.3/site-packages + $ python -m site --user-site + /home/user/.local/lib/python2.7/site-packages .. program:: site diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index b0b58e8..5b2808d 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -24,15 +24,20 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). A :class:`SMTP` instance encapsulates an SMTP connection. It has methods that support a full repertoire of SMTP and ESMTP operations. If the optional - host and port parameters are given, the SMTP :meth:`connect` method is called - with those parameters during initialization. An :exc:`SMTPConnectError` is - raised if the specified host doesn't respond correctly. The optional + host and port parameters are given, the SMTP :meth:`connect` method is + called with those parameters during initialization. If specified, + *local_hostname* is used as the FQDN of the local host in the HELO/EHLO + command. Otherwise, the local hostname is found using + :func:`socket.getfqdn`. If the :meth:`connect` call returns anything other + than a success code, an :exc:`SMTPConnectError` is raised. The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the connection attempt (if not specified, the global default timeout - setting will be used). + setting will be used). If the timeout expires, :exc:`socket.timeout` + is raised. For normal use, you should only require the initialization/connect, - :meth:`sendmail`, and :meth:`quit` methods. An example is included below. + :meth:`sendmail`, and :meth:`~smtplib.quit` methods. + An example is included below. .. versionchanged:: 2.6 *timeout* was added. @@ -44,12 +49,14 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). :class:`SMTP`. :class:`SMTP_SSL` should be used for situations where SSL is required from the beginning of the connection and using :meth:`starttls` is not appropriate. If *host* is not specified, the local host is used. If - *port* is omitted, the standard SMTP-over-SSL port (465) is used. *keyfile* - and *certfile* are also optional, and can contain a PEM formatted private key - and certificate chain file for the SSL connection. The optional *timeout* - parameter specifies a timeout in seconds for blocking operations like the - connection attempt (if not specified, the global default timeout setting - will be used). + *port* is omitted, the standard SMTP-over-SSL port (465) is used. + *local_hostname* has the same meaning as it does for the :class:`SMTP` + class. *keyfile* and *certfile* are also optional, and can contain a PEM + formatted private key and certificate chain file for the SSL connection. The + optional *timeout* parameter specifies a timeout in seconds for blocking + operations like the connection attempt (if not specified, the global default + timeout setting will be used). If the timeout expires, :exc:`socket.timeout` + is raised. .. versionadded:: 2.6 @@ -57,13 +64,15 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). .. class:: LMTP([host[, port[, local_hostname]]]) The LMTP protocol, which is very similar to ESMTP, is heavily based on the - standard SMTP client. It's common to use Unix sockets for LMTP, so our :meth:`connect` - method must support that as well as a regular host:port server. To specify a - Unix socket, you must use an absolute path for *host*, starting with a '/'. + standard SMTP client. It's common to use Unix sockets for LMTP, so our + :meth:`connect` method must support that as well as a regular host:port + server. *local_hostname* has the same meaning as it does for the + :class:`SMTP` class. To specify a Unix socket, you must use an absolute + path for *host*, starting with a '/'. - Authentication is supported, using the regular SMTP mechanism. When using a Unix - socket, LMTP generally don't support or require any authentication, but your - mileage might vary. + Authentication is supported, using the regular SMTP mechanism. When using a + Unix socket, LMTP generally don't support or require any authentication, but + your mileage might vary. .. versionadded:: 2.6 @@ -72,7 +81,8 @@ A nice selection of exceptions is defined as well: .. exception:: SMTPException - Base exception class for all exceptions raised by this module. + The base exception class for all the other exceptions provided by this + module. .. exception:: SMTPServerDisconnected @@ -151,15 +161,6 @@ An :class:`SMTP` instance has the following methods: for connection and for all messages sent to and received from the server. -.. method:: SMTP.connect([host[, port]]) - - Connect to a host on a given port. The defaults are to connect to the local - host at the standard SMTP port (25). If the hostname ends with a colon (``':'``) - followed by a number, that suffix will be stripped off and the number - interpreted as the port number to use. This method is automatically invoked by - the constructor if a host is specified during instantiation. - - .. method:: SMTP.docmd(cmd, [, argstring]) Send a command *cmd* to the server. The optional argument *argstring* is simply @@ -176,6 +177,17 @@ An :class:`SMTP` instance has the following methods: :exc:`SMTPServerDisconnected` will be raised. +.. method:: SMTP.connect([host[, port]]) + + Connect to a host on a given port. The defaults are to connect to the local + host at the standard SMTP port (25). If the hostname ends with a colon (``':'``) + followed by a number, that suffix will be stripped off and the number + interpreted as the port number to use. This method is automatically invoked by + the constructor if a host is specified during instantiation. Returns a + 2-tuple of the response code and message sent by the server in its + connection response. + + .. method:: SMTP.helo([hostname]) Identify yourself to the SMTP server using ``HELO``. The hostname argument diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 8ac47fb..f6c711e 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -28,7 +28,7 @@ want to refer to :rfc:`3493` titled Basic Socket Interface Extensions for IPv6. The Python interface is a straightforward transliteration of the Unix system call and library interface for sockets to Python's object-oriented style: the -:func:`socket` function returns a :dfn:`socket object` whose methods implement +:func:`.socket` function returns a :dfn:`socket object` whose methods implement the various socket system calls. Parameter types are somewhat higher-level than in the C interface: as with :meth:`read` and :meth:`write` operations on Python files, buffer allocation on receive operations is automatic, and buffer length @@ -38,7 +38,7 @@ Socket addresses are represented as follows: A single string is used for the :const:`AF_UNIX` address family. A pair ``(host, port)`` is used for the :const:`AF_INET` address family, where *host* is a string representing either a hostname in Internet domain notation like ``'daring.cwi.nl'`` or an IPv4 address -like ``'100.50.200.5'``, and *port* is an integral port number. For +like ``'100.50.200.5'``, and *port* is an integer. For :const:`AF_INET6` address family, a four-tuple ``(host, port, flowinfo, scopeid)`` is used, where *flowinfo* and *scopeid* represents ``sin6_flowinfo`` and ``sin6_scope_id`` member in :const:`struct sockaddr_in6` in C. For @@ -72,17 +72,17 @@ numeric address in *host* portion. tuple, and the fields depend on the address type. The general tuple form is ``(addr_type, v1, v2, v3 [, scope])``, where: - - *addr_type* is one of TIPC_ADDR_NAMESEQ, TIPC_ADDR_NAME, or - TIPC_ADDR_ID. - - *scope* is one of TIPC_ZONE_SCOPE, TIPC_CLUSTER_SCOPE, and - TIPC_NODE_SCOPE. - - If *addr_type* is TIPC_ADDR_NAME, then *v1* is the server type, *v2* is + - *addr_type* is one of :const:`TIPC_ADDR_NAMESEQ`, :const:`TIPC_ADDR_NAME`, + or :const:`TIPC_ADDR_ID`. + - *scope* is one of :const:`TIPC_ZONE_SCOPE`, :const:`TIPC_CLUSTER_SCOPE`, + and :const:`TIPC_NODE_SCOPE`. + - If *addr_type* is :const:`TIPC_ADDR_NAME`, then *v1* is the server type, *v2* is the port identifier, and *v3* should be 0. - If *addr_type* is TIPC_ADDR_NAMESEQ, then *v1* is the server type, *v2* + If *addr_type* is :const:`TIPC_ADDR_NAMESEQ`, then *v1* is the server type, *v2* is the lower port number, and *v3* is the upper port number. - If *addr_type* is TIPC_ADDR_ID, then *v1* is the node, *v2* is the + If *addr_type* is :const:`TIPC_ADDR_ID`, then *v1* is the node, *v2* is the reference, and *v3* should be set to 0. @@ -146,7 +146,7 @@ The module :mod:`socket` exports the following constants and functions: AF_INET6 These constants represent the address (and protocol) families, used for the - first argument to :func:`socket`. If the :const:`AF_UNIX` constant is not + first argument to :func:`.socket`. If the :const:`AF_UNIX` constant is not defined then this protocol is unsupported. @@ -186,7 +186,7 @@ The module :mod:`socket` exports the following constants and functions: RCVALL_* Constants for Windows' WSAIoctl(). The constants are used as arguments to the - :meth:`ioctl` method of socket objects. + :meth:`~socket.socket.ioctl` method of socket objects. .. versionadded:: 2.6 @@ -230,7 +230,7 @@ The module :mod:`socket` exports the following constants and functions: *source_address* was added. -.. function:: getaddrinfo(host, port, family=0, socktype=0, proto=0, flags=0) +.. function:: getaddrinfo(host, port[, family[, socktype[, proto[, flags]]]]) Translate the *host*/*port* argument into a sequence of 5-tuples that contain all the necessary arguments for creating a socket connected to that service. @@ -240,19 +240,19 @@ The module :mod:`socket` exports the following constants and functions: and *port*, you can pass ``NULL`` to the underlying C API. The *family*, *socktype* and *proto* arguments can be optionally specified - in order to narrow the list of addresses returned. Passing zero as a - value for each of these arguments selects the full range of results. + in order to narrow the list of addresses returned. By default, their value + is ``0``, meaning that the full range of results is selected. The *flags* argument can be one or several of the ``AI_*`` constants, - and will influence how results are computed and returned. - For example, :const:`AI_NUMERICHOST` will disable domain name resolution - and will raise an error if *host* is a domain name. + and will influence how results are computed and returned. Its default value + is ``0``. For example, :const:`AI_NUMERICHOST` will disable domain name + resolution and will raise an error if *host* is a domain name. The function returns a list of 5-tuples with the following structure: ``(family, socktype, proto, canonname, sockaddr)`` In these tuples, *family*, *socktype*, *proto* are all integers and are - meant to be passed to the :func:`socket` function. *canonname* will be + meant to be passed to the :func:`.socket` function. *canonname* will be a string representing the canonical name of the *host* if :const:`AI_CANONNAME` is part of the *flags* argument; else *canonname* will be empty. *sockaddr* is a tuple describing a socket address, whose @@ -343,7 +343,7 @@ The module :mod:`socket` exports the following constants and functions: .. function:: getprotobyname(protocolname) Translate an Internet protocol name (for example, ``'icmp'``) to a constant - suitable for passing as the (optional) third argument to the :func:`socket` + suitable for passing as the (optional) third argument to the :func:`.socket` function. This is usually only needed for sockets opened in "raw" mode (:const:`SOCK_RAW`); for the normal socket modes, the correct protocol is chosen automatically if the protocol is omitted or zero. @@ -377,7 +377,7 @@ The module :mod:`socket` exports the following constants and functions: Build a pair of connected socket objects using the given address family, socket type, and protocol number. Address family, socket type, and protocol number are - as for the :func:`socket` function above. The default family is :const:`AF_UNIX` + as for the :func:`.socket` function above. The default family is :const:`AF_UNIX` if defined on the platform; otherwise, the default is :const:`AF_INET`. Availability: Unix. @@ -388,7 +388,7 @@ The module :mod:`socket` exports the following constants and functions: Duplicate the file descriptor *fd* (an integer as returned by a file object's :meth:`fileno` method) and build a socket object from the result. Address - family, socket type and protocol number are as for the :func:`socket` function + family, socket type and protocol number are as for the :func:`.socket` function above. The file descriptor should refer to a socket, but this is not checked --- subsequent operations on the object may fail if the file descriptor is invalid. This function is rarely needed, but can be used to get or set socket options on @@ -562,6 +562,7 @@ correspond to Unix system calls applicable to sockets. automatically closed when they are garbage-collected. .. note:: + :meth:`close()` releases the resource associated with a connection but does not necessarily close the connection immediately. If you want to close the connection in a timely fashion, call :meth:`shutdown()` @@ -739,7 +740,8 @@ correspond to Unix system calls applicable to sockets. much data, if any, was successfully sent. -.. method:: socket.sendto(string[, flags], address) +.. method:: socket.sendto(string, address) + socket.sendto(string, flags, address) Send data to the socket. The socket should not be connected to a remote socket, since the destination socket is specified by *address*. The optional *flags* @@ -860,10 +862,10 @@ Example Here are four minimal example programs using the TCP/IP protocol: a server that echoes all data that it receives back (servicing only one client), and a client -using it. Note that a server must perform the sequence :func:`socket`, +using it. Note that a server must perform the sequence :func:`.socket`, :meth:`~socket.bind`, :meth:`~socket.listen`, :meth:`~socket.accept` (possibly repeating the :meth:`~socket.accept` to service more than one client), while a -client only needs the sequence :func:`socket`, :meth:`~socket.connect`. Also +client only needs the sequence :func:`.socket`, :meth:`~socket.connect`. Also note that the server does not :meth:`~socket.sendall`/:meth:`~socket.recv` on the socket it is listening on but on the new socket returned by :meth:`~socket.accept`. @@ -919,13 +921,13 @@ sends traffic to the first one connected successfully. :: af, socktype, proto, canonname, sa = res try: s = socket.socket(af, socktype, proto) - except socket.error, msg: + except socket.error as msg: s = None continue try: s.bind(sa) s.listen(1) - except socket.error, msg: + except socket.error as msg: s.close() s = None continue @@ -954,12 +956,12 @@ sends traffic to the first one connected successfully. :: af, socktype, proto, canonname, sa = res try: s = socket.socket(af, socktype, proto) - except socket.error, msg: + except socket.error as msg: s = None continue try: s.connect(sa) - except socket.error, msg: + except socket.error as msg: s.close() s = None continue diff --git a/Doc/library/socketserver.rst b/Doc/library/socketserver.rst index 62caf2b..a9053d1 100644 --- a/Doc/library/socketserver.rst +++ b/Doc/library/socketserver.rst @@ -7,8 +7,8 @@ .. note:: The :mod:`SocketServer` module has been renamed to :mod:`socketserver` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. **Source code:** :source:`Lib/SocketServer.py` @@ -116,13 +116,13 @@ can be implemented by using a synchronous server and doing an explicit fork in the request handler class :meth:`handle` method. Another approach to handling multiple simultaneous requests in an environment -that supports neither threads nor :func:`fork` (or where these are too expensive -or inappropriate for the service) is to maintain an explicit table of partially -finished requests and to use :func:`select` to decide which request to work on -next (or whether to handle a new incoming request). This is particularly -important for stream services where each client can potentially be connected for -a long time (if threads or subprocesses cannot be used). See :mod:`asyncore` for -another way to manage this. +that supports neither threads nor :func:`~os.fork` (or where these are too +expensive or inappropriate for the service) is to maintain an explicit table of +partially finished requests and to use :func:`~select.select` to decide which +request to work on next (or whether to handle a new incoming request). This is +particularly important for stream services where each client can potentially be +connected for a long time (if threads or subprocesses cannot be used). See +:mod:`asyncore` for another way to manage this. .. XXX should data and methods be intermingled, or separate? how should the distinction between class and instance variables be drawn? @@ -306,8 +306,8 @@ request. .. method:: RequestHandler.finish() Called after the :meth:`handle` method to perform any clean-up actions - required. The default implementation does nothing. If :meth:`setup` or - :meth:`handle` raise an exception, this function will not be called. + required. The default implementation does nothing. If :meth:`setup` + raises an exception, this function will not be called. .. method:: RequestHandler.handle() diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 80803aa..ff634c0 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -3,7 +3,7 @@ .. module:: sqlite3 :synopsis: A DB-API 2.0 implementation using SQLite 3.x. -.. sectionauthor:: Gerhard Häring <gh@ghaering.de> +.. sectionauthor:: Gerhard Häring <gh@ghaering.de> .. versionadded:: 2.5 @@ -15,15 +15,15 @@ SQLite for internal data storage. It's also possible to prototype an application using SQLite and then port the code to a larger database such as PostgreSQL or Oracle. -sqlite3 was written by Gerhard Häring and provides a SQL interface compliant -with the DB-API 2.0 specification described by :pep:`249`. +The sqlite3 module was written by Gerhard Häring. It provides a SQL interface +compliant with the DB-API 2.0 specification described by :pep:`249`. To use the module, you must first create a :class:`Connection` object that represents the database. Here the data will be stored in the -:file:`/tmp/example` file:: +:file:`example.db` file:: import sqlite3 - conn = sqlite3.connect('/tmp/example') + conn = sqlite3.connect('example.db') You can also supply the special name ``:memory:`` to create a database in RAM. @@ -33,23 +33,29 @@ and call its :meth:`~Cursor.execute` method to perform SQL commands:: c = conn.cursor() # Create table - c.execute('''create table stocks - (date text, trans text, symbol text, - qty real, price real)''') + c.execute('''CREATE TABLE stocks + (date text, trans text, symbol text, qty real, price real)''') # Insert a row of data - c.execute("""insert into stocks - values ('2006-01-05','BUY','RHAT',100,35.14)""") + c.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)") # Save (commit) the changes conn.commit() - # We can also close the cursor if we are done with it - c.close() + # We can also close the connection if we are done with it. + # Just be sure any changes have been committed or they will be lost. + conn.close() + +The data you've saved is persistent and is available in subsequent sessions:: + + import sqlite3 + conn = sqlite3.connect('example.db') + c = conn.cursor() Usually your SQL operations will need to use values from Python variables. You shouldn't assemble your query using Python's string operations because doing so -is insecure; it makes your program vulnerable to an SQL injection attack. +is insecure; it makes your program vulnerable to an SQL injection attack +(see http://xkcd.com/327/ for humorous example of what can go wrong). Instead, use the DB-API's parameter substitution. Put ``?`` as a placeholder wherever you want to use a value, and then provide a tuple of values as the @@ -58,19 +64,20 @@ modules may use a different placeholder, such as ``%s`` or ``:1``.) For example:: # Never do this -- insecure! - symbol = 'IBM' - c.execute("select * from stocks where symbol = '%s'" % symbol) + symbol = 'RHAT' + c.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol) # Do this instead - t = (symbol,) - c.execute('select * from stocks where symbol=?', t) + t = ('RHAT',) + c.execute('SELECT * FROM stocks WHERE symbol=?', t) + print c.fetchone() - # Larger example - for t in [('2006-03-28', 'BUY', 'IBM', 1000, 45.00), - ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00), - ('2006-04-06', 'SELL', 'IBM', 500, 53.00), - ]: - c.execute('insert into stocks values (?,?,?,?,?)', t) + # Larger example that inserts many records at a time + purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00), + ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00), + ('2006-04-06', 'SELL', 'IBM', 500, 53.00), + ] + c.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases) To retrieve data after executing a SELECT statement, you can either treat the cursor as an :term:`iterator`, call the cursor's :meth:`~Cursor.fetchone` method to @@ -79,21 +86,18 @@ matching rows. This example uses the iterator form:: - >>> c = conn.cursor() - >>> c.execute('select * from stocks order by price') - >>> for row in c: - ... print row - ... + >>> for row in c.execute('SELECT * FROM stocks ORDER BY price'): + print row + (u'2006-01-05', u'BUY', u'RHAT', 100, 35.14) (u'2006-03-28', u'BUY', u'IBM', 1000, 45.0) (u'2006-04-06', u'SELL', u'IBM', 500, 53.0) (u'2006-04-05', u'BUY', u'MSFT', 1000, 72.0) - >>> .. seealso:: - http://code.google.com/p/pysqlite/ + https://github.com/ghaering/pysqlite The pysqlite web page -- sqlite3 is developed externally under the name "pysqlite". @@ -101,6 +105,9 @@ This example uses the iterator form:: The SQLite web page; the documentation describes the syntax and the available data types for the supported SQL dialect. + http://www.w3schools.com/sql/ + Tutorial, reference and examples for learning SQL syntax. + :pep:`249` - Database API Specification 2.0 PEP written by Marc-André Lemburg. @@ -111,6 +118,24 @@ Module functions and constants ------------------------------ +.. data:: version + + The version number of this module, as a string. This is not the version of + the SQLite library. + +.. data:: version_info + + The version number of this module, as a tuple of integers. This is not the + version of the SQLite library. + +.. data:: sqlite_version + + The version number of the run-time SQLite library, as a string. + +.. data:: sqlite_version_info + + The version number of the run-time SQLite library, as a tuple of integers. + .. data:: PARSE_DECLTYPES This constant is meant to be used with the *detect_types* parameter of the @@ -154,7 +179,7 @@ Module functions and constants For the *isolation_level* parameter, please see the :attr:`Connection.isolation_level` property of :class:`Connection` objects. - SQLite natively supports only the types TEXT, INTEGER, FLOAT, BLOB and NULL. If + SQLite natively supports only the types TEXT, INTEGER, REAL, BLOB and NULL. If you want to use other types you must add support for them yourself. The *detect_types* parameter and the using custom **converters** registered with the module-level :func:`register_converter` function allow you to easily do that. @@ -209,10 +234,10 @@ Module functions and constants .. function:: enable_callback_tracebacks(flag) By default you will not get any tracebacks in user-defined functions, - aggregates, converters, authorizer callbacks etc. If you want to debug them, you - can call this function with *flag* as True. Afterwards, you will get tracebacks - from callbacks on ``sys.stderr``. Use :const:`False` to disable the feature - again. + aggregates, converters, authorizer callbacks etc. If you want to debug them, + you can call this function with *flag* set to ``True``. Afterwards, you will + get tracebacks from callbacks on ``sys.stderr``. Use :const:`False` to + disable the feature again. .. _sqlite3-connection-objects: @@ -224,237 +249,236 @@ Connection Objects A SQLite database connection has the following attributes and methods: -.. attribute:: Connection.isolation_level + .. attribute:: isolation_level - Get or set the current isolation level. :const:`None` for autocommit mode or - one of "DEFERRED", "IMMEDIATE" or "EXCLUSIVE". See section - :ref:`sqlite3-controlling-transactions` for a more detailed explanation. + Get or set the current isolation level. :const:`None` for autocommit mode or + one of "DEFERRED", "IMMEDIATE" or "EXCLUSIVE". See section + :ref:`sqlite3-controlling-transactions` for a more detailed explanation. -.. method:: Connection.cursor([cursorClass]) + .. method:: cursor([cursorClass]) - The cursor method accepts a single optional parameter *cursorClass*. If - supplied, this must be a custom cursor class that extends - :class:`sqlite3.Cursor`. + The cursor method accepts a single optional parameter *cursorClass*. If + supplied, this must be a custom cursor class that extends + :class:`sqlite3.Cursor`. + .. method:: commit() -.. method:: Connection.commit() + This method commits the current transaction. If you don't call this method, + anything you did since the last call to ``commit()`` is not visible from + other database connections. If you wonder why you don't see the data you've + written to the database, please check you didn't forget to call this method. - This method commits the current transaction. If you don't call this method, - anything you did since the last call to ``commit()`` is not visible from - other database connections. If you wonder why you don't see the data you've - written to the database, please check you didn't forget to call this method. + .. method:: rollback() -.. method:: Connection.rollback() + This method rolls back any changes to the database since the last call to + :meth:`commit`. - This method rolls back any changes to the database since the last call to - :meth:`commit`. + .. method:: close() -.. method:: Connection.close() + This closes the database connection. Note that this does not automatically + call :meth:`commit`. If you just close your database connection without + calling :meth:`commit` first, your changes will be lost! - This closes the database connection. Note that this does not automatically - call :meth:`commit`. If you just close your database connection without - calling :meth:`commit` first, your changes will be lost! + .. method:: execute(sql, [parameters]) -.. method:: Connection.execute(sql, [parameters]) + This is a nonstandard shortcut that creates an intermediate cursor object by + calling the cursor method, then calls the cursor's :meth:`execute + <Cursor.execute>` method with the parameters given. - This is a nonstandard shortcut that creates an intermediate cursor object by - calling the cursor method, then calls the cursor's :meth:`execute - <Cursor.execute>` method with the parameters given. + .. method:: executemany(sql, [parameters]) -.. method:: Connection.executemany(sql, [parameters]) + This is a nonstandard shortcut that creates an intermediate cursor object by + calling the cursor method, then calls the cursor's :meth:`executemany + <Cursor.executemany>` method with the parameters given. - This is a nonstandard shortcut that creates an intermediate cursor object by - calling the cursor method, then calls the cursor's :meth:`executemany - <Cursor.executemany>` method with the parameters given. + .. method:: executescript(sql_script) -.. method:: Connection.executescript(sql_script) + This is a nonstandard shortcut that creates an intermediate cursor object by + calling the cursor method, then calls the cursor's :meth:`executescript + <Cursor.executescript>` method with the parameters given. - This is a nonstandard shortcut that creates an intermediate cursor object by - calling the cursor method, then calls the cursor's :meth:`executescript - <Cursor.executescript>` method with the parameters given. + .. method:: create_function(name, num_params, func) -.. method:: Connection.create_function(name, num_params, func) + Creates a user-defined function that you can later use from within SQL + statements under the function name *name*. *num_params* is the number of + parameters the function accepts, and *func* is a Python callable that is called + as the SQL function. - Creates a user-defined function that you can later use from within SQL - statements under the function name *name*. *num_params* is the number of - parameters the function accepts, and *func* is a Python callable that is called - as the SQL function. + The function can return any of the types supported by SQLite: unicode, str, int, + long, float, buffer and None. - The function can return any of the types supported by SQLite: unicode, str, int, - long, float, buffer and None. + Example: - Example: + .. literalinclude:: ../includes/sqlite3/md5func.py - .. literalinclude:: ../includes/sqlite3/md5func.py + .. method:: create_aggregate(name, num_params, aggregate_class) -.. method:: Connection.create_aggregate(name, num_params, aggregate_class) + Creates a user-defined aggregate function. - Creates a user-defined aggregate function. + The aggregate class must implement a ``step`` method, which accepts the number + of parameters *num_params*, and a ``finalize`` method which will return the + final result of the aggregate. - The aggregate class must implement a ``step`` method, which accepts the number - of parameters *num_params*, and a ``finalize`` method which will return the - final result of the aggregate. + The ``finalize`` method can return any of the types supported by SQLite: + unicode, str, int, long, float, buffer and None. - The ``finalize`` method can return any of the types supported by SQLite: - unicode, str, int, long, float, buffer and None. + Example: - Example: + .. literalinclude:: ../includes/sqlite3/mysumaggr.py - .. literalinclude:: ../includes/sqlite3/mysumaggr.py + .. method:: create_collation(name, callable) -.. method:: Connection.create_collation(name, callable) + Creates a collation with the specified *name* and *callable*. The callable will + be passed two string arguments. It should return -1 if the first is ordered + lower than the second, 0 if they are ordered equal and 1 if the first is ordered + higher than the second. Note that this controls sorting (ORDER BY in SQL) so + your comparisons don't affect other SQL operations. - Creates a collation with the specified *name* and *callable*. The callable will - be passed two string arguments. It should return -1 if the first is ordered - lower than the second, 0 if they are ordered equal and 1 if the first is ordered - higher than the second. Note that this controls sorting (ORDER BY in SQL) so - your comparisons don't affect other SQL operations. + Note that the callable will get its parameters as Python bytestrings, which will + normally be encoded in UTF-8. - Note that the callable will get its parameters as Python bytestrings, which will - normally be encoded in UTF-8. + The following example shows a custom collation that sorts "the wrong way": - The following example shows a custom collation that sorts "the wrong way": + .. literalinclude:: ../includes/sqlite3/collation_reverse.py - .. literalinclude:: ../includes/sqlite3/collation_reverse.py + To remove a collation, call ``create_collation`` with None as callable:: - To remove a collation, call ``create_collation`` with None as callable:: + con.create_collation("reverse", None) - con.create_collation("reverse", None) + .. method:: interrupt() -.. method:: Connection.interrupt() + You can call this method from a different thread to abort any queries that might + be executing on the connection. The query will then abort and the caller will + get an exception. - You can call this method from a different thread to abort any queries that might - be executing on the connection. The query will then abort and the caller will - get an exception. + .. method:: set_authorizer(authorizer_callback) -.. method:: Connection.set_authorizer(authorizer_callback) + This routine registers a callback. The callback is invoked for each attempt to + access a column of a table in the database. The callback should return + :const:`SQLITE_OK` if access is allowed, :const:`SQLITE_DENY` if the entire SQL + statement should be aborted with an error and :const:`SQLITE_IGNORE` if the + column should be treated as a NULL value. These constants are available in the + :mod:`sqlite3` module. - This routine registers a callback. The callback is invoked for each attempt to - access a column of a table in the database. The callback should return - :const:`SQLITE_OK` if access is allowed, :const:`SQLITE_DENY` if the entire SQL - statement should be aborted with an error and :const:`SQLITE_IGNORE` if the - column should be treated as a NULL value. These constants are available in the - :mod:`sqlite3` module. + The first argument to the callback signifies what kind of operation is to be + authorized. The second and third argument will be arguments or :const:`None` + depending on the first argument. The 4th argument is the name of the database + ("main", "temp", etc.) if applicable. The 5th argument is the name of the + inner-most trigger or view that is responsible for the access attempt or + :const:`None` if this access attempt is directly from input SQL code. - The first argument to the callback signifies what kind of operation is to be - authorized. The second and third argument will be arguments or :const:`None` - depending on the first argument. The 4th argument is the name of the database - ("main", "temp", etc.) if applicable. The 5th argument is the name of the - inner-most trigger or view that is responsible for the access attempt or - :const:`None` if this access attempt is directly from input SQL code. + Please consult the SQLite documentation about the possible values for the first + argument and the meaning of the second and third argument depending on the first + one. All necessary constants are available in the :mod:`sqlite3` module. - Please consult the SQLite documentation about the possible values for the first - argument and the meaning of the second and third argument depending on the first - one. All necessary constants are available in the :mod:`sqlite3` module. + .. method:: set_progress_handler(handler, n) -.. method:: Connection.set_progress_handler(handler, n) + This routine registers a callback. The callback is invoked for every *n* + instructions of the SQLite virtual machine. This is useful if you want to + get called from SQLite during long-running operations, for example to update + a GUI. - .. versionadded:: 2.6 + If you want to clear any previously installed progress handler, call the + method with :const:`None` for *handler*. - This routine registers a callback. The callback is invoked for every *n* - instructions of the SQLite virtual machine. This is useful if you want to - get called from SQLite during long-running operations, for example to update - a GUI. - - If you want to clear any previously installed progress handler, call the - method with :const:`None` for *handler*. + .. versionadded:: 2.6 -.. method:: Connection.enable_load_extension(enabled) + .. method:: enable_load_extension(enabled) - .. versionadded:: 2.7 + This routine allows/disallows the SQLite engine to load SQLite extensions + from shared libraries. SQLite extensions can define new functions, + aggregates or whole new virtual table implementations. One well-known + extension is the fulltext-search extension distributed with SQLite. - This routine allows/disallows the SQLite engine to load SQLite extensions - from shared libraries. SQLite extensions can define new functions, - aggregates or whole new virtual table implementations. One well-known - extension is the fulltext-search extension distributed with SQLite. + Loadable extensions are disabled by default. See [#f1]_. - .. literalinclude:: ../includes/sqlite3/load_extension.py + .. versionadded:: 2.7 - Loadable extensions are disabled by default. See [#f1]_ + .. literalinclude:: ../includes/sqlite3/load_extension.py -.. method:: Connection.load_extension(path) + .. method:: load_extension(path) - .. versionadded:: 2.7 + This routine loads a SQLite extension from a shared library. You have to + enable extension loading with :meth:`enable_load_extension` before you can + use this routine. - This routine loads a SQLite extension from a shared library. You have to - enable extension loading with :meth:`enable_load_extension` before you can - use this routine. + Loadable extensions are disabled by default. See [#f1]_. - Loadable extensions are disabled by default. See [#f1]_ + .. versionadded:: 2.7 -.. attribute:: Connection.row_factory + .. attribute:: row_factory - You can change this attribute to a callable that accepts the cursor and the - original row as a tuple and will return the real result row. This way, you can - implement more advanced ways of returning results, such as returning an object - that can also access columns by name. + You can change this attribute to a callable that accepts the cursor and the + original row as a tuple and will return the real result row. This way, you can + implement more advanced ways of returning results, such as returning an object + that can also access columns by name. - Example: + Example: - .. literalinclude:: ../includes/sqlite3/row_factory.py + .. literalinclude:: ../includes/sqlite3/row_factory.py - If returning a tuple doesn't suffice and you want name-based access to - columns, you should consider setting :attr:`row_factory` to the - highly-optimized :class:`sqlite3.Row` type. :class:`Row` provides both - index-based and case-insensitive name-based access to columns with almost no - memory overhead. It will probably be better than your own custom - dictionary-based approach or even a db_row based solution. + If returning a tuple doesn't suffice and you want name-based access to + columns, you should consider setting :attr:`row_factory` to the + highly-optimized :class:`sqlite3.Row` type. :class:`Row` provides both + index-based and case-insensitive name-based access to columns with almost no + memory overhead. It will probably be better than your own custom + dictionary-based approach or even a db_row based solution. - .. XXX what's a db_row-based solution? + .. XXX what's a db_row-based solution? -.. attribute:: Connection.text_factory + .. attribute:: text_factory - Using this attribute you can control what objects are returned for the ``TEXT`` - data type. By default, this attribute is set to :class:`unicode` and the - :mod:`sqlite3` module will return Unicode objects for ``TEXT``. If you want to - return bytestrings instead, you can set it to :class:`str`. + Using this attribute you can control what objects are returned for the ``TEXT`` + data type. By default, this attribute is set to :class:`unicode` and the + :mod:`sqlite3` module will return Unicode objects for ``TEXT``. If you want to + return bytestrings instead, you can set it to :class:`str`. - For efficiency reasons, there's also a way to return Unicode objects only for - non-ASCII data, and bytestrings otherwise. To activate it, set this attribute to - :const:`sqlite3.OptimizedUnicode`. + For efficiency reasons, there's also a way to return Unicode objects only for + non-ASCII data, and bytestrings otherwise. To activate it, set this attribute to + :const:`sqlite3.OptimizedUnicode`. - You can also set it to any other callable that accepts a single bytestring - parameter and returns the resulting object. + You can also set it to any other callable that accepts a single bytestring + parameter and returns the resulting object. - See the following example code for illustration: + See the following example code for illustration: - .. literalinclude:: ../includes/sqlite3/text_factory.py + .. literalinclude:: ../includes/sqlite3/text_factory.py -.. attribute:: Connection.total_changes + .. attribute:: total_changes - Returns the total number of database rows that have been modified, inserted, or - deleted since the database connection was opened. + Returns the total number of database rows that have been modified, inserted, or + deleted since the database connection was opened. -.. attribute:: Connection.iterdump + .. attribute:: iterdump - Returns an iterator to dump the database in an SQL text format. Useful when - saving an in-memory database for later restoration. This function provides - the same capabilities as the :kbd:`.dump` command in the :program:`sqlite3` - shell. + Returns an iterator to dump the database in an SQL text format. Useful when + saving an in-memory database for later restoration. This function provides + the same capabilities as the :kbd:`.dump` command in the :program:`sqlite3` + shell. - .. versionadded:: 2.6 + .. versionadded:: 2.6 - Example:: + Example:: - # Convert file existing_db.db to SQL dump file dump.sql - import sqlite3, os + # Convert file existing_db.db to SQL dump file dump.sql + import sqlite3, os - con = sqlite3.connect('existing_db.db') - with open('dump.sql', 'w') as f: - for line in con.iterdump(): - f.write('%s\n' % line) + con = sqlite3.connect('existing_db.db') + with open('dump.sql', 'w') as f: + for line in con.iterdump(): + f.write('%s\n' % line) .. _sqlite3-cursor-objects: @@ -466,114 +490,110 @@ Cursor Objects A :class:`Cursor` instance has the following attributes and methods. -.. method:: Cursor.execute(sql, [parameters]) - - Executes an SQL statement. The SQL statement may be parametrized (i. e. - placeholders instead of SQL literals). The :mod:`sqlite3` module supports two - kinds of placeholders: question marks (qmark style) and named placeholders - (named style). + .. method:: execute(sql, [parameters]) - This example shows how to use parameters with qmark style: + Executes an SQL statement. The SQL statement may be parameterized (i. e. + placeholders instead of SQL literals). The :mod:`sqlite3` module supports two + kinds of placeholders: question marks (qmark style) and named placeholders + (named style). - .. literalinclude:: ../includes/sqlite3/execute_1.py + Here's an example of both styles: - This example shows how to use the named style: + .. literalinclude:: ../includes/sqlite3/execute_1.py - .. literalinclude:: ../includes/sqlite3/execute_2.py + :meth:`execute` will only execute a single SQL statement. If you try to execute + more than one statement with it, it will raise a Warning. Use + :meth:`executescript` if you want to execute multiple SQL statements with one + call. - :meth:`execute` will only execute a single SQL statement. If you try to execute - more than one statement with it, it will raise a Warning. Use - :meth:`executescript` if you want to execute multiple SQL statements with one - call. + .. method:: executemany(sql, seq_of_parameters) -.. method:: Cursor.executemany(sql, seq_of_parameters) + Executes an SQL command against all parameter sequences or mappings found in + the sequence *sql*. The :mod:`sqlite3` module also allows using an + :term:`iterator` yielding parameters instead of a sequence. - Executes an SQL command against all parameter sequences or mappings found in - the sequence *sql*. The :mod:`sqlite3` module also allows using an - :term:`iterator` yielding parameters instead of a sequence. + .. literalinclude:: ../includes/sqlite3/executemany_1.py - .. literalinclude:: ../includes/sqlite3/executemany_1.py + Here's a shorter example using a :term:`generator`: - Here's a shorter example using a :term:`generator`: + .. literalinclude:: ../includes/sqlite3/executemany_2.py - .. literalinclude:: ../includes/sqlite3/executemany_2.py + .. method:: executescript(sql_script) -.. method:: Cursor.executescript(sql_script) + This is a nonstandard convenience method for executing multiple SQL statements + at once. It issues a ``COMMIT`` statement first, then executes the SQL script it + gets as a parameter. - This is a nonstandard convenience method for executing multiple SQL statements - at once. It issues a ``COMMIT`` statement first, then executes the SQL script it - gets as a parameter. + *sql_script* can be a bytestring or a Unicode string. - *sql_script* can be a bytestring or a Unicode string. + Example: - Example: + .. literalinclude:: ../includes/sqlite3/executescript.py - .. literalinclude:: ../includes/sqlite3/executescript.py + .. method:: fetchone() -.. method:: Cursor.fetchone() + Fetches the next row of a query result set, returning a single sequence, + or :const:`None` when no more data is available. - Fetches the next row of a query result set, returning a single sequence, - or :const:`None` when no more data is available. + .. method:: fetchmany([size=cursor.arraysize]) -.. method:: Cursor.fetchmany([size=cursor.arraysize]) + Fetches the next set of rows of a query result, returning a list. An empty + list is returned when no more rows are available. - Fetches the next set of rows of a query result, returning a list. An empty - list is returned when no more rows are available. + The number of rows to fetch per call is specified by the *size* parameter. + If it is not given, the cursor's arraysize determines the number of rows + to be fetched. The method should try to fetch as many rows as indicated by + the size parameter. If this is not possible due to the specified number of + rows not being available, fewer rows may be returned. - The number of rows to fetch per call is specified by the *size* parameter. - If it is not given, the cursor's arraysize determines the number of rows - to be fetched. The method should try to fetch as many rows as indicated by - the size parameter. If this is not possible due to the specified number of - rows not being available, fewer rows may be returned. + Note there are performance considerations involved with the *size* parameter. + For optimal performance, it is usually best to use the arraysize attribute. + If the *size* parameter is used, then it is best for it to retain the same + value from one :meth:`fetchmany` call to the next. - Note there are performance considerations involved with the *size* parameter. - For optimal performance, it is usually best to use the arraysize attribute. - If the *size* parameter is used, then it is best for it to retain the same - value from one :meth:`fetchmany` call to the next. + .. method:: fetchall() -.. method:: Cursor.fetchall() + Fetches all (remaining) rows of a query result, returning a list. Note that + the cursor's arraysize attribute can affect the performance of this operation. + An empty list is returned when no rows are available. - Fetches all (remaining) rows of a query result, returning a list. Note that - the cursor's arraysize attribute can affect the performance of this operation. - An empty list is returned when no rows are available. + .. attribute:: rowcount -.. attribute:: Cursor.rowcount + Although the :class:`Cursor` class of the :mod:`sqlite3` module implements this + attribute, the database engine's own support for the determination of "rows + affected"/"rows selected" is quirky. - Although the :class:`Cursor` class of the :mod:`sqlite3` module implements this - attribute, the database engine's own support for the determination of "rows - affected"/"rows selected" is quirky. + For :meth:`executemany` statements, the number of modifications are summed up + into :attr:`rowcount`. - For :meth:`executemany` statements, the number of modifications are summed up - into :attr:`rowcount`. + As required by the Python DB API Spec, the :attr:`rowcount` attribute "is -1 in + case no ``executeXX()`` has been performed on the cursor or the rowcount of the + last operation is not determinable by the interface". This includes ``SELECT`` + statements because we cannot determine the number of rows a query produced + until all rows were fetched. - As required by the Python DB API Spec, the :attr:`rowcount` attribute "is -1 in - case no ``executeXX()`` has been performed on the cursor or the rowcount of the - last operation is not determinable by the interface". This includes ``SELECT`` - statements because we cannot determine the number of rows a query produced - until all rows were fetched. + With SQLite versions before 3.6.5, :attr:`rowcount` is set to 0 if + you make a ``DELETE FROM table`` without any condition. - With SQLite versions before 3.6.5, :attr:`rowcount` is set to 0 if - you make a ``DELETE FROM table`` without any condition. + .. attribute:: lastrowid -.. attribute:: Cursor.lastrowid + This read-only attribute provides the rowid of the last modified row. It is + only set if you issued a ``INSERT`` statement using the :meth:`execute` + method. For operations other than ``INSERT`` or when :meth:`executemany` is + called, :attr:`lastrowid` is set to :const:`None`. - This read-only attribute provides the rowid of the last modified row. It is - only set if you issued a ``INSERT`` statement using the :meth:`execute` - method. For operations other than ``INSERT`` or when :meth:`executemany` is - called, :attr:`lastrowid` is set to :const:`None`. + .. attribute:: description -.. attribute:: Cursor.description + This read-only attribute provides the column names of the last query. To + remain compatible with the Python DB API, it returns a 7-tuple for each + column where the last six items of each tuple are :const:`None`. - This read-only attribute provides the column names of the last query. To - remain compatible with the Python DB API, it returns a 7-tuple for each - column where the last six items of each tuple are :const:`None`. - - It is set for ``SELECT`` statements without any matching rows as well. + It is set for ``SELECT`` statements without any matching rows as well. .. _sqlite3-row-objects: @@ -597,7 +617,7 @@ Row Objects .. method:: keys - This method returns a tuple of column names. Immediately after a query, + This method returns a list of column names. Immediately after a query, it is the first member of each tuple in :attr:`Cursor.description`. .. versionadded:: 2.6 @@ -633,7 +653,8 @@ Now we plug :class:`Row` in:: ['date', 'trans', 'symbol', 'qty', 'price'] >>> r['qty'] 100.0 - >>> for member in r: print member + >>> for member in r: + ... print member ... 2006-01-05 BUY @@ -706,9 +727,6 @@ use other Python types with SQLite, you must **adapt** them to one of the sqlite3 module's supported types for SQLite: one of NoneType, int, long, float, str, unicode, buffer. -The :mod:`sqlite3` module uses Python object adaptation, as described in -:pep:`246` for this. The protocol to use is :class:`PrepareProtocol`. - There are two ways to enable the :mod:`sqlite3` module to adapt a custom Python type to one of the supported ones. @@ -811,6 +829,10 @@ The following example demonstrates this. .. literalinclude:: ../includes/sqlite3/pysqlite_datetime.py +If a timestamp stored in SQLite has a fractional part longer than 6 +numbers, its value will be truncated to microsecond precision by the +timestamp converter. + .. _sqlite3-controlling-transactions: diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 8782439..c115976 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -30,6 +30,18 @@ probably additional platforms, as long as OpenSSL is installed on that platform. operating system socket APIs. The installed version of OpenSSL may also cause variations in behavior. +.. warning:: + The ssl module won't validate certificates by default. When used in + client mode, this means you are vulnerable to man-in-the-middle attacks. + +.. warning:: + + OpenSSL's internal random number generator does not properly handle fork. + Applications must change the PRNG state of the parent process if they use + any SSL feature with :func:`os.fork`. Any successful call of + :func:`~ssl.RAND_add`, :func:`~ssl.RAND_bytes` or + :func:`~ssl.RAND_pseudo_bytes` is sufficient. + This section documents the objects and functions in the ``ssl`` module; for more general information about TLS, SSL, and certificates, the reader is referred to the documents in the "See Also" section at the bottom. @@ -57,13 +69,16 @@ Functions, Constants, and Exceptions Takes an instance ``sock`` of :class:`socket.socket`, and returns an instance of :class:`ssl.SSLSocket`, a subtype of :class:`socket.socket`, which wraps - the underlying socket in an SSL context. For client-side sockets, the - context construction is lazy; if the underlying socket isn't connected yet, - the context construction will be performed after :meth:`connect` is called on - the socket. For server-side sockets, if the socket has no remote peer, it is - assumed to be a listening socket, and the server-side SSL wrapping is - automatically performed on client connections accepted via the :meth:`accept` - method. :func:`wrap_socket` may raise :exc:`SSLError`. + the underlying socket in an SSL context. ``sock`` must be a + :data:`~socket.SOCK_STREAM` socket; other socket types are unsupported. + + For client-side sockets, the context construction is lazy; if the + underlying socket isn't connected yet, the context construction will be + performed after :meth:`connect` is called on the socket. For + server-side sockets, if the socket has no remote peer, it is assumed + to be a listening socket, and the server-side SSL wrapping is + automatically performed on client connections accepted via the + :meth:`accept` method. :func:`wrap_socket` may raise :exc:`SSLError`. The ``keyfile`` and ``certfile`` parameters specify optional files which contain a certificate to be used to identify the local side of the @@ -154,7 +169,7 @@ Functions, Constants, and Exceptions .. function:: RAND_status() - Returns True if the SSL pseudo-random number generator has been seeded with + Returns ``True`` if the SSL pseudo-random number generator has been seeded with 'enough' randomness, and False otherwise. You can use :func:`ssl.RAND_egd` and :func:`ssl.RAND_add` to increase the randomness of the pseudo-random number generator. @@ -298,21 +313,37 @@ Functions, Constants, and Exceptions SSLSocket Objects ----------------- -.. method:: SSLSocket.read([nbytes=1024]) - - Reads up to ``nbytes`` bytes from the SSL-encrypted channel and returns them. - -.. method:: SSLSocket.write(data) - - Writes the ``data`` to the other side of the connection, using the SSL - channel to encrypt. Returns the number of bytes written. +SSL sockets provide the following methods of :ref:`socket-objects`: + +- :meth:`~socket.socket.accept()` +- :meth:`~socket.socket.bind()` +- :meth:`~socket.socket.close()` +- :meth:`~socket.socket.connect()` +- :meth:`~socket.socket.fileno()` +- :meth:`~socket.socket.getpeername()`, :meth:`~socket.socket.getsockname()` +- :meth:`~socket.socket.getsockopt()`, :meth:`~socket.socket.setsockopt()` +- :meth:`~socket.socket.gettimeout()`, :meth:`~socket.socket.settimeout()`, + :meth:`~socket.socket.setblocking()` +- :meth:`~socket.socket.listen()` +- :meth:`~socket.socket.makefile()` +- :meth:`~socket.socket.recv()`, :meth:`~socket.socket.recv_into()` + (but passing a non-zero ``flags`` argument is not allowed) +- :meth:`~socket.socket.send()`, :meth:`~socket.socket.sendall()` (with + the same limitation) +- :meth:`~socket.socket.shutdown()` + +However, since the SSL (and TLS) protocol has its own framing atop +of TCP, the SSL sockets abstraction can, in certain respects, diverge from +the specification of normal, OS-level sockets. + +SSL sockets also have the following additional methods and attributes: .. method:: SSLSocket.getpeercert(binary_form=False) If there is no certificate for the peer on the other end of the connection, returns ``None``. - If the parameter ``binary_form`` is :const:`False`, and a certificate was + If the ``binary_form`` parameter is :const:`False`, and a certificate was received from the peer, this method returns a :class:`dict` instance. If the certificate was not validated, the dict is empty. If the certificate was validated, it returns a dict with the keys ``subject`` (the principal for @@ -338,10 +369,16 @@ SSLSocket Objects If the ``binary_form`` parameter is :const:`True`, and a certificate was provided, this method returns the DER-encoded form of the entire certificate as a sequence of bytes, or :const:`None` if the peer did not provide a - certificate. This return value is independent of validation; if validation - was required (:const:`CERT_OPTIONAL` or :const:`CERT_REQUIRED`), it will have - been validated, but if :const:`CERT_NONE` was used to establish the - connection, the certificate, if present, will not have been validated. + certificate. Whether the peer provides a certificate depends on the SSL + socket's role: + + * for a client SSL socket, the server will always provide a certificate, + regardless of whether validation was required; + + * for a server SSL socket, the client will only provide a certificate + when requested by the server; therefore :meth:`getpeercert` will return + :const:`None` if you used :const:`CERT_NONE` (rather than + :const:`CERT_OPTIONAL` or :const:`CERT_REQUIRED`). .. method:: SSLSocket.cipher() @@ -361,7 +398,7 @@ SSLSocket Objects try: s.do_handshake() break - except ssl.SSLError, err: + except ssl.SSLError as err: if err.args[0] == ssl.SSL_ERROR_WANT_READ: select.select([s], [], []) elif err.args[0] == ssl.SSL_ERROR_WANT_WRITE: @@ -453,8 +490,7 @@ these chains concatenated together. For validation, Python will use the first chain it finds in the file which matches. Some "standard" root certificates are available from various certification -authorities: `CACert.org <http://www.cacert.org/index.php?id=3>`_, `Thawte -<http://www.thawte.com/roots/>`_, `Verisign +authorities: `Thawte <http://www.thawte.com/roots/>`_, `Verisign <http://www.verisign.com/support/roots.html>`_, `Positive SSL <http://www.PositiveSSL.com/ssl-certificate-support/cert_installation/UTN-USERFirst-Hardware.crt>`_ (used by python.org), `Equifax and GeoTrust @@ -619,10 +655,10 @@ And go back to listening for new client connections. .. seealso:: Class :class:`socket.socket` - Documentation of underlying :mod:`socket` class + Documentation of underlying :mod:`socket` class - `TLS (Transport Layer Security) and SSL (Secure Socket Layer) <http://www3.rad.com/networks/applications/secure/tls.htm>`_ - Debby Koren + `SSL/TLS Strong Encryption: An Introduction <http://httpd.apache.org/docs/trunk/en/ssl/ssl_intro.html>`_ + Intro from the Apache webserver documentation `RFC 1422: Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management <http://www.ietf.org/rfc/rfc1422>`_ Steve Kent diff --git a/Doc/library/stat.rst b/Doc/library/stat.rst index 6b0e1b4..a8f411a 100644 --- a/Doc/library/stat.rst +++ b/Doc/library/stat.rst @@ -1,5 +1,5 @@ -:mod:`stat` --- Interpreting :func:`stat` results -================================================= +:mod:`stat` --- Interpreting :func:`~os.stat` results +===================================================== .. module:: stat :synopsis: Utilities for interpreting the results of os.stat(), os.lstat() and os.fstat(). @@ -171,10 +171,6 @@ The variables below define the flags used in the :data:`ST_MODE` field. Use of the functions above is more portable than use of the first set of flags: -.. data:: S_IFMT - - Bit mask for the file type bit fields. - .. data:: S_IFSOCK Socket. diff --git a/Doc/library/statvfs.rst b/Doc/library/statvfs.rst index 748b7f9..6f44b2c 100644 --- a/Doc/library/statvfs.rst +++ b/Doc/library/statvfs.rst @@ -6,7 +6,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`statvfs` module has been deprecated for removal in Python 3.0. + The :mod:`statvfs` module has been removed in Python 3. .. sectionauthor:: Moshe Zadka <moshez@zadka.site.co.il> diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index e560360..f229967 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -26,7 +26,7 @@ instances and exceptions. Some operations are supported by several object types; in particular, practically all objects can be compared, tested for truth value, and converted -to a string (with the :func:`repr` function or the slightly different +to a string (with the :ref:`repr() <func-repr>` function or the slightly different :func:`str` function). The latter function is implicitly used when an object is written by the :func:`print` function. @@ -189,11 +189,22 @@ such objects are ordered arbitrarily but consistently. The ``<``, ``<=``, ``>`` and ``>=`` operators will raise a :exc:`TypeError` exception when any operand is a complex number. -.. index:: single: __cmp__() (instance method) - -Instances of a class normally compare as non-equal unless the class defines the -:meth:`__cmp__` method. Refer to :ref:`customization`) for information on the -use of this method to effect object comparisons. +.. index:: + single: __cmp__() (instance method) + single: __eq__() (instance method) + single: __ne__() (instance method) + single: __lt__() (instance method) + single: __le__() (instance method) + single: __gt__() (instance method) + single: __ge__() (instance method) + +Non-identical instances of a class normally compare as non-equal unless the +class defines the :meth:`__eq__` method or the :meth:`__cmp__` method. + +Instances of a class cannot be ordered with respect to other instances of the +same class, or other types of object, unless the class defines either enough of +the rich comparison methods (:meth:`__lt__`, :meth:`__le__`, :meth:`__gt__`, and +:meth:`__ge__`) or the :meth:`__cmp__` method. .. impl-detail:: @@ -388,8 +399,8 @@ All :class:`numbers.Real` types (:class:`int`, :class:`long`, and | ``math.trunc(x)`` | *x* truncated to Integral | | +--------------------+------------------------------------+--------+ | ``round(x[, n])`` | *x* rounded to n digits, | | -| | rounding half to even. If n is | | -| | omitted, it defaults to 0. | | +| | rounding ties away from zero. If n | | +| | is omitted, it defaults to 0. | | +--------------------+------------------------------------+--------+ | ``math.floor(x)`` | the greatest integral float <= *x* | | +--------------------+------------------------------------+--------+ @@ -614,7 +625,7 @@ support: iterators for those iteration types. (An example of an object supporting multiple forms of iteration would be a tree structure which supports both breadth-first and depth-first traversal.) This method corresponds to the - :attr:`tp_iter` slot of the type structure for Python objects in the Python/C + :c:member:`~PyTypeObject.tp_iter` slot of the type structure for Python objects in the Python/C API. The iterator objects themselves are required to support the following two @@ -625,7 +636,7 @@ methods, which together form the :dfn:`iterator protocol`: Return the iterator object itself. This is required to allow both containers and iterators to be used with the :keyword:`for` and :keyword:`in` statements. - This method corresponds to the :attr:`tp_iter` slot of the type structure for + This method corresponds to the :c:member:`~PyTypeObject.tp_iter` slot of the type structure for Python objects in the Python/C API. @@ -633,7 +644,7 @@ methods, which together form the :dfn:`iterator protocol`: Return the next item from the container. If there are no further items, raise the :exc:`StopIteration` exception. This method corresponds to the - :attr:`tp_iternext` slot of the type structure for Python objects in the + :c:member:`~PyTypeObject.tp_iternext` slot of the type structure for Python objects in the Python/C API. Python defines several iterator objects to support iteration over general and @@ -743,11 +754,11 @@ are sequences of the same type; *n*, *i* and *j* are integers: +------------------+--------------------------------+----------+ | ``max(s)`` | largest item of *s* | | +------------------+--------------------------------+----------+ -| ``s.index(i)`` | index of the first occurence | | -| | of *i* in *s* | | +| ``s.index(x)`` | index of the first occurrence | | +| | of *x* in *s* | | +------------------+--------------------------------+----------+ -| ``s.count(i)`` | total number of occurences of | | -| | *i* in *s* | | +| ``s.count(x)`` | total number of occurrences of | | +| | *x* in *s* | | +------------------+--------------------------------+----------+ Sequence types also support comparisons. In particular, tuples and lists @@ -931,10 +942,22 @@ string functions based on regular expressions. .. method:: str.expandtabs([tabsize]) Return a copy of the string where all tab characters are replaced by one or - more spaces, depending on the current column and the given tab size. The - column number is reset to zero after each newline occurring in the string. - If *tabsize* is not given, a tab size of ``8`` characters is assumed. This - doesn't understand other non-printing characters or escape sequences. + more spaces, depending on the current column and the given tab size. Tab + positions occur every *tabsize* characters (default is 8, giving tab + positions at columns 0, 8, 16 and so on). To expand the string, the current + column is set to zero and the string is examined character by character. If + the character is a tab (``\t``), one or more space characters are inserted + in the result until the current column is equal to the next tab position. + (The tab character itself is not copied.) If the character is a newline + (``\n``) or return (``\r``), it is copied and the current column is reset to + zero. Any other character is copied unchanged and the current column is + incremented by one regardless of how the character is represented when + printed. + + >>> '01\t012\t0123\t01234'.expandtabs() + '01 012 0123 01234' + >>> '01\t012\t0123\t01234'.expandtabs(4) + '01 012 0123 01234' .. method:: str.find(sub[, start[, end]]) @@ -969,7 +992,7 @@ string functions based on regular expressions. See :ref:`formatstrings` for a description of the various formatting options that can be specified in format strings. - This method of string formatting is the new standard in Python 3.0, and + This method of string formatting is the new standard in Python 3, and should be preferred to the ``%`` formatting described in :ref:`string-formatting` in new code. @@ -1161,8 +1184,8 @@ string functions based on regular expressions. Return a list of the words in the string, using *sep* as the delimiter string. If *maxsplit* is given, at most *maxsplit* splits are done (thus, the list will have at most ``maxsplit+1`` elements). If *maxsplit* is not - specified, then there is no limit on the number of splits (all possible - splits are made). + specified or ``-1``, then there is no limit on the number of splits + (all possible splits are made). If *sep* is given, consecutive delimiters are not grouped together and are deemed to delimit empty strings (for example, ``'1,,2'.split(',')`` returns @@ -1181,11 +1204,23 @@ string functions based on regular expressions. ``' 1 2 3 '.split(None, 1)`` returns ``['1', '2 3 ']``. +.. index:: + single: universal newlines; str.splitlines method + .. method:: str.splitlines([keepends]) - Return a list of the lines in the string, breaking at line boundaries. Line - breaks are not included in the resulting list unless *keepends* is given and - true. + Return a list of the lines in the string, breaking at line boundaries. + This method uses the :term:`universal newlines` approach to splitting lines. + Line breaks are not included in the resulting list unless *keepends* is + given and true. + + For example, ``'ab c\n\nde fg\rkl\r\n'.splitlines()`` returns + ``['ab c', '', 'de fg', 'kl']``, while the same call with ``splitlines(True)`` + returns ``['ab c\n', '\n', 'de fg\r', 'kl\r\n']``. + + Unlike :meth:`~str.split` when a delimiter string *sep* is given, this + method returns an empty list for the empty string, and a terminal line + break does not result in an extra line. .. method:: str.startswith(prefix[, start[, end]]) @@ -1241,11 +1276,11 @@ string functions based on regular expressions. >>> import re >>> def titlecase(s): - return re.sub(r"[A-Za-z]+('[A-Za-z]+)?", - lambda mo: mo.group(0)[0].upper() + - mo.group(0)[1:].lower(), - s) - + ... return re.sub(r"[A-Za-z]+('[A-Za-z]+)?", + ... lambda mo: mo.group(0)[0].upper() + + ... mo.group(0)[1:].lower(), + ... s) + ... >>> titlecase("they're bill's friends.") "They're Bill's Friends." @@ -1440,7 +1475,7 @@ The conversion types are: | | character string). | | +------------+-----------------------------------------------------+-------+ | ``'r'`` | String (converts any Python object using | \(5) | -| | :func:`repr`). | | +| | :ref:`repr() <func-repr>`). | | +------------+-----------------------------------------------------+-------+ | ``'s'`` | String (converts any Python object using | \(6) | | | :func:`str`). | | @@ -1633,9 +1668,8 @@ Notes: Previously, all negative indices were truncated to zero. (6) - The :meth:`pop` method is only supported by the list and array types. The - optional argument *i* defaults to ``-1``, so that by default the last item is - removed and returned. + The :meth:`pop` method's optional argument *i* defaults to ``-1``, so that + by default the last item is removed and returned. (7) The :meth:`sort` and :meth:`reverse` methods modify the list in place for @@ -1710,11 +1744,11 @@ other sequence-like behavior. There are currently two built-in set types, :class:`set` and :class:`frozenset`. The :class:`set` type is mutable --- the contents can be changed using methods -like :meth:`add` and :meth:`remove`. Since it is mutable, it has no hash value -and cannot be used as either a dictionary key or as an element of another set. -The :class:`frozenset` type is immutable and :term:`hashable` --- its contents -cannot be altered after it is created; it can therefore be used as a dictionary -key or as an element of another set. +like :meth:`~set.add` and :meth:`~set.remove`. Since it is mutable, it has no +hash value and cannot be used as either a dictionary key or as an element of +another set. The :class:`frozenset` type is immutable and :term:`hashable` --- +its contents cannot be altered after it is created; it can therefore be used as +a dictionary key or as an element of another set. As of Python 2.7, non-empty sets (not frozensets) can be created by placing a comma-separated list of elements within braces, for example: ``{'jack', @@ -1726,9 +1760,10 @@ The constructors for both classes work the same: frozenset([iterable]) Return a new set or frozenset object whose elements are taken from - *iterable*. The elements of a set must be hashable. To represent sets of - sets, the inner sets must be :class:`frozenset` objects. If *iterable* is - not specified, a new empty set is returned. + *iterable*. The elements of a set must be :term:`hashable`. To + represent sets of sets, the inner sets must be :class:`frozenset` + objects. If *iterable* is not specified, a new empty set is + returned. Instances of :class:`set` and :class:`frozenset` provide the following operations: @@ -1747,7 +1782,7 @@ The constructors for both classes work the same: .. method:: isdisjoint(other) - Return True if the set has no elements in common with *other*. Sets are + Return ``True`` if the set has no elements in common with *other*. Sets are disjoint if and only if their intersection is the empty set. .. versionadded:: 2.6 @@ -1759,7 +1794,7 @@ The constructors for both classes work the same: .. method:: set < other - Test whether the set is a true subset of *other*, that is, + Test whether the set is a proper subset of *other*, that is, ``set <= other and set != other``. .. method:: issuperset(other) @@ -1769,7 +1804,7 @@ The constructors for both classes work the same: .. method:: set > other - Test whether the set is a true superset of *other*, that is, ``set >= + Test whether the set is a proper superset of *other*, that is, ``set >= other and set != other``. .. method:: union(other, ...) @@ -1824,8 +1859,8 @@ The constructors for both classes work the same: based on their members. For example, ``set('abc') == frozenset('abc')`` returns ``True`` and so does ``set('abc') in set([frozenset('abc')])``. - The subset and equality comparisons do not generalize to a complete ordering - function. For example, any two disjoint sets are not equal and are not + The subset and equality comparisons do not generalize to a total ordering + function. For example, any two non-empty disjoint sets are not equal and are not subsets of each other, so *all* of the following return ``False``: ``a<b``, ``a==b``, or ``a>b``. Accordingly, sets do not implement the :meth:`__cmp__` method. @@ -1925,7 +1960,7 @@ Mapping Types --- :class:`dict` statement: del builtin: len -A :dfn:`mapping` object maps :term:`hashable` values to arbitrary objects. +A :term:`mapping` object maps :term:`hashable` values to arbitrary objects. Mappings are mutable objects. There is currently only one standard mapping type, the :dfn:`dictionary`. (For other containers see the built in :class:`list`, :class:`set`, and :class:`tuple` classes, and the @@ -1944,32 +1979,41 @@ Dictionaries can be created by placing a comma-separated list of ``key: value`` pairs within braces, for example: ``{'jack': 4098, 'sjoerd': 4127}`` or ``{4098: 'jack', 4127: 'sjoerd'}``, or by the :class:`dict` constructor. -.. class:: dict([arg]) - - Return a new dictionary initialized from an optional positional argument or from - a set of keyword arguments. If no arguments are given, return a new empty - dictionary. If the positional argument *arg* is a mapping object, return a - dictionary mapping the same keys to the same values as does the mapping object. - Otherwise the positional argument must be a sequence, a container that supports - iteration, or an iterator object. The elements of the argument must each also - be of one of those kinds, and each must in turn contain exactly two objects. - The first is used as a key in the new dictionary, and the second as the key's - value. If a given key is seen more than once, the last value associated with it - is retained in the new dictionary. - - If keyword arguments are given, the keywords themselves with their associated - values are added as items to the dictionary. If a key is specified both in the - positional argument and as a keyword argument, the value associated with the - keyword is retained in the dictionary. For example, these all return a - dictionary equal to ``{"one": 1, "two": 2}``: - - * ``dict(one=1, two=2)`` - * ``dict({'one': 1, 'two': 2})`` - * ``dict(zip(('one', 'two'), (1, 2)))`` - * ``dict([['two', 2], ['one', 1]])`` - - The first example only works for keys that are valid Python - identifiers; the others work with any valid keys. +.. class:: dict(**kwarg) + dict(mapping, **kwarg) + dict(iterable, **kwarg) + + Return a new dictionary initialized from an optional positional argument + and a possibly empty set of keyword arguments. + + If no positional argument is given, an empty dictionary is created. + If a positional argument is given and it is a mapping object, a dictionary + is created with the same key-value pairs as the mapping object. Otherwise, + the positional argument must be an :term:`iterable` object. Each item in + the iterable must itself be an iterable with exactly two objects. The + first object of each item becomes a key in the new dictionary, and the + second object the corresponding value. If a key occurs more than once, the + last value for that key becomes the corresponding value in the new + dictionary. + + If keyword arguments are given, the keyword arguments and their values are + added to the dictionary created from the positional argument. If a key + being added is already present, the value from the keyword argument + replaces the value from the positional argument. + + To illustrate, the following examples all return a dictionary equal to + ``{"one": 1, "two": 2, "three": 3}``:: + + >>> a = dict(one=1, two=2, three=3) + >>> b = {'one': 1, 'two': 2, 'three': 3} + >>> c = dict(zip(['one', 'two', 'three'], [1, 2, 3])) + >>> d = dict([('two', 2), ('one', 1), ('three', 3)]) + >>> e = dict({'three': 3, 'one': 1, 'two': 2}) + >>> a == b == c == d == e + True + + Providing keyword arguments as in the first example only works for keys that + are valid Python identifiers. Otherwise, any valid keys can be used. .. versionadded:: 2.2 @@ -2309,7 +2353,7 @@ Files have the following methods: with open("hello.txt") as f: for line in f: - print line + print line, In older versions of Python, you would have needed to do this to get the same effect:: @@ -2317,7 +2361,7 @@ Files have the following methods: f = open("hello.txt") try: for line in f: - print line + print line, finally: f.close() @@ -2371,14 +2415,14 @@ Files have the following methods: A file object is its own iterator, for example ``iter(f)`` returns *f* (unless *f* is closed). When a file is used as an iterator, typically in a - :keyword:`for` loop (for example, ``for line in f: print line``), the + :keyword:`for` loop (for example, ``for line in f: print line.strip()``), the :meth:`~file.next` method is called repeatedly. This method returns the next input line, or raises :exc:`StopIteration` when EOF is hit when the file is open for reading (behavior is undefined when the file is open for writing). In order to make a :keyword:`for` loop the most efficient way of looping over the lines of a file (a very common operation), the :meth:`~file.next` method uses a hidden read-ahead buffer. As a consequence of using a read-ahead buffer, combining :meth:`~file.next` - with other file methods (like :meth:`readline`) does not work right. However, + with other file methods (like :meth:`~file.readline`) does not work right. However, using :meth:`seek` to reposition the file to an absolute position will flush the read-ahead buffer. @@ -2420,7 +2464,7 @@ Files have the following methods: .. method:: file.readlines([sizehint]) - Read until EOF using :meth:`readline` and return a list containing the lines + Read until EOF using :meth:`~file.readline` and return a list containing the lines thus read. If the optional *sizehint* argument is present, instead of reading up to EOF, whole lines totalling approximately *sizehint* bytes (possibly after rounding up to an internal buffer size) are read. Objects @@ -2500,7 +2544,7 @@ Files have the following methods: add line separators.) Files support the iterator protocol. Each iteration returns the same result as -``file.readline()``, and iteration ends when the :meth:`readline` method returns +:meth:`~file.readline`, and iteration ends when the :meth:`~file.readline` method returns an empty string. File objects also offer a number of other interesting attributes. These are not @@ -2549,16 +2593,19 @@ the particular object. form ``<...>``. This is a read-only attribute and may not be present on all file-like objects. + .. index:: + single: universal newlines; file.newlines attribute + .. attribute:: file.newlines - If Python was built with universal newlines enabled (the default) this + If Python was built with :term:`universal newlines` enabled (the default) this read-only attribute exists, and for files opened in universal newline read mode it keeps track of the types of newlines encountered while reading the file. The values it can take are ``'\r'``, ``'\n'``, ``'\r\n'``, ``None`` (unknown, no newlines read yet) or a tuple containing all the newline types seen, to indicate that multiple newline conventions were encountered. For - files not opened in universal newline read mode the value of this attribute + files not opened in universal newlines read mode the value of this attribute will be ``None``. @@ -2791,12 +2838,12 @@ statement is not, strictly speaking, an operation on a module object; ``import foo`` does not require a module object named *foo* to exist, rather it requires an (external) *definition* for a module named *foo* somewhere.) -A special attribute of every module is :attr:`__dict__`. This is the dictionary -containing the module's symbol table. Modifying this dictionary will actually -change the module's symbol table, but direct assignment to the :attr:`__dict__` -attribute is not possible (you can write ``m.__dict__['a'] = 1``, which defines -``m.a`` to be ``1``, but you can't write ``m.__dict__ = {}``). Modifying -:attr:`__dict__` directly is not recommended. +A special attribute of every module is :attr:`~object.__dict__`. This is the +dictionary containing the module's symbol table. Modifying this dictionary will +actually change the module's symbol table, but direct assignment to the +:attr:`__dict__` attribute is not possible (you can write +``m.__dict__['a'] = 1``, which defines ``m.a`` to be ``1``, but you can't write +``m.__dict__ = {}``). Modifying :attr:`__dict__` directly is not recommended. Modules built into the interpreter are written like this: ``<module 'sys' (built-in)>``. If loaded from a file, they are written as ``<module 'os' from @@ -2854,16 +2901,23 @@ that class), otherwise a :exc:`TypeError` is raised. Like function objects, methods objects support getting arbitrary attributes. However, since method attributes are actually stored on the underlying function object (``meth.im_func``), setting method attributes on either bound or unbound -methods is disallowed. Attempting to set a method attribute results in a -:exc:`TypeError` being raised. In order to set a method attribute, you need to -explicitly set it on the underlying function object:: - - class C: - def method(self): - pass +methods is disallowed. Attempting to set an attribute on a method results in +an :exc:`AttributeError` being raised. In order to set a method attribute, you +need to explicitly set it on the underlying function object:: + + >>> class C: + ... def method(self): + ... pass + ... + >>> c = C() + >>> c.method.whoami = 'my name is method' # can't set on the method + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + AttributeError: 'instancemethod' object has no attribute 'whoami' + >>> c.method.im_func.whoami = 'my name is method' + >>> c.method.whoami + 'my name is method' - c = C() - c.method.im_func.whoami = 'my name is c' See :ref:`types` for more information. @@ -3033,7 +3087,7 @@ The following attributes are only supported by :term:`new-style class`\ es. This method can be overridden by a metaclass to customize the method resolution order for its instances. It is called at class instantiation, and - its result is stored in :attr:`__mro__`. + its result is stored in :attr:`~class.__mro__`. .. method:: class.__subclasses__ diff --git a/Doc/library/string.rst b/Doc/library/string.rst index 4c3abcd..b0ffb6a 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -123,8 +123,8 @@ string formatting behaviors using the same implementation as the built-in .. method:: format(format_string, *args, **kwargs) - :meth:`format` is the primary API method. It takes a format template - string, and an arbitrary set of positional and keyword argument. + :meth:`format` is the primary API method. It takes a format string and + an arbitrary set of positional and keyword arguments. :meth:`format` is just a wrapper that calls :meth:`vformat`. .. method:: vformat(format_string, args, kwargs) @@ -132,9 +132,9 @@ string formatting behaviors using the same implementation as the built-in This function does the actual work of formatting. It is exposed as a separate function for cases where you want to pass in a predefined dictionary of arguments, rather than unpacking and repacking the - dictionary as individual arguments using the ``*args`` and ``**kwds`` - syntax. :meth:`vformat` does the work of breaking up the format template - string into character data and replacement fields. It calls the various + dictionary as individual arguments using the ``*args`` and ``**kwargs`` + syntax. :meth:`vformat` does the work of breaking up the format string + into character data and replacement fields. It calls the various methods described below. In addition, the :class:`Formatter` defines a number of methods that are @@ -205,7 +205,8 @@ string formatting behaviors using the same implementation as the built-in Converts the value (returned by :meth:`get_field`) given a conversion type (as in the tuple returned by the :meth:`parse` method). The default - version understands 'r' (repr) and 's' (str) conversion types. + version understands 's' (str), 'r' (repr) and 'a' (ascii) conversion + types. .. _formatstrings: @@ -322,18 +323,18 @@ The general form of a *standard format specifier* is: .. productionlist:: sf format_spec: [[`fill`]`align`][`sign`][#][0][`width`][,][.`precision`][`type`] - fill: <a character other than '}'> + fill: <any character> align: "<" | ">" | "=" | "^" sign: "+" | "-" | " " width: `integer` precision: `integer` type: "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%" -The *fill* character can be any character other than '{' or '}'. The presence -of a fill character is signaled by the character following it, which must be -one of the alignment options. If the second character of *format_spec* is not -a valid alignment option, then it is assumed that both the fill character and -the alignment option are absent. +If a valid *align* value is specified, it can be preceded by a *fill* +character that can be any character and defaults to a space if omitted. +Note that it is not possible to use ``{`` and ``}`` as *fill* char while +using the :meth:`str.format` method; this limitation however doesn't +affect the :func:`format` function. The meaning of the various alignment options is as follows: @@ -389,9 +390,9 @@ instead. *width* is a decimal integer defining the minimum field width. If not specified, then the field width will be determined by the content. -If the *width* field is preceded by a zero (``'0'``) character, this enables -zero-padding. This is equivalent to an *alignment* type of ``'='`` and a *fill* -character of ``'0'``. +Preceding the *width* field by a zero (``'0'``) character enables +sign-aware zero-padding for numeric types. This is equivalent to a *fill* +character of ``'0'`` with an *alignment* type of ``'='``. The *precision* is a decimal number indicating how many digits should be displayed after the decimal point for a floating point value formatted with @@ -452,12 +453,13 @@ The available presentation types for floating point and decimal values are: +=========+==========================================================+ | ``'e'`` | Exponent notation. Prints the number in scientific | | | notation using the letter 'e' to indicate the exponent. | + | | The default precision is ``6``. | +---------+----------------------------------------------------------+ | ``'E'`` | Exponent notation. Same as ``'e'`` except it uses an | | | upper case 'E' as the separator character. | +---------+----------------------------------------------------------+ | ``'f'`` | Fixed point. Displays the number as a fixed-point | - | | number. | + | | number. The default precision is ``6``. | +---------+----------------------------------------------------------+ | ``'F'`` | Fixed point. Same as ``'f'``. | +---------+----------------------------------------------------------+ @@ -483,7 +485,7 @@ The available presentation types for floating point and decimal values are: | | the precision. | | | | | | A precision of ``0`` is treated as equivalent to a | - | | precision of ``1``. | + | | precision of ``1``. The default precision is ``6``. | +---------+----------------------------------------------------------+ | ``'G'`` | General format. Same as ``'g'`` except switches to | | | ``'E'`` if the number gets too large. The | @@ -706,7 +708,7 @@ these rules. The methods of :class:`Template` are: This is the object passed to the constructor's *template* argument. In general, you shouldn't change it, but read-only access is not enforced. -Here is an example of how to use a Template: +Here is an example of how to use a Template:: >>> from string import Template >>> s = Template('$who likes $what') @@ -715,11 +717,11 @@ Here is an example of how to use a Template: >>> d = dict(who='tim') >>> Template('Give $who $100').substitute(d) Traceback (most recent call last): - [...] - ValueError: Invalid placeholder in string: line 1, col 10 + ... + ValueError: Invalid placeholder in string: line 1, col 11 >>> Template('$who likes $what').substitute(d) Traceback (most recent call last): - [...] + ... KeyError: 'what' >>> Template('$who likes $what').safe_substitute(d) 'tim likes $what' @@ -793,7 +795,7 @@ Deprecated string functions The following list of functions are also defined as methods of string and Unicode objects; see section :ref:`string-methods` for more information on those. You should consider these functions as deprecated, although they will -not be removed until Python 3.0. The functions defined in this module are: +not be removed until Python 3. The functions defined in this module are: .. function:: atof(s) @@ -905,14 +907,15 @@ not be removed until Python 3.0. The functions defined in this module are: Return a list of the words of the string *s*. If the optional second argument *sep* is absent or ``None``, the words are separated by arbitrary strings of - whitespace characters (space, tab, newline, return, formfeed). If the second + whitespace characters (space, tab, newline, return, formfeed). If the second argument *sep* is present and not ``None``, it specifies a string to be used as the word separator. The returned list will then have one more item than the - number of non-overlapping occurrences of the separator in the string. The - optional third argument *maxsplit* defaults to 0. If it is nonzero, at most - *maxsplit* number of splits occur, and the remainder of the string is returned - as the final element of the list (thus, the list will have at most - ``maxsplit+1`` elements). + number of non-overlapping occurrences of the separator in the string. + If *maxsplit* is given, at most *maxsplit* number of splits occur, and the + remainder of the string is returned as the final element of the list (thus, + the list will have at most ``maxsplit+1`` elements). If *maxsplit* is not + specified or ``-1``, then there is no limit on the number of splits (all + possible splits are made). The behavior of split on an empty string depends on the value of *sep*. If *sep* is not specified, or specified as ``None``, the result will be an empty list. @@ -925,7 +928,7 @@ not be removed until Python 3.0. The functions defined in this module are: Return a list of the words of the string *s*, scanning *s* from the end. To all intents and purposes, the resulting list of words is the same as returned by :func:`split`, except when the optional third argument *maxsplit* is explicitly - specified and nonzero. When *maxsplit* is nonzero, at most *maxsplit* number of + specified and nonzero. If *maxsplit* is given, at most *maxsplit* number of splits -- the *rightmost* ones -- occur, and the remainder of the string is returned as the first element of the list (thus, the list will have at most ``maxsplit+1`` elements). @@ -1023,13 +1026,14 @@ not be removed until Python 3.0. The functions defined in this module are: .. function:: zfill(s, width) - Pad a numeric string on the left with zero digits until the given width is - reached. Strings starting with a sign are handled correctly. + Pad a numeric string *s* on the left with zero digits until the + given *width* is reached. Strings starting with a sign are handled + correctly. -.. function:: replace(str, old, new[, maxreplace]) +.. function:: replace(s, old, new[, maxreplace]) - Return a copy of string *str* with all occurrences of substring *old* replaced + Return a copy of string *s* with all occurrences of substring *old* replaced by *new*. If the optional argument *maxreplace* is given, the first *maxreplace* occurrences are replaced. diff --git a/Doc/library/stringprep.rst b/Doc/library/stringprep.rst index d2f269c..b0944e4 100644 --- a/Doc/library/stringprep.rst +++ b/Doc/library/stringprep.rst @@ -4,7 +4,6 @@ .. module:: stringprep :synopsis: String preparation, as per RFC 3453 - :deprecated: .. moduleauthor:: Martin v. Löwis <martin@v.loewis.de> .. sectionauthor:: Martin v. Löwis <martin@v.loewis.de> diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 4331665..74e3206 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -284,7 +284,7 @@ platforms use 32-bit pointers and will use a Python integer. For the ``'?'`` format character, the return value is either :const:`True` or :const:`False`. When packing, the truth value of the argument object is used. Either 0 or 1 in the native or standard bool representation will be packed, and -any non-zero value will be True when unpacking. +any non-zero value will be ``True`` when unpacking. @@ -386,7 +386,7 @@ The :mod:`struct` module also defines the following type: (``len(string)`` must equal :attr:`self.size`). - .. method:: unpack_from(buffer[, offset=0]) + .. method:: unpack_from(buffer, offset=0) Identical to the :func:`unpack_from` function, using the compiled format. (``len(buffer[offset:])`` must be at least :attr:`self.size`). diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 2ba960a..114907f 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -12,7 +12,7 @@ The :mod:`subprocess` module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to -replace several other, older modules and functions, such as:: +replace several older modules and functions:: os.system os.spawn* @@ -20,20 +20,26 @@ replace several other, older modules and functions, such as:: popen2.* commands.* -Information about how the :mod:`subprocess` module can be used to replace these -modules and functions can be found in the following sections. +Information about how this module can be used to replace the older +functions can be found in the subprocess-replacements_ section. .. seealso:: + POSIX users (Linux, BSD, etc.) are strongly encouraged to install + and use the much more recent subprocess32_ module instead of the + version included with python 2.7. It is a drop in replacement with + better behavior in many situations. + :pep:`324` -- PEP proposing the subprocess module +.. _subprocess32: https://pypi.python.org/pypi/subprocess32/ -Using the subprocess Module ---------------------------- +Using the :mod:`subprocess` Module +---------------------------------- -The recommended approach to invoking subprocesses is to use the following -convenience functions for all use cases they can handle. For more advanced -use cases, the underlying :class:`Popen` interface can be used directly. +The recommended way to launch subprocesses is to use the following +convenience functions. For more advanced use cases when these do not +meet your needs, use the underlying :class:`Popen` interface. .. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False) @@ -57,16 +63,15 @@ use cases, the underlying :class:`Popen` interface can be used directly. .. warning:: - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. .. note:: - Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As - the pipes are not being read in the current process, the child - process may block if it generates enough output to a pipe to fill up - the OS pipe buffer. + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function + as that can deadlock based on the child process output volume. + Use :class:`Popen` with the :meth:`communicate` method when you + need pipes. .. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False) @@ -74,7 +79,7 @@ use cases, the underlying :class:`Popen` interface can be used directly. Run command with arguments. Wait for command to complete. If the return code was zero then return, otherwise raise :exc:`CalledProcessError`. The :exc:`CalledProcessError` object will have the return code in the - :attr:`returncode` attribute. + :attr:`~CalledProcessError.returncode` attribute. The arguments shown above are merely the most common ones, described below in :ref:`frequently-used-arguments` (hence the slightly odd notation in @@ -96,16 +101,15 @@ use cases, the underlying :class:`Popen` interface can be used directly. .. warning:: - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. .. note:: - Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As - the pipes are not being read in the current process, the child - process may block if it generates enough output to a pipe to fill up - the OS pipe buffer. + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function + as that can deadlock based on the child process output volume. + Use :class:`Popen` with the :meth:`communicate` method when you + need pipes. .. function:: check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False) @@ -114,8 +118,8 @@ use cases, the underlying :class:`Popen` interface can be used directly. If the return code was non-zero it raises a :exc:`CalledProcessError`. The :exc:`CalledProcessError` object will have the return code in the - :attr:`returncode` attribute and any output in the :attr:`output` - attribute. + :attr:`~CalledProcessError.returncode` attribute and any output in the + :attr:`~CalledProcessError.output` attribute. The arguments shown above are merely the most common ones, described below in :ref:`frequently-used-arguments` (hence the slightly odd notation in @@ -147,15 +151,14 @@ use cases, the underlying :class:`Popen` interface can be used directly. .. warning:: - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. .. note:: - Do not use ``stderr=PIPE`` with this function. As the pipe is not being - read in the current process, the child process may block if it - generates enough output to the pipe to fill up the OS pipe buffer. + Do not use ``stderr=PIPE`` with this function as that can deadlock + based on the child process error volume. Use :class:`Popen` with + the :meth:`communicate` method when you need a stderr pipe. .. data:: PIPE @@ -172,6 +175,26 @@ use cases, the underlying :class:`Popen` interface can be used directly. output. +.. exception:: CalledProcessError + + Exception raised when a process run by :func:`check_call` or + :func:`check_output` returns a non-zero exit status. + + .. attribute:: returncode + + Exit status of the child process. + + .. attribute:: cmd + + Command that was used to spawn the child process. + + .. attribute:: output + + Output of the child process if this exception is raised by + :func:`check_output`. Otherwise, ``None``. + + + .. _frequently-used-arguments: Frequently Used Arguments @@ -200,15 +223,22 @@ default values. The arguments that are most commonly needed are: the stderr data from the child process should be captured into the same file handle as for stdout. + .. index:: + single: universal newlines; subprocess module + When *stdout* or *stderr* are pipes and *universal_newlines* is - :const:`True` then all line endings will be converted to ``'\n'`` as - described for the universal newlines `'U'`` mode argument to :func:`open`. + ``True`` then all line endings will be converted to ``'\n'`` as described + for the :term:`universal newlines` ``'U'`` mode argument to :func:`open`. - If *shell* is :const:`True`, the specified command will be executed through - the shell. This can be useful if you are using Python primarily for the + If *shell* is ``True``, the specified command will be executed through + the shell. This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want - access to other shell features such as filename wildcards, shell pipes and - environment variable expansion. + convenient access to other shell features such as shell pipes, filename + wildcards, environment variable expansion, and expansion of ``~`` to a + user's home directory. However, note that Python itself offers + implementations of many shell-like features (in particular, :mod:`glob`, + :mod:`fnmatch`, :func:`os.walk`, :func:`os.path.expandvars`, + :func:`os.path.expanduser`, and :mod:`shutil`). .. warning:: @@ -216,8 +246,8 @@ default values. The arguments that are most commonly needed are: untrusted source makes a program vulnerable to `shell injection <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_, a serious security flaw which can result in arbitrary command execution. - For this reason, the use of *shell=True* is **strongly discouraged** in cases - where the command string is constructed from external input:: + For this reason, the use of ``shell=True`` is **strongly discouraged** + in cases where the command string is constructed from external input:: >>> from subprocess import call >>> filename = input("What file would you like to display?\n") @@ -229,6 +259,10 @@ default values. The arguments that are most commonly needed are: from this vulnerability; see the Note in the :class:`Popen` constructor documentation for helpful hints in getting ``shell=False`` to work. + When using ``shell=True``, :func:`pipes.quote` can be used to properly + escape whitespace and shell metacharacters in strings that are going to + be used to construct shell commands. + These options, along with all of the other options, are described in more detail in the :class:`Popen` constructor documentation. @@ -242,23 +276,26 @@ are able to handle the less common cases not covered by the convenience functions. -.. class:: Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0) +.. class:: Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, \ + stderr=None, preexec_fn=None, close_fds=False, shell=False, \ + cwd=None, env=None, universal_newlines=False, \ + startupinfo=None, creationflags=0) - Arguments are: + Execute a child program in a new process. On Unix, the class uses + :meth:`os.execvp`-like behavior to execute the child program. On Windows, + the class uses the Windows ``CreateProcess()`` function. The arguments to + :class:`Popen` are as follows. - *args* should be a string, or a sequence of program arguments. The program - to execute is normally the first item in the args sequence or the string if - a string is given, but can be explicitly set by using the *executable* - argument. When *executable* is given, the first item in the args sequence - is still treated by most programs as the command name, which can then be - different from the actual executable name. On Unix, it becomes the display - name for the executing program in utilities such as :program:`ps`. + *args* should be a sequence of program arguments or else a single string. + By default, the program to execute is the first item in *args* if *args* is + a sequence. If *args* is a string, the interpretation is + platform-dependent and described below. See the *shell* and *executable* + arguments for additional differences from the default behavior. Unless + otherwise stated, it is recommended to pass *args* as a sequence. - On Unix, with *shell=False* (default): In this case, the Popen class uses - :meth:`os.execvp` to execute the child program. *args* should normally be a - sequence. If a string is specified for *args*, it will be used as the name - or path of the program to execute; this will only work if the program is - being given no arguments. + On Unix, if *args* is a string, the string is interpreted as the name or + path of the program to execute. However, this can only be done if not + passing arguments to the program. .. note:: @@ -279,20 +316,36 @@ functions. used in the shell (such as filenames containing spaces or the *echo* command shown above) are single list elements. - On Unix, with *shell=True*: If args is a string, it specifies the command - string to execute through the shell. This means that the string must be + On Windows, if *args* is a sequence, it will be converted to a string in a + manner described in :ref:`converting-argument-sequence`. This is because + the underlying ``CreateProcess()`` operates on strings. + + The *shell* argument (which defaults to *False*) specifies whether to use + the shell as the program to execute. If *shell* is *True*, it is + recommended to pass *args* as a string rather than as a sequence. + + On Unix with ``shell=True``, the shell defaults to :file:`/bin/sh`. If + *args* is a string, the string specifies the command + to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If *args* is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell - itself. That is to say, *Popen* does the equivalent of:: + itself. That is to say, :class:`Popen` does the equivalent of:: Popen(['/bin/sh', '-c', args[0], args[1], ...]) - On Windows: the :class:`Popen` class uses CreateProcess() to execute the child - child program, which operates on strings. If *args* is a sequence, it will - be converted to a string in a manner described in - :ref:`converting-argument-sequence`. + On Windows with ``shell=True``, the :envvar:`COMSPEC` environment variable + specifies the default shell. The only time you need to specify + ``shell=True`` on Windows is when the command you wish to execute is built + into the shell (e.g. :command:`dir` or :command:`copy`). You do not need + ``shell=True`` to run a batch file or console-based executable. + + .. warning:: + + Passing ``shell=True`` can be a security hazard if combined with + untrusted input. See the warning under :ref:`frequently-used-arguments` + for details. *bufsize*, if given, has the same meaning as the corresponding argument to the built-in open() function: :const:`0` means unbuffered, :const:`1` means line @@ -306,15 +359,15 @@ functions. enable buffering by setting *bufsize* to either -1 or a large enough positive value (such as 4096). - The *executable* argument specifies the program to execute. It is very seldom - needed: Usually, the program to execute is defined by the *args* argument. If - ``shell=True``, the *executable* argument specifies which shell to use. On Unix, - the default shell is :file:`/bin/sh`. On Windows, the default shell is - specified by the :envvar:`COMSPEC` environment variable. The only reason you - would need to specify ``shell=True`` on Windows is where the command you - wish to execute is actually built in to the shell, eg ``dir``, ``copy``. - You don't need ``shell=True`` to run a batch file, nor to run a console-based - executable. + The *executable* argument specifies a replacement program to execute. It + is very seldom needed. When ``shell=False``, *executable* replaces the + program to execute specified by *args*. However, the original *args* is + still passed to the program. Most programs treat the program specified + by *args* as the command name, which can then be different from the program + actually executed. On Unix, the *args* name + becomes the display name for the executable in utilities such as + :program:`ps`. If ``shell=True``, on Unix the *executable* argument + specifies a replacement shell for the default :file:`/bin/sh`. *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values @@ -335,15 +388,6 @@ functions. child process. Note that on Windows, you cannot set *close_fds* to true and also redirect the standard handles by setting *stdin*, *stdout* or *stderr*. - If *shell* is :const:`True`, the specified command will be executed through the - shell. - - .. warning:: - - Enabling this option can be a security hazard if combined with untrusted - input. See the warning under :ref:`frequently-used-arguments` - for details. - If *cwd* is not ``None``, the child's current directory will be changed to *cwd* before it is executed. Note that this directory is not considered when searching the executable, so you can't specify the program's path relative to @@ -362,11 +406,11 @@ functions. .. _side-by-side assembly: http://en.wikipedia.org/wiki/Side-by-Side_Assembly - If *universal_newlines* is :const:`True`, the file objects stdout and stderr are - opened as text files, but lines may be terminated by any of ``'\n'``, the Unix - end-of-line convention, ``'\r'``, the old Macintosh convention or ``'\r\n'``, the - Windows convention. All of these external representations are seen as ``'\n'`` - by the Python program. + If *universal_newlines* is ``True``, the file objects *stdout* and *stderr* + are opened as text files in :term:`universal newlines` mode. Lines may be + terminated by any of ``'\n'``, the Unix end-of-line convention, ``'\r'``, + the old Macintosh convention or ``'\r\n'``, the Windows convention. All of + these external representations are seen as ``'\n'`` by the Python program. .. note:: @@ -419,14 +463,14 @@ Instances of the :class:`Popen` class have the following methods: .. method:: Popen.poll() - Check if child process has terminated. Set and return :attr:`returncode` - attribute. + Check if child process has terminated. Set and return + :attr:`~Popen.returncode` attribute. .. method:: Popen.wait() - Wait for child process to terminate. Set and return :attr:`returncode` - attribute. + Wait for child process to terminate. Set and return + :attr:`~Popen.returncode` attribute. .. warning:: @@ -490,8 +534,8 @@ The following attributes are also available: .. warning:: - Use :meth:`communicate` rather than :attr:`.stdin.write <stdin>`, - :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` to avoid + Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`, + :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process. @@ -639,8 +683,8 @@ The :mod:`subprocess` module exposes the following constants. .. _subprocess-replacements: -Replacing Older Functions with the subprocess Module ----------------------------------------------------- +Replacing Older Functions with the :mod:`subprocess` Module +----------------------------------------------------------- In this section, "a becomes b" means that b can be used as a replacement for a. @@ -652,11 +696,11 @@ In this section, "a becomes b" means that b can be used as a replacement for a. In addition, the replacements using :func:`check_output` will fail with a :exc:`CalledProcessError` if the requested operation produces a non-zero - return code. The output is still available as the ``output`` attribute of - the raised exception. + return code. The output is still available as the + :attr:`~CalledProcessError.output` attribute of the raised exception. In the following examples, we assume that the relevant functions have already -been imported from the subprocess module. +been imported from the :mod:`subprocess` module. Replacing /bin/sh shell backquote @@ -685,7 +729,7 @@ The p1.stdout.close() call after starting the p2 is important in order for p1 to receive a SIGPIPE if p2 exits before p1. Alternatively, for trusted input, the shell's own pipeline support may still -be used directly: +be used directly:: output=`dmesg | grep hda` # becomes @@ -697,9 +741,9 @@ Replacing :func:`os.system` :: - sts = os.system("mycmd" + " myarg") + status = os.system("mycmd" + " myarg") # becomes - sts = call("mycmd" + " myarg", shell=True) + status = subprocess.call("mycmd" + " myarg", shell=True) Notes: @@ -713,7 +757,7 @@ A more realistic example would look like this:: print >>sys.stderr, "Child was terminated by signal", -retcode else: print >>sys.stderr, "Child returned", retcode - except OSError, e: + except OSError as e: print >>sys.stderr, "Execution failed:", e @@ -822,7 +866,7 @@ Replacing functions from the :mod:`popen2` module (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) ==> - p = Popen(["somestring"], shell=True, bufsize=bufsize, + p = Popen("somestring", shell=True, bufsize=bufsize, stdin=PIPE, stdout=PIPE, close_fds=True) (child_stdout, child_stdin) = (p.stdout, p.stdin) diff --git a/Doc/library/sunaudio.rst b/Doc/library/sunaudio.rst index 148eb5e..187204e 100644 --- a/Doc/library/sunaudio.rst +++ b/Doc/library/sunaudio.rst @@ -8,7 +8,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`sunaudiodev` module has been deprecated for removal in Python 3.0. + The :mod:`sunaudiodev` module has been removed in Python 3. @@ -153,7 +153,7 @@ the SIGPOLL signal. Here's an example of how you might enable this in Python:: :deprecated: .. deprecated:: 2.6 - The :mod:`SUNAUDIODEV` module has been deprecated for removal in Python 3.0. + The :mod:`SUNAUDIODEV` module has been removed in Python 3. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 3873eb8..48d2b5b 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1,4 +1,3 @@ - :mod:`sys` --- System-specific parameters and functions ======================================================= @@ -208,7 +207,7 @@ always available. be set at build time with the ``--exec-prefix`` argument to the :program:`configure` script. Specifically, all configuration files (e.g. the :file:`pyconfig.h` header file) are installed in the directory - :file:`{exec_prefix}/lib/python{X.Y}/config', and shared library modules are + :file:`{exec_prefix}/lib/python{X.Y}/config`, and shared library modules are installed in :file:`{exec_prefix}/lib/python{X.Y}/lib-dynload`, where *X.Y* is the version number of Python, for example ``2.7``. @@ -291,6 +290,8 @@ always available. .. versionadded:: 2.6 + .. versionadded:: 2.7.3 + The ``hash_randomization`` attribute. .. data:: float_info @@ -301,6 +302,8 @@ always available. 5.2.4.2.2 of the 1999 ISO/IEC C standard [C99]_, 'Characteristics of floating types', for details. + .. tabularcolumns:: |l|l|L| + +---------------------+----------------+--------------------------------------------------+ | attribute | float.h macro | explanation | +=====================+================+==================================================+ @@ -598,6 +601,8 @@ always available. A struct sequence that holds information about Python's internal representation of integers. The attributes are read only. + .. tabularcolumns:: |l|L| + +-------------------------+----------------------------------------------+ | Attribute | Explanation | +=========================+==============================================+ @@ -773,9 +778,9 @@ always available. independent Python files are installed; by default, this is the string ``'/usr/local'``. This can be set at build time with the ``--prefix`` argument to the :program:`configure` script. The main collection of Python - library modules is installed in the directory :file:`{prefix}/lib/python{X.Y}`` + library modules is installed in the directory :file:`{prefix}/lib/python{X.Y}` while the platform independent header files (all except :file:`pyconfig.h`) are - stored in :file:`{prefix}/include/python{X.Y}``, where *X.Y* is the version + stored in :file:`{prefix}/include/python{X.Y}`, where *X.Y* is the version number of Python, for example ``2.7``. @@ -796,10 +801,10 @@ always available. .. data:: py3kwarning - Bool containing the status of the Python 3.0 warning flag. It's ``True`` + Bool containing the status of the Python 3 warning flag. It's ``True`` when Python is started with the -3 option. (This should be considered read-only; setting it to a different value doesn't have an effect on - Python 3.0 warnings.) + Python 3 warnings.) .. versionadded:: 2.6 @@ -1072,5 +1077,5 @@ always available. .. rubric:: Citations -.. [C99] ISO/IEC 9899:1999. "Programming languages -- C." A public draft of this standard is available at http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf . +.. [C99] ISO/IEC 9899:1999. "Programming languages -- C." A public draft of this standard is available at http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf\ . diff --git a/Doc/library/sysconfig.rst b/Doc/library/sysconfig.rst index 5ba6fa2..a745a3d 100644 --- a/Doc/library/sysconfig.rst +++ b/Doc/library/sysconfig.rst @@ -129,7 +129,7 @@ identifier. Python currently uses eight paths: one may call this function and get the default value. If *scheme* is provided, it must be a value from the list returned by - :func:`get_path_names`. Otherwise, the default scheme for the current + :func:`get_scheme_names`. Otherwise, the default scheme for the current platform is used. If *vars* is provided, it must be a dictionary of variables that will update diff --git a/Doc/library/syslog.rst b/Doc/library/syslog.rst index 21eee1e..9b66abf 100644 --- a/Doc/library/syslog.rst +++ b/Doc/library/syslog.rst @@ -17,7 +17,8 @@ library that can speak to a syslog server is available in the The module defines the following functions: -.. function:: syslog([priority,] message) +.. function:: syslog(message) + syslog(priority, message) Send the string *message* to the system logger. A trailing newline is added if necessary. Each message is tagged with a priority composed of a @@ -73,7 +74,8 @@ Priority levels (high to low): Facilities: :const:`LOG_KERN`, :const:`LOG_USER`, :const:`LOG_MAIL`, :const:`LOG_DAEMON`, :const:`LOG_AUTH`, :const:`LOG_LPR`, :const:`LOG_NEWS`, :const:`LOG_UUCP`, - :const:`LOG_CRON` and :const:`LOG_LOCAL0` to :const:`LOG_LOCAL7`. + :const:`LOG_CRON`, :const:`LOG_SYSLOG` and :const:`LOG_LOCAL0` to + :const:`LOG_LOCAL7`. Log options: :const:`LOG_PID`, :const:`LOG_CONS`, :const:`LOG_NDELAY`, :const:`LOG_NOWAIT` diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst index 5502adc..bd218e1 100644 --- a/Doc/library/tarfile.rst +++ b/Doc/library/tarfile.rst @@ -16,7 +16,8 @@ The :mod:`tarfile` module makes it possible to read and write tar archives, including those using gzip or bz2 compression. -(:file:`.zip` files can be read and written using the :mod:`zipfile` module.) +Use the :mod:`zipfile` module to read or write :file:`.zip` files, or the +higher-level functions in :ref:`shutil <archiving-operations>`. Some facts and figures: @@ -76,6 +77,10 @@ Some facts and figures: If *fileobj* is specified, it is used as an alternative to a file object opened for *name*. It is supposed to be at position 0. + For modes ``'w:gz'``, ``'r:gz'``, ``'w:bz2'``, ``'r:bz2'``, :func:`tarfile.open` + accepts the keyword argument *compresslevel* to specify the compression level of + the file. + For special purposes, there is a second format for *mode*: ``'filemode|[compression]'``. :func:`tarfile.open` will return a :class:`TarFile` object that processes its data as a stream of blocks. No random seeking will @@ -142,7 +147,7 @@ Some facts and figures: .. deprecated:: 2.6 - The :class:`TarFileCompat` class has been deprecated for removal in Python 3.0. + The :class:`TarFileCompat` class has been removed in Python 3. .. exception:: TarError @@ -304,7 +309,7 @@ be finalized; only the internally used file object will be closed. See the .. versionadded:: 2.6 -.. method:: TarFile.open(...) +.. classmethod:: TarFile.open(...) Alternative constructor. The :func:`tarfile.open` function is actually a shortcut to this classmethod. @@ -543,7 +548,7 @@ A ``TarInfo`` object has the following public data attributes: :const:`AREGTYPE`, :const:`LNKTYPE`, :const:`SYMTYPE`, :const:`DIRTYPE`, :const:`FIFOTYPE`, :const:`CONTTYPE`, :const:`CHRTYPE`, :const:`BLKTYPE`, :const:`GNUTYPE_SPARSE`. To determine the type of a :class:`TarInfo` object - more conveniently, use the ``is_*()`` methods below. + more conveniently, use the ``is*()`` methods below. .. attribute:: TarInfo.linkname diff --git a/Doc/library/telnetlib.rst b/Doc/library/telnetlib.rst index f6340a9..a3019f5 100644 --- a/Doc/library/telnetlib.rst +++ b/Doc/library/telnetlib.rst @@ -189,7 +189,7 @@ Telnet Objects Read until one from a list of a regular expressions matches. The first argument is a list of regular expressions, either compiled - (:class:`re.RegexObject` instances) or uncompiled (strings). The optional second + (:class:`regex objects <re-objects>`) or uncompiled (strings). The optional second argument is a timeout, in seconds; the default is to block indefinitely. Return a tuple of three items: the index in the list of the first regular @@ -208,7 +208,7 @@ Telnet Objects .. method:: Telnet.set_option_negotiation_callback(callback) Each time a telnet option is read on the input flow, this *callback* (if set) is - called with the following parameters : callback(telnet socket, command + called with the following parameters: callback(telnet socket, command (DO/DONT/WILL/WONT), option). No other action is done afterwards by telnetlib. diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index 936f06a..827f5f5 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -86,13 +86,14 @@ The module defines the following user-callable functions: data is spooled in memory until the file size exceeds *max_size*, or until the file's :func:`fileno` method is called, at which point the contents are written to disk and operation proceeds as with - :func:`TemporaryFile`. + :func:`TemporaryFile`. Also, it's ``truncate`` method does not + accept a ``size`` argument. The resulting file has one additional method, :func:`rollover`, which causes the file to roll over to an on-disk file regardless of its size. The returned object is a file-like object whose :attr:`_file` attribute - is either a :class:`StringIO` object or a true file object, depending on + is either a :class:`~StringIO.StringIO` object or a true file object, depending on whether :func:`rollover` has been called. This file-like object can be used in a :keyword:`with` statement, just like a normal file. diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 3a0b5cb..434ef5a 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -169,10 +169,10 @@ be passed to the script. Specifying a single regression test (:program:`python the test passed or failed and thus minimize output. Running :mod:`test.regrtest` directly allows what resources are available for -tests to use to be set. You do this by using the :option:`-u` command-line -option. Run :program:`python -m test.regrtest -uall` to turn on all -resources; specifying ``all`` as an option for ``-u`` enables all -possible resources. If all but one resource is desired (a more common case), a +tests to use to be set. You do this by using the ``-u`` command-line +option. Specifying ``all`` as the value for the ``-u`` option enables all +possible resources: :program:`python -m test -uall`. +If all but one resource is desired (a more common case), a comma-separated list of resources that are not desired may be listed after ``all``. The command :program:`python -m test.regrtest -uall,-audio,-largefile` will run :mod:`test.regrtest` with all resources except the ``audio`` and @@ -380,7 +380,7 @@ The :mod:`test.test_support` module defines the following functions: with captured_stdout() as s: print "hello" - assert s.getvalue() == "hello" + assert s.getvalue() == "hello\n" .. versionadded:: 2.6 diff --git a/Doc/library/textwrap.rst b/Doc/library/textwrap.rst index 84e6ee1..a50600e 100644 --- a/Doc/library/textwrap.rst +++ b/Doc/library/textwrap.rst @@ -26,6 +26,9 @@ otherwise, you should use an instance of :class:`TextWrapper` for efficiency. Optional keyword arguments correspond to the instance attributes of :class:`TextWrapper`, documented below. *width* defaults to ``70``. + See the :meth:`TextWrapper.wrap` method for additional details on how + :func:`wrap` behaves. + .. function:: fill(text[, width[, ...]]) @@ -112,9 +115,11 @@ indentation from strings that have unwanted whitespace to the left of the text. .. attribute:: replace_whitespace - (default: ``True``) If true, each whitespace character (as defined by - ``string.whitespace``) remaining after tab expansion will be replaced by a - single space. + (default: ``True``) If true, after tab expansion but before wrapping, + the :meth:`wrap` method will replace each whitespace character + with a single space. The whitespace characters replaced are + as follows: tab, newline, vertical tab, formfeed, and carriage + return (``'\t\n\v\f\r'``). .. note:: @@ -132,9 +137,11 @@ indentation from strings that have unwanted whitespace to the left of the text. .. attribute:: drop_whitespace - (default: ``True``) If true, whitespace that, after wrapping, happens to - end up at the beginning or end of a line is dropped (leading whitespace in - the first line is always preserved, though). + (default: ``True``) If true, whitespace at the beginning and ending of + every line (after wrapping but before indenting) is dropped. + Whitespace at the beginning of the paragraph, however, is not dropped + if non-whitespace follows it. If whitespace being dropped takes up an + entire line, the whole line is dropped. .. versionadded:: 2.6 Whitespace was always dropped in earlier versions. @@ -143,7 +150,8 @@ indentation from strings that have unwanted whitespace to the left of the text. .. attribute:: initial_indent (default: ``''``) String that will be prepended to the first line of - wrapped output. Counts towards the length of the first line. + wrapped output. Counts towards the length of the first line. The empty + string is not indented. .. attribute:: subsequent_indent @@ -206,8 +214,9 @@ indentation from strings that have unwanted whitespace to the left of the text. Wraps the single paragraph in *text* (a string) so every line is at most :attr:`width` characters long. All wrapping options are taken from - instance attributes of the :class:`TextWrapper` instance. Returns a list - of output lines, without final newlines. + instance attributes of the :class:`TextWrapper` instance. Returns a list + of output lines, without final newlines. If the wrapped output has no + content, the returned list is empty. .. method:: fill(text) diff --git a/Doc/library/thread.rst b/Doc/library/thread.rst index 7e8d5c8..15859d1 100644 --- a/Doc/library/thread.rst +++ b/Doc/library/thread.rst @@ -5,9 +5,9 @@ :synopsis: Create multiple threads of control within one interpreter. .. note:: - The :mod:`thread` module has been renamed to :mod:`_thread` in Python 3.0. + The :mod:`thread` module has been renamed to :mod:`_thread` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0; however, you should consider using the high-level + sources to Python 3; however, you should consider using the high-level :mod:`threading` module instead. diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index 28a3f81..a24c385 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -31,10 +31,10 @@ The :mod:`dummy_threading` module is provided for situations where .. impl-detail:: - Due to the :term:`Global Interpreter Lock`, in CPython only one thread + In CPython, due to the :term:`Global Interpreter Lock`, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). - If you want your application to make better of use of the computational + If you want your application to make better use of the computational resources of multi-core machines, you are advised to use :mod:`multiprocessing`. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously. @@ -48,6 +48,9 @@ This module defines the following functions and objects: Return the number of :class:`Thread` objects currently alive. The returned count is equal to the length of the list returned by :func:`.enumerate`. + .. versionchanged:: 2.6 + Added ``active_count()`` spelling. + .. function:: Condition() :noindex: @@ -67,6 +70,9 @@ This module defines the following functions and objects: :mod:`threading` module, a dummy thread object with limited functionality is returned. + .. versionchanged:: 2.6 + Added ``current_thread()`` spelling. + .. function:: enumerate() @@ -167,7 +173,7 @@ This module defines the following functions and objects: Set a trace function for all threads started from the :mod:`threading` module. The *func* will be passed to :func:`sys.settrace` for each thread, before its - :meth:`run` method is called. + :meth:`~Thread.run` method is called. .. versionadded:: 2.3 @@ -178,7 +184,7 @@ This module defines the following functions and objects: Set a profile function for all threads started from the :mod:`threading` module. The *func* will be passed to :func:`sys.setprofile` for each thread, before its - :meth:`run` method is called. + :meth:`~Thread.run` method is called. .. versionadded:: 2.3 @@ -202,6 +208,13 @@ This module defines the following functions and objects: .. versionadded:: 2.5 + +.. exception:: ThreadError + + Raised for various threading-related errors as described below. Note that + many interfaces use :exc:`RuntimeError` instead of :exc:`ThreadError`. + + Detailed interfaces for the objects are documented below. The design of this module is loosely based on Java's threading model. However, @@ -247,6 +260,12 @@ that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the :attr:`daemon` property. +.. note:: + Daemon threads are abruptly stopped at shutdown. Their resources (such + as open files, database transactions, etc.) may not be released properly. + If you want your threads to stop gracefully, make them non-daemonic and + use a suitable signalling mechanism such as an :class:`Event`. + There is a "main thread" object; this corresponds to the initial thread of control in the Python program. It is not a daemon thread. @@ -322,17 +341,19 @@ impossible to detect the termination of alien threads. :meth:`join` a thread before it has been started and attempts to do so raises the same exception. - .. method:: getName() - setName() - - Old API for :attr:`~Thread.name`. - .. attribute:: name A string used for identification purposes only. It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor. + .. versionadded:: 2.6 + + .. method:: getName() + setName() + + Pre-2.6 API for :attr:`~Thread.name`. + .. attribute:: ident The 'thread identifier' of this thread or ``None`` if the thread has not @@ -352,10 +373,8 @@ impossible to detect the termination of alien threads. until just after the :meth:`run` method terminates. The module function :func:`.enumerate` returns a list of all alive threads. - .. method:: isDaemon() - setDaemon() - - Old API for :attr:`~Thread.daemon`. + .. versionchanged:: 2.6 + Added ``is_alive()`` spelling. .. attribute:: daemon @@ -368,6 +387,13 @@ impossible to detect the termination of alien threads. The entire Python program exits when no alive non-daemon threads are left. + .. versionadded:: 2.6 + + .. method:: isDaemon() + setDaemon() + + Pre-2.6 API for :attr:`~Thread.daemon`. + .. _lock-objects: @@ -387,7 +413,7 @@ blocks until a call to :meth:`release` in another thread changes it to unlocked, then the :meth:`acquire` call resets it to locked and returns. The :meth:`release` method should only be called in the locked state; it changes the state to unlocked and returns immediately. If an attempt is made to release an -unlocked lock, a :exc:`RuntimeError` will be raised. +unlocked lock, a :exc:`ThreadError` will be raised. When more than one thread is blocked in :meth:`acquire` waiting for the state to turn to unlocked, only one thread proceeds when a :meth:`release` call resets @@ -401,15 +427,12 @@ All methods are executed atomically. Acquire a lock, blocking or non-blocking. - When invoked without arguments, block until the lock is unlocked, then set it to - locked, and return true. + When invoked with the *blocking* argument set to ``True`` (the default), + block until the lock is unlocked, then set it to locked and return ``True``. - When invoked with the *blocking* argument set to true, do the same thing as when - called without arguments, and return true. - - When invoked with the *blocking* argument set to false, do not block. If a call - without an argument would block, return false immediately; otherwise, do the - same thing as when called without arguments, and return true. + When invoked with the *blocking* argument set to ``False``, do not block. + If a call with *blocking* set to ``True`` would block, return ``False`` + immediately; otherwise, set the lock to locked and return ``True``. .. method:: Lock.release() @@ -420,7 +443,7 @@ All methods are executed atomically. are blocked waiting for the lock to become unlocked, allow exactly one of them to proceed. - Do not call this method when the lock is unlocked. + When invoked on an unlocked lock, a :exc:`ThreadError` is raised. There is no return value. @@ -599,6 +622,9 @@ needs to wake up one consumer thread. calling thread has not acquired the lock when this method is called, a :exc:`RuntimeError` is raised. + .. versionchanged:: 2.6 + Added ``notify_all()`` spelling. + .. _semaphore-objects: @@ -698,7 +724,7 @@ An event object manages an internal flag that can be set to true with the Return true if and only if the internal flag is true. .. versionchanged:: 2.6 - The ``is_set()`` syntax is new. + Added ``is_set()`` spelling. .. method:: set() @@ -739,10 +765,11 @@ This class represents an action that should be run only after a certain amount of time has passed --- a timer. :class:`Timer` is a subclass of :class:`Thread` and as such also functions as an example of creating custom threads. -Timers are started, as with threads, by calling their :meth:`start` method. The -timer can be stopped (before its action has begun) by calling the :meth:`cancel` -method. The interval the timer will wait before executing its action may not be -exactly the same as the interval specified by the user. +Timers are started, as with threads, by calling their :meth:`~Timer.start` +method. The timer can be stopped (before its action has begun) by calling the +:meth:`~Timer.cancel` method. The interval the timer will wait before +executing its action may not be exactly the same as the interval specified by +the user. For example:: diff --git a/Doc/library/time.rst b/Doc/library/time.rst index 56e9019..304bf0c 100644 --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -71,9 +71,9 @@ An explanation of some terminology and conventions is in order. the units in which their value or argument is expressed. E.g. on most Unix systems, the clock "ticks" only 50 or 100 times a second. -* On the other hand, the precision of :func:`time` and :func:`sleep` is better +* On the other hand, the precision of :func:`.time` and :func:`sleep` is better than their Unix equivalents: times are expressed as floating point numbers, - :func:`time` returns the most accurate time available (using Unix + :func:`.time` returns the most accurate time available (using Unix :c:func:`gettimeofday` where available), and :func:`sleep` will accept a time with a nonzero fraction (Unix :c:func:`select` is used to implement this, where available). @@ -164,7 +164,7 @@ The module defines the following functions and data items: Convert a time expressed in seconds since the epoch to a string representing local time. If *secs* is not provided or :const:`None`, the current time as - returned by :func:`time` is used. ``ctime(secs)`` is equivalent to + returned by :func:`.time` is used. ``ctime(secs)`` is equivalent to ``asctime(localtime(secs))``. Locale information is not used by :func:`ctime`. .. versionchanged:: 2.1 @@ -183,7 +183,7 @@ The module defines the following functions and data items: Convert a time expressed in seconds since the epoch to a :class:`struct_time` in UTC in which the dst flag is always zero. If *secs* is not provided or - :const:`None`, the current time as returned by :func:`time` is used. Fractions + :const:`None`, the current time as returned by :func:`.time` is used. Fractions of a second are ignored. See above for a description of the :class:`struct_time` object. See :func:`calendar.timegm` for the inverse of this function. @@ -198,7 +198,7 @@ The module defines the following functions and data items: .. function:: localtime([secs]) Like :func:`gmtime` but converts to local time. If *secs* is not provided or - :const:`None`, the current time as returned by :func:`time` is used. The dst + :const:`None`, the current time as returned by :func:`.time` is used. The dst flag is set to ``1`` when DST applies to the given time. .. versionchanged:: 2.1 @@ -213,7 +213,7 @@ The module defines the following functions and data items: This is the inverse function of :func:`localtime`. Its argument is the :class:`struct_time` or full 9-tuple (since the dst flag is needed; use ``-1`` as the dst flag if it is unknown) which expresses the time in *local* time, not - UTC. It returns a floating point number, for compatibility with :func:`time`. + UTC. It returns a floating point number, for compatibility with :func:`.time`. If the input value cannot be represented as a valid time, either :exc:`OverflowError` or :exc:`ValueError` will be raised (which depends on whether the invalid value is caught by Python or the underlying C libraries). @@ -236,7 +236,9 @@ The module defines the following functions and data items: :func:`gmtime` or :func:`localtime` to a string as specified by the *format* argument. If *t* is not provided, the current time as returned by :func:`localtime` is used. *format* must be a string. :exc:`ValueError` is - raised if any field in *t* is outside of the allowed range. + raised if any field in *t* is outside of the allowed range. :func:`strftime` + returns a locale depedent byte string; the result may be converted to unicode + by doing ``strftime(<myformat>).decode(locale.getlocale()[1])``. .. versionchanged:: 2.1 Allowed *t* to be omitted. @@ -350,8 +352,10 @@ The module defines the following functions and data items: >>> strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime()) 'Thu, 28 Jun 2001 14:17:15 +0000' - Additional directives may be supported on certain platforms, but only the ones - listed here have a meaning standardized by ANSI C. + Additional directives may be supported on certain platforms, but only the + ones listed here have a meaning standardized by ANSI C. To see the full set + of format codes supported on your platform, consult the :manpage:`strftime(3)` + documentation. On some platforms, an optional field width and precision specification can immediately follow the initial ``'%'`` of a directive in the following order; @@ -410,7 +414,7 @@ The module defines the following functions and data items: +-------+-------------------+---------------------------------+ | 4 | :attr:`tm_min` | range [0, 59] | +-------+-------------------+---------------------------------+ - | 5 | :attr:`tm_sec` | range [0, 61]; see **(1)** in | + | 5 | :attr:`tm_sec` | range [0, 61]; see **(2)** in | | | | :func:`strftime` description | +-------+-------------------+---------------------------------+ | 6 | :attr:`tm_wday` | range [0, 6], Monday is 0 | @@ -435,8 +439,8 @@ The module defines the following functions and data items: .. function:: time() - Return the time as a floating point number expressed in seconds since the epoch, - in UTC. Note that even though the time is always returned as a floating point + Return the time in seconds since the epoch as a floating point number. + Note that even though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second. While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between @@ -547,12 +551,12 @@ The module defines the following functions and data items: More object-oriented interface to dates and times. Module :mod:`locale` - Internationalization services. The locale settings can affect the return values - for some of the functions in the :mod:`time` module. + Internationalization services. The locale setting affects the interpretation + of many format specifiers in :func:`strftime` and :func:`strptime`. Module :mod:`calendar` - General calendar-related functions. :func:`timegm` is the inverse of - :func:`gmtime` from this module. + General calendar-related functions. :func:`~calendar.timegm` is the + inverse of :func:`gmtime` from this module. .. rubric:: Footnotes diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst index 7fbe19e..2cb3c9d 100644 --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -16,112 +16,163 @@ -------------- This module provides a simple way to time small bits of Python code. It has both -command line as well as callable interfaces. It avoids a number of common traps -for measuring execution times. See also Tim Peters' introduction to the -"Algorithms" chapter in the Python Cookbook, published by O'Reilly. +a :ref:`command-line-interface` as well as a :ref:`callable <python-interface>` +one. It avoids a number of common traps for measuring execution times. +See also Tim Peters' introduction to the "Algorithms" chapter in the *Python +Cookbook*, published by O'Reilly. -The module defines the following public class: +Basic Examples +-------------- -.. class:: Timer([stmt='pass' [, setup='pass' [, timer=<timer function>]]]) +The following example shows how the :ref:`command-line-interface` +can be used to compare three different expressions: - Class for timing execution speed of small code snippets. +.. code-block:: sh - The constructor takes a statement to be timed, an additional statement used for - setup, and a timer function. Both statements default to ``'pass'``; the timer - function is platform-dependent (see the module doc string). *stmt* and *setup* - may also contain multiple statements separated by ``;`` or newlines, as long as - they don't contain multi-line string literals. + $ python -m timeit '"-".join(str(n) for n in range(100))' + 10000 loops, best of 3: 40.3 usec per loop + $ python -m timeit '"-".join([str(n) for n in range(100)])' + 10000 loops, best of 3: 33.4 usec per loop + $ python -m timeit '"-".join(map(str, range(100)))' + 10000 loops, best of 3: 25.2 usec per loop - To measure the execution time of the first statement, use the :meth:`timeit` - method. The :meth:`repeat` method is a convenience to call :meth:`timeit` - multiple times and return a list of results. +This can be achieved from the :ref:`python-interface` with:: - .. versionchanged:: 2.6 - The *stmt* and *setup* parameters can now also take objects that are callable - without arguments. This will embed calls to them in a timer function that will - then be executed by :meth:`timeit`. Note that the timing overhead is a little - larger in this case because of the extra function calls. + >>> import timeit + >>> timeit.timeit('"-".join(str(n) for n in range(100))', number=10000) + 0.8187260627746582 + >>> timeit.timeit('"-".join([str(n) for n in range(100)])', number=10000) + 0.7288308143615723 + >>> timeit.timeit('"-".join(map(str, range(100)))', number=10000) + 0.5858950614929199 +Note however that :mod:`timeit` will automatically determine the number of +repetitions only when the command-line interface is used. In the +:ref:`timeit-examples` section you can find more advanced examples. -.. method:: Timer.print_exc([file=None]) - Helper to print a traceback from the timed code. +.. _python-interface: - Typical use:: +Python Interface +---------------- - t = Timer(...) # outside the try/except - try: - t.timeit(...) # or t.repeat(...) - except: - t.print_exc() +The module defines three convenience functions and a public class: - The advantage over the standard traceback is that source lines in the compiled - template will be displayed. The optional *file* argument directs where the - traceback is sent; it defaults to ``sys.stderr``. +.. function:: timeit(stmt='pass', setup='pass', timer=<default timer>, number=1000000) -.. method:: Timer.repeat([repeat=3 [, number=1000000]]) + Create a :class:`Timer` instance with the given statement, *setup* code and + *timer* function and run its :meth:`.timeit` method with *number* executions. - Call :meth:`timeit` a few times. + .. versionadded:: 2.6 - This is a convenience function that calls the :meth:`timeit` repeatedly, - returning a list of results. The first argument specifies how many times to - call :meth:`timeit`. The second argument specifies the *number* argument for - :func:`timeit`. - .. note:: +.. function:: repeat(stmt='pass', setup='pass', timer=<default timer>, repeat=3, number=1000000) - It's tempting to calculate mean and standard deviation from the result vector - and report these. However, this is not very useful. In a typical case, the - lowest value gives a lower bound for how fast your machine can run the given - code snippet; higher values in the result vector are typically not caused by - variability in Python's speed, but by other processes interfering with your - timing accuracy. So the :func:`min` of the result is probably the only number - you should be interested in. After that, you should look at the entire vector - and apply common sense rather than statistics. + Create a :class:`Timer` instance with the given statement, *setup* code and + *timer* function and run its :meth:`.repeat` method with the given *repeat* + count and *number* executions. + .. versionadded:: 2.6 -.. method:: Timer.timeit([number=1000000]) - Time *number* executions of the main statement. This executes the setup - statement once, and then returns the time it takes to execute the main statement - a number of times, measured in seconds as a float. The argument is the number - of times through the loop, defaulting to one million. The main statement, the - setup statement and the timer function to be used are passed to the constructor. +.. function:: default_timer() - .. note:: + Define a default timer, in a platform-specific manner. On Windows, + :func:`time.clock` has microsecond granularity, but :func:`time.time`'s + granularity is 1/60th of a second. On Unix, :func:`time.clock` has 1/100th of + a second granularity, and :func:`time.time` is much more precise. On either + platform, :func:`default_timer` measures wall clock time, not the CPU + time. This means that other processes running on the same computer may + interfere with the timing. - By default, :meth:`timeit` temporarily turns off :term:`garbage collection` - during the timing. The advantage of this approach is that it makes - independent timings more comparable. This disadvantage is that GC may be - an important component of the performance of the function being measured. - If so, GC can be re-enabled as the first statement in the *setup* string. - For example:: - timeit.Timer('for i in xrange(10): oct(i)', 'gc.enable()').timeit() +.. class:: Timer(stmt='pass', setup='pass', timer=<timer function>) -Starting with version 2.6, the module also defines two convenience functions: + Class for timing execution speed of small code snippets. + The constructor takes a statement to be timed, an additional statement used + for setup, and a timer function. Both statements default to ``'pass'``; + the timer function is platform-dependent (see the module doc string). + *stmt* and *setup* may also contain multiple statements separated by ``;`` + or newlines, as long as they don't contain multi-line string literals. -.. function:: repeat(stmt[, setup[, timer[, repeat=3 [, number=1000000]]]]) + To measure the execution time of the first statement, use the :meth:`.timeit` + method. The :meth:`.repeat` method is a convenience to call :meth:`.timeit` + multiple times and return a list of results. - Create a :class:`Timer` instance with the given statement, setup code and timer - function and run its :meth:`repeat` method with the given repeat count and - *number* executions. + .. versionchanged:: 2.6 + The *stmt* and *setup* parameters can now also take objects that are + callable without arguments. This will embed calls to them in a timer + function that will then be executed by :meth:`.timeit`. Note that the + timing overhead is a little larger in this case because of the extra + function calls. - .. versionadded:: 2.6 + .. method:: Timer.timeit(number=1000000) -.. function:: timeit(stmt[, setup[, timer[, number=1000000]]]) + Time *number* executions of the main statement. This executes the setup + statement once, and then returns the time it takes to execute the main + statement a number of times, measured in seconds as a float. + The argument is the number of times through the loop, defaulting to one + million. The main statement, the setup statement and the timer function + to be used are passed to the constructor. - Create a :class:`Timer` instance with the given statement, setup code and timer - function and run its :meth:`timeit` method with *number* executions. + .. note:: - .. versionadded:: 2.6 + By default, :meth:`.timeit` temporarily turns off :term:`garbage + collection` during the timing. The advantage of this approach is that + it makes independent timings more comparable. This disadvantage is + that GC may be an important component of the performance of the + function being measured. If so, GC can be re-enabled as the first + statement in the *setup* string. For example:: + + timeit.Timer('for i in xrange(10): oct(i)', 'gc.enable()').timeit() + + + .. method:: Timer.repeat(repeat=3, number=1000000) + + Call :meth:`.timeit` a few times. + + This is a convenience function that calls the :meth:`.timeit` repeatedly, + returning a list of results. The first argument specifies how many times + to call :meth:`.timeit`. The second argument specifies the *number* + argument for :meth:`.timeit`. + + .. note:: + + It's tempting to calculate mean and standard deviation from the result + vector and report these. However, this is not very useful. + In a typical case, the lowest value gives a lower bound for how fast + your machine can run the given code snippet; higher values in the + result vector are typically not caused by variability in Python's + speed, but by other processes interfering with your timing accuracy. + So the :func:`min` of the result is probably the only number you + should be interested in. After that, you should look at the entire + vector and apply common sense rather than statistics. + + + .. method:: Timer.print_exc(file=None) + + Helper to print a traceback from the timed code. + Typical use:: -Command Line Interface + t = Timer(...) # outside the try/except + try: + t.timeit(...) # or t.repeat(...) + except: + t.print_exc() + + The advantage over the standard traceback is that source lines in the + compiled template will be displayed. The optional *file* argument directs + where the traceback is sent; it defaults to :data:`sys.stderr`. + + +.. _command-line-interface: + +Command-Line Interface ---------------------- When called as a program from the command line, the following form is used:: @@ -168,13 +219,9 @@ similarly. If :option:`-n` is not given, a suitable number of loops is calculated by trying successive powers of 10 until the total time is at least 0.2 seconds. -The default timer function is platform dependent. On Windows, -:func:`time.clock` has microsecond granularity but :func:`time.time`'s -granularity is 1/60th of a second; on Unix, :func:`time.clock` has 1/100th of a -second granularity and :func:`time.time` is much more precise. On either -platform, the default timer functions measure wall clock time, not the CPU time. -This means that other processes running on the same computer may interfere with -the timing. The best thing to do when accurate timing is necessary is to repeat +:func:`default_timer` measurations can be affected by other programs running on +the same machine, so +the best thing to do when accurate timing is necessary is to repeat the timing a few times and use the best time. The :option:`-r` option is good for this; the default of 3 repetitions is probably enough in most cases. On Unix, you can use :func:`time.clock` to measure CPU time. @@ -183,25 +230,55 @@ Unix, you can use :func:`time.clock` to measure CPU time. There is a certain baseline overhead associated with executing a pass statement. The code here doesn't try to hide it, but you should be aware of it. The - baseline overhead can be measured by invoking the program without arguments. + baseline overhead can be measured by invoking the program without arguments, and + it might differ between Python versions. Also, to fairly compare older Python + versions to Python 2.3, you may want to use Python's :option:`-O` option for + the older versions to avoid timing ``SET_LINENO`` instructions. -The baseline overhead differs between Python versions! Also, to fairly compare -older Python versions to Python 2.3, you may want to use Python's :option:`-O` -option for the older versions to avoid timing ``SET_LINENO`` instructions. +.. _timeit-examples: Examples -------- -Here are two example sessions (one using the command line, one using the module -interface) that compare the cost of using :func:`hasattr` vs. -:keyword:`try`/:keyword:`except` to test for missing and present object -attributes. :: +It is possible to provide a setup statement that is executed only once at the beginning: + +.. code-block:: sh + + $ python -m timeit -s 'text = "sample string"; char = "g"' 'char in text' + 10000000 loops, best of 3: 0.0877 usec per loop + $ python -m timeit -s 'text = "sample string"; char = "g"' 'text.find(char)' + 1000000 loops, best of 3: 0.342 usec per loop + +:: + + >>> import timeit + >>> timeit.timeit('char in text', setup='text = "sample string"; char = "g"') + 0.41440500499993504 + >>> timeit.timeit('text.find(char)', setup='text = "sample string"; char = "g"') + 1.7246671520006203 + +The same can be done using the :class:`Timer` class and its methods:: + + >>> import timeit + >>> t = timeit.Timer('char in text', setup='text = "sample string"; char = "g"') + >>> t.timeit() + 0.3955516149999312 + >>> t.repeat() + [0.40193588800002544, 0.3960157959998014, 0.39594301399984033] + + +The following examples show how to time expressions that contain multiple lines. +Here we compare the cost of using :func:`hasattr` vs. :keyword:`try`/:keyword:`except` +to test for missing and present object attributes: + +.. code-block:: sh $ python -m timeit 'try:' ' str.__nonzero__' 'except AttributeError:' ' pass' 100000 loops, best of 3: 15.7 usec per loop $ python -m timeit 'if hasattr(str, "__nonzero__"): pass' 100000 loops, best of 3: 4.26 usec per loop + $ python -m timeit 'try:' ' int.__nonzero__' 'except AttributeError:' ' pass' 1000000 loops, best of 3: 1.43 usec per loop $ python -m timeit 'if hasattr(int, "__nonzero__"): pass' @@ -210,39 +287,34 @@ attributes. :: :: >>> import timeit + >>> # attribute is missing >>> s = """\ ... try: ... str.__nonzero__ ... except AttributeError: ... pass ... """ - >>> t = timeit.Timer(stmt=s) - >>> print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000) - 17.09 usec/pass - >>> s = """\ - ... if hasattr(str, '__nonzero__'): pass - ... """ - >>> t = timeit.Timer(stmt=s) - >>> print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000) - 4.85 usec/pass + >>> timeit.timeit(stmt=s, number=100000) + 0.9138244460009446 + >>> s = "if hasattr(str, '__bool__'): pass" + >>> timeit.timeit(stmt=s, number=100000) + 0.5829014980008651 + >>> + >>> # attribute is present >>> s = """\ ... try: ... int.__nonzero__ ... except AttributeError: ... pass ... """ - >>> t = timeit.Timer(stmt=s) - >>> print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000) - 1.97 usec/pass - >>> s = """\ - ... if hasattr(int, '__nonzero__'): pass - ... """ - >>> t = timeit.Timer(stmt=s) - >>> print "%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000) - 3.15 usec/pass + >>> timeit.timeit(stmt=s, number=100000) + 0.04215312199994514 + >>> s = "if hasattr(int, '__bool__'): pass" + >>> timeit.timeit(stmt=s, number=100000) + 0.08588060699912603 To give the :mod:`timeit` module access to functions you define, you can pass a -``setup`` parameter which contains an import statement:: +*setup* parameter which contains an import statement:: def test(): """Stupid test function""" @@ -251,7 +323,5 @@ To give the :mod:`timeit` module access to functions you define, you can pass a L.append(i) if __name__ == '__main__': - from timeit import Timer - t = Timer("test()", "from __main__ import test") - print t.timeit() - + import timeit + print(timeit.timeit("test()", setup="from __main__ import test")) diff --git a/Doc/library/tix.rst b/Doc/library/tix.rst index 8b5355d..a2f31a0 100644 --- a/Doc/library/tix.rst +++ b/Doc/library/tix.rst @@ -24,9 +24,9 @@ special needs of your application and users. .. note:: - :mod:`Tix` has been renamed to :mod:`tkinter.tix` in Python 3.0. The + :mod:`Tix` has been renamed to :mod:`tkinter.tix` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. .. seealso:: @@ -514,7 +514,7 @@ Tix Commands print root.tix_configure() -.. method:: tixCommand.tix_configure([cnf,] **kw) +.. method:: tixCommand.tix_configure(cnf=None **kw) Query or modify the configuration options of the Tix application context. If no option is specified, returns a dictionary all of the available options. If diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index 3431f86..ddaeec7 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -13,22 +13,34 @@ is maintained at ActiveState.) .. note:: - :mod:`Tkinter` has been renamed to :mod:`tkinter` in Python 3.0. The + :mod:`Tkinter` has been renamed to :mod:`tkinter` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting your - sources to 3.0. + sources to Python 3. .. seealso:: - `Python Tkinter Resources <http://www.python.org/topics/tkinter/>`_ + `Python Tkinter Resources <https://wiki.python.org/moin/TkInter>`_ The Python Tkinter Topic Guide provides a great deal of information on using Tk from Python and links to other sources of information on Tk. - `An Introduction to Tkinter <http://www.pythonware.com/library/an-introduction-to-tkinter.htm>`_ - Fredrik Lundh's on-line reference material. + `TKDocs <http://www.tkdocs.com/>`_ + Extensive tutorial plus friendlier widget pages for some of the widgets. - `Tkinter reference: a GUI for Python <http://infohost.nmt.edu/tcc/help/pubs/lang.html>`_ + `Tkinter reference: a GUI for Python <http://infohost.nmt.edu/tcc/help/pubs/tkinter/>`_ On-line reference material. + `Tkinter docs from effbot <http://effbot.org/tkinterbook/>`_ + Online reference for tkinter supported by effbot.org. + + `Tcl/Tk manual <http://www.tcl.tk/man/tcl8.5/>`_ + Official manual for the latest tcl/tk version. + + `Programming Python <http://www.amazon.com/Programming-Python-Mark-Lutz/dp/0596158106/>`_ + Book by Mark Lutz, has excellent coverage of Tkinter. + + `Modern Tkinter for Busy Python Developers <http://www.amazon.com/Modern-Tkinter-Python-Developers-ebook/dp/B0071QDNLO/>`_ + Book by Mark Rozerman about building attractive and modern graphical user interfaces with Python and Tkinter. + `Python and Tkinter Programming <http://www.amazon.com/exec/obidos/ASIN/1884777813>`_ The book by John Grayson (ISBN 1-884777-81-3). @@ -109,7 +121,7 @@ Other modules that provide Tk support include: :mod:`turtle` Turtle graphics in a Tk window. -These have been renamed as well in Python 3.0; they were all made submodules of +These have been renamed as well in Python 3; they were all made submodules of the new ``tkinter`` package. @@ -176,7 +188,7 @@ documentation that exists. Here are some hints: The Tk/Tcl development is largely taking place at ActiveState. `Tcl and the Tk Toolkit <http://www.amazon.com/exec/obidos/ASIN/020163337X>`_ - The book by John Ousterhout, the inventor of Tcl . + The book by John Ousterhout, the inventor of Tcl. `Practical Programming in Tcl and Tk <http://www.amazon.com/exec/obidos/ASIN/0130220280>`_ Brent Welch's encyclopedic book. @@ -440,7 +452,7 @@ back will contain the name of the synonym and the "real" option (such as Example:: >>> print fred.config() - {'relief' : ('relief', 'relief', 'Relief', 'raised', 'groove')} + {'relief': ('relief', 'relief', 'Relief', 'raised', 'groove')} Of course, the dictionary printed will include all the options available and their values. This is meant only as an example. @@ -613,7 +625,7 @@ bitmap preceded with an ``@``, as in ``"@/usr/contrib/bitmap/gumby.bit"``. boolean - You can pass integers 0 or 1 or the strings ``"yes"`` or ``"no"`` . + You can pass integers 0 or 1 or the strings ``"yes"`` or ``"no"``. callback This is any Python function that takes no arguments. For example:: diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst index 7075035..3f25a2c 100644 --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -17,9 +17,10 @@ for on-screen displays. To simplify token stream handling, all :ref:`operators` and :ref:`delimiters` tokens are returned using the generic :data:`token.OP` token type. The exact -type can be determined by checking the token ``string`` field on the -:term:`named tuple` returned from :func:`tokenize.tokenize` for the character -sequence that identifies a specific operator token. +type can be determined by checking the second field (containing the actual +token string matched) of the tuple returned from +:func:`tokenize.generate_tokens` for the character sequence that identifies a +specific operator token. The primary entry point is a :term:`generator`: @@ -29,7 +30,8 @@ The primary entry point is a :term:`generator`: which must be a callable object which provides the same interface as the :meth:`readline` method of built-in file objects (see section :ref:`bltin-file-objects`). Each call to the function should return one line - of input as a string. + of input as a string. Alternately, *readline* may be a callable object that + signals completion by raising :exc:`StopIteration`. The generator produces 5-tuples with these members: the token type; the token string; a 2-tuple ``(srow, scol)`` of ints specifying the row and column @@ -96,6 +98,24 @@ back the modified script. .. versionadded:: 2.5 +.. exception:: TokenError + + Raised when either a docstring or expression that may be split over several + lines is not completed anywhere in the file, for example:: + + """Beginning of + docstring + + or:: + + [1, + 2, + 3 + +Note that unclosed single-quoted strings do not cause an error to be +raised. They are tokenized as ``ERRORTOKEN``, followed by the tokenization of +their contents. + Example of a script re-writer that transforms float literals into Decimal objects:: diff --git a/Doc/library/trace.rst b/Doc/library/trace.rst index a2afda1..e00f118 100644 --- a/Doc/library/trace.rst +++ b/Doc/library/trace.rst @@ -41,8 +41,8 @@ Main options At least one of the following options must be specified when invoking :mod:`trace`. The :option:`--listfuncs <-l>` option is mutually exclusive with -the :option:`--trace <-t>` and :option:`--counts <-c>` options . When -:option:`--listfuncs <-l>` is provided, neither :option:`--counts <-c>` nor +the :option:`--trace <-t>` and :option:`--count <-c>` options. When +:option:`--listfuncs <-l>` is provided, neither :option:`--count <-c>` nor :option:`--trace <-t>` are accepted, and vice versa. .. program:: trace @@ -149,7 +149,7 @@ Programmatic Interface the current tracing parameters. *cmd* must be a string or code object, suitable for passing into :func:`exec`. - .. method:: runctx(cmd[, globals=None[, locals=None]]) + .. method:: runctx(cmd, globals=None, locals=None) Execute the command and gather statistics from the execution with the current tracing parameters, in the defined global and local @@ -200,7 +200,7 @@ A simple example demonstrating the use of the programmatic interface:: # run the new command using the given tracer tracer.run('main()') - # make a report, placing output in /tmp + # make a report, placing output in the current directory r = tracer.results() - r.write_results(show_missing=True, coverdir="/tmp") + r.write_results(show_missing=True, coverdir=".") diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst index 15eaa6e..6859d4b 100644 --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -75,7 +75,7 @@ The module defines the following functions: Return a list of up to *limit* "pre-processed" stack trace entries extracted from the traceback object *traceback*. It is useful for alternate formatting of stack traces. If *limit* is omitted or ``None``, all entries are extracted. A - "pre-processed" stack trace entry is a quadruple (*filename*, *line number*, + "pre-processed" stack trace entry is a 4-tuple (*filename*, *line number*, *function name*, *text*) representing the information that is usually printed for a stack trace. The *text* is a string with leading and trailing whitespace stripped; if the source is not available it is ``None``. diff --git a/Doc/library/ttk.rst b/Doc/library/ttk.rst index 0721234..6cab3e0 100644 --- a/Doc/library/ttk.rst +++ b/Doc/library/ttk.rst @@ -265,10 +265,10 @@ Besides the methods described below, the :class:`ttk.Widget` class supports the *x* and *y* are pixel coordinates relative to the widget. - .. method:: instate(statespec[, callback=None[, *args[, **kw]]]) + .. method:: instate(statespec, callback=None, *args, **kw) - Test the widget's state. If a callback is not specified, returns True - if the widget state matches *statespec* and False otherwise. If callback + Test the widget's state. If a callback is not specified, returns ``True`` + if the widget state matches *statespec* and ``False`` otherwise. If callback is specified then it is called with *args* if widget state matches *statespec*. @@ -523,7 +523,7 @@ ttk.Notebook omitted, returns the widget name of the currently selected pane. - .. method:: tab(tab_id[, option=None[, **kw]]) + .. method:: tab(tab_id, option=None, **kw) Query or modify the options of the specific *tab_id*. @@ -846,7 +846,7 @@ ttk.Treeview .. class:: Treeview - .. method:: bbox(item[, column=None]) + .. method:: bbox(item, column=None) Returns the bounding box (relative to the treeview widget's window) of the specified *item* in the form (x, y, width, height). @@ -873,7 +873,7 @@ ttk.Treeview *item*'s children. - .. method:: column(column[, option=None[, **kw]]) + .. method:: column(column, option=None, **kw) Query or modify the options for the specified *column*. @@ -919,7 +919,7 @@ ttk.Treeview .. method:: exists(item) - Returns True if the specified *item* is present in the tree. + Returns ``True`` if the specified *item* is present in the tree. .. method:: focus([item=None]) @@ -928,7 +928,7 @@ ttk.Treeview the current focus item, or '' if there is none. - .. method:: heading(column[, option=None[, **kw]]) + .. method:: heading(column, option=None, **kw) Query or modify the heading options for the specified *column*. @@ -1001,7 +1001,7 @@ ttk.Treeview Returns the integer index of *item* within its parent's list of children. - .. method:: insert(parent, index[, iid=None[, **kw]]) + .. method:: insert(parent, index, iid=None, **kw) Creates a new item and returns the item identifier of the newly created item. @@ -1065,7 +1065,7 @@ ttk.Treeview Ensure that *item* is visible. - Sets all of *item*'s ancestors open option to True, and scrolls the + Sets all of *item*'s ancestors open option to ``True``, and scrolls the widget if necessary so that *item* is within the visible portion of the tree. @@ -1096,7 +1096,7 @@ ttk.Treeview Toggle the selection state of each item in *items*. - .. method:: set(item[, column=None[, value=None]]) + .. method:: set(item, column=None, value=None) With one argument, returns a dictionary of column/value pairs for the specified *item*. With two arguments, returns the current value of the @@ -1104,14 +1104,14 @@ ttk.Treeview *column* in given *item* to the specified *value*. - .. method:: tag_bind(tagname[, sequence=None[, callback=None]]) + .. method:: tag_bind(tagname, sequence=None, callback=None) Bind a callback for the given event *sequence* to the tag *tagname*. When an event is delivered to an item, the callbacks for each of the item's tags option are called. - .. method:: tag_configure(tagname[, option=None[, **kw]]) + .. method:: tag_configure(tagname, option=None, **kw) Query or modify the options for the specified *tagname*. @@ -1220,7 +1220,7 @@ option. If the class name of a widget is unknown, use the method foreground option, for example, you would get a blue foreground when the widget is in the active or pressed states. - .. method:: lookup(style, option[, state=None[, default=None]]) + .. method:: lookup(style, option, state=None, default=None) Returns the value specified for *option* in *style*. @@ -1235,7 +1235,7 @@ option. If the class name of a widget is unknown, use the method print ttk.Style().lookup("TButton", "font") - .. method:: layout(style[, layoutspec=None]) + .. method:: layout(style, layoutspec=None) Define the widget layout for given *style*. If *layoutspec* is omitted, return the layout specification for given style. @@ -1318,7 +1318,7 @@ option. If the class name of a widget is unknown, use the method Returns the list of *elementname*'s options. - .. method:: theme_create(themename[, parent=None[, settings=None]]) + .. method:: theme_create(themename, parent=None, settings=None) Create a new theme. diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index e05305a..f04a07a 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -1049,8 +1049,8 @@ More drawing control Write text - the string representation of *arg* - at the current turtle position according to *align* ("left", "center" or right") and with the given - font. If *move* is True, the pen is moved to the bottom-right corner of the - text. By default, *move* is False. + font. If *move* is true, the pen is moved to the bottom-right corner of the + text. By default, *move* is ``False``. >>> turtle.write("Home = ", True, align="center") >>> turtle.write((0,0), True) @@ -1086,7 +1086,7 @@ Visibility .. function:: isvisible() - Return True if the Turtle is shown, False if it's hidden. + Return ``True`` if the Turtle is shown, ``False`` if it's hidden. >>> turtle.hideturtle() >>> turtle.isvisible() @@ -2173,9 +2173,11 @@ It contains: The demoscripts are: +.. tabularcolumns:: |l|L|L| + +----------------+------------------------------+-----------------------+ | Name | Description | Features | -+----------------+------------------------------+-----------------------+ ++================+==============================+=======================+ | bytedesign | complex classical | :func:`tracer`, delay,| | | turtlegraphics pattern | :func:`update` | +----------------+------------------------------+-----------------------+ diff --git a/Doc/library/unicodedata.rst b/Doc/library/unicodedata.rst index 4d9b3c5..a3a7c96 100644 --- a/Doc/library/unicodedata.rst +++ b/Doc/library/unicodedata.rst @@ -66,7 +66,7 @@ It defines the following functions: .. function:: bidirectional(unichr) - Returns the bidirectional category assigned to the Unicode character *unichr* as + Returns the bidirectional class assigned to the Unicode character *unichr* as string. If no such value is defined, an empty string is returned. diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index b53c029..d82c407 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -91,7 +91,7 @@ need to derive from a specific class. Third-party unittest frameworks with a lighter-weight syntax for writing tests. For example, ``assert func(10) == 42``. - `The Python Testing Tools Taxonomy <http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy>`_ + `The Python Testing Tools Taxonomy <http://wiki.python.org/moin/PythonTestingToolsTaxonomy>`_ An extensive list of Python testing tools including functional testing frameworks and mock object libraries. @@ -279,15 +279,15 @@ The ``discover`` sub-command has the following options: Verbose output -.. cmdoption:: -s directory +.. cmdoption:: -s, --start-directory directory - Directory to start discovery ('.' default) + Directory to start discovery (``.`` default) -.. cmdoption:: -p pattern +.. cmdoption:: -p, --pattern pattern - Pattern to match test files ('test*.py' default) + Pattern to match test files (``test*.py`` default) -.. cmdoption:: -t directory +.. cmdoption:: -t, --top-level-directory directory Top level directory of project (defaults to start directory) @@ -584,7 +584,7 @@ that is broken and will fail, but shouldn't be counted as a failure on a Skipping a test is simply a matter of using the :func:`skip` :term:`decorator` or one of its conditional variants. -Basic skipping looks like this: :: +Basic skipping looks like this:: class MyTestCase(unittest.TestCase): @@ -603,7 +603,7 @@ Basic skipping looks like this: :: # windows specific testing code pass -This is the output of running the example above in verbose mode: :: +This is the output of running the example above in verbose mode:: test_format (__main__.MyTestCase) ... skipped 'not supported in this library version' test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping' @@ -614,9 +614,9 @@ This is the output of running the example above in verbose mode: :: OK (skipped=3) -Classes can be skipped just like methods: :: +Classes can be skipped just like methods:: - @skip("showing class skipping") + @unittest.skip("showing class skipping") class MySkippedTestCase(unittest.TestCase): def test_not_run(self): pass @@ -633,12 +633,12 @@ Expected failures use the :func:`expectedFailure` decorator. :: It's easy to roll your own skipping decorators by making a decorator that calls :func:`skip` on the test when it wants it to be skipped. This decorator skips -the test unless the passed object has a certain attribute: :: +the test unless the passed object has a certain attribute:: def skipUnlessHasattr(obj, attr): if hasattr(obj, attr): return lambda func: func - return unittest.skip("{0!r} doesn't have {1!r}".format(obj, attr)) + return unittest.skip("{!r} doesn't have {!r}".format(obj, attr)) The following decorators implement test skipping and expected failures: @@ -660,6 +660,13 @@ The following decorators implement test skipping and expected failures: Mark the test as an expected failure. If the test fails when run, the test is not counted as a failure. +.. exception:: SkipTest(reason) + + This exception is raised to skip a test. + + Usually you can use :meth:`TestCase.skipTest` or one of the skipping + decorators instead of raising this directly. + Skipped tests will not have :meth:`setUp` or :meth:`tearDown` run around them. Skipped classes will not have :meth:`setUpClass` or :meth:`tearDownClass` run. @@ -712,9 +719,9 @@ Test cases .. method:: setUp() Method called to prepare the test fixture. This is called immediately - before calling the test method; any exception raised by this method will - be considered an error rather than a test failure. The default - implementation does nothing. + before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, + any exception raised by this method will be considered an error rather than + a test failure. The default implementation does nothing. .. method:: tearDown() @@ -722,10 +729,10 @@ Test cases Method called immediately after the test method has been called and the result recorded. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly - careful about checking internal state. Any exception raised by this - method will be considered an error rather than a test failure. This - method will only be called if the :meth:`setUp` succeeds, regardless of - the outcome of the test method. The default implementation does nothing. + careful about checking internal state. Any exception, other than :exc:`AssertionError` + or :exc:`SkipTest`, raised by this method will be considered an error rather than a + test failure. This method will only be called if the :meth:`setUp` succeeds, + regardless of the outcome of the test method. The default implementation does nothing. .. method:: setUpClass() @@ -909,8 +916,8 @@ Test cases | :meth:`assertRaises(exc, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | | | <TestCase.assertRaises>` | | | +---------------------------------------------------------+--------------------------------------+------------+ - | :meth:`assertRaisesRegexp(exc, re, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | 2.7 | - | <TestCase.assertRaisesRegexp>` | and the message matches *re* | | + | :meth:`assertRaisesRegexp(exc, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | 2.7 | + | <TestCase.assertRaisesRegexp>` | and the message matches regex *r* | | +---------------------------------------------------------+--------------------------------------+------------+ .. method:: assertRaises(exception, callable, *args, **kwds) @@ -951,7 +958,7 @@ Test cases a regular expression object or a string containing a regular expression suitable for use by :func:`re.search`. Examples:: - self.assertRaisesRegexp(ValueError, 'invalid literal for.*XYZ$', + self.assertRaisesRegexp(ValueError, "invalid literal for.*XYZ'$", int, 'XYZ') or:: @@ -986,10 +993,10 @@ Test cases | :meth:`assertLessEqual(a, b) | ``a <= b`` | 2.7 | | <TestCase.assertLessEqual>` | | | +---------------------------------------+--------------------------------+--------------+ - | :meth:`assertRegexpMatches(s, re) | ``regex.search(s)`` | 2.7 | + | :meth:`assertRegexpMatches(s, r) | ``r.search(s)`` | 2.7 | | <TestCase.assertRegexpMatches>` | | | +---------------------------------------+--------------------------------+--------------+ - | :meth:`assertNotRegexpMatches(s, re) | ``not regex.search(s)`` | 2.7 | + | :meth:`assertNotRegexpMatches(s, r) | ``not r.search(s)`` | 2.7 | | <TestCase.assertNotRegexpMatches>` | | | +---------------------------------------+--------------------------------+--------------+ | :meth:`assertItemsEqual(a, b) | sorted(a) == sorted(b) and | 2.7 | @@ -1010,7 +1017,7 @@ Test cases like the :func:`round` function) and not *significant digits*. If *delta* is supplied instead of *places* then the difference - between *first* and *second* must be less (or more) than *delta*. + between *first* and *second* must be less or equal to (or greater than) *delta*. Supplying both *delta* and *places* raises a ``TypeError``. @@ -1068,6 +1075,8 @@ Test cases sorted(actual))`` but it works with sequences of unhashable objects as well. + In Python 3, this method is named ``assertCountEqual``. + .. versionadded:: 2.7 @@ -1157,7 +1166,7 @@ Test cases .. method:: assertListEqual(list1, list2, msg=None) assertTupleEqual(tuple1, tuple2, msg=None) - Tests that two lists or tuples are equal. If not an error message is + Tests that two lists or tuples are equal. If not, an error message is constructed that shows only the differences between the two. An error is also raised if either of the parameters are of the wrong type. These methods are used by default when comparing lists or tuples with @@ -1426,8 +1435,8 @@ Loading and running tests The :class:`TestLoader` class is used to create test suites from classes and modules. Normally, there is no need to create an instance of this class; the :mod:`unittest` module provides an instance that can be shared as - ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows - customization of some configurable properties. + :data:`unittest.defaultTestLoader`. Using a subclass or instance, however, + allows customization of some configurable properties. :class:`TestLoader` objects have the following methods: @@ -1501,11 +1510,11 @@ Loading and running tests .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None) - Find and return all test modules from the specified start directory, - recursing into subdirectories to find them. Only test files that match - *pattern* will be loaded. (Using shell style pattern matching.) Only - module names that are importable (i.e. are valid Python identifiers) will - be loaded. + Find all the test modules by recursing into subdirectories from the + specified start directory, and return a TestSuite object containing them. + Only test files that match *pattern* will be loaded. (Using shell style + pattern matching.) Only module names that are importable (i.e. are valid + Python identifiers) will be loaded. All test modules must be importable from the top level of the project. If the start directory is not the top level directory then the top level @@ -1594,8 +1603,7 @@ Loading and running tests A list containing 2-tuples of :class:`TestCase` instances and strings holding formatted tracebacks. Each tuple represents a test where a failure - was explicitly signalled using the :meth:`TestCase.fail\*` or - :meth:`TestCase.assert\*` methods. + was explicitly signalled using the :meth:`TestCase.assert\*` methods. .. versionchanged:: 2.2 Contains formatted tracebacks instead of :func:`sys.exc_info` results. @@ -1679,14 +1687,14 @@ Loading and running tests Called after the test case *test* has been executed, regardless of the outcome. - .. method:: startTestRun(test) + .. method:: startTestRun() Called once before any tests are executed. .. versionadded:: 2.7 - .. method:: stopTestRun(test) + .. method:: stopTestRun() Called once after all tests are executed. @@ -1695,7 +1703,7 @@ Loading and running tests .. method:: addError(test, err) - Called when the test case *test* raises an unexpected exception *err* is a + Called when the test case *test* raises an unexpected exception. *err* is a tuple of the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``. @@ -1784,11 +1792,12 @@ Loading and running tests stream, descriptions, verbosity -.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader[, exit[, verbosity[, failfast[, catchbreak[,buffer]]]]]]]]]]) +.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader[, exit[, verbosity[, failfast[, catchbreak[, buffer]]]]]]]]]]) - A command-line program that runs a set of tests; this is primarily for making - test modules conveniently executable. The simplest use for this function is to - include the following line at the end of a test script:: + A command-line program that loads a set of tests from *module* and runs them; + this is primarily for making test modules conveniently executable. + The simplest use for this function is to include the following line at the + end of a test script:: if __name__ == '__main__': unittest.main() @@ -1799,10 +1808,21 @@ Loading and running tests if __name__ == '__main__': unittest.main(verbosity=2) + The *defaultTest* argument is the name of the test to run if no test names + are specified via *argv*. If not specified or ``None`` and no test names are + provided via *argv*, all tests found in *module* are run. + + The *argv* argument can be a list of options passed to the program, with the + first element being the program name. If not specified or ``None``, + the values of :data:`sys.argv` are used. + The *testRunner* argument can either be a test runner class or an already created instance of it. By default ``main`` calls :func:`sys.exit` with an exit code indicating success or failure of the tests run. + The *testLoader* argument has to be a :class:`TestLoader` instance, + and defaults to :data:`defaultTestLoader`. + ``main`` supports being used from the interactive interpreter by passing in the argument ``exit=False``. This displays the result on standard output without calling :func:`sys.exit`:: @@ -1810,14 +1830,14 @@ Loading and running tests >>> from unittest import main >>> main(module='test_module', exit=False) - The ``failfast``, ``catchbreak`` and ``buffer`` parameters have the same + The *failfast*, *catchbreak* and *buffer* parameters have the same effect as the same-name `command-line options`_. Calling ``main`` actually returns an instance of the ``TestProgram`` class. This stores the result of the tests run as the ``result`` attribute. .. versionchanged:: 2.7 - The ``exit``, ``verbosity``, ``failfast``, ``catchbreak`` and ``buffer`` + The *exit*, *verbosity*, *failfast*, *catchbreak* and *buffer* parameters were added. @@ -1860,10 +1880,10 @@ name then the package :file:`__init__.py` will be checked for ``load_tests``. .. note:: - The default pattern is 'test*.py'. This matches all Python files - that start with 'test' but *won't* match any test directories. + The default pattern is ``'test*.py'``. This matches all Python files + that start with ``'test'`` but *won't* match any test directories. - A pattern like 'test*' will match test packages as well as + A pattern like ``'test*'`` will match test packages as well as modules. If the package :file:`__init__.py` defines ``load_tests`` then it will be @@ -1948,7 +1968,7 @@ then you must call up to them yourself. The implementations in If an exception is raised during a ``setUpClass`` then the tests in the class are not run and the ``tearDownClass`` is not run. Skipped classes will not have ``setUpClass`` or ``tearDownClass`` run. If the exception is a -``SkipTest`` exception then the class will be reported as having been skipped +:exc:`SkipTest` exception then the class will be reported as having been skipped instead of as an error. @@ -1965,7 +1985,7 @@ These should be implemented as functions:: If an exception is raised in a ``setUpModule`` then none of the tests in the module will be run and the ``tearDownModule`` will not be run. If the exception is a -``SkipTest`` exception then the module will be reported as having been skipped +:exc:`SkipTest` exception then the module will be reported as having been skipped instead of as an error. diff --git a/Doc/library/urllib.rst b/Doc/library/urllib.rst index 1f5d994..62f198f 100644 --- a/Doc/library/urllib.rst +++ b/Doc/library/urllib.rst @@ -6,11 +6,12 @@ .. note:: The :mod:`urllib` module has been split into parts and renamed in - Python 3.0 to :mod:`urllib.request`, :mod:`urllib.parse`, + Python 3 to :mod:`urllib.request`, :mod:`urllib.parse`, and :mod:`urllib.error`. The :term:`2to3` tool will automatically adapt - imports when converting your sources to 3.0. - Also note that the :func:`urllib.urlopen` function has been removed in - Python 3.0 in favor of :func:`urllib2.urlopen`. + imports when converting your sources to Python 3. + Also note that the :func:`urllib.request.urlopen` function in Python 3 is + equivalent to :func:`urllib2.urlopen` and that :func:`urllib.urlopen` has + been removed. .. index:: single: WWW @@ -32,16 +33,17 @@ High-level interface .. function:: urlopen(url[, data[, proxies]]) - Open a network object denoted by a URL for reading. If the URL does not have a - scheme identifier, or if it has :file:`file:` as its scheme identifier, this - opens a local file (without universal newlines); otherwise it opens a socket to - a server somewhere on the network. If the connection cannot be made the - :exc:`IOError` exception is raised. If all went well, a file-like object is - returned. This supports the following methods: :meth:`read`, :meth:`readline`, - :meth:`readlines`, :meth:`fileno`, :meth:`close`, :meth:`info`, :meth:`getcode` and - :meth:`geturl`. It also has proper support for the :term:`iterator` protocol. One - caveat: the :meth:`read` method, if the size argument is omitted or negative, - may not read until the end of the data stream; there is no good way to determine + Open a network object denoted by a URL for reading. If the URL does not + have a scheme identifier, or if it has :file:`file:` as its scheme + identifier, this opens a local file (without :term:`universal newlines`); + otherwise it opens a socket to a server somewhere on the network. If the + connection cannot be made the :exc:`IOError` exception is raised. If all + went well, a file-like object is returned. This supports the following + methods: :meth:`read`, :meth:`readline`, :meth:`readlines`, :meth:`fileno`, + :meth:`close`, :meth:`info`, :meth:`getcode` and :meth:`geturl`. It also + has proper support for the :term:`iterator` protocol. One caveat: the + :meth:`read` method, if the size argument is omitted or negative, may not + read until the end of the data stream; there is no good way to determine that the entire stream from a socket has been read in the general case. Except for the :meth:`info`, :meth:`getcode` and :meth:`geturl` methods, @@ -131,7 +133,7 @@ High-level interface :envvar:`no_proxy` environment variable. .. deprecated:: 2.6 - The :func:`urlopen` function has been removed in Python 3.0 in favor + The :func:`urlopen` function has been removed in Python 3 in favor of :func:`urllib2.urlopen`. @@ -279,6 +281,13 @@ Utility functions find it, looks for proxy information from Mac OSX System Configuration for Mac OS X and Windows Systems Registry for Windows. +.. note:: + urllib also exposes certain utility functions like splittype, splithost and + others parsing url into various components. But it is recommended to use + :mod:`urlparse` for parsing urls than using these functions directly. + Python 3 does not expose these helper functions from :mod:`urllib.parse` + module. + URL Opener objects ------------------ diff --git a/Doc/library/urllib2.rst b/Doc/library/urllib2.rst index b66ebd7..0411e18 100644 --- a/Doc/library/urllib2.rst +++ b/Doc/library/urllib2.rst @@ -9,9 +9,9 @@ .. note:: The :mod:`urllib2` module has been split across several modules in - Python 3.0 named :mod:`urllib.request` and :mod:`urllib.error`. + Python 3 named :mod:`urllib.request` and :mod:`urllib.error`. The :term:`2to3` tool will automatically adapt imports when converting - your sources to 3.0. + your sources to Python 3. The :mod:`urllib2` module defines functions and classes which help in opening @@ -43,7 +43,7 @@ The :mod:`urllib2` module defines the following functions: timeout setting will be used). This actually only works for HTTP, HTTPS and FTP connections. - This function returns a file-like object with two additional methods: + This function returns a file-like object with three additional methods: * :meth:`geturl` --- return the URL of the resource retrieved, commonly used to determine if a redirect was followed @@ -52,14 +52,18 @@ The :mod:`urllib2` module defines the following functions: in the form of an :class:`mimetools.Message` instance (see `Quick Reference to HTTP Headers <http://www.cs.tut.fi/~jkorpela/http.html>`_) + * :meth:`getcode` --- return the HTTP status code of the response. + Raises :exc:`URLError` on errors. Note that ``None`` may be returned if no handler handles the request (though the default installed global :class:`OpenerDirector` uses :class:`UnknownHandler` to ensure this never happens). - In addition, default installed :class:`ProxyHandler` makes sure the requests - are handled through the proxy when they are set. + In addition, if proxy settings are detected (for example, when a ``*_proxy`` + environment variable like :envvar:`http_proxy` is set), + :class:`ProxyHandler` is default installed and makes sure the requests are + handled through the proxy. .. versionchanged:: 2.6 *timeout* was added. @@ -81,7 +85,8 @@ The :mod:`urllib2` module defines the following functions: subclasses of :class:`BaseHandler` (in which case it must be possible to call the constructor without any parameters). Instances of the following classes will be in front of the *handler*\s, unless the *handler*\s contain them, - instances of them or subclasses of them: :class:`ProxyHandler`, + instances of them or subclasses of them: :class:`ProxyHandler` (if proxy + settings are detected), :class:`UnknownHandler`, :class:`HTTPHandler`, :class:`HTTPDefaultErrorHandler`, :class:`HTTPRedirectHandler`, :class:`FTPHandler`, :class:`FileHandler`, :class:`HTTPErrorProcessor`. @@ -121,7 +126,10 @@ The following exceptions are raised as appropriate: This numeric value corresponds to a value found in the dictionary of codes as found in :attr:`BaseHTTPServer.BaseHTTPRequestHandler.responses`. + .. attribute:: reason + The reason for this error. It can be a message string or another exception + instance. The following classes are provided: @@ -158,7 +166,7 @@ The following classes are provided: should be the request-host of the request for the page containing the image. *unverifiable* should indicate whether the request is unverifiable, as defined - by RFC 2965. It defaults to False. An unverifiable request is one whose URL + by RFC 2965. It defaults to ``False``. An unverifiable request is one whose URL the user did not have the option to approve. For example, if the request is for an image in an HTML document, and the user had no option to approve the automatic fetching of the image, this should be true. @@ -197,9 +205,9 @@ The following classes are provided: Cause requests to go through a proxy. If *proxies* is given, it must be a dictionary mapping protocol names to URLs of proxies. The default is to read the list of proxies from the environment variables - :envvar:`<protocol>_proxy`. If no proxy environment variables are set, in a - Windows environment, proxy settings are obtained from the registry's - Internet Settings section and in a Mac OS X environment, proxy information + :envvar:`<protocol>_proxy`. If no proxy environment variables are set, then + in a Windows environment proxy settings are obtained from the registry's + Internet Settings section, and in a Mac OS X environment proxy information is retrieved from the OS X System Configuration Framework. To disable autodetected proxy pass an empty dictionary. @@ -380,6 +388,17 @@ so all must be overridden in subclasses. Return the selector --- the part of the URL that is sent to the server. +.. method:: Request.get_header(header_name, default=None) + + Return the value of the given header. If the header is not present, return + the default value. + + +.. method:: Request.header_items() + + Return a list of tuples (header_name, header_value) of the Request headers. + + .. method:: Request.set_proxy(host, type) Prepare the request by connecting to a proxy server. The *host* and *type* will diff --git a/Doc/library/urlparse.rst b/Doc/library/urlparse.rst index f118845..1bc361d 100644 --- a/Doc/library/urlparse.rst +++ b/Doc/library/urlparse.rst @@ -13,9 +13,9 @@ pair: relative; URL .. note:: - The :mod:`urlparse` module is renamed to :mod:`urllib.parse` in Python 3.0. + The :mod:`urlparse` module is renamed to :mod:`urllib.parse` in Python 3. The :term:`2to3` tool will automatically adapt imports when converting - your sources to 3.0. + your sources to Python 3. **Source code:** :source:`Lib/urlparse.py` @@ -27,11 +27,11 @@ combine the components back into a URL string, and to convert a "relative URL" to an absolute URL given a "base URL." The module has been designed to match the Internet RFC on Relative Uniform -Resource Locators (and discovered a bug in an earlier draft!). It supports the -following URL schemes: ``file``, ``ftp``, ``gopher``, ``hdl``, ``http``, -``https``, ``imap``, ``mailto``, ``mms``, ``news``, ``nntp``, ``prospero``, -``rsync``, ``rtsp``, ``rtspu``, ``sftp``, ``shttp``, ``sip``, ``sips``, -``snews``, ``svn``, ``svn+ssh``, ``telnet``, ``wais``. +Resource Locators. It supports the following URL schemes: ``file``, ``ftp``, +``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``mailto``, ``mms``, +``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtspu``, ``sftp``, +``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, ``telnet``, +``wais``. .. versionadded:: 2.5 Support for the ``sftp`` and ``sips`` schemes. @@ -71,8 +71,8 @@ The :mod:`urlparse` module defines the following functions: >>> urlparse('//www.cwi.nl:80/%7Eguido/Python.html') ParseResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', params='', query='', fragment='') - >>> urlparse('www.cwi.nl:80/%7Eguido/Python.html') - ParseResult(scheme='', netloc='', path='www.cwi.nl:80/%7Eguido/Python.html', + >>> urlparse('www.cwi.nl/%7Eguido/Python.html') + ParseResult(scheme='', netloc='', path='www.cwi.nl/%7Eguido/Python.html', params='', query='', fragment='') >>> urlparse('help/Python.html') ParseResult(scheme='', netloc='', path='help/Python.html', params='', diff --git a/Doc/library/user.rst b/Doc/library/user.rst index 920f429..5acd7ce 100644 --- a/Doc/library/user.rst +++ b/Doc/library/user.rst @@ -7,7 +7,7 @@ :deprecated: .. deprecated:: 2.6 - The :mod:`user` module has been removed in Python 3.0. + The :mod:`user` module has been removed in Python 3. .. index:: pair: .pythonrc.py; file diff --git a/Doc/library/userdict.rst b/Doc/library/userdict.rst index 2e14c12..0585bda 100644 --- a/Doc/library/userdict.rst +++ b/Doc/library/userdict.rst @@ -114,8 +114,8 @@ The :mod:`UserList` module defines the :class:`UserList` class: .. note:: The :class:`UserList` class has been moved to the :mod:`collections` - module in Python 3.0. The :term:`2to3` tool will automatically adapt - imports when converting your sources to 3.0. + module in Python 3. The :term:`2to3` tool will automatically adapt + imports when converting your sources to Python 3. In addition to supporting the methods and operations of mutable sequences (see @@ -128,7 +128,7 @@ attribute: A real Python list object used to store the contents of the :class:`UserList` class. -**Subclassing requirements:** Subclasses of :class:`UserList` are expect to +**Subclassing requirements:** Subclasses of :class:`UserList` are expected to offer a constructor which can be called with either no arguments or one argument. List operations which return a new sequence attempt to create an instance of the actual implementation class. To do so, it assumes that the @@ -187,8 +187,8 @@ The :mod:`UserString` module defines the following classes: .. note:: The :class:`UserString` class has been moved to the :mod:`collections` - module in Python 3.0. The :term:`2to3` tool will automatically adapt - imports when converting your sources to 3.0. + module in Python 3. The :term:`2to3` tool will automatically adapt + imports when converting your sources to Python 3. @@ -203,7 +203,7 @@ The :mod:`UserString` module defines the following classes: hard to track down. .. deprecated:: 2.6 - The :class:`MutableString` class has been removed in Python 3.0. + The :class:`MutableString` class has been removed in Python 3. In addition to supporting the methods and operations of string and Unicode objects (see section :ref:`string-methods`), :class:`UserString` instances diff --git a/Doc/library/warnings.rst b/Doc/library/warnings.rst index 6f3c105..27658d6 100644 --- a/Doc/library/warnings.rst +++ b/Doc/library/warnings.rst @@ -57,6 +57,8 @@ There are a number of built-in exceptions that represent warning categories. This categorization is useful to be able to filter out groups of warnings. The following warnings category classes are currently defined: +.. tabularcolumns:: |l|p{0.6\linewidth}| + +----------------------------------+-----------------------------------------------+ | Class | Description | +==================================+===============================================+ @@ -167,7 +169,8 @@ By default, Python installs several warning filters, which can be overridden by the command-line options passed to :option:`-W` and calls to :func:`filterwarnings`. -* :exc:`PendingDeprecationWarning`, and :exc:`ImportWarning` are ignored. +* :exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning`, and + :exc:`ImportWarning` are ignored. * :exc:`BytesWarning` is ignored unless the :option:`-b` option is given once or twice; in this case this warning is either printed (``-b``) or turned into an @@ -418,7 +421,7 @@ Available Context Managers .. note:: - In Python 3.0, the arguments to the constructor for + In Python 3, the arguments to the constructor for :class:`catch_warnings` are keyword-only arguments. .. versionadded:: 2.6 diff --git a/Doc/library/weakref.rst b/Doc/library/weakref.rst index 7929c51..1c3cdbb 100644 --- a/Doc/library/weakref.rst +++ b/Doc/library/weakref.rst @@ -53,12 +53,6 @@ own weak references directly. The low-level machinery used by the weak dictionary implementations is exposed by the :mod:`weakref` module for the benefit of advanced uses. -.. note:: - - Weak references to an object are cleared before the object's :meth:`__del__` - is called, to ensure that the weak reference callback (if any) finds the - object still alive. - Not all objects can be weakly referenced; those objects which can include class instances, functions written in Python (but not in C), methods (both bound and unbound), sets, frozensets, file objects, :term:`generator`\s, type objects, @@ -169,7 +163,7 @@ than needed. .. method:: WeakKeyDictionary.iterkeyrefs() - Return an :term:`iterator` that yields the weak references to the keys. + Return an iterable of the weak references to the keys. .. versionadded:: 2.5 @@ -201,7 +195,7 @@ methods of :class:`WeakKeyDictionary` objects. .. method:: WeakValueDictionary.itervaluerefs() - Return an :term:`iterator` that yields the weak references to the values. + Return an iterable of the weak references to the values. .. versionadded:: 2.5 diff --git a/Doc/library/webbrowser.rst b/Doc/library/webbrowser.rst index 1a27d07..e6e7ea2 100644 --- a/Doc/library/webbrowser.rst +++ b/Doc/library/webbrowser.rst @@ -36,7 +36,9 @@ The script :program:`webbrowser` can be used as a command-line interface for the module. It accepts an URL as the argument. It accepts the following optional parameters: ``-n`` opens the URL in a new browser window, if possible; ``-t`` opens the URL in a new browser page ("tab"). The options are, -naturally, mutually exclusive. +naturally, mutually exclusive. Usage example:: + + python -m webbrowser -t "http://www.python.org" The following exception is defined: @@ -48,7 +50,7 @@ The following exception is defined: The following functions are defined: -.. function:: open(url[, new=0[, autoraise=True]]) +.. function:: open(url, new=0, autoraise=True) Display *url* using the default browser. If *new* is 0, the *url* is opened in the same browser window if possible. If *new* is 1, a new browser window @@ -138,9 +140,9 @@ for the controller classes, all defined in this module. +-----------------------+-----------------------------------------+-------+ | ``'windows-default'`` | :class:`WindowsDefault` | \(2) | +-----------------------+-----------------------------------------+-------+ -| ``'internet-config'`` | :class:`InternetConfig` | \(3) | +| ``'macosx'`` | :class:`MacOSX('default')` | \(3) | +-----------------------+-----------------------------------------+-------+ -| ``'macosx'`` | :class:`MacOSX('default')` | \(4) | +| ``'safari'`` | :class:`MacOSX('safari')` | \(3) | +-----------------------+-----------------------------------------+-------+ Notes: @@ -156,9 +158,6 @@ Notes: Only on Windows platforms. (3) - Only on Mac OS platforms; requires the standard MacPython :mod:`ic` module. - -(4) Only on Mac OS X platform. Here are some simple examples:: @@ -181,7 +180,7 @@ Browser controllers provide these methods which parallel three of the module-level convenience functions: -.. method:: controller.open(url[, new=0[, autoraise=True]]) +.. method:: controller.open(url, new=0, autoraise=True) Display *url* using the browser handled by this controller. If *new* is 1, a new browser window is opened if possible. If *new* is 2, a new browser page ("tab") diff --git a/Doc/library/whichdb.rst b/Doc/library/whichdb.rst index 7048a0e..3bcb57c 100644 --- a/Doc/library/whichdb.rst +++ b/Doc/library/whichdb.rst @@ -6,8 +6,8 @@ .. note:: The :mod:`whichdb` module's only function has been put into the :mod:`dbm` - module in Python 3.0. The :term:`2to3` tool will automatically adapt imports - when converting your sources to 3.0. + module in Python 3. The :term:`2to3` tool will automatically adapt imports + when converting your sources to Python 3. The single function in this module attempts to guess which of the several simple diff --git a/Doc/library/winsound.rst b/Doc/library/winsound.rst index 2325081..5c590ff 100644 --- a/Doc/library/winsound.rst +++ b/Doc/library/winsound.rst @@ -133,6 +133,10 @@ provided by Windows platforms. It includes functions and several constants. Return immediately if the sound driver is busy. + .. note:: + + This flag is not supported on modern Windows platforms. + .. data:: MB_ICONASTERISK diff --git a/Doc/library/wsgiref.rst b/Doc/library/wsgiref.rst index 3163497..0b0c7c2 100644 --- a/Doc/library/wsgiref.rst +++ b/Doc/library/wsgiref.rst @@ -59,7 +59,7 @@ parameter expect a WSGI-compliant dictionary to be supplied; please see found, and "http" otherwise. -.. function:: request_uri(environ [, include_query=1]) +.. function:: request_uri(environ, include_query=1) Return the full request URI, optionally including the query string, using the algorithm found in the "URL Reconstruction" section of :pep:`333`. If @@ -148,7 +148,7 @@ also provides these miscellaneous utilities: :rfc:`2616`. -.. class:: FileWrapper(filelike [, blksize=8192]) +.. class:: FileWrapper(filelike, blksize=8192) A wrapper to convert a file-like object to an :term:`iterator`. The resulting objects support both :meth:`__getitem__` and :meth:`__iter__` iteration styles, for @@ -271,7 +271,7 @@ request. (E.g., using the :func:`shift_path_info` function from :mod:`wsgiref.util`.) -.. function:: make_server(host, port, app [, server_class=WSGIServer [, handler_class=WSGIRequestHandler]]) +.. function:: make_server(host, port, app, server_class=WSGIServer, handler_class=WSGIRequestHandler) Create a new WSGI server listening on *host* and *port*, accepting connections for *app*. The return value is an instance of the supplied *server_class*, and @@ -460,7 +460,7 @@ input, output, and error streams. environment. -.. class:: BaseCGIHandler(stdin, stdout, stderr, environ [, multithread=True [, multiprocess=False]]) +.. class:: BaseCGIHandler(stdin, stdout, stderr, environ, multithread=True, multiprocess=False) Similar to :class:`CGIHandler`, but instead of using the :mod:`sys` and :mod:`os` modules, the CGI environment and I/O streams are specified explicitly. @@ -475,7 +475,7 @@ input, output, and error streams. instead of :class:`SimpleHandler`. -.. class:: SimpleHandler(stdin, stdout, stderr, environ [,multithread=True [, multiprocess=False]]) +.. class:: SimpleHandler(stdin, stdout, stderr, environ, multithread=True, multiprocess=False) Similar to :class:`BaseCGIHandler`, but designed for use with HTTP origin servers. If you are writing an HTTP server implementation, you will probably diff --git a/Doc/library/xdrlib.rst b/Doc/library/xdrlib.rst index e56650c..6f05306 100644 --- a/Doc/library/xdrlib.rst +++ b/Doc/library/xdrlib.rst @@ -274,6 +274,5 @@ Here is an example of how you would catch one of these exceptions:: p = xdrlib.Packer() try: p.pack_double(8.01) - except xdrlib.ConversionError, instance: + except xdrlib.ConversionError as instance: print 'packing the double failed:', instance.msg - diff --git a/Doc/library/xml.dom.minidom.rst b/Doc/library/xml.dom.minidom.rst index 69e9e56..1ff7024 100644 --- a/Doc/library/xml.dom.minidom.rst +++ b/Doc/library/xml.dom.minidom.rst @@ -1,8 +1,8 @@ -:mod:`xml.dom.minidom` --- Lightweight DOM implementation -========================================================= +:mod:`xml.dom.minidom` --- Minimal DOM implementation +===================================================== .. module:: xml.dom.minidom - :synopsis: Lightweight Document Object Model (DOM) implementation. + :synopsis: Minimal Document Object Model (DOM) implementation. .. moduleauthor:: Paul Prescod <paul@prescod.net> .. sectionauthor:: Paul Prescod <paul@prescod.net> .. sectionauthor:: Martin v. Löwis <martin@v.loewis.de> @@ -14,9 +14,19 @@ -------------- -:mod:`xml.dom.minidom` is a light-weight implementation of the Document Object -Model interface. It is intended to be simpler than the full DOM and also -significantly smaller. +:mod:`xml.dom.minidom` is a minimal implementation of the Document Object +Model interface, with an API similar to that in other languages. It is intended +to be simpler than the full DOM and also significantly smaller. Users who are +not already proficient with the DOM should consider using the +:mod:`xml.etree.ElementTree` module for their XML processing instead + + +.. warning:: + + The :mod:`xml.dom.minidom` module is not secure against + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see :ref:`xml-vulnerabilities`. + DOM applications typically start by parsing some XML into a DOM. With :mod:`xml.dom.minidom`, this is done through the parse functions:: @@ -48,7 +58,7 @@ instead: .. function:: parseString(string[, parser]) Return a :class:`Document` that represents the *string*. This method creates a - :class:`StringIO` object for the string and passes that on to :func:`parse`. + :class:`~StringIO.StringIO` object for the string and passes that on to :func:`parse`. Both functions return a :class:`Document` object representing the content of the document. @@ -121,7 +131,7 @@ module documentation. This section lists the differences between the API and to discard children of that node. -.. method:: Node.writexml(writer[, indent=""[, addindent=""[, newl=""]]]) +.. method:: Node.writexml(writer, indent="", addindent="", newl="") Write XML to the writer object. The writer should have a :meth:`write` method which matches that of the file object interface. The *indent* parameter is the @@ -266,4 +276,4 @@ utility to most DOM users. .. [#] The encoding string included in XML output should conform to the appropriate standards. For example, "UTF-8" is valid, but "UTF8" is not. See http://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl - and http://www.iana.org/assignments/character-sets . + and http://www.iana.org/assignments/character-sets\ . diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst index bad0daa..9032706 100644 --- a/Doc/library/xml.dom.pulldom.rst +++ b/Doc/library/xml.dom.pulldom.rst @@ -16,6 +16,13 @@ Object Model representation of a document from SAX events. +.. warning:: + + The :mod:`xml.dom.pulldom` module is not secure against + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see :ref:`xml-vulnerabilities`. + + .. class:: PullDOM([documentFactory]) :class:`xml.sax.handler.ContentHandler` implementation that ... diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst index 1069615..541d649 100644 --- a/Doc/library/xml.dom.rst +++ b/Doc/library/xml.dom.rst @@ -374,7 +374,7 @@ All of the components of an XML document are subclasses of :class:`Node`. Add a new child node to this node at the end of the list of children, returning *newChild*. If the node was already in - in the tree, it is removed first. + the tree, it is removed first. .. method:: Node.insertBefore(newChild, refChild) @@ -441,14 +441,15 @@ objects: In addition, the Python DOM interface requires that some additional support is provided to allow :class:`NodeList` objects to be used as Python sequences. All -:class:`NodeList` implementations must include support for :meth:`__len__` and -:meth:`__getitem__`; this allows iteration over the :class:`NodeList` in +:class:`NodeList` implementations must include support for +:meth:`~object.__len__` and +:meth:`~object.__getitem__`; this allows iteration over the :class:`NodeList` in :keyword:`for` statements and proper support for the :func:`len` built-in function. If a DOM implementation supports modification of the document, the -:class:`NodeList` implementation must also support the :meth:`__setitem__` and -:meth:`__delitem__` methods. +:class:`NodeList` implementation must also support the +:meth:`~object.__setitem__` and :meth:`~object.__delitem__` methods. .. _dom-documenttype-objects: diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst index 5fbbf20..f14742e 100644 --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -16,6 +16,14 @@ The :class:`Element` type is a flexible container object, designed to store hierarchical data structures in memory. The type can be described as a cross between a list and a dictionary. + +.. warning:: + + The :mod:`xml.etree.ElementTree` module is not secure against + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see :ref:`xml-vulnerabilities`. + + Each element has a number of properties associated with it: * a tag which is a string identifying what kind of data this element represents @@ -46,11 +54,315 @@ the xml.etree.ElementTree. `Introducing ElementTree 1.3 <http://effbot.org/zone/elementtree-13-intro.htm>`_. +Tutorial +-------- + +This is a short tutorial for using :mod:`xml.etree.ElementTree` (``ET`` in +short). The goal is to demonstrate some of the building blocks and basic +concepts of the module. + +XML tree and elements +^^^^^^^^^^^^^^^^^^^^^ + +XML is an inherently hierarchical data format, and the most natural way to +represent it is with a tree. ``ET`` has two classes for this purpose - +:class:`ElementTree` represents the whole XML document as a tree, and +:class:`Element` represents a single node in this tree. Interactions with +the whole document (reading and writing to/from files) are usually done +on the :class:`ElementTree` level. Interactions with a single XML element +and its sub-elements are done on the :class:`Element` level. + +.. _elementtree-parsing-xml: + +Parsing XML +^^^^^^^^^^^ + +We'll be using the following XML document as the sample data for this section: + +.. code-block:: xml + + <?xml version="1.0"?> + <data> + <country name="Liechtenstein"> + <rank>1</rank> + <year>2008</year> + <gdppc>141100</gdppc> + <neighbor name="Austria" direction="E"/> + <neighbor name="Switzerland" direction="W"/> + </country> + <country name="Singapore"> + <rank>4</rank> + <year>2011</year> + <gdppc>59900</gdppc> + <neighbor name="Malaysia" direction="N"/> + </country> + <country name="Panama"> + <rank>68</rank> + <year>2011</year> + <gdppc>13600</gdppc> + <neighbor name="Costa Rica" direction="W"/> + <neighbor name="Colombia" direction="E"/> + </country> + </data> + +We have a number of ways to import the data. Reading the file from disk:: + + import xml.etree.ElementTree as ET + tree = ET.parse('country_data.xml') + root = tree.getroot() + +Reading the data from a string:: + + root = ET.fromstring(country_data_as_string) + +:func:`fromstring` parses XML from a string directly into an :class:`Element`, +which is the root element of the parsed tree. Other parsing functions may +create an :class:`ElementTree`. Check the documentation to be sure. + +As an :class:`Element`, ``root`` has a tag and a dictionary of attributes:: + + >>> root.tag + 'data' + >>> root.attrib + {} + +It also has children nodes over which we can iterate:: + + >>> for child in root: + ... print child.tag, child.attrib + ... + country {'name': 'Liechtenstein'} + country {'name': 'Singapore'} + country {'name': 'Panama'} + +Children are nested, and we can access specific child nodes by index:: + + >>> root[0][1].text + '2008' + +Finding interesting elements +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:class:`Element` has some useful methods that help iterate recursively over all +the sub-tree below it (its children, their children, and so on). For example, +:meth:`Element.iter`:: + + >>> for neighbor in root.iter('neighbor'): + ... print neighbor.attrib + ... + {'name': 'Austria', 'direction': 'E'} + {'name': 'Switzerland', 'direction': 'W'} + {'name': 'Malaysia', 'direction': 'N'} + {'name': 'Costa Rica', 'direction': 'W'} + {'name': 'Colombia', 'direction': 'E'} + +:meth:`Element.findall` finds only elements with a tag which are direct +children of the current element. :meth:`Element.find` finds the *first* child +with a particular tag, and :attr:`Element.text` accesses the element's text +content. :meth:`Element.get` accesses the element's attributes:: + + >>> for country in root.findall('country'): + ... rank = country.find('rank').text + ... name = country.get('name') + ... print name, rank + ... + Liechtenstein 1 + Singapore 4 + Panama 68 + +More sophisticated specification of which elements to look for is possible by +using :ref:`XPath <elementtree-xpath>`. + +Modifying an XML File +^^^^^^^^^^^^^^^^^^^^^ + +:class:`ElementTree` provides a simple way to build XML documents and write them to files. +The :meth:`ElementTree.write` method serves this purpose. + +Once created, an :class:`Element` object may be manipulated by directly changing +its fields (such as :attr:`Element.text`), adding and modifying attributes +(:meth:`Element.set` method), as well as adding new children (for example +with :meth:`Element.append`). + +Let's say we want to add one to each country's rank, and add an ``updated`` +attribute to the rank element:: + + >>> for rank in root.iter('rank'): + ... new_rank = int(rank.text) + 1 + ... rank.text = str(new_rank) + ... rank.set('updated', 'yes') + ... + >>> tree.write('output.xml') + +Our XML now looks like this: + +.. code-block:: xml + + <?xml version="1.0"?> + <data> + <country name="Liechtenstein"> + <rank updated="yes">2</rank> + <year>2008</year> + <gdppc>141100</gdppc> + <neighbor name="Austria" direction="E"/> + <neighbor name="Switzerland" direction="W"/> + </country> + <country name="Singapore"> + <rank updated="yes">5</rank> + <year>2011</year> + <gdppc>59900</gdppc> + <neighbor name="Malaysia" direction="N"/> + </country> + <country name="Panama"> + <rank updated="yes">69</rank> + <year>2011</year> + <gdppc>13600</gdppc> + <neighbor name="Costa Rica" direction="W"/> + <neighbor name="Colombia" direction="E"/> + </country> + </data> + +We can remove elements using :meth:`Element.remove`. Let's say we want to +remove all countries with a rank higher than 50:: + + >>> for country in root.findall('country'): + ... rank = int(country.find('rank').text) + ... if rank > 50: + ... root.remove(country) + ... + >>> tree.write('output.xml') + +Our XML now looks like this: + +.. code-block:: xml + + <?xml version="1.0"?> + <data> + <country name="Liechtenstein"> + <rank updated="yes">2</rank> + <year>2008</year> + <gdppc>141100</gdppc> + <neighbor name="Austria" direction="E"/> + <neighbor name="Switzerland" direction="W"/> + </country> + <country name="Singapore"> + <rank updated="yes">5</rank> + <year>2011</year> + <gdppc>59900</gdppc> + <neighbor name="Malaysia" direction="N"/> + </country> + </data> + +Building XML documents +^^^^^^^^^^^^^^^^^^^^^^ + +The :func:`SubElement` function also provides a convenient way to create new +sub-elements for a given element:: + + >>> a = ET.Element('a') + >>> b = ET.SubElement(a, 'b') + >>> c = ET.SubElement(a, 'c') + >>> d = ET.SubElement(c, 'd') + >>> ET.dump(a) + <a><b /><c><d /></c></a> + +Additional resources +^^^^^^^^^^^^^^^^^^^^ + +See http://effbot.org/zone/element-index.htm for tutorials and links to other +docs. + +.. _elementtree-xpath: + +XPath support +------------- + +This module provides limited support for +`XPath expressions <http://www.w3.org/TR/xpath>`_ for locating elements in a +tree. The goal is to support a small subset of the abbreviated syntax; a full +XPath engine is outside the scope of the module. + +Example +^^^^^^^ + +Here's an example that demonstrates some of the XPath capabilities of the +module. We'll be using the ``countrydata`` XML document from the +:ref:`Parsing XML <elementtree-parsing-xml>` section:: + + import xml.etree.ElementTree as ET + + root = ET.fromstring(countrydata) + + # Top-level elements + root.findall(".") + + # All 'neighbor' grand-children of 'country' children of the top-level + # elements + root.findall("./country/neighbor") + + # Nodes with name='Singapore' that have a 'year' child + root.findall(".//year/..[@name='Singapore']") + + # 'year' nodes that are children of nodes with name='Singapore' + root.findall(".//*[@name='Singapore']/year") + + # All 'neighbor' nodes that are the second child of their parent + root.findall(".//neighbor[2]") + +Supported XPath syntax +^^^^^^^^^^^^^^^^^^^^^^ + +.. tabularcolumns:: |l|L| + ++-----------------------+------------------------------------------------------+ +| Syntax | Meaning | ++=======================+======================================================+ +| ``tag`` | Selects all child elements with the given tag. | +| | For example, ``spam`` selects all child elements | +| | named ``spam``, and ``spam/egg`` selects all | +| | grandchildren named ``egg`` in all children named | +| | ``spam``. | ++-----------------------+------------------------------------------------------+ +| ``*`` | Selects all child elements. For example, ``*/egg`` | +| | selects all grandchildren named ``egg``. | ++-----------------------+------------------------------------------------------+ +| ``.`` | Selects the current node. This is mostly useful | +| | at the beginning of the path, to indicate that it's | +| | a relative path. | ++-----------------------+------------------------------------------------------+ +| ``//`` | Selects all subelements, on all levels beneath the | +| | current element. For example, ``.//egg`` selects | +| | all ``egg`` elements in the entire tree. | ++-----------------------+------------------------------------------------------+ +| ``..`` | Selects the parent element. | ++-----------------------+------------------------------------------------------+ +| ``[@attrib]`` | Selects all elements that have the given attribute. | ++-----------------------+------------------------------------------------------+ +| ``[@attrib='value']`` | Selects all elements for which the given attribute | +| | has the given value. The value cannot contain | +| | quotes. | ++-----------------------+------------------------------------------------------+ +| ``[tag]`` | Selects all elements that have a child named | +| | ``tag``. Only immediate children are supported. | ++-----------------------+------------------------------------------------------+ +| ``[position]`` | Selects all elements that are located at the given | +| | position. The position can be either an integer | +| | (1 is the first position), the expression ``last()`` | +| | (for the last position), or a position relative to | +| | the last position (e.g. ``last()-1``). | ++-----------------------+------------------------------------------------------+ + +Predicates (expressions within square brackets) must be preceded by a tag +name, an asterisk, or another predicate. ``position`` predicates must be +preceded by a tag name. + +Reference +--------- .. _elementtree-functions: Functions ---------- +^^^^^^^^^ .. function:: Comment(text=None) @@ -101,8 +413,9 @@ Functions going on to the user. *source* is a filename or file object containing XML data. *events* is a list of events to report back. If omitted, only "end" events are reported. *parser* is an optional parser instance. If not - given, the standard :class:`XMLParser` parser is used. Returns an - :term:`iterator` providing ``(event, elem)`` pairs. + given, the standard :class:`XMLParser` parser is used. *parser* is not + supported by ``cElementTree``. Returns an :term:`iterator` providing + ``(event, elem)`` pairs. .. note:: @@ -196,8 +509,7 @@ Functions .. _elementtree-element-objects: Element Objects ---------------- - +^^^^^^^^^^^^^^^ .. class:: Element(tag, attrib={}, **extra) @@ -368,8 +680,9 @@ Element Objects or contents. :class:`Element` objects also support the following sequence type methods - for working with subelements: :meth:`__delitem__`, :meth:`__getitem__`, - :meth:`__setitem__`, :meth:`__len__`. + for working with subelements: :meth:`~object.__delitem__`, + :meth:`~object.__getitem__`, :meth:`~object.__setitem__`, + :meth:`~object.__len__`. Caution: Elements with no subelements will test as ``False``. This behavior will change in future versions. Use specific ``len(elem)`` or ``elem is @@ -387,7 +700,7 @@ Element Objects .. _elementtree-elementtree-objects: ElementTree Objects -------------------- +^^^^^^^^^^^^^^^^^^^ .. class:: ElementTree(element=None, file=None) @@ -409,26 +722,17 @@ ElementTree Objects .. method:: find(match) - Finds the first toplevel element matching *match*. *match* may be a tag - name or path. Same as getroot().find(match). Returns the first matching - element, or ``None`` if no element was found. + Same as :meth:`Element.find`, starting at the root of the tree. .. method:: findall(match) - Finds all matching subelements, by tag name or path. Same as - getroot().findall(match). *match* may be a tag name or path. Returns a - list containing all matching elements, in document order. + Same as :meth:`Element.findall`, starting at the root of the tree. .. method:: findtext(match, default=None) - Finds the element text for the first toplevel element with given tag. - Same as getroot().findtext(match). *match* may be a tag name or path. - *default* is the value to return if the element was not found. Returns - the text content of the first matching element, or the default value no - element was found. Note that if the element is found, but has no text - content, this method returns an empty string. + Same as :meth:`Element.findtext`, starting at the root of the tree. .. method:: getiterator(tag=None) @@ -466,13 +770,15 @@ ElementTree Objects root element. - .. method:: write(file, encoding="us-ascii", xml_declaration=None, method="xml") + .. method:: write(file, encoding="us-ascii", xml_declaration=None, \ + default_namespace=None, method="xml") Writes the element tree to a file, as XML. *file* is a file name, or a file object opened for writing. *encoding* [1]_ is the output encoding (default is US-ASCII). *xml_declaration* controls if an XML declaration should be added to the file. Use False for never, True for always, None - for only if not US-ASCII or UTF-8 (default is None). *method* is either + for only if not US-ASCII or UTF-8 (default is None). *default_namespace* + sets the default XML namespace (for "xmlns"). *method* is either ``"xml"``, ``"html"`` or ``"text"`` (default is ``"xml"``). Returns an encoded string. @@ -507,7 +813,7 @@ Example of changing the attribute "target" of every link in first paragraph:: .. _elementtree-qname-objects: QName Objects -------------- +^^^^^^^^^^^^^ .. class:: QName(text_or_uri, tag=None) @@ -523,7 +829,7 @@ QName Objects .. _elementtree-treebuilder-objects: TreeBuilder Objects -------------------- +^^^^^^^^^^^^^^^^^^^ .. class:: TreeBuilder(element_factory=None) @@ -574,7 +880,7 @@ TreeBuilder Objects .. _elementtree-xmlparser-objects: XMLParser Objects ------------------ +^^^^^^^^^^^^^^^^^ .. class:: XMLParser(html=0, target=None, encoding=None) diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst new file mode 100644 index 0000000..e56eb2c --- /dev/null +++ b/Doc/library/xml.rst @@ -0,0 +1,136 @@ +.. _xml: + +XML Processing Modules +====================== + +.. module:: xml + :synopsis: Package containing XML processing modules +.. sectionauthor:: Christian Heimes <christian@python.org> +.. sectionauthor:: Georg Brandl <georg@python.org> + + +Python's interfaces for processing XML are grouped in the ``xml`` package. + +.. warning:: + + The XML modules are not secure against erroneous or maliciously + constructed data. If you need to parse untrusted or unauthenticated data see + :ref:`xml-vulnerabilities`. + +It is important to note that modules in the :mod:`xml` package require that +there be at least one SAX-compliant XML parser available. The Expat parser is +included with Python, so the :mod:`xml.parsers.expat` module will always be +available. + +The documentation for the :mod:`xml.dom` and :mod:`xml.sax` packages are the +definition of the Python bindings for the DOM and SAX interfaces. + +The XML handling submodules are: + +* :mod:`xml.etree.ElementTree`: the ElementTree API, a simple and lightweight + XML processor + +.. + +* :mod:`xml.dom`: the DOM API definition +* :mod:`xml.dom.minidom`: a minimal DOM implementation +* :mod:`xml.dom.pulldom`: support for building partial DOM trees + +.. + +* :mod:`xml.sax`: SAX2 base classes and convenience functions +* :mod:`xml.parsers.expat`: the Expat parser binding + + +.. _xml-vulnerabilities: + +XML vulnerabilities +=================== + +The XML processing modules are not secure against maliciously constructed data. +An attacker can abuse vulnerabilities for e.g. denial of service attacks, to +access local files, to generate network connections to other machines, or +to or circumvent firewalls. The attacks on XML abuse unfamiliar features +like inline `DTD`_ (document type definition) with entities. + +The following table gives an overview of the known attacks and if the various +modules are vulnerable to them. + +========================= ======== ========= ========= ======== ========= +kind sax etree minidom pulldom xmlrpc +========================= ======== ========= ========= ======== ========= +billion laughs **Yes** **Yes** **Yes** **Yes** **Yes** +quadratic blowup **Yes** **Yes** **Yes** **Yes** **Yes** +external entity expansion **Yes** No (1) No (2) **Yes** No (3) +DTD retrieval **Yes** No No **Yes** No +decompression bomb No No No No **Yes** +========================= ======== ========= ========= ======== ========= + +1. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a + ParserError when an entity occurs. +2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns + the unexpanded entity verbatim. +3. :mod:`xmlrpclib` doesn't expand external entities and omits them. + + +billion laughs / exponential entity expansion + The `Billion Laughs`_ attack -- also known as exponential entity expansion -- + uses multiple levels of nested entities. Each entity refers to another entity + several times, the final entity definition contains a small string. Eventually + the small string is expanded to several gigabytes. The exponential expansion + consumes lots of CPU time, too. + +quadratic blowup entity expansion + A quadratic blowup attack is similar to a `Billion Laughs`_ attack; it abuses + entity expansion, too. Instead of nested entities it repeats one large entity + with a couple of thousand chars over and over again. The attack isn't as + efficient as the exponential case but it avoids triggering countermeasures of + parsers against heavily nested entities. + +external entity expansion + Entity declarations can contain more than just text for replacement. They can + also point to external resources by public identifiers or system identifiers. + System identifiers are standard URIs or can refer to local files. The XML + parser retrieves the resource with e.g. HTTP or FTP requests and embeds the + content into the XML document. + +DTD retrieval + Some XML libraries like Python's :mod:`xml.dom.pulldom` retrieve document type + definitions from remote or local locations. The feature has similar + implications as the external entity expansion issue. + +decompression bomb + The issue of decompression bombs (aka `ZIP bomb`_) apply to all XML libraries + that can parse compressed XML stream like gzipped HTTP streams or LZMA-ed + files. For an attacker it can reduce the amount of transmitted data by three + magnitudes or more. + +The documentation of `defusedxml`_ on PyPI has further information about +all known attack vectors with examples and references. + +defused packages +---------------- + +These external packages are recommended for any code that parses +untrusted XML data. + +`defusedxml`_ is a pure Python package with modified subclasses of all stdlib +XML parsers that prevent any potentially malicious operation. The +package also ships with example exploits and extended documentation on more +XML exploits like xpath injection. + +`defusedexpat`_ provides a modified libexpat and patched replacement +:mod:`pyexpat` extension module with countermeasures against entity expansion +DoS attacks. Defusedexpat still allows a sane and configurable amount of entity +expansions. The modifications will be merged into future releases of Python. + +The workarounds and modifications are not included in patch releases as they +break backward compatibility. After all inline DTD and entity expansion are +well-defined XML features. + + +.. _defusedxml: https://pypi.python.org/pypi/defusedxml/ +.. _defusedexpat: https://pypi.python.org/pypi/defusedexpat/ +.. _Billion Laughs: http://en.wikipedia.org/wiki/Billion_laughs +.. _ZIP bomb: http://en.wikipedia.org/wiki/Zip_bomb +.. _DTD: http://en.wikipedia.org/wiki/Document_Type_Definition diff --git a/Doc/library/xml.sax.handler.rst b/Doc/library/xml.sax.handler.rst index 23f429e..15140a3 100644 --- a/Doc/library/xml.sax.handler.rst +++ b/Doc/library/xml.sax.handler.rst @@ -240,7 +240,8 @@ events in the input document: Signals the start of an element in non-namespace mode. The *name* parameter contains the raw XML 1.0 name of the element type as a - string and the *attrs* parameter holds an object of the :class:`Attributes` + string and the *attrs* parameter holds an object of the + :class:`~xml.sax.xmlreader.Attributes` interface (see :ref:`attributes-objects`) containing the attributes of the element. The object passed as *attrs* may be re-used by the parser; holding on to a reference to it is not a reliable way to keep a copy of the attributes. @@ -263,7 +264,8 @@ events in the input document: The *name* parameter contains the name of the element type as a ``(uri, localname)`` tuple, the *qname* parameter contains the raw XML 1.0 name used in the source document, and the *attrs* parameter holds an instance of the - :class:`AttributesNS` interface (see :ref:`attributes-ns-objects`) + :class:`~xml.sax.xmlreader.AttributesNS` interface (see + :ref:`attributes-ns-objects`) containing the attributes of the element. If no namespace is associated with the element, the *uri* component of *name* will be ``None``. The object passed as *attrs* may be re-used by the parser; holding on to a reference to it is not @@ -379,8 +381,9 @@ ErrorHandler Objects -------------------- Objects with this interface are used to receive error and warning information -from the :class:`XMLReader`. If you create an object that implements this -interface, then register the object with your :class:`XMLReader`, the parser +from the :class:`~xml.sax.xmlreader.XMLReader`. If you create an object that +implements this interface, then register the object with your +:class:`~xml.sax.xmlreader.XMLReader`, the parser will call the methods in your object to report all warnings and errors. There are three levels of errors available: warnings, (possibly) recoverable errors, and unrecoverable errors. All methods take a :exc:`SAXParseException` as the diff --git a/Doc/library/xml.sax.reader.rst b/Doc/library/xml.sax.reader.rst index 4b3c18a..6956cd1 100644 --- a/Doc/library/xml.sax.reader.rst +++ b/Doc/library/xml.sax.reader.rst @@ -109,47 +109,50 @@ The :class:`XMLReader` interface supports the following methods: .. method:: XMLReader.getContentHandler() - Return the current :class:`ContentHandler`. + Return the current :class:`~xml.sax.handler.ContentHandler`. .. method:: XMLReader.setContentHandler(handler) - Set the current :class:`ContentHandler`. If no :class:`ContentHandler` is set, - content events will be discarded. + Set the current :class:`~xml.sax.handler.ContentHandler`. If no + :class:`~xml.sax.handler.ContentHandler` is set, content events will be + discarded. .. method:: XMLReader.getDTDHandler() - Return the current :class:`DTDHandler`. + Return the current :class:`~xml.sax.handler.DTDHandler`. .. method:: XMLReader.setDTDHandler(handler) - Set the current :class:`DTDHandler`. If no :class:`DTDHandler` is set, DTD + Set the current :class:`~xml.sax.handler.DTDHandler`. If no + :class:`~xml.sax.handler.DTDHandler` is set, DTD events will be discarded. .. method:: XMLReader.getEntityResolver() - Return the current :class:`EntityResolver`. + Return the current :class:`~xml.sax.handler.EntityResolver`. .. method:: XMLReader.setEntityResolver(handler) - Set the current :class:`EntityResolver`. If no :class:`EntityResolver` is set, + Set the current :class:`~xml.sax.handler.EntityResolver`. If no + :class:`~xml.sax.handler.EntityResolver` is set, attempts to resolve an external entity will result in opening the system identifier for the entity, and fail if it is not available. .. method:: XMLReader.getErrorHandler() - Return the current :class:`ErrorHandler`. + Return the current :class:`~xml.sax.handler.ErrorHandler`. .. method:: XMLReader.setErrorHandler(handler) - Set the current error handler. If no :class:`ErrorHandler` is set, errors will - be raised as exceptions, and warnings will be printed. + Set the current error handler. If no :class:`~xml.sax.handler.ErrorHandler` + is set, errors will be raised as exceptions, and warnings will be printed. .. method:: XMLReader.setLocale(locale) @@ -326,8 +329,13 @@ The :class:`Attributes` Interface --------------------------------- :class:`Attributes` objects implement a portion of the mapping protocol, -including the methods :meth:`copy`, :meth:`get`, :meth:`has_key`, :meth:`items`, -:meth:`keys`, and :meth:`values`. The following methods are also provided: +including the methods :meth:`~collections.Mapping.copy`, +:meth:`~collections.Mapping.get`, +:meth:`~collections.Mapping.has_key`, +:meth:`~collections.Mapping.items`, +:meth:`~collections.Mapping.keys`, +and :meth:`~collections.Mapping.values`. The following methods +are also provided: .. method:: Attributes.getLength() diff --git a/Doc/library/xml.sax.rst b/Doc/library/xml.sax.rst index 43d17c2..25e4fa9 100644 --- a/Doc/library/xml.sax.rst +++ b/Doc/library/xml.sax.rst @@ -16,12 +16,21 @@ Simple API for XML (SAX) interface for Python. The package itself provides the SAX exceptions and the convenience functions which will be most used by users of the SAX API. + +.. warning:: + + The :mod:`xml.sax` module is not secure against maliciously + constructed data. If you need to parse untrusted or unauthenticated data see + :ref:`xml-vulnerabilities`. + + The convenience functions are: .. function:: make_parser([parser_list]) - Create and return a SAX :class:`XMLReader` object. The first parser found will + Create and return a SAX :class:`~xml.sax.xmlreader.XMLReader` object. The + first parser found will be used. If *parser_list* is provided, it must be a sequence of strings which name modules that have a function named :func:`create_parser`. Modules listed in *parser_list* will be used before modules in the default list of parsers. @@ -31,8 +40,9 @@ The convenience functions are: Create a SAX parser and use it to parse a document. The document, passed in as *filename_or_stream*, can be a filename or a file object. The *handler* - parameter needs to be a SAX :class:`ContentHandler` instance. If - *error_handler* is given, it must be a SAX :class:`ErrorHandler` instance; if + parameter needs to be a SAX :class:`~handler.ContentHandler` instance. If + *error_handler* is given, it must be a SAX :class:`~handler.ErrorHandler` + instance; if omitted, :exc:`SAXParseException` will be raised on all errors. There is no return value; all work must be done by the *handler* passed in. @@ -57,10 +67,12 @@ For these objects, only the interfaces are relevant; they are normally not instantiated by the application itself. Since Python does not have an explicit notion of interface, they are formally introduced as classes, but applications may use implementations which do not inherit from the provided classes. The -:class:`InputSource`, :class:`Locator`, :class:`Attributes`, -:class:`AttributesNS`, and :class:`XMLReader` interfaces are defined in the +:class:`~xml.sax.xmlreader.InputSource`, :class:`~xml.sax.xmlreader.Locator`, +:class:`~xml.sax.xmlreader.Attributes`, :class:`~xml.sax.xmlreader.AttributesNS`, +and :class:`~xml.sax.xmlreader.XMLReader` interfaces are defined in the module :mod:`xml.sax.xmlreader`. The handler interfaces are defined in -:mod:`xml.sax.handler`. For convenience, :class:`InputSource` (which is often +:mod:`xml.sax.handler`. For convenience, +:class:`~xml.sax.xmlreader.InputSource` (which is often instantiated directly) and the handler classes are also available from :mod:`xml.sax`. These interfaces are described below. @@ -73,7 +85,8 @@ classes. Encapsulate an XML error or warning. This class can contain basic error or warning information from either the XML parser or the application: it can be subclassed to provide additional functionality or to add localization. Note - that although the handlers defined in the :class:`ErrorHandler` interface + that although the handlers defined in the + :class:`~xml.sax.handler.ErrorHandler` interface receive instances of this exception, it is not required to actually raise the exception --- it is also useful as a container for information. @@ -86,22 +99,26 @@ classes. .. exception:: SAXParseException(msg, exception, locator) - Subclass of :exc:`SAXException` raised on parse errors. Instances of this class - are passed to the methods of the SAX :class:`ErrorHandler` interface to provide - information about the parse error. This class supports the SAX :class:`Locator` - interface as well as the :class:`SAXException` interface. + Subclass of :exc:`SAXException` raised on parse errors. Instances of this + class are passed to the methods of the SAX + :class:`~xml.sax.handler.ErrorHandler` interface to provide information + about the parse error. This class supports the SAX + :class:`~xml.sax.xmlreader.Locator` interface as well as the + :class:`SAXException` interface. .. exception:: SAXNotRecognizedException(msg[, exception]) - Subclass of :exc:`SAXException` raised when a SAX :class:`XMLReader` is + Subclass of :exc:`SAXException` raised when a SAX + :class:`~xml.sax.xmlreader.XMLReader` is confronted with an unrecognized feature or property. SAX applications and extensions may use this class for similar purposes. .. exception:: SAXNotSupportedException(msg[, exception]) - Subclass of :exc:`SAXException` raised when a SAX :class:`XMLReader` is asked to + Subclass of :exc:`SAXException` raised when a SAX + :class:`~xml.sax.xmlreader.XMLReader` is asked to enable a feature that is not supported, or to set a property to a value that the implementation does not support. SAX applications and extensions may use this class for similar purposes. diff --git a/Doc/library/xml.sax.utils.rst b/Doc/library/xml.sax.utils.rst index c796ca8..e774b6b 100644 --- a/Doc/library/xml.sax.utils.rst +++ b/Doc/library/xml.sax.utils.rst @@ -59,7 +59,8 @@ or as base classes. .. class:: XMLGenerator([out[, encoding]]) - This class implements the :class:`ContentHandler` interface by writing SAX + This class implements the :class:`~xml.sax.handler.ContentHandler` interface + by writing SAX events back into an XML document. In other words, using an :class:`XMLGenerator` as the content handler will reproduce the original document being parsed. *out* should be a file-like object which will default to *sys.stdout*. *encoding* is @@ -68,7 +69,8 @@ or as base classes. .. class:: XMLFilterBase(base) - This class is designed to sit between an :class:`XMLReader` and the client + This class is designed to sit between an + :class:`~xml.sax.xmlreader.XMLReader` and the client application's event handlers. By default, it does nothing but pass requests up to the reader and events on to the handlers unmodified, but subclasses can override specific methods to modify the event stream or the configuration @@ -77,9 +79,10 @@ or as base classes. .. function:: prepare_input_source(source[, base]) - This function takes an input source and an optional base URL and returns a fully - resolved :class:`InputSource` object ready for reading. The input source can be - given as a string, a file-like object, or an :class:`InputSource` object; - parsers will use this function to implement the polymorphic *source* argument to - their :meth:`parse` method. + This function takes an input source and an optional base URL and returns a + fully resolved :class:`~xml.sax.xmlreader.InputSource` object ready for + reading. The input source can be given as a string, a file-like object, or + an :class:`~xml.sax.xmlreader.InputSource` object; parsers will use this + function to implement the polymorphic *source* argument to their + :meth:`parse` method. diff --git a/Doc/library/xmlrpclib.rst b/Doc/library/xmlrpclib.rst index 183be92..0e9ff4b 100644 --- a/Doc/library/xmlrpclib.rst +++ b/Doc/library/xmlrpclib.rst @@ -8,8 +8,8 @@ .. note:: The :mod:`xmlrpclib` module has been renamed to :mod:`xmlrpc.client` in - Python 3.0. The :term:`2to3` tool will automatically adapt imports when - converting your sources to 3.0. + Python 3. The :term:`2to3` tool will automatically adapt imports when + converting your sources to Python 3. .. XXX Not everything is documented yet. It might be good to describe @@ -28,6 +28,13 @@ supports writing XML-RPC client code; it handles all the details of translating between conformable Python objects and XML on the wire. +.. warning:: + + The :mod:`xmlrpclib` module is not secure against maliciously + constructed data. If you need to parse untrusted or unauthenticated data see + :ref:`xml-vulnerabilities`. + + .. class:: ServerProxy(uri[, transport[, encoding[, verbose[, allow_none[, use_datetime]]]]]) A :class:`ServerProxy` instance is an object that manages communication with a @@ -380,7 +387,7 @@ The client code for the preceding server:: proxy = xmlrpclib.ServerProxy("http://localhost:8000/") try: proxy.add(2, 5) - except xmlrpclib.Fault, err: + except xmlrpclib.Fault as err: print "A fault occurred" print "Fault code: %d" % err.faultCode print "Fault string: %s" % err.faultString @@ -427,7 +434,7 @@ by providing an URI that doesn't point to an XMLRPC server:: try: proxy.some_method() - except xmlrpclib.ProtocolError, err: + except xmlrpclib.ProtocolError as err: print "A protocol error occurred" print "URL: %s" % err.url print "HTTP/HTTPS headers: %s" % err.headers @@ -545,7 +552,7 @@ Example of Client Usage try: print server.examples.getStateName(41) - except Error, v: + except Error as v: print "ERROR", v To access an XML-RPC server through a proxy, you need to define a custom diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst index 14e40c8..261747a 100644 --- a/Doc/library/zipfile.rst +++ b/Doc/library/zipfile.rst @@ -25,9 +25,6 @@ decryption of encrypted files in ZIP archives, but it currently cannot create an encrypted file. Decryption is extremely slow as it is implemented in native Python rather than C. -For other archive formats, see the :mod:`bz2`, :mod:`gzip`, and -:mod:`tarfile` modules. - The module defines the following items: .. exception:: BadZipfile @@ -56,7 +53,7 @@ The module defines the following items: .. class:: ZipInfo([filename[, date_time]]) Class used to represent information about a member of an archive. Instances - of this class are returned by the :meth:`getinfo` and :meth:`infolist` + of this class are returned by the :meth:`.getinfo` and :meth:`.infolist` methods of :class:`ZipFile` objects. Most users of the :mod:`zipfile` module will not need to create these, but only use those created by this module. *filename* should be the full name of the archive member, and @@ -81,7 +78,7 @@ The module defines the following items: .. data:: ZIP_DEFLATED The numeric constant for the usual ZIP compression method. This requires the - zlib module. No other compression methods are currently supported. + :mod:`zlib` module. No other compression methods are currently supported. .. seealso:: @@ -128,7 +125,7 @@ ZipFile Objects .. versionchanged:: 2.7.1 If the file is created with mode ``'a'`` or ``'w'`` and then - :meth:`close`\ d without adding any files to the archive, the appropriate + :meth:`closed <close>` without adding any files to the archive, the appropriate ZIP structures for an empty archive will be written to the file. ZipFile is also a context manager and therefore supports the @@ -166,21 +163,26 @@ ZipFile Objects Return a list of archive members by name. + .. index:: + single: universal newlines; zipfile.ZipFile.open method + .. method:: ZipFile.open(name[, mode[, pwd]]) Extract a member from the archive as a file-like object (ZipExtFile). *name* is the name of the file in the archive, or a :class:`ZipInfo` object. The *mode* - parameter, if included, must be one of the following: ``'r'`` (the default), - ``'U'``, or ``'rU'``. Choosing ``'U'`` or ``'rU'`` will enable universal newline + parameter, if included, must be one of the following: ``'r'`` (the default), + ``'U'``, or ``'rU'``. Choosing ``'U'`` or ``'rU'`` will enable + :term:`universal newline <universal newlines>` support in the read-only object. *pwd* is the password used for encrypted files. - Calling :meth:`open` on a closed ZipFile will raise a :exc:`RuntimeError`. + Calling :meth:`.open` on a closed ZipFile will raise a :exc:`RuntimeError`. .. note:: The file-like object is read-only and provides the following methods: - :meth:`!read`, :meth:`!readline`, :meth:`!readlines`, :meth:`!__iter__`, - :meth:`!next`. + :meth:`~file.read`, :meth:`~file.readline`, + :meth:`~file.readlines`, :meth:`__iter__`, + :meth:`~object.next`. .. note:: @@ -195,7 +197,7 @@ ZipFile Objects .. note:: - The :meth:`open`, :meth:`read` and :meth:`extract` methods can take a filename + The :meth:`.open`, :meth:`read` and :meth:`extract` methods can take a filename or a :class:`ZipInfo` object. You will appreciate this when trying to read a ZIP file that contains members with duplicate names. @@ -212,6 +214,16 @@ ZipFile Objects .. versionadded:: 2.6 + .. note:: + + If a member filename is an absolute path, a drive/UNC sharepoint and + leading (back)slashes will be stripped, e.g.: ``///foo/bar`` becomes + ``foo/bar`` on Unix, and ``C:\foo\bar`` becomes ``foo\bar`` on Windows. + And all ``".."`` components in a member filename will be removed, e.g.: + ``../../foo../../ba..r`` becomes ``foo../ba..r``. On Windows illegal + characters (``:``, ``<``, ``>``, ``|``, ``"``, ``?``, and ``*``) + replaced by underscore (``_``). + .. method:: ZipFile.extractall([path[, members[, pwd]]]) @@ -227,6 +239,9 @@ ZipFile Objects that have absolute filenames starting with ``"/"`` or filenames with two dots ``".."``. + .. versionchanged:: 2.7.4 + The zipfile module attempts to prevent that. See :meth:`extract` note. + .. versionadded:: 2.6 @@ -312,7 +327,7 @@ ZipFile Objects :class:`ZipInfo` constructor sets this member to :const:`ZIP_STORED`. .. versionchanged:: 2.7 - The *compression_type* argument. + The *compress_type* argument. The following data attributes are also available: @@ -328,7 +343,7 @@ The following data attributes are also available: The comment text associated with the ZIP file. If assigning a comment to a :class:`ZipFile` instance created with mode 'a' or 'w', this should be a string no longer than 65535 bytes. Comments longer than this will be - truncated in the written archive when :meth:`ZipFile.close` is called. + truncated in the written archive when :meth:`.close` is called. .. _pyzipfile-objects: @@ -368,8 +383,8 @@ The :class:`PyZipFile` constructor takes the same parameters as the ZipInfo Objects --------------- -Instances of the :class:`ZipInfo` class are returned by the :meth:`getinfo` and -:meth:`infolist` methods of :class:`ZipFile` objects. Each object stores +Instances of the :class:`ZipInfo` class are returned by the :meth:`.getinfo` and +:meth:`.infolist` methods of :class:`ZipFile` objects. Each object stores information about a single member of the ZIP archive. Instances have the following attributes: diff --git a/Doc/library/zipimport.rst b/Doc/library/zipimport.rst index af18d15..828e9fc 100644 --- a/Doc/library/zipimport.rst +++ b/Doc/library/zipimport.rst @@ -19,7 +19,7 @@ Typically, :data:`sys.path` is a list of directory names as strings. This modul also allows an item of :data:`sys.path` to be a string naming a ZIP file archive. The ZIP archive can contain a subdirectory structure to support package imports, and a path within the archive can be specified to only import from a -subdirectory. For example, the path :file:`/tmp/example.zip/lib/` would only +subdirectory. For example, the path :file:`example.zip/lib/` would only import from the :file:`lib/` subdirectory within the archive. Any files may be present in the ZIP archive, but only files :file:`.py` and @@ -115,7 +115,7 @@ zipimporter Objects .. method:: is_package(fullname) - Return True if the module specified by *fullname* is a package. Raise + Return ``True`` if the module specified by *fullname* is a package. Raise :exc:`ZipImportError` if the module couldn't be found. @@ -151,8 +151,8 @@ Examples Here is an example that imports a module from a ZIP archive - note that the :mod:`zipimport` module is not explicitly used. :: - $ unzip -l /tmp/example.zip - Archive: /tmp/example.zip + $ unzip -l example.zip + Archive: example.zip Length Date Time Name -------- ---- ---- ---- 8467 11-26-02 22:30 jwzthreading.py @@ -161,8 +161,8 @@ Here is an example that imports a module from a ZIP archive - note that the $ ./python Python 2.3 (#1, Aug 1 2003, 19:54:32) >>> import sys - >>> sys.path.insert(0, '/tmp/example.zip') # Add .zip file to front of path + >>> sys.path.insert(0, 'example.zip') # Add .zip file to front of path >>> import jwzthreading >>> jwzthreading.__file__ - '/tmp/example.zip/jwzthreading.py' + 'example.zip/jwzthreading.py' diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst index 92a3197..192bd4d 100644 --- a/Doc/library/zlib.rst +++ b/Doc/library/zlib.rst @@ -19,9 +19,7 @@ order. This documentation doesn't attempt to cover all of the permutations; consult the zlib manual at http://www.zlib.net/manual.html for authoritative information. -For reading and writing ``.gz`` files see the :mod:`gzip` module. For -other archive formats, see the :mod:`bz2`, :mod:`zipfile`, and -:mod:`tarfile` modules. +For reading and writing ``.gz`` files see the :mod:`gzip` module. The available exception and functions in this module are: @@ -64,18 +62,34 @@ The available exception and functions in this module are: .. function:: compress(string[, level]) Compresses the data in *string*, returning a string contained compressed data. - *level* is an integer from ``1`` to ``9`` controlling the level of compression; + *level* is an integer from ``0`` to ``9`` controlling the level of compression; ``1`` is fastest and produces the least compression, ``9`` is slowest and - produces the most. The default value is ``6``. Raises the :exc:`error` - exception if any error occurs. + produces the most. ``0`` is no compression. The default value is ``6``. + Raises the :exc:`error` exception if any error occurs. -.. function:: compressobj([level]) +.. function:: compressobj([level[, method[, wbits[, memlevel[, strategy]]]]]) Returns a compression object, to be used for compressing data streams that won't - fit into memory at once. *level* is an integer from ``1`` to ``9`` controlling + fit into memory at once. *level* is an integer from ``0`` to ``9`` controlling the level of compression; ``1`` is fastest and produces the least compression, - ``9`` is slowest and produces the most. The default value is ``6``. + ``9`` is slowest and produces the most. ``0`` is no compression. The default + value is ``6``. + + *method* is the compression algorithm. Currently, the only supported value is + ``DEFLATED``. + + *wbits* is the base two logarithm of the size of the window buffer. This + should be an integer from ``8`` to ``15``. Higher values give better + compression, but use more memory. The default is 15. + + *memlevel* controls the amount of memory used for internal compression state. + Valid values range from ``1`` to ``9``. Higher values using more memory, + but are faster and produce smaller output. The default is 8. + + *strategy* is used to tune the compression algorithm. Possible values are + ``Z_DEFAULT_STRATEGY``, ``Z_FILTERED``, and ``Z_HUFFMAN_ONLY``. The default + is ``Z_DEFAULT_STRATEGY``. .. function:: crc32(data[, value]) diff --git a/Doc/license.rst b/Doc/license.rst index 3b0c10e..5c5b269 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -50,59 +50,11 @@ been GPL-compatible; the table below summarizes the various releases. +----------------+--------------+-----------+------------+-----------------+ | 2.1.1 | 2.1+2.0.1 | 2001 | PSF | yes | +----------------+--------------+-----------+------------+-----------------+ -| 2.2 | 2.1.1 | 2001 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ | 2.1.2 | 2.1.1 | 2002 | PSF | yes | +----------------+--------------+-----------+------------+-----------------+ | 2.1.3 | 2.1.2 | 2002 | PSF | yes | +----------------+--------------+-----------+------------+-----------------+ -| 2.2.1 | 2.2 | 2002 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.2.2 | 2.2.1 | 2002 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.2.3 | 2.2.2 | 2002-2003 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.3 | 2.2.2 | 2002-2003 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.3.1 | 2.3 | 2002-2003 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.3.2 | 2.3.1 | 2003 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.3.3 | 2.3.2 | 2003 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.3.4 | 2.3.3 | 2004 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.3.5 | 2.3.4 | 2005 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.4 | 2.3 | 2004 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.4.1 | 2.4 | 2005 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.4.2 | 2.4.1 | 2005 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.4.3 | 2.4.2 | 2006 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.4.4 | 2.4.3 | 2006 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.5 | 2.4 | 2006 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.5.1 | 2.5 | 2007 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.5.2 | 2.5.1 | 2008 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.5.3 | 2.5.2 | 2008 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.6 | 2.5 | 2008 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.6.1 | 2.6 | 2008 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.6.2 | 2.6.1 | 2009 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.6.3 | 2.6.2 | 2009 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.6.4 | 2.6.3 | 2010 | PSF | yes | -+----------------+--------------+-----------+------------+-----------------+ -| 2.7 | 2.6 | 2010 | PSF | yes | +| 2.2 and above | 2.1.1 | 2001-now | PSF | yes | +----------------+--------------+-----------+------------+-----------------+ .. note:: @@ -132,7 +84,7 @@ Terms and conditions for accessing or otherwise using Python analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python |release| alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of - copyright, i.e., "Copyright © 2001-2012 Python Software Foundation; All Rights + copyright, i.e., "Copyright © 2001-2014 Python Software Foundation; All Rights Reserved" are retained in Python |release| alone or in any derivative version prepared by Licensee. @@ -309,7 +261,7 @@ Mersenne Twister ---------------- The :mod:`_random` module includes code based on a download from -http://www.math.keio.ac.jp/ matumoto/MT2002/emt19937ar.html. The following are +http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/emt19937ar.html. The following are the verbatim comments from the original code:: A C-program for MT19937, with initialization improved 2002/1/26. @@ -350,8 +302,8 @@ the verbatim comments from the original code:: Any feedback is very welcome. - http://www.math.keio.ac.jp/matumoto/emt.html - email: matumoto@math.keio.ac.jp + http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html + email: m-mat @ math.sci.hiroshima-u.ac.jp (remove space) Sockets @@ -794,8 +746,8 @@ OpenSSL license here:: * */ - Original SSLeay License - ----------------------- + Original SSLeay License + ----------------------- /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com) * All rights reserved. diff --git a/Doc/make.bat b/Doc/make.bat index d234e77..d3f01b5 100644 --- a/Doc/make.bat +++ b/Doc/make.bat @@ -34,7 +34,7 @@ echo. goto end :checkout -svn co %SVNROOT%/external/Sphinx-0.6.7/sphinx tools/sphinx +svn co %SVNROOT%/external/Sphinx-1.0.7/sphinx tools/sphinx svn co %SVNROOT%/external/docutils-0.6/docutils tools/docutils svn co %SVNROOT%/external/Jinja-2.3.1/jinja2 tools/jinja2 svn co %SVNROOT%/external/Pygments-1.3.1/pygments tools/pygments diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index c99b65a..08230fa 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -238,10 +238,7 @@ present, must be last; it matches any exception. For an except clause with an expression, that expression is evaluated, and the clause matches the exception if the resulting object is "compatible" with the exception. An object is compatible with an exception if it is the class or a base class of the exception -object, a tuple containing an item compatible with the exception, or, in the -(deprecated) case of string exceptions, is the raised string itself (note that -the object identities must match, i.e. it must be the same string object, not -just a string with the same value). +object, or a tuple containing an item compatible with the exception. If no except clause matches the exception, the search for an exception handler continues in the surrounding code and on the invocation stack. [#]_ @@ -297,8 +294,19 @@ not handled, the exception is temporarily saved. The :keyword:`finally` clause is executed. If there is a saved exception, it is re-raised at the end of the :keyword:`finally` clause. If the :keyword:`finally` clause raises another exception or executes a :keyword:`return` or :keyword:`break` statement, the -saved exception is lost. The exception information is not available to the -program during execution of the :keyword:`finally` clause. +saved exception is discarded:: + + >>> def f(): + ... try: + ... 1/0 + ... finally: + ... return 42 + ... + >>> f() + 42 + +The exception information is not available to the program during execution of +the :keyword:`finally` clause. .. index:: statement: return @@ -312,6 +320,20 @@ statement, the :keyword:`finally` clause is also executed 'on the way out.' A reason is a problem with the current implementation --- this restriction may be lifted in the future). +The return value of a function is determined by the last :keyword:`return` +statement executed. Since the :keyword:`finally` clause always executes, a +:keyword:`return` statement executed in the :keyword:`finally` clause will +always be the last one executed:: + + >>> def foo(): + ... try: + ... return 'try' + ... finally: + ... return 'finally' + ... + >>> foo() + 'finally' + Additional information on exceptions can be found in section :ref:`exceptions`, and information on using the :keyword:`raise` statement to generate exceptions may be found in section :ref:`raise`. @@ -323,7 +345,9 @@ may be found in section :ref:`raise`. The :keyword:`with` statement ============================= -.. index:: statement: with +.. index:: + statement: with + single: as; with statement .. versionadded:: 2.5 @@ -399,6 +423,9 @@ is equivalent to :: statement. +.. index:: + single: parameter; function definition + .. _function: .. _def: @@ -423,7 +450,7 @@ A function definition defines a user-defined function object (see section funcdef: "def" `funcname` "(" [`parameter_list`] ")" ":" `suite` dotted_name: `identifier` ("." `identifier`)* parameter_list: (`defparameter` ",")* - : ( "*" `identifier` [, "**" `identifier`] + : ( "*" `identifier` ["," "**" `identifier`] : | "**" `identifier` : | `defparameter` [","] ) defparameter: `parameter` ["=" `expression`] @@ -459,12 +486,15 @@ is equivalent to:: def func(): pass func = f1(arg)(f2(func)) -.. index:: triple: default; parameter; value - -When one or more top-level parameters have the form *parameter* ``=`` -*expression*, the function is said to have "default parameter values." For a -parameter with a default value, the corresponding argument may be omitted from a -call, in which case the parameter's default value is substituted. If a +.. index:: + triple: default; parameter; value + single: argument; function definition + +When one or more top-level :term:`parameters <parameter>` have the form +*parameter* ``=`` *expression*, the function is said to have "default parameter +values." For a parameter with a default value, the corresponding +:term:`argument` may be omitted from a call, in which +case the parameter's default value is substituted. If a parameter has a default value, all following parameters must also have a default value --- this is a syntactic restriction that is not expressed by the grammar. @@ -495,14 +525,14 @@ receiving any excess positional parameters, defaulting to the empty tuple. If the form "``**identifier``" is present, it is initialized to a new dictionary receiving any excess keyword arguments, defaulting to a new empty dictionary. -.. index:: pair: lambda; form +.. index:: pair: lambda; expression It is also possible to create anonymous functions (functions not bound to a -name), for immediate use in expressions. This uses lambda forms, described in -section :ref:`lambda`. Note that the lambda form is merely a shorthand for a +name), for immediate use in expressions. This uses lambda expressions, described in +section :ref:`lambda`. Note that the lambda expression is merely a shorthand for a simplified function definition; a function defined in a ":keyword:`def`" statement can be passed around or assigned to another name just like a function -defined by a lambda form. The ":keyword:`def`" form is actually more powerful +defined by a lambda expression. The ":keyword:`def`" form is actually more powerful since it allows the execution of multiple statements. **Programmer's note:** Functions are first-class objects. A "``def``" form diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 0d87873..fdcd2fd 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -207,7 +207,7 @@ Ellipsis single: True These represent the truth values False and True. The two objects - representing the values False and True are the only Boolean objects. + representing the values ``False`` and ``True`` are the only Boolean objects. The Boolean type is a subtype of plain integers, and Boolean values behave like the values 0 and 1, respectively, in almost all contexts, the exception being that when converted to a string, the strings @@ -356,8 +356,6 @@ Sequences object: mutable sequence object: mutable pair: assignment; statement - single: delete - statement: del single: subscription single: slicing @@ -411,7 +409,7 @@ Set types These represent a mutable set. They are created by the built-in :func:`set` constructor and can be modified afterwards by several methods, such as - :meth:`add`. + :meth:`~set.add`. Frozen sets .. index:: object: frozenset @@ -481,47 +479,44 @@ Callable types Special attributes: + .. tabularcolumns:: |l|L|l| + +-----------------------+-------------------------------+-----------+ | Attribute | Meaning | | +=======================+===============================+===========+ - | :attr:`func_doc` | The function's documentation | Writable | - | | string, or ``None`` if | | - | | unavailable | | - +-----------------------+-------------------------------+-----------+ - | :attr:`__doc__` | Another way of spelling | Writable | - | | :attr:`func_doc` | | - +-----------------------+-------------------------------+-----------+ - | :attr:`func_name` | The function's name | Writable | + | :attr:`__doc__` | The function's documentation | Writable | + | :attr:`func_doc` | string, or ``None`` if | | + | | unavailable. | | +-----------------------+-------------------------------+-----------+ - | :attr:`__name__` | Another way of spelling | Writable | - | | :attr:`func_name` | | + | :attr:`__name__` | The function's name. | Writable | + | :attr:`func_name` | | | +-----------------------+-------------------------------+-----------+ | :attr:`__module__` | The name of the module the | Writable | | | function was defined in, or | | | | ``None`` if unavailable. | | +-----------------------+-------------------------------+-----------+ - | :attr:`func_defaults` | A tuple containing default | Writable | - | | argument values for those | | + | :attr:`__defaults__` | A tuple containing default | Writable | + | :attr:`func_defaults` | argument values for those | | | | arguments that have defaults, | | | | or ``None`` if no arguments | | - | | have a default value | | + | | have a default value. | | +-----------------------+-------------------------------+-----------+ - | :attr:`func_code` | The code object representing | Writable | - | | the compiled function body. | | + | :attr:`__code__` | The code object representing | Writable | + | :attr:`func_code` | the compiled function body. | | +-----------------------+-------------------------------+-----------+ - | :attr:`func_globals` | A reference to the dictionary | Read-only | - | | that holds the function's | | + | :attr:`__globals__` | A reference to the dictionary | Read-only | + | :attr:`func_globals` | that holds the function's | | | | global variables --- the | | | | global namespace of the | | | | module in which the function | | | | was defined. | | +-----------------------+-------------------------------+-----------+ - | :attr:`func_dict` | The namespace supporting | Writable | - | | arbitrary function | | + | :attr:`__dict__` | The namespace supporting | Writable | + | :attr:`func_dict` | arbitrary function | | | | attributes. | | +-----------------------+-------------------------------+-----------+ - | :attr:`func_closure` | ``None`` or a tuple of cells | Read-only | - | | that contain bindings for the | | + | :attr:`__closure__` | ``None`` or a tuple of cells | Read-only | + | :attr:`func_closure` | that contain bindings for the | | | | function's free variables. | | +-----------------------+-------------------------------+-----------+ @@ -530,6 +525,12 @@ Callable types .. versionchanged:: 2.4 ``func_name`` is now writable. + .. versionchanged:: 2.6 + The double-underscore attributes ``__closure__``, ``__code__``, + ``__defaults__``, and ``__globals__`` were introduced as aliases for + the corresponding ``func_*`` attributes for forwards compatibility + with Python 3. + Function objects also support getting and setting arbitrary attributes, which can be used, for example, to attach metadata to functions. Regular attribute dot-notation is used to get and set such attributes. *Note that the current @@ -540,16 +541,21 @@ Callable types code object; see the description of internal types below. .. index:: - single: func_doc (function attribute) single: __doc__ (function attribute) single: __name__ (function attribute) single: __module__ (function attribute) single: __dict__ (function attribute) + single: __defaults__ (function attribute) + single: __code__ (function attribute) + single: __globals__ (function attribute) + single: __closure__ (function attribute) + single: func_doc (function attribute) + single: func_name (function attribute) + single: func_dict (function attribute) single: func_defaults (function attribute) - single: func_closure (function attribute) single: func_code (function attribute) single: func_globals (function attribute) - single: func_dict (function attribute) + single: func_closure (function attribute) pair: global; namespace User-defined methods @@ -573,7 +579,7 @@ Callable types :attr:`im_self` used to refer to the class that defined the method. .. versionchanged:: 2.6 - For 3.0 forward-compatibility, :attr:`im_func` is also available as + For Python 3 forward-compatibility, :attr:`im_func` is also available as :attr:`__func__`, and :attr:`im_self` as :attr:`__self__`. .. index:: @@ -621,9 +627,8 @@ Callable types single: im_self (method attribute) When a user-defined method object is created by retrieving a class method object - from a class or instance, its :attr:`im_self` attribute is the class itself (the - same as the :attr:`im_class` attribute), and its :attr:`im_func` attribute is - the function object underlying the class method. + from a class or instance, its :attr:`im_self` attribute is the class itself, and + its :attr:`im_func` attribute is the function object underlying the class method. When an unbound user-defined method object is called, the underlying function (:attr:`im_func`) is called, with the restriction that the first argument must @@ -660,7 +665,8 @@ Callable types :ref:`yield`) is called a :dfn:`generator function`. Such a function, when called, always returns an iterator object which can be used to execute the body of the function: calling the iterator's - :meth:`next` method will cause the function to execute until it provides a value + :meth:`~iterator.next` method will cause the function to execute until + it provides a value using the :keyword:`yield` statement. When the function executes a :keyword:`return` statement or falls off the end, a :exc:`StopIteration` exception is raised and the iterator will have reached the end of the set of @@ -795,8 +801,8 @@ Classes associated class is either :class:`C` or one of its base classes, it is transformed into an unbound user-defined method object whose :attr:`im_class` attribute is :class:`C`. When it would yield a class method object, it is - transformed into a bound user-defined method object whose :attr:`im_class` - and :attr:`im_self` attributes are both :class:`C`. When it would yield a + transformed into a bound user-defined method object whose + :attr:`im_self` attribute is :class:`C`. When it would yield a static method object, it is transformed into the object wrapped by the static method object. See section :ref:`descriptors` for another way in which attributes retrieved from a class may differ from those actually contained in @@ -820,10 +826,10 @@ Classes Special attributes: :attr:`__name__` is the class name; :attr:`__module__` is the module name in which the class was defined; :attr:`__dict__` is the - dictionary containing the class's namespace; :attr:`__bases__` is a tuple - (possibly empty or a singleton) containing the base classes, in the order of - their occurrence in the base class list; :attr:`__doc__` is the class's - documentation string, or None if undefined. + dictionary containing the class's namespace; :attr:`~class.__bases__` is a + tuple (possibly empty or a singleton) containing the base classes, in the + order of their occurrence in the base class list; :attr:`__doc__` is the + class's documentation string, or None if undefined. Class instances .. index:: @@ -868,8 +874,8 @@ Class instances single: __dict__ (instance attribute) single: __class__ (instance attribute) - Special attributes: :attr:`__dict__` is the attribute dictionary; - :attr:`__class__` is the instance's class. + Special attributes: :attr:`~object.__dict__` is the attribute dictionary; + :attr:`~instance.__class__` is the instance's class. Files .. index:: @@ -1068,9 +1074,9 @@ Internal types single: stop (slice object attribute) single: step (slice object attribute) - Special read-only attributes: :attr:`start` is the lower bound; :attr:`stop` is - the upper bound; :attr:`step` is the step value; each is ``None`` if omitted. - These attributes can have any type. + Special read-only attributes: :attr:`~slice.start` is the lower bound; + :attr:`~slice.stop` is the upper bound; :attr:`~slice.step` is the step + value; each is ``None`` if omitted. These attributes can have any type. Slice objects support one method: @@ -1149,7 +1155,7 @@ sources of additional information. single: class; classic single: class; old-style -Old-style classes are removed in Python 3.0, leaving only the semantics of +Old-style classes are removed in Python 3, leaving only the semantics of new-style classes. @@ -1177,7 +1183,8 @@ When implementing a class that emulates any built-in type, it is important that the emulation only be implemented to the degree that it makes sense for the object being modelled. For example, some sequences may work well with retrieval of individual elements, but extracting a slice may not make sense. (One example -of this is the :class:`NodeList` interface in the W3C's Document Object Model.) +of this is the :class:`~xml.dom.NodeList` interface in the W3C's Document +Object Model.) .. _customization: @@ -1413,7 +1420,7 @@ Basic customization User-defined classes have :meth:`__cmp__` and :meth:`__hash__` methods by default; with them, all objects compare unequal (except with themselves) - and ``x.__hash__()`` returns ``id(x)``. + and ``x.__hash__()`` returns a result derived from ``id(x)``. Classes which inherit a :meth:`__hash__` method from a parent class but change the meaning of :meth:`__cmp__` or :meth:`__eq__` such that the hash @@ -1809,10 +1816,10 @@ case the instance is itself a class. :pep:`3119` - Introducing Abstract Base Classes Includes the specification for customizing :func:`isinstance` and - :func:`issubclass` behavior through :meth:`__instancecheck__` and - :meth:`__subclasscheck__`, with motivation for this functionality in the - context of adding Abstract Base Classes (see the :mod:`abc` module) to the - language. + :func:`issubclass` behavior through :meth:`~class.__instancecheck__` and + :meth:`~class.__subclasscheck__`, with motivation for this functionality + in the context of adding Abstract Base Classes (see the :mod:`abc` + module) to the language. .. _callable-types: @@ -1845,7 +1852,7 @@ range of items. (For backwards compatibility, the method :meth:`__getslice__` is also recommended that mappings provide the methods :meth:`keys`, :meth:`values`, :meth:`items`, :meth:`has_key`, :meth:`get`, :meth:`clear`, :meth:`setdefault`, :meth:`iterkeys`, :meth:`itervalues`, :meth:`iteritems`, -:meth:`pop`, :meth:`popitem`, :meth:`copy`, and :meth:`update` behaving similar +:meth:`pop`, :meth:`popitem`, :meth:`!copy`, and :meth:`update` behaving similar to those for Python's standard dictionary objects. The :mod:`UserDict` module provides a :class:`DictMixin` class to help create those methods from a base set of :meth:`__getitem__`, :meth:`__setitem__`, :meth:`__delitem__`, and @@ -2235,7 +2242,7 @@ Coercion rules This section used to document the rules for coercion. As the language has evolved, the coercion rules have become hard to document precisely; documenting what one version of one particular implementation does is undesirable. Instead, -here are some informal guidelines regarding coercion. In Python 3.0, coercion +here are some informal guidelines regarding coercion. In Python 3, coercion will not be supported. * diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index a7c66d3..d3dc111 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -96,14 +96,13 @@ exception. definition begins with two or more underscore characters and does not end in two or more underscores, it is considered a :dfn:`private name` of that class. Private names are transformed to a longer form before code is generated for -them. The transformation inserts the class name in front of the name, with -leading underscores removed, and a single underscore inserted in front of the -class name. For example, the identifier ``__spam`` occurring in a class named -``Ham`` will be transformed to ``_Ham__spam``. This transformation is -independent of the syntactical context in which the identifier is used. If the -transformed name is extremely long (longer than 255 characters), implementation -defined truncation may happen. If the class name consists only of underscores, -no transformation is done. +them. The transformation inserts the class name, with leading underscores +removed and a single underscore inserted, in front of the name. For example, +the identifier ``__spam`` occurring in a class named ``Ham`` will be transformed +to ``_Ham__spam``. This transformation is independent of the syntactical +context in which the identifier is used. If the transformed name is extremely +long (longer than 255 characters), implementation defined truncation may happen. +If the class name consists only of underscores, no transformation is done. @@ -185,7 +184,7 @@ brackets: list_comprehension: `expression` `list_for` list_for: "for" `target_list` "in" `old_expression_list` [`list_iter`] old_expression_list: `old_expression` [("," `old_expression`)+ [","]] - old_expression: `or_test` | `old_lambda_form` + old_expression: `or_test` | `old_lambda_expr` list_iter: `list_for` | `list_if` list_if: "if" `old_expression` [`list_iter`] @@ -421,10 +420,18 @@ transferred to the generator's caller. .. index:: object: generator -The following generator's methods can be used to control the execution of a -generator function: + +Generator-iterator methods +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This subsection describes the methods of a generator iterator. They can +be used to control the execution of a generator function. + +Note that calling any of the generator methods below when the generator +is already executing raises a :exc:`ValueError` exception. .. index:: exception: StopIteration +.. class:: generator .. method:: generator.next() @@ -438,6 +445,7 @@ generator function: exits without yielding another value, a :exc:`StopIteration` exception is raised. +.. class:: . .. method:: generator.send(value) @@ -654,23 +662,24 @@ is a tuple containing the conversion of the slice items; otherwise, the conversion of the lone slice item is the key. The conversion of a slice item that is an expression is that expression. The conversion of an ellipsis slice item is the built-in ``Ellipsis`` object. The conversion of a proper slice is a -slice object (see section :ref:`types`) whose :attr:`start`, :attr:`stop` and -:attr:`step` attributes are the values of the expressions given as lower bound, -upper bound and stride, respectively, substituting ``None`` for missing -expressions. +slice object (see section :ref:`types`) whose :attr:`~slice.start`, +:attr:`~slice.stop` and :attr:`~slice.step` attributes are the values of the +expressions given as lower bound, upper bound and stride, respectively, +substituting ``None`` for missing expressions. +.. index:: + object: callable + single: call + single: argument; call semantics + .. _calls: Calls ----- -.. index:: single: call - -.. index:: object: callable - -A call calls a callable object (e.g., a function) with a possibly empty series -of arguments: +A call calls a callable object (e.g., a :term:`function`) with a possibly empty +series of :term:`arguments <argument>`: .. productionlist:: call: `primary` "(" [`argument_list` [","] @@ -689,12 +698,15 @@ of arguments: A trailing comma may be present after the positional and keyword arguments but does not affect the semantics. +.. index:: + single: parameter; call semantics + The primary must evaluate to a callable object (user-defined functions, built-in functions, methods of built-in objects, class objects, methods of class instances, and certain class instances themselves are callable; extensions may define additional callable object types). All argument expressions are evaluated before the call is attempted. Please refer to section :ref:`function` -for the syntax of formal parameter lists. +for the syntax of formal :term:`parameter` lists. If keyword arguments are present, they are first converted to positional arguments, as follows. First, a list of unfilled slots is created for the @@ -1245,7 +1257,7 @@ Conditional Expressions .. productionlist:: conditional_expression: `or_test` ["if" `or_test` "else" `expression`] - expression: `conditional_expression` | `lambda_form` + expression: `conditional_expression` | `lambda_expr` Conditional expressions (sometimes called a "ternary operator") have the lowest priority of all Python operations. @@ -1265,14 +1277,13 @@ Lambdas .. index:: pair: lambda; expression - pair: lambda; form pair: anonymous; function .. productionlist:: - lambda_form: "lambda" [`parameter_list`]: `expression` - old_lambda_form: "lambda" [`parameter_list`]: `old_expression` + lambda_expr: "lambda" [`parameter_list`]: `expression` + old_lambda_expr: "lambda" [`parameter_list`]: `old_expression` -Lambda forms (lambda expressions) have the same syntactic position as +Lambda expressions (sometimes called lambda forms) have the same syntactic position as expressions. They are a shorthand to create anonymous functions; the expression ``lambda arguments: expression`` yields a function object. The unnamed object behaves like a function object defined with :: @@ -1281,7 +1292,7 @@ behaves like a function object defined with :: return expression See section :ref:`function` for the syntax of parameter lists. Note that -functions created with lambda forms cannot contain statements. +functions created with lambda expressions cannot contain statements. .. _exprlists: @@ -1332,8 +1343,8 @@ their suffixes:: .. _operator-summary: -Summary -======= +Operator precedence +=================== .. index:: pair: operator; precedence @@ -1356,10 +1367,10 @@ groups from right to left). +-----------------------------------------------+-------------------------------------+ | :keyword:`and` | Boolean AND | +-----------------------------------------------+-------------------------------------+ -| :keyword:`not` *x* | Boolean NOT | +| :keyword:`not` ``x`` | Boolean NOT | +-----------------------------------------------+-------------------------------------+ -| :keyword:`in`, :keyword:`not` :keyword:`in`, | Comparisons, including membership | -| :keyword:`is`, :keyword:`is not`, ``<``, | tests and identity tests, | +| :keyword:`in`, :keyword:`not in`, | Comparisons, including membership | +| :keyword:`is`, :keyword:`is not`, ``<``, | tests and identity tests | | ``<=``, ``>``, ``>=``, ``<>``, ``!=``, ``==`` | | +-----------------------------------------------+-------------------------------------+ | ``|`` | Bitwise OR | @@ -1384,7 +1395,7 @@ groups from right to left). +-----------------------------------------------+-------------------------------------+ | ``(expressions...)``, | Binding or tuple display, | | ``[expressions...]``, | list display, | -| ``{key:datum...}``, | dictionary display, | +| ``{key: value...}``, | dictionary display, | | ```expressions...``` | string conversion | +-----------------------------------------------+-------------------------------------+ @@ -1392,7 +1403,7 @@ groups from right to left). .. [#] In Python 2.3 and later releases, a list comprehension "leaks" the control variables of each ``for`` it contains into the containing scope. However, this - behavior is deprecated, and relying on it will not work in Python 3.0 + behavior is deprecated, and relying on it will not work in Python 3. .. [#] While ``abs(x%y) < abs(y)`` is true mathematically, for floats it may not be true numerically due to roundoff. For example, and assuming a platform on which diff --git a/Doc/reference/index.rst b/Doc/reference/index.rst index bd1a281..513e445 100644 --- a/Doc/reference/index.rst +++ b/Doc/reference/index.rst @@ -4,9 +4,6 @@ The Python Language Reference ################################# -:Release: |version| -:Date: |today| - This reference manual describes the syntax and "core semantics" of the language. It is terse, but attempts to be exact and complete. The semantics of non-essential built-in object types and of the built-in functions and modules diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index ea92b8e..52a09a8 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -104,9 +104,7 @@ are ignored by the syntax; they are not tokens. Encoding declarations --------------------- -.. index:: - single: source character set - single: encodings +.. index:: source character set, encoding declarations (source file) If a comment in the first or second line of the Python script matches the regular expression ``coding[=:]\s*([-\w.]+)``, this comment is processed as an @@ -529,8 +527,7 @@ Notes: (2) Any Unicode character can be encoded this way, but characters outside the Basic Multilingual Plane (BMP) will be encoded using a surrogate pair if Python is - compiled to use 16-bit code units (the default). Individual code units which - form parts of a surrogate pair can be encoded using this escape sequence. + compiled to use 16-bit code units (the default). (3) As in Standard C, up to three octal digits are accepted. diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 05b78a0..d8e539d 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -72,6 +72,7 @@ Assignment statements ===================== .. index:: + single: =; assignment statement pair: assignment; statement pair: binding; name pair: rebinding; name @@ -241,6 +242,18 @@ Augmented assignment statements .. index:: pair: augmented; assignment single: statement; assignment, augmented + single: +=; augmented assignment + single: -=; augmented assignment + single: *=; augmented assignment + single: /=; augmented assignment + single: %=; augmented assignment + single: &=; augmented assignment + single: ^=; augmented assignment + single: |=; augmented assignment + single: **=; augmented assignment + single: //=; augmented assignment + single: >>=; augmented assignment + single: <<=; augmented assignment Augmented assignment is the combination, in a single statement, of a binary operation and an assignment statement: @@ -511,6 +524,9 @@ reference count or by being garbage collected), the generator-iterator's :meth:`close` method will be called, allowing any pending :keyword:`finally` clauses to execute. +For full details of :keyword:`yield` semantics, refer to the :ref:`yieldexpr` +section. + .. note:: In Python 2.2, the :keyword:`yield` statement was only allowed when the @@ -650,6 +666,7 @@ The :keyword:`import` statement single: module; importing pair: name; binding keyword: from + single: as; import statement .. productionlist:: import_stmt: "import" `module` ["as" `name`] ( "," `module` ["as" `name`] )* @@ -737,7 +754,7 @@ can be found but the path exists then a value of ``None`` is stored in :data:`sys.path_importer_cache` to signify that an implicit, file-based finder that handles modules stored as individual files should be used for that path. If the path does not exist then a finder which always -returns `None`` is placed in the cache for the path. +returns ``None`` is placed in the cache for the path. .. index:: single: loader @@ -978,21 +995,32 @@ The :keyword:`exec` statement exec_stmt: "exec" `or_expr` ["in" `expression` ["," `expression`]] This statement supports dynamic execution of Python code. The first expression -should evaluate to either a string, an open file object, or a code object. If -it is a string, the string is parsed as a suite of Python statements which is -then executed (unless a syntax error occurs). [#]_ If it is an open file, the file -is parsed until EOF and executed. If it is a code object, it is simply -executed. In all cases, the code that's executed is expected to be valid as -file input (see section :ref:`file-input`). Be aware that the +should evaluate to either a Unicode string, a *Latin-1* encoded string, an open +file object, a code object, or a tuple. If it is a string, the string is parsed +as a suite of Python statements which is then executed (unless a syntax error +occurs). [#]_ If it is an open file, the file is parsed until EOF and executed. +If it is a code object, it is simply executed. For the interpretation of a +tuple, see below. In all cases, the code that's executed is expected to be +valid as file input (see section :ref:`file-input`). Be aware that the :keyword:`return` and :keyword:`yield` statements may not be used outside of function definitions even within the context of code passed to the :keyword:`exec` statement. In all cases, if the optional parts are omitted, the code is executed in the -current scope. If only the first expression after :keyword:`in` is specified, +current scope. If only the first expression after ``in`` is specified, it should be a dictionary, which will be used for both the global and the local variables. If two expressions are given, they are used for the global and local variables, respectively. If provided, *locals* can be any mapping object. +Remember that at module level, globals and locals are the same dictionary. If +two separate objects are given as *globals* and *locals*, the code will be +executed as if it were embedded in a class definition. + +The first expression may also be a tuple of length 2 or 3. In this case, the +optional parts must be omitted. The form ``exec(expr, globals)`` is equivalent +to ``exec expr in globals``, while the form ``exec(expr, globals, locals)`` is +equivalent to ``exec expr in globals, locals``. The tuple form of ``exec`` +provides compatibility with Python 3, where ``exec`` is a function rather than +a statement. .. versionchanged:: 2.4 Formerly, *locals* was required to be a dictionary. @@ -1021,5 +1049,5 @@ which may be useful to pass around for use by :keyword:`exec`. .. rubric:: Footnotes .. [#] Note that the parser only accepts the Unix-style end of line convention. - If you are reading the code from a file, make sure to use universal - newline mode to convert Windows or Mac-style newlines. + If you are reading the code from a file, make sure to use + :term:`universal newlines` mode to convert Windows or Mac-style newlines. diff --git a/Doc/tools/dailybuild.py b/Doc/tools/dailybuild.py index 1a471e6..786d5ab 100755 --- a/Doc/tools/dailybuild.py +++ b/Doc/tools/dailybuild.py @@ -5,12 +5,12 @@ # # Usages: # -# dailybuild.py +# dailybuild.py [-q] # # without any arguments builds docs for all branches configured in the global -# BRANCHES value. +# BRANCHES value. -q selects "quick build", which means to build only HTML. # -# dailybuild.py [-d] <checkout> <target> +# dailybuild.py [-q] [-d] <checkout> <target> # # builds one version, where <checkout> is an SVN checkout directory of the # Python branch to build docs for, and <target> is the directory where the @@ -29,31 +29,79 @@ import getopt BUILDROOT = '/home/gbrandl/docbuild' +SPHINXBUILD = os.path.join(BUILDROOT, 'sphinx-env/bin/sphinx-build') WWWROOT = '/data/ftp.python.org/pub/docs.python.org' BRANCHES = [ # checkout, target, isdev - (BUILDROOT + '/python33', WWWROOT + '/dev', True), - (BUILDROOT + '/python27', WWWROOT, False), - (BUILDROOT + '/python32', WWWROOT + '/py3k', False), + (BUILDROOT + '/python34', WWWROOT + '/3.4', False), + (BUILDROOT + '/python35', WWWROOT + '/3.5', True), + (BUILDROOT + '/python27', WWWROOT + '/2.7', False), ] -def build_one(checkout, target, isdev): +def _files_changed(old, new): + with open(old, 'rb') as fp1, open(new, 'rb') as fp2: + st1 = os.fstat(fp1.fileno()) + st2 = os.fstat(fp2.fileno()) + if st1.st_size != st2.st_size: + return False + if st1.st_mtime >= st2.st_mtime: + return True + while True: + one = fp1.read(4096) + two = fp2.read(4096) + if one != two: + return False + if one == '': + break + return True + +def build_one(checkout, target, isdev, quick): print 'Doc autobuild started in %s' % checkout os.chdir(checkout) - print 'Running svn update' - os.system('svn update') + print 'Running hg pull --update' + os.system('hg pull --update') print 'Running make autobuild' - if os.WEXITSTATUS(os.system( - 'cd Doc; make autobuild-%s' % (isdev and 'dev' or 'stable'))) == 2: + maketarget = 'autobuild-' + ('html' if quick else + ('dev' if isdev else 'stable')) + if os.WEXITSTATUS(os.system('cd Doc; make SPHINXBUILD=%s %s' % (SPHINXBUILD, maketarget))) == 2: print '*' * 80 return - print 'Copying HTML files' + print('Computing changed files') + changed = [] + for dirpath, dirnames, filenames in os.walk('Doc/build/html/'): + dir_rel = dirpath[len('Doc/build/html/'):] + for fn in filenames: + local_path = os.path.join(dirpath, fn) + rel_path = os.path.join(dir_rel, fn) + target_path = os.path.join(target, rel_path) + if (os.path.exists(target_path) and + not _files_changed(target_path, local_path)): + changed.append(rel_path) + print 'Copying HTML files to %s' % target os.system('cp -a Doc/build/html/* %s' % target) - print 'Copying dist files' - os.system('mkdir -p %s/archives' % target) - os.system('cp -a Doc/dist/* %s/archives' % target) + if not quick: + print 'Copying dist files' + os.system('mkdir -p %s/archives' % target) + os.system('cp -a Doc/dist/* %s/archives' % target) + changed.append('archives/') + for fn in os.listdir(os.path.join(target, 'archives')): + changed.append('archives/' + fn) + print '%s files changed' % len(changed) + if changed: + target_ino = os.stat(target).st_ino + targets_dir = os.path.dirname(target) + prefixes = [] + for fn in os.listdir(targets_dir): + if os.stat(os.path.join(targets_dir, fn)).st_ino == target_ino: + prefixes.append(fn) + to_purge = [] + for prefix in prefixes: + to_purge.extend(prefix + "/" + p for p in changed) + purge_cmd = 'curl -X PURGE "https://docs.python.org/{%s}"' % ','.join(to_purge) + print("Running CDN purge") + os.system(purge_cmd) print 'Finished' print '=' * 80 @@ -67,15 +115,21 @@ def usage(): if __name__ == '__main__': try: - opts, args = getopt.getopt(sys.argv[1:], 'd') + opts, args = getopt.getopt(sys.argv[1:], 'dq') except getopt.error: usage() - if opts and not args: + quick = devel = False + for opt, _ in opts: + if opt == '-q': + quick = True + if opt == '-d': + devel = True + if devel and not args: usage() if args: if len(args) != 2: usage() - build_one(args[0], args[1], bool(opts)) + build_one(os.path.abspath(args[0]), os.path.abspath(args[1]), devel, quick) else: - for branch in BRANCHES: - build_one(*branch) + for checkout, dest, devel in BRANCHES: + build_one(checkout, dest, devel, quick) diff --git a/Doc/tools/sphinx-build.py b/Doc/tools/sphinx-build.py index a0bd7fa..d3fe702 100644 --- a/Doc/tools/sphinx-build.py +++ b/Doc/tools/sphinx-build.py @@ -15,13 +15,13 @@ warnings.filterwarnings('ignore', category=UserWarning, module='jinja2') if __name__ == '__main__': - if sys.version_info[:3] < (2, 4, 0): - print >>sys.stderr, """\ -Error: Sphinx needs to be executed with Python 2.4 or newer + if sys.version_info[:3] < (2, 4, 0) or sys.version_info[:3] > (3, 0, 0): + sys.stderr.write("""\ +Error: Sphinx needs to be executed with Python 2.4 or newer (not 3.x though). (If you run this from the Makefile, you can set the PYTHON variable to the path of an alternative interpreter executable, e.g., ``make html PYTHON=python2.5``). -""" +""") sys.exit(1) from sphinx import main diff --git a/Doc/tools/sphinxext/download.html b/Doc/tools/sphinxext/download.html index 4fca138..3adf2e9 100644 --- a/Doc/tools/sphinxext/download.html +++ b/Doc/tools/sphinxext/download.html @@ -35,9 +35,13 @@ in the table are the size of the download files in megabytes.</p> </tr> </table> - <p>These archives contain all the content in the documentation.</p> +<p>HTML Help (<tt>.chm</tt>) files are made available in the "Windows" section +on the <a href="http://python.org/download/releases/{{ release[:5] }}/">Python +download page</a>.</p> + + <h2>Unpacking</h2> <p>Unix users should download the .tar.bz2 archives; these are bzipped tar diff --git a/Doc/tools/sphinxext/indexsidebar.html b/Doc/tools/sphinxext/indexsidebar.html index 8d3feef..5c4c75d 100644 --- a/Doc/tools/sphinxext/indexsidebar.html +++ b/Doc/tools/sphinxext/indexsidebar.html @@ -1,23 +1,17 @@ - <h3>Download</h3> - <p><a href="{{ pathto('download') }}">Download these documents</a></p> - <h3>Docs for other versions</h3> - <ul> - <li><a href="http://docs.python.org/2.6/">Python 2.6 (stable)</a></li> - <li><a href="http://docs.python.org/3.2/">Python 3.2 (stable)</a></li> - <li><a href="http://docs.python.org/dev/">Python 3.3 (in development)</a></li> - <li><a href="http://www.python.org/doc/versions/">Old versions</a></li> - </ul> +<h3>Download</h3> +<p><a href="{{ pathto('download') }}">Download these documents</a></p> +<h3>Docs for other versions</h3> +<ul> + <li><a href="http://docs.python.org/3.4/">Python 3.4 (stable)</a></li> + <li><a href="http://docs.python.org/3.5/">Python 3.5 (in development)</a></li> + <li><a href="http://www.python.org/doc/versions/">Old versions</a></li> +</ul> - <h3>Other resources</h3> - <ul> - {# XXX: many of these should probably be merged in the main docs #} - <li><a href="http://www.python.org/doc/faq/">FAQs</a></li> - <li><a href="http://www.python.org/doc/essays/">Guido's Essays</a></li> - <li><a href="http://www.python.org/doc/newstyle/">New-style Classes</a></li> - <li><a href="http://www.python.org/dev/peps/">PEP Index</a></li> - <li><a href="http://wiki.python.org/moin/BeginnersGuide">Beginner's Guide</a></li> - <li><a href="http://wiki.python.org/moin/PythonBooks">Book List</a></li> - <li><a href="http://www.python.org/doc/av/">Audio/Visual Talks</a></li> - <li><a href="http://www.python.org/doc/other/">Other Doc Collections</a></li> - <li><a href="{{ pathto('bugs') }}">Report a Bug</a></li> - </ul> +<h3>Other resources</h3> +<ul> + {# XXX: many of these should probably be merged in the main docs #} + <li><a href="http://www.python.org/dev/peps/">PEP Index</a></li> + <li><a href="http://wiki.python.org/moin/BeginnersGuide">Beginner's Guide</a></li> + <li><a href="http://wiki.python.org/moin/PythonBooks">Book List</a></li> + <li><a href="http://www.python.org/doc/av/">Audio/Visual Talks</a></li> +</ul> diff --git a/Doc/tools/sphinxext/layout.html b/Doc/tools/sphinxext/layout.html index d4bb105..f24dc8c 100644 --- a/Doc/tools/sphinxext/layout.html +++ b/Doc/tools/sphinxext/layout.html @@ -2,18 +2,35 @@ {% block rootrellink %} <li><img src="{{ pathto('_static/py.png', 1) }}" alt="" style="vertical-align: middle; margin-top: -1px"/></li> - <li><a href="{{ pathto('index') }}">{{ shorttitle }}</a>{{ reldelim1 }}</li> + <li><a href="http://www.python.org/">Python</a>{{ reldelim1 }}</li> + <li> + {%- if versionswitcher is defined %} + <span class="version_switcher_placeholder">{{ release }}</span> + <a href="{{ pathto('index') }}">Documentation</a>{{ reldelim1 }} + {%- else %} + <a href="{{ pathto('index') }}">{{ shorttitle }}</a>{{ reldelim1 }} + {%- endif %} + </li> {% endblock %} +{% block relbar1 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} +{% block relbar2 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} {% block extrahead %} <link rel="shortcut icon" type="image/png" href="{{ pathto('_static/py.png', 1) }}" /> {% if not embedded %}<script type="text/javascript" src="{{ pathto('_static/copybutton.js', 1) }}"></script>{% endif %} + {% if versionswitcher is defined and not embedded %}<script type="text/javascript" src="{{ pathto('_static/version_switch.js', 1) }}"></script>{% endif %} {{ super() }} + {% if builder == 'qthelp' %} + <style type="text/css"> + body { background-color: white; } + div.document { background-color: white; } + </style> + {% endif %} {% endblock %} {% block footer %} <div class="footer"> © <a href="{{ pathto('copyright') }}">Copyright</a> {{ copyright|e }}. <br /> - The Python Software Foundation is a non-profit corporation. + The Python Software Foundation is a non-profit corporation. <a href="http://www.python.org/psf/donations/">Please donate.</a> <br /> Last updated on {{ last_updated|e }}. diff --git a/Doc/tools/sphinxext/pyspecific.py b/Doc/tools/sphinxext/pyspecific.py index 4b91253..b90969e 100644 --- a/Doc/tools/sphinxext/pyspecific.py +++ b/Doc/tools/sphinxext/pyspecific.py @@ -5,7 +5,7 @@ Sphinx extension with Python doc-specific markup. - :copyright: 2008, 2009, 2010 by Georg Brandl. + :copyright: 2008-2013 by Georg Brandl. :license: Python license. """ @@ -13,7 +13,12 @@ ISSUE_URI = 'http://bugs.python.org/issue%s' SOURCE_URI = 'http://hg.python.org/cpython/file/2.7/%s' from docutils import nodes, utils + +import sphinx from sphinx.util.nodes import split_explicit_title +from sphinx.writers.html import HTMLTranslator +from sphinx.writers.latex import LaTeXTranslator +from sphinx.locale import versionlabels # monkey-patch reST parser to disable alphabetic and roman enumerated lists from docutils.parsers.rst.states import Body @@ -22,20 +27,45 @@ Body.enum.converters['loweralpha'] = \ Body.enum.converters['lowerroman'] = \ Body.enum.converters['upperroman'] = lambda x: None -# monkey-patch HTML translator to give versionmodified paragraphs a class -def new_visit_versionmodified(self, node): - self.body.append(self.starttag(node, 'p', CLASS=node['type'])) - text = versionlabels[node['type']] % node['version'] - if len(node): - text += ': ' - else: - text += '.' - self.body.append('<span class="versionmodified">%s</span>' % text) - -from sphinx.writers.html import HTMLTranslator -from sphinx.locale import versionlabels -HTMLTranslator.visit_versionmodified = new_visit_versionmodified - +if sphinx.__version__[:3] < '1.2': + # monkey-patch HTML translator to give versionmodified paragraphs a class + def new_visit_versionmodified(self, node): + self.body.append(self.starttag(node, 'p', CLASS=node['type'])) + text = versionlabels[node['type']] % node['version'] + if len(node): + text += ': ' + else: + text += '.' + self.body.append('<span class="versionmodified">%s</span>' % text) + HTMLTranslator.visit_versionmodified = new_visit_versionmodified + +# monkey-patch HTML and LaTeX translators to keep doctest blocks in the +# doctest docs themselves +orig_visit_literal_block = HTMLTranslator.visit_literal_block +def new_visit_literal_block(self, node): + meta = self.builder.env.metadata[self.builder.current_docname] + old_trim_doctest_flags = self.highlighter.trim_doctest_flags + if 'keepdoctest' in meta: + self.highlighter.trim_doctest_flags = False + try: + orig_visit_literal_block(self, node) + finally: + self.highlighter.trim_doctest_flags = old_trim_doctest_flags + +HTMLTranslator.visit_literal_block = new_visit_literal_block + +orig_depart_literal_block = LaTeXTranslator.depart_literal_block +def new_depart_literal_block(self, node): + meta = self.builder.env.metadata[self.curfilestack[-1]] + old_trim_doctest_flags = self.highlighter.trim_doctest_flags + if 'keepdoctest' in meta: + self.highlighter.trim_doctest_flags = False + try: + orig_depart_literal_block(self, node) + finally: + self.highlighter.trim_doctest_flags = old_trim_doctest_flags + +LaTeXTranslator.depart_literal_block = new_depart_literal_block # Support for marking up and linking to bugs.python.org issues @@ -154,11 +184,11 @@ pydoc_topic_labels = [ 'bltin-null-object', 'bltin-type-objects', 'booleans', 'break', 'callable-types', 'calls', 'class', 'comparisons', 'compound', 'context-managers', 'continue', 'conversions', 'customization', 'debugger', - 'del', 'dict', 'dynamic-features', 'else', 'exceptions', 'execmodel', + 'del', 'dict', 'dynamic-features', 'else', 'exceptions', 'exec', 'execmodel', 'exprlists', 'floating', 'for', 'formatstrings', 'function', 'global', 'id-classes', 'identifiers', 'if', 'imaginary', 'import', 'in', 'integers', - 'lambda', 'lists', 'naming', 'nonlocal', 'numbers', 'numeric-types', - 'objects', 'operator-summary', 'pass', 'power', 'raise', 'return', + 'lambda', 'lists', 'naming', 'numbers', 'numeric-types', + 'objects', 'operator-summary', 'pass', 'power', 'print', 'raise', 'return', 'sequence-types', 'shifting', 'slicings', 'specialattrs', 'specialnames', 'string-methods', 'strings', 'subscriptions', 'truth', 'try', 'types', 'typesfunctions', 'typesmapping', 'typesmethods', 'typesmodules', diff --git a/Doc/tools/sphinxext/static/basic.css b/Doc/tools/sphinxext/static/basic.css index 2b47622..21c3db2 100644 --- a/Doc/tools/sphinxext/static/basic.css +++ b/Doc/tools/sphinxext/static/basic.css @@ -38,7 +38,10 @@ div.related li.right { /* -- sidebar --------------------------------------------------------------- */ div.sphinxsidebarwrapper { + position: relative; + top: 0; padding: 10px 5px 0 10px; + word-wrap: break-word; } div.sphinxsidebar { @@ -253,8 +256,8 @@ table.docutils { table.docutils td, table.docutils th { padding: 2px 5px 2px 5px; - border-left: 0; - background-color: #eef; + border-left: 0; + background-color: #eef; } table.docutils td p.last, table.docutils th p.last { @@ -270,7 +273,7 @@ table.footnote td, table.footnote th { } table.docutils th { - border-top: 1px solid #cac; + border-top: 1px solid #cac; background-color: #ede; } @@ -280,7 +283,7 @@ th { } th.head { - text-align: center; + text-align: center; } /* -- other body styles ----------------------------------------------------- */ @@ -333,10 +336,14 @@ dl.glossary dt { font-style: italic; } -p.deprecated { +.deprecated { background-color: #ffe4e4; border: 1px solid #f66; - padding: 7px + padding: 7px; +} + +div.deprecated p { + margin-bottom: 0; } .system-message { diff --git a/Doc/tools/sphinxext/static/sidebar.js b/Doc/tools/sphinxext/static/sidebar.js new file mode 100644 index 0000000..1bdd829 --- /dev/null +++ b/Doc/tools/sphinxext/static/sidebar.js @@ -0,0 +1,186 @@ +/* + * sidebar.js + * ~~~~~~~~~~ + * + * This script makes the Sphinx sidebar collapsible and implements + * intelligent scrolling. + * + * .sphinxsidebar contains .sphinxsidebarwrapper. This script adds + * in .sphixsidebar, after .sphinxsidebarwrapper, the #sidebarbutton + * used to collapse and expand the sidebar. + * + * When the sidebar is collapsed the .sphinxsidebarwrapper is hidden + * and the width of the sidebar and the margin-left of the document + * are decreased. When the sidebar is expanded the opposite happens. + * This script saves a per-browser/per-session cookie used to + * remember the position of the sidebar among the pages. + * Once the browser is closed the cookie is deleted and the position + * reset to the default (expanded). + * + * :copyright: Copyright 2007-2011 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ + +$(function() { + // global elements used by the functions. + // the 'sidebarbutton' element is defined as global after its + // creation, in the add_sidebar_button function + var jwindow = $(window); + var jdocument = $(document); + var bodywrapper = $('.bodywrapper'); + var sidebar = $('.sphinxsidebar'); + var sidebarwrapper = $('.sphinxsidebarwrapper'); + + // original margin-left of the bodywrapper and width of the sidebar + // with the sidebar expanded + var bw_margin_expanded = bodywrapper.css('margin-left'); + var ssb_width_expanded = sidebar.width(); + + // margin-left of the bodywrapper and width of the sidebar + // with the sidebar collapsed + var bw_margin_collapsed = '.8em'; + var ssb_width_collapsed = '.8em'; + + // colors used by the current theme + var dark_color = $('.related').css('background-color'); + var light_color = $('.document').css('background-color'); + + function get_viewport_height() { + if (window.innerHeight) + return window.innerHeight; + else + return jwindow.height(); + } + + function sidebar_is_collapsed() { + return sidebarwrapper.is(':not(:visible)'); + } + + function toggle_sidebar() { + if (sidebar_is_collapsed()) + expand_sidebar(); + else + collapse_sidebar(); + // adjust the scrolling of the sidebar + scroll_sidebar(); + } + + function collapse_sidebar() { + sidebarwrapper.hide(); + sidebar.css('width', ssb_width_collapsed); + bodywrapper.css('margin-left', bw_margin_collapsed); + sidebarbutton.css({ + 'margin-left': '0', + 'height': bodywrapper.height() + }); + sidebarbutton.find('span').text('»'); + sidebarbutton.attr('title', _('Expand sidebar')); + document.cookie = 'sidebar=collapsed'; + } + + function expand_sidebar() { + bodywrapper.css('margin-left', bw_margin_expanded); + sidebar.css('width', ssb_width_expanded); + sidebarwrapper.show(); + sidebarbutton.css({ + 'margin-left': ssb_width_expanded-12, + 'height': bodywrapper.height() + }); + sidebarbutton.find('span').text('«'); + sidebarbutton.attr('title', _('Collapse sidebar')); + document.cookie = 'sidebar=expanded'; + } + + function add_sidebar_button() { + sidebarwrapper.css({ + 'float': 'left', + 'margin-right': '0', + 'width': ssb_width_expanded - 28 + }); + // create the button + sidebar.append( + '<div id="sidebarbutton"><span>«</span></div>' + ); + var sidebarbutton = $('#sidebarbutton'); + light_color = sidebarbutton.css('background-color'); + // find the height of the viewport to center the '<<' in the page + var viewport_height = get_viewport_height(); + sidebarbutton.find('span').css({ + 'display': 'block', + 'margin-top': (viewport_height - sidebar.position().top - 20) / 2 + }); + + sidebarbutton.click(toggle_sidebar); + sidebarbutton.attr('title', _('Collapse sidebar')); + sidebarbutton.css({ + 'color': '#FFFFFF', + 'border-left': '1px solid ' + dark_color, + 'font-size': '1.2em', + 'cursor': 'pointer', + 'height': bodywrapper.height(), + 'padding-top': '1px', + 'margin-left': ssb_width_expanded - 12 + }); + + sidebarbutton.hover( + function () { + $(this).css('background-color', dark_color); + }, + function () { + $(this).css('background-color', light_color); + } + ); + } + + function set_position_from_cookie() { + if (!document.cookie) + return; + var items = document.cookie.split(';'); + for(var k=0; k<items.length; k++) { + var key_val = items[k].split('='); + var key = key_val[0]; + if (key == 'sidebar') { + var value = key_val[1]; + if ((value == 'collapsed') && (!sidebar_is_collapsed())) + collapse_sidebar(); + else if ((value == 'expanded') && (sidebar_is_collapsed())) + expand_sidebar(); + } + } + } + + add_sidebar_button(); + var sidebarbutton = $('#sidebarbutton'); + set_position_from_cookie(); + + + /* intelligent scrolling */ + function scroll_sidebar() { + var sidebar_height = sidebarwrapper.height(); + var viewport_height = get_viewport_height(); + var offset = sidebar.position()['top']; + var wintop = jwindow.scrollTop(); + var winbot = wintop + viewport_height; + var curtop = sidebarwrapper.position()['top']; + var curbot = curtop + sidebar_height; + // does sidebar fit in window? + if (sidebar_height < viewport_height) { + // yes: easy case -- always keep at the top + sidebarwrapper.css('top', $u.min([$u.max([0, wintop - offset - 10]), + jdocument.height() - sidebar_height - 200])); + } + else { + // no: only scroll if top/bottom edge of sidebar is at + // top/bottom edge of window + if (curtop > wintop && curbot > winbot) { + sidebarwrapper.css('top', $u.max([wintop - offset - 10, 0])); + } + else if (curtop < wintop && curbot < winbot) { + sidebarwrapper.css('top', $u.min([winbot - sidebar_height - offset - 20, + jdocument.height() - sidebar_height - 200])); + } + } + } + jwindow.scroll(scroll_sidebar); +}); diff --git a/Doc/tools/sphinxext/static/version_switch.js b/Doc/tools/sphinxext/static/version_switch.js new file mode 100644 index 0000000..e5528eb --- /dev/null +++ b/Doc/tools/sphinxext/static/version_switch.js @@ -0,0 +1,67 @@ +(function() { + 'use strict'; + + var all_versions = { + '3.5': 'dev (3.5)', + '3.4': '3.4', + '3.3': '3.3', + '3.2': '3.2', + '2.7': '2.7', + '2.6': '2.6' + }; + + function build_select(current_version, current_release) { + var buf = ['<select>']; + + $.each(all_versions, function(version, title) { + buf.push('<option value="' + version + '"'); + if (version == current_version) + buf.push(' selected="selected">' + current_release + '</option>'); + else + buf.push('>' + title + '</option>'); + }); + + buf.push('</select>'); + return buf.join(''); + } + + function patch_url(url, new_version) { + var url_re = /\.org\/(\d|py3k|dev|((release\/)?\d\.\d[\w\d\.]*))\//, + new_url = url.replace(url_re, '.org/' + new_version + '/'); + + if (new_url == url && !new_url.match(url_re)) { + // python 2 url without version? + new_url = url.replace(/\.org\//, '.org/' + new_version + '/'); + } + return new_url; + } + + function on_switch() { + var selected = $(this).children('option:selected').attr('value'); + + var url = window.location.href, + new_url = patch_url(url, selected); + + if (new_url != url) { + // check beforehand if url exists, else redirect to version's start page + $.ajax({ + url: new_url, + success: function() { + window.location.href = new_url; + }, + error: function() { + window.location.href = 'http://docs.python.org/' + selected; + } + }); + } + } + + $(document).ready(function() { + var release = DOCUMENTATION_OPTIONS.VERSION; + var version = release.substr(0, 3); + var select = build_select(version, release); + + $('.version_switcher_placeholder').html(select); + $('.version_switcher_placeholder select').bind('change', on_switch); + }); +})(); diff --git a/Doc/tools/sphinxext/susp-ignored.csv b/Doc/tools/sphinxext/susp-ignored.csv index 3785e96..dc719e5 100644 --- a/Doc/tools/sphinxext/susp-ignored.csv +++ b/Doc/tools/sphinxext/susp-ignored.csv @@ -1,9 +1,6 @@ c-api/arg,,:ref,"PyArg_ParseTuple(args, ""O|O:ref"", &object, &callback)" c-api/list,,:high,list[low:high] -c-api/list,,:high,list[low:high] = itemlist c-api/sequence,,:i2,o[i1:i2] -c-api/sequence,,:i2,o[i1:i2] = v -c-api/sequence,,:i2,del o[i1:i2] c-api/unicode,,:end,str[start:end] distutils/setupscript,,::, extending/embedding,,:numargs,"if(!PyArg_ParseTuple(args, "":numargs""))" @@ -11,7 +8,10 @@ extending/extending,,:set,"if (PyArg_ParseTuple(args, ""O:set_callback"", &temp) extending/extending,,:myfunction,"PyArg_ParseTuple(args, ""D:myfunction"", &c);" extending/newtypes,,:call,"if (!PyArg_ParseTuple(args, ""sss:call"", &arg1, &arg2, &arg3)) {" extending/windows,,:initspam,/export:initspam -howto/cporting,,:add,"if (!PyArg_ParseTuple(args, ""ii:add_ints"", &one, &two))" +faq/programming,,:reduce,"print (lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y," +faq/programming,,:reduce,"Sx=Sx,Sy=Sy:reduce(lambda x,y:x+y,map(lambda x,xc=Ru,yc=yc,Ru=Ru,Ro=Ro," +faq/programming,,:chr,">=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr(" +faq/programming,,::,for x in sequence[::-1]: howto/cporting,,:encode,"if (!PyArg_ParseTuple(args, ""O:encode_object"", &myobj))" howto/cporting,,:say,"if (!PyArg_ParseTuple(args, ""U:say_hello"", &name))" howto/curses,,:black,"They are: 0:black, 1:red, 2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and" @@ -22,11 +22,36 @@ howto/curses,,:magenta,"They are: 0:black, 1:red, 2:green, 3:yellow, 4:blue, 5:m howto/curses,,:red,"They are: 0:black, 1:red, 2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and" howto/curses,,:white,"7:white." howto/curses,,:yellow,"They are: 0:black, 1:red, 2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and" +howto/logging,,:root,WARNING:root:Watch out! +howto/logging,,:Watch,WARNING:root:Watch out! +howto/logging,,:root,DEBUG:root:This message should go to the log file +howto/logging,,:root,INFO:root:So should this +howto/logging,,:So,INFO:root:So should this +howto/logging,,:root,"WARNING:root:And this, too" +howto/logging,,:And,"WARNING:root:And this, too" +howto/logging,,:root,INFO:root:Started +howto/logging,,:Started,INFO:root:Started +howto/logging,,:root,INFO:root:Doing something +howto/logging,,:Doing,INFO:root:Doing something +howto/logging,,:root,INFO:root:Finished +howto/logging,,:Finished,INFO:root:Finished +howto/logging,,:root,WARNING:root:Look before you leap! +howto/logging,,:Look,WARNING:root:Look before you leap! +howto/logging,,:This,DEBUG:This message should appear on the console +howto/logging,,:So,INFO:So should this +howto/logging,,:And,"WARNING:And this, too" +howto/logging,,:logger,severity:logger name:message +howto/logging,,:message,severity:logger name:message +howto/logging,,:This,DEBUG:root:This message should go to the log file +howto/pyporting,,::,Programming Language :: Python :: 2 +howto/pyporting,,::,Programming Language :: Python :: 3 howto/regex,,::, howto/regex,,:foo,(?:foo) howto/urllib2,,:example,"for example ""joe@password:example.com""" -howto/webservers,,.. image:,.. image:: http.png library/audioop,,:ipos,"# factor = audioop.findfactor(in_test[ipos*2:ipos*2+len(out_test)]," +library/bisect,,:hi,all(val >= x for val in a[i:hi]) +library/bisect,,:hi,all(val > x for val in a[i:hi]) +library/cookie,,`,!#$%&'*+-.^_`|~ library/datetime,,:MM, library/datetime,,:SS, library/decimal,,:optional,"trailneg:optional trailing minus indicator" @@ -40,22 +65,18 @@ library/dis,,`,TOS = `TOS` library/doctest,,`,``factorial`` from the ``example`` module: library/doctest,,`,The ``example`` module library/doctest,,`,Using ``factorial`` +library/exceptions,,:err,err.object[err.start:err.end] library/functions,,:step,a[start:stop:step] library/functions,,:stop,"a[start:stop, i]" library/functions,,:stop,a[start:stop:step] library/hotshot,,:lineno,"ncalls tottime percall cumtime percall filename:lineno(function)" library/httplib,,:port,host:port -library/imaplib,,:MM,"""DD-Mmm-YYYY HH:MM:SS +HHMM""" -library/imaplib,,:SS,"""DD-Mmm-YYYY HH:MM:SS +HHMM""" +library/imaplib,,:MM,"""DD-Mmm-YYYY HH:MM:SS" +library/imaplib,,:SS,"""DD-Mmm-YYYY HH:MM:SS" library/itertools,,:stop,elements from seq[start:stop:step] library/itertools,,:step,elements from seq[start:stop:step] library/linecache,,:sys,"sys:x:3:3:sys:/dev:/bin/sh" -library/logging,,:And, -library/logging,,:package1, -library/logging,,:package2, -library/logging,,:root, -library/logging,,:This, -library/logging,,:port,host:port +library/logging.handlers,,:port,host:port library/mmap,,:i2,obj[i1:i2] library/multiprocessing,,:queue,">>> QueueManager.register('get_queue', callable=lambda:queue)" library/multiprocessing,,`,">>> l._callmethod('__getitem__', (20,)) # equiv to `l[20]`" @@ -65,31 +86,26 @@ library/multiprocessing,,`,# `Pool.imap()` (which will save on the amount of cod library/multiprocessing,,`,# A test file for the `multiprocessing` package library/multiprocessing,,`,# A test of `multiprocessing.Pool` class library/multiprocessing,,`,# Add more tasks using `put()` -library/multiprocessing,,`,# create server for a `HostManager` object -library/multiprocessing,,`,# Depends on `multiprocessing` package -- tested with `processing-0.60` library/multiprocessing,,`,# in the original order then consider using `Pool.map()` or library/multiprocessing,,`,# Not sure if we should synchronize access to `socket.accept()` method by library/multiprocessing,,`,# object. (We import `multiprocessing.reduction` to enable this pickling.) library/multiprocessing,,`,# register the Foo class; make `f()` and `g()` accessible via proxy library/multiprocessing,,`,# register the Foo class; make `g()` and `_h()` accessible via proxy library/multiprocessing,,`,# register the generator function baz; use `GeneratorProxy` to make proxies -library/multiprocessing,,`,`Cluster` is a subclass of `SyncManager` so it allows creation of -library/multiprocessing,,`,`hostname` gives the name of the host. If hostname is not -library/multiprocessing,,`,`slots` is used to specify the number of slots for processes on library/optparse,,:len,"del parser.rargs[:len(value)]" library/os.path,,:foo,c:foo -library/parser,,`,"""Make a function that raises an argument to the exponent `exp`.""" +library/pdb,,:lineno,filename:lineno library/posix,,`,"CFLAGS=""`getconf LFS_CFLAGS`"" OPT=""-g -O2 $CFLAGS""" library/profile,,:lineno,ncalls tottime percall cumtime percall filename:lineno(function) library/profile,,:lineno,filename:lineno(function) library/pyexpat,,:elem1,<py:elem1 /> library/pyexpat,,:py,"xmlns:py = ""http://www.python.org/ns/"">" -library/repr,,`,"return `obj`" -library/smtplib,,:port,"as well as a regular host:port server." +library/smtplib,,:port,method must support that as well as a regular host:port library/socket,,::,'5aef:2b::8' +library/socket,,::,"(10, 1, 6, '', ('2001:888:2000:d::a2', 80, 0, 0))]" library/sqlite3,,:memory, -library/sqlite3,,:age,"select name_last, age from people where name_last=:who and age=:age" -library/sqlite3,,:who,"select name_last, age from people where name_last=:who and age=:age" +library/sqlite3,,:who,"cur.execute(""select * from people where name_last=:who and age=:age"", {""who"": who, ""age"": age})" +library/sqlite3,,:age,"cur.execute(""select * from people where name_last=:who and age=:age"", {""who"": who, ""age"": age})" library/ssl,,:My,"Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Organization, Inc." library/ssl,,:My,"Organizational Unit Name (eg, section) []:My Group" library/ssl,,:myserver,"Common Name (eg, YOUR name) []:myserver.mygroup.myorganization.com" @@ -98,8 +114,7 @@ library/ssl,,:ops,Email Address []:ops@myserver.mygroup.myorganization.com library/ssl,,:Some,"Locality Name (eg, city) []:Some City" library/ssl,,:US,Country Name (2 letter code) [AU]:US library/stdtypes,,:len,s[len(s):len(s)] -library/stdtypes,,:len,s[len(s):len(s)] -library/string,,:end,s[start:end] +library/stdtypes,,:end,s[start:end] library/string,,:end,s[start:end] library/subprocess,,`,"output=`mycmd myarg`" library/subprocess,,`,"output=`dmesg | grep hda`" @@ -111,6 +126,7 @@ library/time,,:ss, library/turtle,,::,Example:: library/urllib,,:port,:port library/urllib2,,:password,"""joe:password@python.org""" +library/urllib2,,:close,Connection:close library/uuid,,:uuid,urn:uuid:12345678-1234-5678-1234-567812345678 library/xmlrpclib,,:pass,http://user:pass@host:port/path library/xmlrpclib,,:pass,user:pass @@ -118,14 +134,15 @@ library/xmlrpclib,,:port,http://user:pass@host:port/path license,,`,THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND license,,:zooko,mailto:zooko@zooko.com license,,`,THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +license,,`,* THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY +license,,`,* THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND +license,,`,"``Software''), to deal in the Software without restriction, including" +license,,`,"THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND," reference/datamodel,,:step,a[i:j:step] reference/datamodel,,:max, reference/expressions,,:index,x[index:index] -reference/expressions,,:datum,{key:datum...} reference/expressions,,`,`expressions...` -reference/grammar,,:output,#diagram:output -reference/grammar,,:rules,#diagram:rules -reference/grammar,,:token,#diagram:token +reference/expressions,,`,"""`""" reference/grammar,,`,'`' testlist1 '`' reference/lexical_analysis,,:fileencoding,# vim:fileencoding=<encoding-name> reference/lexical_analysis,,`,", : . ` = ;" @@ -148,8 +165,7 @@ using/cmdline,,:line,action:message:category:module:line using/cmdline,,:message,action:message:category:module:line using/cmdline,,:module,action:message:category:module:line using/cmdline,,:errorhandler,:errorhandler -using/windows,162,`,`` this fixes syntax highlighting errors in some editors due to the \\\\ hackery -using/windows,170,`,`` +using/unix,,:Packaging,http://en.opensuse.org/Portal:Packaging whatsnew/2.0,418,:len, whatsnew/2.3,,::, whatsnew/2.3,,:config, @@ -163,36 +179,8 @@ whatsnew/2.4,,:System, whatsnew/2.5,,:memory,:memory: whatsnew/2.5,,:step,[start:stop:step] whatsnew/2.5,,:stop,[start:stop:step] -distutils/examples,267,`,This is the description of the ``foobar`` package. -faq/programming,,:reduce,"print (lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y," -faq/programming,,:reduce,"Sx=Sx,Sy=Sy:reduce(lambda x,y:x+y,map(lambda x,xc=Ru,yc=yc,Ru=Ru,Ro=Ro," -faq/programming,,:chr,">=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr(" -faq/programming,,::,for x in sequence[::-1]: -faq/windows,229,:EOF,@setlocal enableextensions & python -x %~f0 %* & goto :EOF -faq/windows,393,:REG,.py :REG_SZ: c:\<path to python>\python.exe -u %s %s -library/bisect,,:hi,all(val >= x for val in a[i:hi]) -library/bisect,,:hi,all(val > x for val in a[i:hi]) -library/http.client,52,:port,host:port -library/nntplib,,:bytes,:bytes -library/nntplib,,:lines,:lines -library/nntplib,,:lines,"['xref', 'from', ':lines', ':bytes', 'references', 'date', 'message-id', 'subject']" -library/nntplib,,:bytes,"['xref', 'from', ':lines', ':bytes', 'references', 'date', 'message-id', 'subject']" -library/pickle,567,:memory,"conn = sqlite3.connect("":memory:"")" -library/profile,293,:lineno,"(sort by filename:lineno)," -library/socket,,::,"(10, 1, 6, '', ('2001:888:2000:d::a2', 80, 0, 0))]" -library/stdtypes,,:end,s[start:end] -library/stdtypes,,:end,s[start:end] -license,,`,* THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY -license,,`,* THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND -license,,`,"``Software''), to deal in the Software without restriction, including" -license,,`,"THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND," -reference/lexical_analysis,704,`,$ ? ` whatsnew/2.7,735,:Sunday,'2009:4:Sunday' whatsnew/2.7,862,::,"export PYTHONWARNINGS=all,error:::Cookie:0" whatsnew/2.7,862,:Cookie,"export PYTHONWARNINGS=all,error:::Cookie:0" whatsnew/2.7,,::,>>> urlparse.urlparse('http://[1080::8:800:200C:417A]/foo') whatsnew/2.7,,::,"ParseResult(scheme='http', netloc='[1080::8:800:200C:417A]'," -howto/pyporting,75,::,# make sure to use :: Python *and* :: Python :: 3 so -howto/pyporting,75,::,"'Programming Language :: Python'," -howto/pyporting,75,::,'Programming Language :: Python :: 3' -library/urllib2,67,:close,Connection:close diff --git a/Doc/tools/sphinxext/suspicious.py b/Doc/tools/sphinxext/suspicious.py index f15e931..e397560 100644 --- a/Doc/tools/sphinxext/suspicious.py +++ b/Doc/tools/sphinxext/suspicious.py @@ -66,6 +66,10 @@ class Rule: # None -> don't care self.issue = issue # the markup fragment that triggered this rule self.line = line # text of the container element (single line only) + self.used = False + + def __repr__(self): + return '{0.docname},,{0.issue},{0.line}'.format(self) @@ -105,6 +109,12 @@ class CheckSuspiciousMarkupBuilder(Builder): doctree.walk(visitor) def finish(self): + unused_rules = [rule for rule in self.rules if not rule.used] + if unused_rules: + self.warn('Found %s/%s unused rules:' % + (len(unused_rules), len(self.rules))) + for rule in unused_rules: + self.info(repr(rule)) return def check_issue(self, line, lineno, issue): @@ -129,6 +139,7 @@ class CheckSuspiciousMarkupBuilder(Builder): if (rule.lineno is not None) and \ abs(rule.lineno - lineno) > 5: continue # if it came this far, the rule matched + rule.used = True return True return False diff --git a/Doc/tutorial/classes.rst b/Doc/tutorial/classes.rst index 9f115b0..60d382c 100644 --- a/Doc/tutorial/classes.rst +++ b/Doc/tutorial/classes.rst @@ -337,6 +337,77 @@ object and the argument list, and the function object is called with this new argument list. +.. _tut-class-and-instance-variables: + +Class and Instance Variables +---------------------------- + +Generally speaking, instance variables are for data unique to each instance +and class variables are for attributes and methods shared by all instances +of the class:: + + class Dog: + + kind = 'canine' # class variable shared by all instances + + def __init__(self, name): + self.name = name # instance variable unique to each instance + + >>> d = Dog('Fido') + >>> e = Dog('Buddy') + >>> d.kind # shared by all dogs + 'canine' + >>> e.kind # shared by all dogs + 'canine' + >>> d.name # unique to d + 'Fido' + >>> e.name # unique to e + 'Buddy' + +As discussed in :ref:`tut-object`, shared data can have possibly surprising +effects with involving :term:`mutable` objects such as lists and dictionaries. +For example, the *tricks* list in the following code should not be used as a +class variable because just a single list would be shared by all *Dog* +instances:: + + class Dog: + + tricks = [] # mistaken use of a class variable + + def __init__(self, name): + self.name = name + + def add_trick(self, trick): + self.tricks.append(trick) + + >>> d = Dog('Fido') + >>> e = Dog('Buddy') + >>> d.add_trick('roll over') + >>> e.add_trick('play dead') + >>> d.tricks # unexpectedly shared by all dogs + ['roll over', 'play dead'] + +Correct design of the class should use an instance variable instead:: + + class Dog: + + def __init__(self, name): + self.name = name + self.tricks = [] # creates a new empty list for each dog + + def add_trick(self, trick): + self.tricks.append(trick) + + >>> d = Dog('Fido') + >>> e = Dog('Buddy') + >>> d.add_trick('roll over') + >>> e.add_trick('play dead') + >>> d.tricks + ['roll over'] + >>> e.tricks + ['play dead'] + + .. _tut-remarks: Random Remarks @@ -534,8 +605,8 @@ http://www.python.org/download/releases/2.3/mro/. .. _tut-private: -Private Variables -================= +Private Variables and Class-local References +============================================ "Private" instance variables that cannot be accessed except from inside an object don't exist in Python. However, there is a convention that is followed @@ -609,7 +680,7 @@ will do nicely:: A piece of Python code that expects a particular abstract data type can often be passed a class that emulates the methods of that data type instead. For instance, if you have a function that formats some data from a file object, you -can define a class with methods :meth:`read` and :meth:`readline` that get the +can define a class with methods :meth:`read` and :meth:`!readline` that get the data from a string buffer instead, and pass it as an argument. .. (Unfortunately, this technique has its limitations: a class can't define @@ -688,14 +759,15 @@ using a :keyword:`for` statement:: for char in "123": print char for line in open("myfile.txt"): - print line + print line, This style of access is clear, concise, and convenient. The use of iterators pervades and unifies Python. Behind the scenes, the :keyword:`for` statement calls :func:`iter` on the container object. The function returns an iterator -object that defines the method :meth:`next` which accesses elements in the -container one at a time. When there are no more elements, :meth:`next` raises a -:exc:`StopIteration` exception which tells the :keyword:`for` loop to terminate. +object that defines the method :meth:`~iterator.next` which accesses elements +in the container one at a time. When there are no more elements, +:meth:`~iterator.next` raises a :exc:`StopIteration` exception which tells the +:keyword:`for` loop to terminate. This example shows how it all works:: >>> s = 'abc' diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index 24c02eb..8ffaf3f 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -19,14 +19,14 @@ example:: >>> x = int(raw_input("Please enter an integer: ")) Please enter an integer: 42 >>> if x < 0: - ... x = 0 - ... print 'Negative changed to zero' + ... x = 0 + ... print 'Negative changed to zero' ... elif x == 0: - ... print 'Zero' + ... print 'Zero' ... elif x == 1: - ... print 'Single' + ... print 'Single' ... else: - ... print 'More' + ... print 'More' ... More @@ -59,24 +59,24 @@ they appear in the sequence. For example (no pun intended): :: >>> # Measure some strings: - ... a = ['cat', 'window', 'defenestrate'] - >>> for x in a: - ... print x, len(x) + ... words = ['cat', 'window', 'defenestrate'] + >>> for w in words: + ... print w, len(w) ... cat 3 window 6 defenestrate 12 -It is not safe to modify the sequence being iterated over in the loop (this can -only happen for mutable sequence types, such as lists). If you need to modify -the list you are iterating over (for example, to duplicate selected items) you -must iterate over a copy. The slice notation makes this particularly -convenient:: +If you need to modify the sequence you are iterating over while inside the loop +(for example to duplicate selected items), it is recommended that you first +make a copy. Iterating over a sequence does not implicitly make a copy. The +slice notation makes this especially convenient:: - >>> for x in a[:]: # make a slice copy of the entire list - ... if len(x) > 6: a.insert(0, x) + >>> for w in words[:]: # Loop over a slice copy of the entire list. + ... if len(w) > 6: + ... words.insert(0, w) ... - >>> a + >>> words ['defenestrate', 'cat', 'window', 'defenestrate'] @@ -129,9 +129,6 @@ function, see :ref:`tut-loopidioms`. The :keyword:`break` statement, like in C, breaks out of the smallest enclosing :keyword:`for` or :keyword:`while` loop. -The :keyword:`continue` statement, also borrowed from C, continues with the next -iteration of the loop. - Loop statements may have an ``else`` clause; it is executed when the loop terminates through exhaustion of the list (with :keyword:`for`) or when the condition becomes false (with :keyword:`while`), but not when the loop is @@ -159,6 +156,30 @@ following loop, which searches for prime numbers:: (Yes, this is the correct code. Look closely: the ``else`` clause belongs to the :keyword:`for` loop, **not** the :keyword:`if` statement.) +When used with a loop, the ``else`` clause has more in common with the +``else`` clause of a :keyword:`try` statement than it does that of +:keyword:`if` statements: a :keyword:`try` statement's ``else`` clause runs +when no exception occurs, and a loop's ``else`` clause runs when no ``break`` +occurs. For more on the :keyword:`try` statement and exceptions, see +:ref:`tut-handling`. + +The :keyword:`continue` statement, also borrowed from C, continues with the next +iteration of the loop:: + + >>> for num in range(2, 10): + ... if num % 2 == 0: + ... print "Found an even number", num + ... continue + ... print "Found a number", num + Found an even number 2 + Found a number 3 + Found an even number 4 + Found a number 5 + Found an even number 6 + Found a number 7 + Found an even number 8 + Found a number 9 + .. _tut-pass: @@ -520,17 +541,16 @@ In the same fashion, dictionaries can deliver keyword arguments with the ``**``\ .. _tut-lambda: -Lambda Forms ------------- +Lambda Expressions +------------------ -By popular demand, a few features commonly found in functional programming -languages like Lisp have been added to Python. With the :keyword:`lambda` -keyword, small anonymous functions can be created. Here's a function that -returns the sum of its two arguments: ``lambda a, b: a+b``. Lambda forms can be -used wherever function objects are required. They are syntactically restricted -to a single expression. Semantically, they are just syntactic sugar for a -normal function definition. Like nested function definitions, lambda forms can -reference variables from the containing scope:: +Small anonymous functions can be created with the :keyword:`lambda` keyword. +This function returns the sum of its two arguments: ``lambda a, b: a+b``. +Lambda functions can be used wherever function objects are required. They are +syntactically restricted to a single expression. Semantically, they are just +syntactic sugar for a normal function definition. Like nested function +definitions, lambda functions can reference variables from the containing +scope:: >>> def make_incrementor(n): ... return lambda x: x + n @@ -541,6 +561,14 @@ reference variables from the containing scope:: >>> f(1) 43 +The above example uses a lambda expression to return a function. Another use +is to pass a small function as an argument:: + + >>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')] + >>> pairs.sort(key=lambda pair: pair[1]) + >>> pairs + [(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')] + .. _tut-docstrings: diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst index 489a336..acc2cc1 100644 --- a/Doc/tutorial/datastructures.rst +++ b/Doc/tutorial/datastructures.rst @@ -68,10 +68,11 @@ objects: Return the number of times *x* appears in the list. -.. method:: list.sort() +.. method:: list.sort(cmp=None, key=None, reverse=False) :noindex: - Sort the items of the list, in place. + Sort the items of the list in place (the arguments can be used for sort + customization, see :func:`sorted` for their explanation). .. method:: list.reverse() @@ -99,6 +100,15 @@ An example that uses most of the list methods:: >>> a.sort() >>> a [-1, 1, 66.25, 333, 333, 1234.5] + >>> a.pop() + 1234.5 + >>> a + [-1, 1, 66.25, 333, 333] + +You might have noticed that methods like ``insert``, ``remove`` or ``sort`` that +only modify the list have no return value printed -- they return the default +``None``. [1]_ This is a design principle for all mutable data structures in +Python. .. _tut-lists-as-stacks: @@ -171,7 +181,7 @@ There are three built-in functions that are very useful when used with lists: the sequence for which ``function(item)`` is true. If *sequence* is a :class:`string` or :class:`tuple`, the result will be of the same type; otherwise, it is always a :class:`list`. For example, to compute a sequence of -numbers not divisible by 2 and 3:: +numbers not divisible by 2 or 3:: >>> def f(x): return x % 2 != 0 and x % 3 != 0 ... @@ -229,8 +239,7 @@ Don't use this example's definition of :func:`sum`: since summing numbers is such a common need, a built-in function ``sum(sequence)`` is already provided, and works exactly like this. -.. versionadded:: 2.3 - +.. _tut-listcomps: List Comprehensions ------------------- @@ -423,17 +432,31 @@ A tuple consists of a number of values separated by commas, for instance:: ... u = t, (1, 2, 3, 4, 5) >>> u ((12345, 54321, 'hello!'), (1, 2, 3, 4, 5)) + >>> # Tuples are immutable: + ... t[0] = 88888 + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + TypeError: 'tuple' object does not support item assignment + >>> # but they can contain mutable objects: + ... v = ([1, 2, 3], [3, 2, 1]) + >>> v + ([1, 2, 3], [3, 2, 1]) + As you see, on output tuples are always enclosed in parentheses, so that nested tuples are interpreted correctly; they may be input with or without surrounding parentheses, although often parentheses are necessary anyway (if the tuple is -part of a larger expression). - -Tuples have many uses. For example: (x, y) coordinate pairs, employee records -from a database, etc. Tuples, like strings, are immutable: it is not possible -to assign to the individual items of a tuple (you can simulate much of the same -effect with slicing and concatenation, though). It is also possible to create -tuples which contain mutable objects, such as lists. +part of a larger expression). It is not possible to assign to the individual +items of a tuple, however it is possible to create tuples which contain mutable +objects, such as lists. + +Though tuples may seem similar to lists, they are often used in different +situations and for different purposes. +Tuples are :term:`immutable`, and usually contain an heterogeneous sequence of +elements that are accessed via unpacking (see later in this section) or indexing +(or even by attribute in the case of :func:`namedtuples <collections.namedtuple>`). +Lists are :term:`mutable`, and their elements are usually homogeneous and are +accessed by iterating over the list. A special problem is the construction of tuples containing 0 or 1 items: the syntax has some extra quirks to accommodate these. Empty tuples are constructed @@ -462,8 +485,6 @@ variables on the left to have the same number of elements as the length of the sequence. Note that multiple assignment is really just a combination of tuple packing and sequence unpacking. -.. XXX Add a bit on the difference between tuples and lists. - .. _tut-sets: @@ -475,6 +496,10 @@ with no duplicate elements. Basic uses include membership testing and eliminating duplicate entries. Set objects also support mathematical operations like union, intersection, difference, and symmetric difference. +Curly braces or the :func:`set` function can be used to create sets. Note: to +create an empty set you have to use ``set()``, not ``{}``; the latter creates an +empty dictionary, a data structure that we discuss in the next section. + Here is a brief demonstration:: >>> basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana'] @@ -501,6 +526,13 @@ Here is a brief demonstration:: >>> a ^ b # letters in a or b but not both set(['r', 'd', 'b', 'm', 'z', 'l']) +Similarly to :ref:`list comprehensions <tut-listcomps>`, set comprehensions +are also supported:: + + >>> a = {x for x in 'abracadabra' if x not in 'abc'} + >>> a + set(['r', 'd']) + .. _tut-dictionaries: @@ -552,18 +584,17 @@ Here is a small example using a dictionary:: >>> 'guido' in tel True -The :func:`dict` constructor builds dictionaries directly from lists of -key-value pairs stored as tuples. When the pairs form a pattern, list -comprehensions can compactly specify the key-value list. :: +The :func:`dict` constructor builds dictionaries directly from sequences of +key-value pairs:: >>> dict([('sape', 4139), ('guido', 4127), ('jack', 4098)]) {'sape': 4139, 'jack': 4098, 'guido': 4127} - >>> dict([(x, x**2) for x in (2, 4, 6)]) # use a list comprehension - {2: 4, 4: 16, 6: 36} -Later in the tutorial, we will learn about Generator Expressions which are even -better suited for the task of supplying key-values pairs to the :func:`dict` -constructor. +In addition, dict comprehensions can be used to create dictionaries from +arbitrary key and value expressions:: + + >>> {x: x**2 for x in (2, 4, 6)} + {2: 4, 4: 16, 6: 36} When the keys are simple strings, it is sometimes easier to specify pairs using keyword arguments:: @@ -577,16 +608,6 @@ keyword arguments:: Looping Techniques ================== -When looping through dictionaries, the key and corresponding value can be -retrieved at the same time using the :meth:`iteritems` method. :: - - >>> knights = {'gallahad': 'the pure', 'robin': 'the brave'} - >>> for k, v in knights.iteritems(): - ... print k, v - ... - gallahad the pure - robin the brave - When looping through a sequence, the position index and corresponding value can be retrieved at the same time using the :func:`enumerate` function. :: @@ -633,6 +654,29 @@ returns a new sorted list while leaving the source unaltered. :: orange pear +When looping through dictionaries, the key and corresponding value can be +retrieved at the same time using the :meth:`iteritems` method. :: + + >>> knights = {'gallahad': 'the pure', 'robin': 'the brave'} + >>> for k, v in knights.iteritems(): + ... print k, v + ... + gallahad the pure + robin the brave + +To change a sequence you are iterating over while inside the loop (for +example to duplicate certain items), it is recommended that you first make +a copy. Looping over a sequence does not implicitly make a copy. The slice +notation makes this especially convenient:: + + >>> words = ['cat', 'window', 'defenestrate'] + >>> for w in words[:]: # Loop over a slice copy of the entire list. + ... if len(w) > 6: + ... words.insert(0, w) + ... + >>> words + ['defenestrate', 'cat', 'window', 'defenestrate'] + .. _tut-conditions: diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index 1351957..6d14cb3 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -120,6 +120,14 @@ name multiple exceptions as a parenthesized tuple, for example:: ... except (RuntimeError, TypeError, NameError): ... pass +Note that the parentheses around this tuple are required, because +``except ValueError, e:`` was the syntax used for what is normally +written as ``except ValueError as e:`` in modern Python (described +below). The old syntax is still supported for backwards compatibility. +This means ``except RuntimeError, TypeError`` is not equivalent to +``except (RuntimeError, TypeError):`` but to ``except RuntimeError as +TypeError:`` which is not what you want. + The last except clause may omit the exception name(s), to serve as a wildcard. Use this with extreme caution, since it is easy to mask a real programming error in this way! It can also be used to print an error message and then re-raise @@ -131,8 +139,8 @@ the exception (allowing a caller to handle the exception as well):: f = open('myfile.txt') s = f.readline() i = int(s.strip()) - except IOError as (errno, strerror): - print "I/O error({0}): {1}".format(errno, strerror) + except IOError as e: + print "I/O error({0}): {1}".format(e.errno, e.strerror) except ValueError: print "Could not convert data to an integer." except: @@ -177,7 +185,7 @@ attributes to it as desired. :: ... print type(inst) # the exception instance ... print inst.args # arguments stored in .args ... print inst # __str__ allows args to printed directly - ... x, y = inst # __getitem__ allows args to be unpacked directly + ... x, y = inst.args ... print 'x =', x ... print 'y =', y ... @@ -389,7 +397,7 @@ succeeded or failed. Look at the following example, which tries to open a file and print its contents to the screen. :: for line in open("myfile.txt"): - print line + print line, The problem with this code is that it leaves the file open for an indeterminate amount of time after the code has finished executing. This is not an issue in @@ -399,7 +407,7 @@ ensures they are always cleaned up promptly and correctly. :: with open("myfile.txt") as f: for line in f: - print line + print line, After the statement is executed, the file *f* is always closed, even if a problem was encountered while processing the lines. Other objects which provide diff --git a/Doc/tutorial/index.rst b/Doc/tutorial/index.rst index 7d0bfc2..604cff8 100644 --- a/Doc/tutorial/index.rst +++ b/Doc/tutorial/index.rst @@ -4,9 +4,6 @@ The Python Tutorial ###################### -:Release: |version| -:Date: |today| - Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, diff --git a/Doc/tutorial/inputoutput.rst b/Doc/tutorial/inputoutput.rst index daad58d..6fdc5f0 100644 --- a/Doc/tutorial/inputoutput.rst +++ b/Doc/tutorial/inputoutput.rst @@ -37,7 +37,7 @@ or :func:`str` functions. The :func:`str` function is meant to return representations of values which are fairly human-readable, while :func:`repr` is meant to generate representations which can be read by the interpreter (or will force a :exc:`SyntaxError` if -there is not equivalent syntax). For objects which don't have a particular +there is no equivalent syntax). For objects which don't have a particular representation for human consumption, :func:`str` will return the same value as :func:`repr`. Many values, such as numbers or structures like lists and dictionaries, have the same representation using either function. Strings and @@ -215,10 +215,6 @@ operation. For example:: >>> print 'The value of PI is approximately %5.3f.' % math.pi The value of PI is approximately 3.142. -Since :meth:`str.format` is quite new, a lot of Python code still uses the ``%`` -operator. However, because this old style of formatting will eventually be -removed from the language, :meth:`str.format` should generally be used. - More information can be found in the :ref:`string-formatting` section. @@ -236,9 +232,9 @@ arguments: ``open(filename, mode)``. :: - >>> f = open('/tmp/workfile', 'w') + >>> f = open('workfile', 'w') >>> print f - <open file '/tmp/workfile', mode 'w' at 80a0960> + <open file 'workfile', mode 'w' at 80a0960> The first argument is a string containing the filename. The second argument is another string containing a few characters describing the way in which the file @@ -295,18 +291,8 @@ containing only a single newline. :: >>> f.readline() '' -``f.readlines()`` returns a list containing all the lines of data in the file. -If given an optional parameter *sizehint*, it reads that many bytes from the -file and enough more to complete a line, and returns the lines from that. This -is often used to allow efficient reading of a large file by lines, but without -having to load the entire file in memory. Only complete lines will be returned. -:: - - >>> f.readlines() - ['This is the first line of the file.\n', 'Second line of the file\n'] - -An alternative approach to reading lines is to loop over the file object. This is -memory efficient, fast, and leads to simpler code:: +For reading lines from a file, you can loop over the file object. This is memory +efficient, fast, and leads to simple code:: >>> for line in f: print line, @@ -314,9 +300,8 @@ memory efficient, fast, and leads to simpler code:: This is the first line of the file. Second line of the file -The alternative approach is simpler but does not provide as fine-grained -control. Since the two approaches manage line buffering differently, they -should not be mixed. +If you want to read all the lines of a file in a list you can also use +``list(f)`` or ``f.readlines()``. ``f.write(string)`` writes the contents of *string* to the file, returning ``None``. :: @@ -339,7 +324,7 @@ of the file, 1 uses the current file position, and 2 uses the end of the file as the reference point. *from_what* can be omitted and defaults to 0, using the beginning of the file as the reference point. :: - >>> f = open('/tmp/workfile', 'r+') + >>> f = open('workfile', 'r+') >>> f.write('0123456789abcdef') >>> f.seek(5) # Go to the 6th byte in the file >>> f.read(1) @@ -363,7 +348,7 @@ objects. This has the advantage that the file is properly closed after its suite finishes, even if an exception is raised on the way. It is also much shorter than writing equivalent :keyword:`try`\ -\ :keyword:`finally` blocks:: - >>> with open('/tmp/workfile', 'r') as f: + >>> with open('workfile', 'r') as f: ... read_data = f.read() >>> f.closed True @@ -373,47 +358,64 @@ File objects have some additional methods, such as :meth:`~file.isatty` and Reference for a complete guide to file objects. -.. _tut-pickle: +.. _tut-json: -The :mod:`pickle` Module ------------------------- +Saving structured data with :mod:`json` +--------------------------------------- -.. index:: module: pickle +.. index:: module: json -Strings can easily be written to and read from a file. Numbers take a bit more +Strings can easily be written to and read from a file. Numbers take a bit more effort, since the :meth:`read` method only returns strings, which will have to be passed to a function like :func:`int`, which takes a string like ``'123'`` -and returns its numeric value 123. However, when you want to save more complex -data types like lists, dictionaries, or class instances, things get a lot more -complicated. - -Rather than have users be constantly writing and debugging code to save -complicated data types, Python provides a standard module called :mod:`pickle`. -This is an amazing module that can take almost any Python object (even some -forms of Python code!), and convert it to a string representation; this process -is called :dfn:`pickling`. Reconstructing the object from the string -representation is called :dfn:`unpickling`. Between pickling and unpickling, -the string representing the object may have been stored in a file or data, or +and returns its numeric value 123. When you want to save more complex data +types like nested lists and dictionaries, parsing and serializing by hand +becomes complicated. + +Rather than having users constantly writing and debugging code to save +complicated data types to files, Python allows you to use the popular data +interchange format called `JSON (JavaScript Object Notation) +<http://json.org>`_. The standard module called :mod:`json` can take Python +data hierarchies, and convert them to string representations; this process is +called :dfn:`serializing`. Reconstructing the data from the string representation +is called :dfn:`deserializing`. Between serializing and deserializing, the +string representing the object may have been stored in a file or data, or sent over a network connection to some distant machine. -If you have an object ``x``, and a file object ``f`` that's been opened for -writing, the simplest way to pickle the object takes only one line of code:: +.. note:: + The JSON format is commonly used by modern applications to allow for data + exchange. Many programmers are already familiar with it, which makes + it a good choice for interoperability. + +If you have an object ``x``, you can view its JSON string representation with a +simple line of code:: + + >>> json.dumps([1, 'simple', 'list']) + '[1, "simple", "list"]' + +Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`, +simply serializes the object to a file. So if ``f`` is a :term:`file object` +opened for writing, we can do this:: + + json.dump(x, f) - pickle.dump(x, f) +To decode the object again, if ``f`` is a :term:`file object` which has +been opened for reading:: -To unpickle the object again, if ``f`` is a file object which has been opened -for reading:: + x = json.load(f) - x = pickle.load(f) +This simple serialization technique can handle lists and dictionaries, but +serializing arbitrary class instances in JSON requires a bit of extra effort. +The reference for the :mod:`json` module contains an explanation of this. -(There are other variants of this, used when pickling many objects or when you -don't want to write the pickled data to a file; consult the complete -documentation for :mod:`pickle` in the Python Library Reference.) +.. seealso:: -:mod:`pickle` is the standard way to make Python objects which can be stored and -reused by other programs or by a future invocation of the same program; the -technical term for this is a :dfn:`persistent` object. Because :mod:`pickle` is -so widely used, many authors who write Python extensions take care to ensure -that new data types such as matrices can be properly pickled and unpickled. + :mod:`pickle` - the pickle module + Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows + the serialization of arbitrarily complex Python objects. As such, it is + specific to Python and cannot be used to communicate with applications + written in other languages. It is also insecure by default: + deserializing pickle data coming from an untrusted source can execute + arbitrary code, if the data was crafted by a skilled attacker. diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst index 6bdc1c5..5140b43 100644 --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -185,8 +185,8 @@ encodings can be found in the Python Library Reference, in the section on For example, to write Unicode literals including the Euro currency symbol, the ISO-8859-15 encoding can be used, with the Euro symbol having the ordinal value -164. This script will print the value 8364 (the Unicode codepoint corresponding -to the Euro symbol) and then exit:: +164. This script, when saved in the ISO-8859-15 encoding, will print the value +8364 (the Unicode codepoint corresponding to the Euro symbol) and then exit:: # -*- coding: iso-8859-15 -*- @@ -252,7 +252,7 @@ of your user site-packages directory. Start Python and run this code: >>> import site >>> site.getusersitepackages() - '/home/user/.local/lib/python3.2/site-packages' + '/home/user/.local/lib/python2.7/site-packages' Now you can create a file named :file:`usercustomize.py` in that directory and put anything you want in it. It will affect every invocation of Python, unless diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index c99915f..1871dd1 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -82,8 +82,7 @@ A value can be assigned to several variables simultaneously:: Variables must be "defined" (assigned a value) before they can be used, or an error will occur:: - >>> # try to access an undefined variable - ... n + >>> n # try to access an undefined variable Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'n' is not defined diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst index 62b9bb5..f5b8114 100644 --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -71,7 +71,8 @@ More on Modules A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only -the *first* time the module is imported somewhere. [#]_ +the *first* time the module name is encountered in an import statement. [#]_ +(They are also run if the file is executed as a script.) Each module has its own private symbol table, which is used as the global symbol table by all functions defined in the module. Thus, the author of a module can @@ -242,7 +243,7 @@ modules are built into the interpreter; these provide access to operations that are not part of the core of the language but are nevertheless built in, either for efficiency or to provide access to operating system primitives such as system calls. The set of such modules is a configuration option which also -depends on the underlying platform For example, the :mod:`winreg` module is only +depends on the underlying platform. For example, the :mod:`winreg` module is only provided on Windows systems. One particular module deserves some attention: :mod:`sys`, which is built into every Python interpreter. The variables ``sys.ps1`` and ``sys.ps2`` define the strings used as primary and secondary @@ -282,16 +283,21 @@ defines. It returns a sorted list of strings:: >>> import fibo, sys >>> dir(fibo) ['__name__', 'fib', 'fib2'] - >>> dir(sys) - ['__displayhook__', '__doc__', '__excepthook__', '__name__', '__stderr__', - '__stdin__', '__stdout__', '_getframe', 'api_version', 'argv', - 'builtin_module_names', 'byteorder', 'callstats', 'copyright', - 'displayhook', 'exc_clear', 'exc_info', 'exc_type', 'excepthook', - 'exec_prefix', 'executable', 'exit', 'getdefaultencoding', 'getdlopenflags', - 'getrecursionlimit', 'getrefcount', 'hexversion', 'maxint', 'maxunicode', - 'meta_path', 'modules', 'path', 'path_hooks', 'path_importer_cache', - 'platform', 'prefix', 'ps1', 'ps2', 'setcheckinterval', 'setdlopenflags', - 'setprofile', 'setrecursionlimit', 'settrace', 'stderr', 'stdin', 'stdout', + >>> dir(sys) # doctest: +NORMALIZE_WHITESPACE + ['__displayhook__', '__doc__', '__excepthook__', '__name__', '__package__', + '__stderr__', '__stdin__', '__stdout__', '_clear_type_cache', + '_current_frames', '_getframe', '_mercurial', 'api_version', 'argv', + 'builtin_module_names', 'byteorder', 'call_tracing', 'callstats', + 'copyright', 'displayhook', 'dont_write_bytecode', 'exc_clear', 'exc_info', + 'exc_traceback', 'exc_type', 'exc_value', 'excepthook', 'exec_prefix', + 'executable', 'exit', 'flags', 'float_info', 'float_repr_style', + 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags', + 'getfilesystemencoding', 'getobjects', 'getprofile', 'getrecursionlimit', + 'getrefcount', 'getsizeof', 'gettotalrefcount', 'gettrace', 'hexversion', + 'long_info', 'maxint', 'maxsize', 'maxunicode', 'meta_path', 'modules', + 'path', 'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1', + 'py3kwarning', 'setcheckinterval', 'setdlopenflags', 'setprofile', + 'setrecursionlimit', 'settrace', 'stderr', 'stdin', 'stdout', 'subversion', 'version', 'version_info', 'warnoptions'] Without arguments, :func:`dir` lists the names you have defined currently:: @@ -300,7 +306,7 @@ Without arguments, :func:`dir` lists the names you have defined currently:: >>> import fibo >>> fib = fibo.fib >>> dir() - ['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib', 'fibo', 'sys'] + ['__builtins__', '__name__', '__package__', 'a', 'fib', 'fibo', 'sys'] Note that it lists all types of names: variables, modules, functions, etc. @@ -311,10 +317,11 @@ want a list of those, they are defined in the standard module :mod:`__builtin__`:: >>> import __builtin__ - >>> dir(__builtin__) - ['ArithmeticError', 'AssertionError', 'AttributeError', 'DeprecationWarning', - 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', - 'FloatingPointError', 'FutureWarning', 'IOError', 'ImportError', + >>> dir(__builtin__) # doctest: +NORMALIZE_WHITESPACE + ['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', + 'BufferError', 'BytesWarning', 'DeprecationWarning', 'EOFError', + 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FloatingPointError', + 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError', 'None', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', @@ -323,18 +330,19 @@ want a list of those, they are defined in the standard module 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', - 'UserWarning', 'ValueError', 'Warning', 'WindowsError', + 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '_', '__debug__', '__doc__', '__import__', - '__name__', 'abs', 'apply', 'basestring', 'bool', 'buffer', - 'callable', 'chr', 'classmethod', 'cmp', 'coerce', 'compile', - 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', - 'enumerate', 'eval', 'execfile', 'exit', 'file', 'filter', 'float', - 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', - 'id', 'input', 'int', 'intern', 'isinstance', 'issubclass', 'iter', - 'len', 'license', 'list', 'locals', 'long', 'map', 'max', 'memoryview', - 'min', 'object', 'oct', 'open', 'ord', 'pow', 'property', 'quit', 'range', - 'raw_input', 'reduce', 'reload', 'repr', 'reversed', 'round', 'set', - 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', + '__name__', '__package__', 'abs', 'all', 'any', 'apply', 'basestring', + 'bin', 'bool', 'buffer', 'bytearray', 'bytes', 'callable', 'chr', + 'classmethod', 'cmp', 'coerce', 'compile', 'complex', 'copyright', + 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', + 'execfile', 'exit', 'file', 'filter', 'float', 'format', 'frozenset', + 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', + 'int', 'intern', 'isinstance', 'issubclass', 'iter', 'len', 'license', + 'list', 'locals', 'long', 'map', 'max', 'memoryview', 'min', 'next', + 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', + 'range', 'raw_input', 'reduce', 'reload', 'repr', 'reversed', 'round', + 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'unichr', 'unicode', 'vars', 'xrange', 'zip'] @@ -360,7 +368,9 @@ There are also many different operations you might want to perform on sound data (such as mixing, adding echo, applying an equalizer function, creating an artificial stereo effect), so in addition you will be writing a never-ending stream of modules to perform these operations. Here's a possible structure for -your package (expressed in terms of a hierarchical filesystem):: +your package (expressed in terms of a hierarchical filesystem): + +.. code-block:: text sound/ Top-level package __init__.py Initialize the sound package @@ -457,7 +467,7 @@ list of module names that should be imported when ``from package import *`` is encountered. It is up to the package author to keep this list up-to-date when a new version of the package is released. Package authors may also decide not to support it, if they don't see a use for importing \* from their package. For -example, the file :file:`sounds/effects/__init__.py` could contain the following +example, the file :file:`sound/effects/__init__.py` could contain the following code:: __all__ = ["echo", "surround", "reverse"] @@ -543,6 +553,6 @@ modules found in a package. .. rubric:: Footnotes .. [#] In fact function definitions are also 'statements' that are 'executed'; the - execution of a module-level function enters the function name in the module's - global symbol table. + execution of a module-level function definition enters the function name in + the module's global symbol table. diff --git a/Doc/tutorial/stdlib.rst b/Doc/tutorial/stdlib.rst index 844f8bc..f6239d6 100644 --- a/Doc/tutorial/stdlib.rst +++ b/Doc/tutorial/stdlib.rst @@ -145,7 +145,7 @@ Internet Access =============== There are a number of modules for accessing the internet and processing internet -protocols. Two of the simplest are :mod:`urllib2` for retrieving data from urls +protocols. Two of the simplest are :mod:`urllib2` for retrieving data from URLs and :mod:`smtplib` for sending mail:: >>> import urllib2 @@ -278,8 +278,10 @@ file:: def test_average(self): self.assertEqual(average([20, 30, 70]), 40.0) self.assertEqual(round(average([1, 5, 7]), 1), 4.3) - self.assertRaises(ZeroDivisionError, average, []) - self.assertRaises(TypeError, average, 20, 30, 70) + with self.assertRaises(ZeroDivisionError): + average([]) + with self.assertRaises(TypeError): + average(20, 30, 70) unittest.main() # Calling from the command line invokes all tests diff --git a/Doc/tutorial/stdlib2.rst b/Doc/tutorial/stdlib2.rst index 514d583..65fb093 100644 --- a/Doc/tutorial/stdlib2.rst +++ b/Doc/tutorial/stdlib2.rst @@ -71,9 +71,9 @@ formatting numbers with group separators:: Templating ========== -The :mod:`string` module includes a versatile :class:`Template` class with a -simplified syntax suitable for editing by end-users. This allows users to -customize their applications without having to alter the application. +The :mod:`string` module includes a versatile :class:`~string.Template` class +with a simplified syntax suitable for editing by end-users. This allows users +to customize their applications without having to alter the application. The format uses placeholder names formed by ``$`` with valid Python identifiers (alphanumeric characters and underscores). Surrounding the placeholder with @@ -85,17 +85,17 @@ spaces. Writing ``$$`` creates a single escaped ``$``:: >>> t.substitute(village='Nottingham', cause='the ditch fund') 'Nottinghamfolk send $10 to the ditch fund.' -The :meth:`substitute` method raises a :exc:`KeyError` when a placeholder is not -supplied in a dictionary or a keyword argument. For mail-merge style -applications, user supplied data may be incomplete and the -:meth:`safe_substitute` method may be more appropriate --- it will leave -placeholders unchanged if data is missing:: +The :meth:`~string.Template.substitute` method raises a :exc:`KeyError` when a +placeholder is not supplied in a dictionary or a keyword argument. For +mail-merge style applications, user supplied data may be incomplete and the +:meth:`~string.Template.safe_substitute` method may be more appropriate --- +it will leave placeholders unchanged if data is missing:: >>> t = Template('Return the $item to $owner.') >>> d = dict(item='unladen swallow') >>> t.substitute(d) Traceback (most recent call last): - . . . + ... KeyError: 'owner' >>> t.safe_substitute(d) 'Return the unladen swallow to $owner.' @@ -132,8 +132,9 @@ templates for XML files, plain text reports, and HTML web reports. Working with Binary Data Record Layouts ======================================= -The :mod:`struct` module provides :func:`pack` and :func:`unpack` functions for -working with variable length binary record formats. The following example shows +The :mod:`struct` module provides :func:`~struct.pack` and +:func:`~struct.unpack` functions for working with variable length binary +record formats. The following example shows how to loop through header information in a ZIP file without using the :mod:`zipfile` module. Pack codes ``"H"`` and ``"I"`` represent two and four byte unsigned numbers respectively. The ``"<"`` indicates that they are @@ -218,7 +219,9 @@ At its simplest, log messages are sent to a file or to ``sys.stderr``:: logging.error('Error occurred') logging.critical('Critical error -- shutting down') -This produces the following output:: +This produces the following output: + +.. code-block:: none WARNING:root:Warning:config file server.conf not found ERROR:root:Error occurred @@ -227,8 +230,9 @@ This produces the following output:: By default, informational and debugging messages are suppressed and the output is sent to standard error. Other output options include routing messages through email, datagrams, sockets, or to an HTTP Server. New filters can select -different routing based on message priority: :const:`DEBUG`, :const:`INFO`, -:const:`WARNING`, :const:`ERROR`, and :const:`CRITICAL`. +different routing based on message priority: :const:`~logging.DEBUG`, +:const:`~logging.INFO`, :const:`~logging.WARNING`, :const:`~logging.ERROR`, +and :const:`~logging.CRITICAL`. The logging system can be configured directly from Python or can be loaded from a user editable configuration file for customized logging without altering the @@ -255,9 +259,9 @@ applications include caching objects that are expensive to create:: >>> import weakref, gc >>> class A: ... def __init__(self, value): - ... self.value = value + ... self.value = value ... def __repr__(self): - ... return str(self.value) + ... return str(self.value) ... >>> a = A(10) # create a reference >>> d = weakref.WeakValueDictionary() @@ -285,11 +289,11 @@ Many data structure needs can be met with the built-in list type. However, sometimes there is a need for alternative implementations with different performance trade-offs. -The :mod:`array` module provides an :class:`array()` object that is like a list -that stores only homogeneous data and stores it more compactly. The following -example shows an array of numbers stored as two byte unsigned binary numbers -(typecode ``"H"``) rather than the usual 16 bytes per entry for regular lists of -Python int objects:: +The :mod:`array` module provides an :class:`~array.array()` object that is like +a list that stores only homogeneous data and stores it more compactly. The +following example shows an array of numbers stored as two byte unsigned binary +numbers (typecode ``"H"``) rather than the usual 16 bytes per entry for regular +lists of Python int objects:: >>> from array import array >>> a = array('H', [4000, 10, 700, 22222]) @@ -298,10 +302,10 @@ Python int objects:: >>> a[1:3] array('H', [10, 700]) -The :mod:`collections` module provides a :class:`deque()` object that is like a -list with faster appends and pops from the left side but slower lookups in the -middle. These objects are well suited for implementing queues and breadth first -tree searches:: +The :mod:`collections` module provides a :class:`~collections.deque()` object +that is like a list with faster appends and pops from the left side but slower +lookups in the middle. These objects are well suited for implementing queues +and breadth first tree searches:: >>> from collections import deque >>> d = deque(["task1", "task2", "task3"]) @@ -309,6 +313,8 @@ tree searches:: >>> print "Handling", d.popleft() Handling task1 +:: + unsearched = deque([starting_node]) def breadth_first_search(unsearched): node = unsearched.popleft() @@ -345,8 +351,8 @@ not want to run a full list sort:: Decimal Floating Point Arithmetic ================================= -The :mod:`decimal` module offers a :class:`Decimal` datatype for decimal -floating point arithmetic. Compared to the built-in :class:`float` +The :mod:`decimal` module offers a :class:`~decimal.Decimal` datatype for +decimal floating point arithmetic. Compared to the built-in :class:`float` implementation of binary floating point, the class is especially helpful for * financial applications and other uses which require exact decimal @@ -370,13 +376,15 @@ becomes significant if the results are rounded to the nearest cent:: >>> round(.70 * 1.05, 2) # same calculation with floats 0.73 -The :class:`Decimal` result keeps a trailing zero, automatically inferring four -place significance from multiplicands with two place significance. Decimal -reproduces mathematics as done by hand and avoids issues that can arise when -binary floating point cannot exactly represent decimal quantities. +The :class:`~decimal.Decimal` result keeps a trailing zero, automatically +inferring four place significance from multiplicands with two place +significance. Decimal reproduces mathematics as done by hand and avoids +issues that can arise when binary floating point cannot exactly represent +decimal quantities. -Exact representation enables the :class:`Decimal` class to perform modulo -calculations and equality tests that are unsuitable for binary floating point:: +Exact representation enables the :class:`~decimal.Decimal` class to perform +modulo calculations and equality tests that are unsuitable for binary floating +point:: >>> Decimal('1.00') % Decimal('.10') Decimal('0.00') diff --git a/Doc/tutorial/whatnow.rst b/Doc/tutorial/whatnow.rst index 157cc9f..48ff5c8 100644 --- a/Doc/tutorial/whatnow.rst +++ b/Doc/tutorial/whatnow.rst @@ -54,9 +54,8 @@ python-list@python.org. The newsgroup and mailing list are gatewayed, so messages posted to one will automatically be forwarded to the other. There are around 120 postings a day (with peaks up to several hundred), asking (and answering) questions, suggesting new features, and announcing new modules. -Before posting, be sure to check the list of `Frequently Asked Questions -<http://www.python.org/doc/faq/>`_ (also called the FAQ), or look for it in the -:file:`Misc/` directory of the Python source distribution. Mailing list +Before posting, be sure to check the list of :ref:`Frequently Asked Questions +<faq-index>` (also called the FAQ). Mailing list archives are available at http://mail.python.org/pipermail/. The FAQ answers many of the questions that come up again and again, and may already contain the solution for your problem. diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 4b2793f..842b266 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -397,18 +397,9 @@ Miscellaneous options .. cmdoption:: -3 - Warn about Python 3.x incompatibilities which cannot be fixed trivially by - :ref:`2to3 <2to3-reference>`. Among these are: - - * :meth:`dict.has_key` - * :func:`apply` - * :func:`callable` - * :func:`coerce` - * :func:`execfile` - * :func:`reduce` - * :func:`reload` - - Using these will emit a :exc:`DeprecationWarning`. + Warn about Python 3.x possible incompatibilities by emitting a + :exc:`DeprecationWarning` for features that are removed or significantly + changed in Python 3. .. versionadded:: 2.6 @@ -442,7 +433,10 @@ Options you shouldn't use Environment variables --------------------- -These environment variables influence Python's behavior. +These environment variables influence Python's behavior, they are processed +before the command-line switches other than -E. It is customary that +command-line switches override environmental variables where there is a +conflict. .. envvar:: PYTHONHOME @@ -541,7 +535,8 @@ These environment variables influence Python's behavior. .. envvar:: PYTHONDONTWRITEBYTECODE If this is set, Python won't try to write ``.pyc`` or ``.pyo`` files on the - import of source modules. + import of source modules. This is equivalent to specifying the :option:`-B` + option. .. versionadded:: 2.6 @@ -637,4 +632,3 @@ if Python was configured with the ``--with-pydebug`` build option. If set, Python will print memory allocation statistics every time a new object arena is created, and on shutdown. - diff --git a/Doc/using/mac.rst b/Doc/using/mac.rst index c5f7f2d..5f29812 100644 --- a/Doc/using/mac.rst +++ b/Doc/using/mac.rst @@ -25,14 +25,14 @@ installers for the latest 2.3 release for Mac OS 9 and related documentation. Getting and Installing MacPython ================================ -Mac OS X 10.5 comes with Python 2.5.1 pre-installed by Apple. If you wish, you +Mac OS X 10.8 comes with Python 2.7 pre-installed by Apple. If you wish, you are invited to install the most recent version of Python from the Python website (http://www.python.org). A current "universal binary" build of Python, which runs natively on the Mac's new Intel and legacy PPC CPU's, is available there. What you get after installing is a number of things: -* A :file:`MacPython 2.5` folder in your :file:`Applications` folder. In here +* A :file:`MacPython 2.7` folder in your :file:`Applications` folder. In here you find IDLE, the development environment that is a standard part of official Python distributions; PythonLauncher, which handles double-clicking Python scripts from the Finder; and the "Build Applet" tool, which allows you to @@ -100,7 +100,7 @@ aware of: programs that talk to the Aqua window manager (in other words, anything that has a GUI) need to be run in a special way. Use :program:`pythonw` instead of :program:`python` to start such scripts. -With Python 2.5, you can use either :program:`python` or :program:`pythonw`. +With Python 2.7, you can use either :program:`python` or :program:`pythonw`. Configuration @@ -133,13 +133,11 @@ Installing Additional Python Packages There are several methods to install additional Python packages: -* http://pythonmac.org/packages/ contains selected compiled packages for Python - 2.5, 2.4, and 2.3. - * Packages can be installed via the standard Python distutils mode (``python setup.py install``). -* Many packages can also be installed via the :program:`setuptools` extension. +* Many packages can also be installed via the :program:`setuptools` extension + or :program:`pip` wrapper, see http://www.pip-installer.org/. GUI Programming on the Mac @@ -167,7 +165,7 @@ http://www.riverbankcomputing.co.uk/software/pyqt/intro. Distributing Python Applications on the Mac =========================================== -The "Build Applet" tool that is placed in the MacPython 2.5 folder is fine for +The "Build Applet" tool that is placed in the MacPython 2.7 folder is fine for packaging small Python scripts on your own machine to run as a standard Mac application. This tool, however, is not robust enough to distribute Python applications to other users. @@ -177,20 +175,6 @@ The standard tool for deploying standalone Python applications on the Mac is at http://undefined.org/python/#py2app. -Application Scripting -===================== - -Python can also be used to script other Mac applications via Apple's Open -Scripting Architecture (OSA); see http://appscript.sourceforge.net. Appscript is -a high-level, user-friendly Apple event bridge that allows you to control -scriptable Mac OS X applications using ordinary Python scripts. Appscript makes -Python a serious alternative to Apple's own *AppleScript* language for -automating your Mac. A related package, *PyOSA*, is an OSA language component -for the Python scripting language, allowing Python code to be executed by any -OSA-enabled application (Script Editor, Mail, iTunes, etc.). PyOSA makes Python -a full peer to AppleScript. - - Other Resources =============== diff --git a/Doc/using/unix.rst b/Doc/using/unix.rst index 1539254..ec5ac6c 100644 --- a/Doc/using/unix.rst +++ b/Doc/using/unix.rst @@ -28,7 +28,7 @@ following links: http://www.debian.org/doc/manuals/maint-guide/first.en.html for Debian users - http://linuxmafia.com/pub/linux/suse-linux-internals/chapter35.html + http://en.opensuse.org/Portal:Packaging for OpenSuse users http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-creating-rpms.html for Fedora users @@ -145,7 +145,7 @@ information on how to code in Python in these editors, look at: * http://sourceforge.net/projects/python-mode Geany is an excellent IDE with support for a lot of languages. For more -information, read: http://geany.uvena.de/ +information, read: http://www.geany.org/ Komodo edit is another extremely good IDE. It also has support for a lot of languages. For more information, read: diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 978440c..b49ba7b 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -84,6 +84,8 @@ In order to run Python flawlessly, you might have to change certain environment settings in Windows. +.. _setting-envvars: + Excursus: Setting environment variables --------------------------------------- diff --git a/Doc/whatsnew/2.2.rst b/Doc/whatsnew/2.2.rst index 412c1d0..5f6fe11 100644 --- a/Doc/whatsnew/2.2.rst +++ b/Doc/whatsnew/2.2.rst @@ -450,9 +450,9 @@ signal that the iterator is done. Python classes can define an :meth:`__iter__` method, which should create and return a new iterator for the object; if the object is its own iterator, this method can just return ``self``. In particular, iterators will usually be their -own iterators. Extension types implemented in C can implement a :attr:`tp_iter` +own iterators. Extension types implemented in C can implement a :c:member:`~PyTypeObject.tp_iter` function in order to return an iterator, and extension types that want to behave -as iterators can define a :attr:`tp_iternext` function. +as iterators can define a :c:member:`~PyTypeObject.tp_iternext` function. So, after all this, what do iterators actually do? They have one required method, :meth:`next`, which takes no arguments and returns the next value. When @@ -478,7 +478,7 @@ there are no more values to be returned, calling :meth:`next` should raise the In 2.2, Python's :keyword:`for` statement no longer expects a sequence; it expects something for which :func:`iter` will return an iterator. For backward compatibility and convenience, an iterator is automatically constructed for -sequences that don't implement :meth:`__iter__` or a :attr:`tp_iter` slot, so +sequences that don't implement :meth:`__iter__` or a :c:member:`~PyTypeObject.tp_iter` slot, so ``for i in [1,2,3]`` will still work. Wherever the Python interpreter loops over a sequence, it's been changed to use the iterator protocol. This means you can do things like this:: diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst index b034eb2..85fea68 100644 --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -366,6 +366,9 @@ Under MacOS, :func:`os.listdir` may now return Unicode filenames. .. ====================================================================== +.. index:: + single: universal newlines; What's new + PEP 278: Universal Newline Support ================================== @@ -376,12 +379,12 @@ mark the ends of lines in text files. Unix uses the linefeed (ASCII character 10), MacOS uses the carriage return (ASCII character 13), and Windows uses a two-character sequence of a carriage return plus a newline. -Python's file objects can now support end of line conventions other than the one -followed by the platform on which Python is running. Opening a file with the -mode ``'U'`` or ``'rU'`` will open a file for reading in universal newline mode. -All three line ending conventions will be translated to a ``'\n'`` in the -strings returned by the various file methods such as :meth:`read` and -:meth:`readline`. +Python's file objects can now support end of line conventions other than the +one followed by the platform on which Python is running. Opening a file with +the mode ``'U'`` or ``'rU'`` will open a file for reading in :term:`universal +newlines` mode. All three line ending conventions will be translated to a +``'\n'`` in the strings returned by the various file methods such as +:meth:`read` and :meth:`readline`. Universal newline support is also used when importing modules and when executing a file with the :func:`execfile` function. This means that Python modules can diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst index 97a0293..fa6c124 100644 --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -411,6 +411,9 @@ error streams will be. You can provide a file object or a file descriptor, or you can use the constant ``subprocess.PIPE`` to create a pipe between the subprocess and the parent. +.. index:: + single: universal newlines; What's new + The constructor has a number of handy options: * *close_fds* requests that all file descriptors be closed before running the @@ -424,7 +427,7 @@ The constructor has a number of handy options: * *preexec_fn* is a function that gets called before the child is started. * *universal_newlines* opens the child's input and output using Python's - universal newline feature. + :term:`universal newlines` feature. Once you've created the :class:`Popen` instance, you can call its :meth:`wait` method to pause until the subprocess has exited, :meth:`poll` to check if it's @@ -843,7 +846,7 @@ Here are all of the changes that Python 2.4 makes to the core Python language. ['A', 'b', 'c', 'D'] Finally, the *reverse* parameter takes a Boolean value. If the value is true, - the list will be sorted into reverse order. Instead of ``L.sort() ; + the list will be sorted into reverse order. Instead of ``L.sort(); L.reverse()``, you can now write ``L.sort(reverse=True)``. The results of sorting are now guaranteed to be stable. This means that two diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index ff599c8..c420a19 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -286,7 +286,7 @@ Python's standard :mod:`string` module? There's no clean way to ignore :mod:`pkg.string` and look for the standard module; generally you had to look at the contents of ``sys.modules``, which is slightly unclean. Holger Krekel's :mod:`py.std` package provides a tidier way to perform imports from the standard -library, ``import py ; py.std.string.join()``, but that package isn't available +library, ``import py; py.std.string.join()``, but that package isn't available on all Python installations. Reading code which relies on relative imports is also less clear, because a @@ -1338,13 +1338,17 @@ complete list of changes, or look through the SVN logs for all the details. .. XXX need to provide some more detail here + .. index:: + single: universal newlines; What's new + * The :mod:`fileinput` module was made more flexible. Unicode filenames are now supported, and a *mode* parameter that defaults to ``"r"`` was added to the - :func:`input` function to allow opening files in binary or universal-newline - mode. Another new parameter, *openhook*, lets you use a function other than - :func:`open` to open the input files. Once you're iterating over the set of - files, the :class:`FileInput` object's new :meth:`fileno` returns the file - descriptor for the currently opened file. (Contributed by Georg Brandl.) + :func:`input` function to allow opening files in binary or :term:`universal + newlines` mode. Another new parameter, *openhook*, lets you use a function + other than :func:`open` to open the input files. Once you're iterating over + the set of files, the :class:`FileInput` object's new :meth:`fileno` returns + the file descriptor for the currently opened file. (Contributed by Georg + Brandl.) * In the :mod:`gc` module, the new :func:`get_count` function returns a 3-tuple containing the current collection counts for the three GC generations. This is diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index 796963a..cefdcaf 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -5,8 +5,6 @@ .. XXX add trademark info for Apple, Microsoft, SourceForge. :Author: A.M. Kuchling (amk at amk.ca) -:Release: |release| -:Date: |today| .. $Id$ Rules for maintenance: @@ -1067,9 +1065,12 @@ the :mod:`io` module: The :class:`BytesIO` class supports reading, writing, and seeking over an in-memory buffer. + .. index:: + single: universal newlines; What's new + * :class:`TextIOBase`: Provides functions for reading and writing strings (remember, strings will be Unicode in Python 3.0), - and supporting universal newlines. :class:`TextIOBase` defines + and supporting :term:`universal newlines`. :class:`TextIOBase` defines the :meth:`readline` method and supports iteration upon objects. @@ -1886,7 +1887,7 @@ changes, or look through the Subversion logs for all the details. >>> dq=deque(maxlen=3) >>> dq deque([], maxlen=3) - >>> dq.append(1) ; dq.append(2) ; dq.append(3) + >>> dq.append(1); dq.append(2); dq.append(3) >>> dq deque([1, 2, 3], maxlen=3) >>> dq.append(4) @@ -2778,12 +2779,12 @@ http://www.json.org. types. The following example encodes and decodes a dictionary:: >>> import json - >>> data = {"spam" : "foo", "parrot" : 42} + >>> data = {"spam": "foo", "parrot": 42} >>> in_json = json.dumps(data) # Encode the data >>> in_json '{"parrot": 42, "spam": "foo"}' >>> json.loads(in_json) # Decode into a Python object - {"spam" : "foo", "parrot" : 42} + {"spam": "foo", "parrot": 42} It's also possible to write your own decoders and encoders to support more types. Pretty-printing of the JSON strings is also supported. diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index 560e2f7..249f6d6 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -3,8 +3,6 @@ **************************** :Author: A.M. Kuchling (amk at amk.ca) -:Release: |release| -:Date: |today| .. hyperlink all the methods & functions. @@ -81,45 +79,91 @@ bug/patch item for each change. The Future for Python 2.x ========================= -Python 2.7 is intended to be the last major release in the 2.x series. -The Python maintainers are planning to focus their future efforts on -the Python 3.x series. - -This means that 2.7 will remain in place for a long time, running -production systems that have not been ported to Python 3.x. -Two consequences of the long-term significance of 2.7 are: - -* It's very likely the 2.7 release will have a longer period of - maintenance compared to earlier 2.x versions. Python 2.7 will - continue to be maintained while the transition to 3.x continues, and - the developers are planning to support Python 2.7 with bug-fix - releases beyond the typical two years. - -* A policy decision was made to silence warnings only of interest to - developers. :exc:`DeprecationWarning` and its - descendants are now ignored unless otherwise requested, preventing - users from seeing warnings triggered by an application. This change - was also made in the branch that will become Python 3.2. (Discussed - on stdlib-sig and carried out in :issue:`7319`.) - - In previous releases, :exc:`DeprecationWarning` messages were - enabled by default, providing Python developers with a clear - indication of where their code may break in a future major version - of Python. - - However, there are increasingly many users of Python-based - applications who are not directly involved in the development of - those applications. :exc:`DeprecationWarning` messages are - irrelevant to such users, making them worry about an application - that's actually working correctly and burdening application developers - with responding to these concerns. - - You can re-enable display of :exc:`DeprecationWarning` messages by - running Python with the :option:`-Wdefault <-W>` (short form: - :option:`-Wd <-W>`) switch, or by setting the :envvar:`PYTHONWARNINGS` - environment variable to ``"default"`` (or ``"d"``) before running - Python. Python code can also re-enable them - by calling ``warnings.simplefilter('default')``. +Python 2.7 is the last major release in the 2.x series, as the Python +maintainers have shifted the focus of their new feature development efforts +to the Python 3.x series. This means that while Python 2 continues to +receive bug fixes, and to be updated to build correctly on new hardware and +versions of supported operated systems, there will be no new full feature +releases for the language or standard library. + +However, while there is a large common subset between Python 2.7 and Python +3, and many of the changes involved in migrating to that common subset, or +directly to Python 3, can be safely automated, some other changes (notably +those associated with Unicode handling) may require careful consideration, +and preferably robust automated regression test suites, to migrate +effectively. + +This means that Python 2.7 will remain in place for a long time, providing a +stable and supported base platform for production systems that have not yet +been ported to Python 3. The full expected lifecycle of the Python 2.7 +series is detailed in :pep:`373`. + +Some key consequences of the long-term significance of 2.7 are: + +* As noted above, the 2.7 release has a much longer period of maintenance + when compared to earlier 2.x versions. Python 2.7 is currently expected to + remain supported by the core development team (receiving security updates + and other bug fixes) until at least 2020 (10 years after its initial + release, compared to the more typical support period of 18-24 months). + +* As the Python 2.7 standard library ages, making effective use of the + Python Package Index (either directly or via a redistributor) becomes + more important for Python 2 users. In addition to a wide variety of third + party packages for various tasks, the available packages include backports + of new modules and features from the Python 3 standard library that are + compatible with Python 2, as well as various tools and libraries that can + make it easier to migrate to Python 3. The `Python Packaging User Guide + <https://packaging.python.org>`__ provides guidance on downloading and + installing software from the Python Package Index. + +* While the preferred approach to enhancing Python 2 is now the publication + of new packages on the Python Package Index, this approach doesn't + necessarily work in all cases, especially those related to network + security. In exceptional cases that cannot be handled adequately by + publishing new or updated packages on PyPI, the Python Enhancement + Proposal process may be used to make the case for adding new features + directly to the Python 2 standard library. Any such additions, and the + maintenance releases where they were added, will be noted in the + :ref:`py27-maintenance-enhancements` section below. + +For projects wishing to migrate from Python 2 to Python 3, or for library +and framework developers wishing to support users on both Python 2 and +Python 3, there are a variety of tools and guides available to help decide +on a suitable approach and manage some of the technical details involved. +The recommended starting point is the :ref:`pyporting-howto` HOWTO guide. + + +Changes to the Handling of Deprecation Warnings +=============================================== + +For Python 2.7, a policy decision was made to silence warnings only of +interest to developers by default. :exc:`DeprecationWarning` and its +descendants are now ignored unless otherwise requested, preventing +users from seeing warnings triggered by an application. This change +was also made in the branch that became Python 3.2. (Discussed +on stdlib-sig and carried out in :issue:`7319`.) + +In previous releases, :exc:`DeprecationWarning` messages were +enabled by default, providing Python developers with a clear +indication of where their code may break in a future major version +of Python. + +However, there are increasingly many users of Python-based +applications who are not directly involved in the development of +those applications. :exc:`DeprecationWarning` messages are +irrelevant to such users, making them worry about an application +that's actually working correctly and burdening application developers +with responding to these concerns. + +You can re-enable display of :exc:`DeprecationWarning` messages by +running Python with the :option:`-Wdefault <-W>` (short form: +:option:`-Wd <-W>`) switch, or by setting the :envvar:`PYTHONWARNINGS` +environment variable to ``"default"`` (or ``"d"``) before running +Python. Python code can also re-enable them +by calling ``warnings.simplefilter('default')``. + +The ``unittest`` module also automatically reenables deprecation warnings +when running tests. Python 3.1 Features @@ -2466,6 +2510,54 @@ For applications that embed Python: .. ====================================================================== +.. _py27-maintenance-enhancements: + +New Features Added to Python 2.7 Maintenance Releases +===================================================== + +New features may be added to Python 2.7 maintenance releases when the +situation genuinely calls for it. Any such additions must go through +the Python Enhancement Proposal process, and make a compelling case for why +they can't be adequately addressed by either adding the new feature solely to +Python 3, or else by publishing it on the Python Package Index. + +In addition to the specific proposals listed below, there is a general +exemption allowing new ``-3`` warnings to be added in any Python 2.7 +maintenance release. + + +PEP 434: IDLE Enhancement Exception for All Branches +---------------------------------------------------- + +:pep:`434` describes a general exemption for changes made to the IDLE +development environment shipped along with Python. This exemption makes it +possible for the IDLE developers to provide a more consistent user +experience across all supported versions of Python 2 and 3. + +For details of any IDLE changes, refer to the NEWS file for the specific +release. + + +PEP 466: Network Security Enhancements for Python 2.7 +----------------------------------------------------- + +:pep:`466` describes a number of network security enhancement proposals +that have been approved for inclusion in Python 2.7 maintenance releases, +with the first of those changes appearing in the Python 2.7.7 release. + +:pep:`466` related features added in Python 2.7.7: + +* :func:`hmac.compare_digest` was added to make a timing attack resistant + comparison operation broadly available to Python 2 applications + (backported by Alex Gaynor in :issue:`21306`) + +* the version of OpenSSL linked with the prebuilt Windows installers + published on python.org was updated to 1.0.1g (contributed by + Zachary Ware in :issue:`21462`) + + +.. ====================================================================== + .. _acks27: Acknowledgements |
