Skip to content

Commit

Permalink
Further tweaks to documentation and release notes in preparation for …
Browse files Browse the repository at this point in the history
…release

of 8.0.1.
  • Loading branch information
anthony-tuininga committed Sep 1, 2020
1 parent 184f4d1 commit aecf006
Show file tree
Hide file tree
Showing 2 changed files with 86 additions and 79 deletions.
14 changes: 8 additions & 6 deletions doc/src/release_notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,18 @@
cx_Oracle Release Notes
=======================

Version 8.0.1 (TBD)
-------------------
Version 8.0.1 (August 2020)
---------------------------

#) Updated embedded ODPI-C to `version 4.0.2
<https://oracle.github.io/odpi/doc/releasenotes.html#
version-4-0-2-TBD>`__. This includes the fix for
version-4-0-2-august-31-2020>`__. This includes the fix for binding and
fetching numbers with 39 or 40 decimal digits
(`issue 459 <https://github.com/oracle/python-cx_Oracle/issues/459>`__).
#) Added metadata (and an exception) specifying that Python 3.5 and higher is
required in order to allow pip (and the exception message) to direct those
using Python 2 to use version 7.3 instead.
#) Added build metadata specifying that Python 3.5 and higher is required in
order to avoid downloading and failing to install with Python 2. The
exception message when running ``setup.py`` directly was updated to inform
those using Python 2 to use version 7.3 instead.
#) Documentation improvements.


Expand Down
151 changes: 78 additions & 73 deletions doc/src/user_guide/tuning.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,93 +91,74 @@ already buffered in the Oracle Client libraries. Reducing round-trips helps
performance and scalability. An overhead of prefetching is the need for an
additional data copy from Oracle Client's prefetch buffers.

To tune queries that return an unknown number of rows, estimate the number of
rows returned and start with an appropriate :attr:`Cursor.arraysize` value. The
default is 100. Then set :attr:`Cursor.prefetchrows` to the ``arraysize``
value. Do not make the sizes unnecessarily large. Keep ``arraysize`` as big,
or bigger than, ``prefetchrows``. Adjust the values as needed for performance,
memory and round-trip usage. An example is:
Choosing values for ``arraysize`` and ``prefetchrows``
++++++++++++++++++++++++++++++++++++++++++++++++++++++

.. code-block:: python
The best :attr:`Cursor.arraysize` and :attr:`Cursor.prefetchrows` values can be
found by experimenting with your application under the expected load of normal
application use. This is because the cost of the extra memory copy from the
prefetch buffers when fetching a large quantity of rows or very "wide" rows may
outweigh the cost of a round-trip for a single cx_Oracle user on a fast network.
However under production application load, the reduction of round-trips may help
performance and overall system scalability. The documentation in
:ref:`round-trips <roundtrips>` shows how to measure round-trips.

cur = connection.cursor()
Here are some suggestions for the starting point to begin your tuning:

cur.prefetchrows = 1000
cur.arraysize = 1000
* To tune queries that return an unknown number of rows, estimate the number of
rows returned and start with an appropriate :attr:`Cursor.arraysize` value.
The default is 100. Then set :attr:`Cursor.prefetchrows` to the ``arraysize``
value. Do not make the sizes unnecessarily large. For example:

for row in cur.execute("SELECT * FROM very_big_table"):
print(row)
.. code-block:: python
For a large quantity of rows or very "wide" rows on fast networks you may prefer
to leave ``prefetchrows`` at its default value of 2. The documentation in
:ref:`roundtrips` shows how to measure round-trips.
cur = connection.cursor()
If you are fetching a fixed number of rows, start your tuning by setting
``arraysize`` to the number of expected rows, and set ``prefetchrows`` to one
greater than this value. (Adding one removes the need for a round-trip to check
for end-of-fetch). For example, if you are querying 20 rows, perhaps to
:ref:`display a page <rowlimit>` of data, set ``prefetchrows`` to 21 and
``arraysize`` to 20:
cur.prefetchrows = 1000
cur.arraysize = 1000
.. code-block:: python
for row in cur.execute("SELECT * FROM very_big_table"):
print(row)
cur = connection.cursor()
Adjust the values as needed for performance, memory and round-trip usage. For
a large quantity of rows or very "wide" rows on fast networks you may prefer
to leave ``prefetchrows`` at its default value of 2. Keep ``arraysize`` as
big, or bigger than, ``prefetchrows``.

cur.prefetchrows = 21
cur.arraysize = 20
* If you are fetching a fixed number of rows, start your tuning by setting
``arraysize`` to the number of expected rows, and set ``prefetchrows`` to one
greater than this value. (Adding one removes the need for a round-trip to check
for end-of-fetch). For example, if you are querying 20 rows, perhaps to
:ref:`display a page <rowlimit>` of data, set ``prefetchrows`` to 21 and
``arraysize`` to 20:

for row in cur.execute("""
SELECT last_name
FROM employees
ORDER BY last_name
OFFSET 0 ROWS FETCH NEXT 20 ROWS ONLY"""):
print(row)
.. code-block:: python
This will return all rows for the query in one round-trip.
cur = connection.cursor()
If you know that a query returns just one row then set :attr:`Cursor.arraysize`
to 1 to minimize memory usage. The default prefetch value of 2 allows minimal
round-trips for single-row queries:
cur.prefetchrows = 21
cur.arraysize = 20
.. code-block:: python
for row in cur.execute("""
SELECT last_name
FROM employees
ORDER BY last_name
OFFSET 0 ROWS FETCH NEXT 20 ROWS ONLY"""):
print(row)
cur = connection.cursor()
cur.arraysize = 1
cur.execute("select * from MyTable where id = 1"):
row = cur.fetchone()
print(row)
This will return all rows for the query in one round-trip.

The best :attr:`Cursor.arraysize` and :attr:`Cursor.prefetchrows` values can be
found by experimenting with your application under the expected load of normal
application use. This is because the cost of the extra memory copy from the
prefetch buffers when fetching a large quantity of rows or very "wide" rows may
outweigh the cost of a round-trip for a single cx_Oracle user on a fast network.
However under production application load, the reduction of round-trips may help
performance and overall system scalability.
* If you know that a query returns just one row then set :attr:`Cursor.arraysize`
to 1 to minimize memory usage. The default prefetch value of 2 allows minimal
round-trips for single-row queries:

Prefetching can also be enabled in an external :ref:`oraaccess.xml
<optclientfiles>` file, which may be useful for tuning an application when
modifying its code is not feasible. Setting the size in ``oraaccess.xml`` will
affect the whole application, so it should not be the first tuning choice.

One place where increasing ``arraysize`` is particularly useful is in copying
data from one database to another:
.. code-block:: python
.. code-block:: python
# setup cursors
sourceCursor = sourceConnection.cursor()
sourceCursor.arraysize = 1000
targetCursor = targetConnection.cursor()
# perform fetch and bulk insertion
sourceCursor.execute("select * from MyTable")
while True:
rows = sourceCursor.fetchmany()
if not rows:
break
targetCursor.executemany("insert into MyTable values (:1, :2)", rows)
targetConnection.commit()
cur = connection.cursor()
cur.arraysize = 1
cur.execute("select * from MyTable where id = 1"):
row = cur.fetchone()
print(row)
In cx_Oracle, the ``arraysize`` and ``prefetchrows`` values are only examined
when a statement is executed the first time. To change the values, create a new
Expand Down Expand Up @@ -207,6 +188,30 @@ to 0:
function before cx_Oracle can return them to the application. Setting
``prefetchrows`` to 0 helps give a consistent flow of data to the application.

Prefetching can also be enabled in an external :ref:`oraaccess.xml
<optclientfiles>` file, which may be useful for tuning an application when
modifying its code is not feasible. Setting the size in ``oraaccess.xml`` will
affect the whole application, so it should not be the first tuning choice.

One place where increasing ``arraysize`` is particularly useful is in copying
data from one database to another:

.. code-block:: python
# setup cursors
sourceCursor = sourceConnection.cursor()
sourceCursor.arraysize = 1000
targetCursor = targetConnection.cursor()
# perform fetch and bulk insertion
sourceCursor.execute("select * from MyTable")
while True:
rows = sourceCursor.fetchmany()
if not rows:
break
targetCursor.executemany("insert into MyTable values (:1, :2)", rows)
targetConnection.commit()
.. _roundtrips:

Database Round-trips
Expand Down Expand Up @@ -305,8 +310,8 @@ cache.

.. _clientresultcache:

Client Result Cache
===================
Client Result Caching
=====================

cx_Oracle applications can use Oracle Database's `Client Result Cache
<https://www.oracle.com/pls/topic/lookup?ctx=dblatest&id=GUID-35CB2592-7588-4C2D-9075-6F639F25425E>`__.
Expand All @@ -332,7 +337,7 @@ restarting the database, for example:
SQL> STARTUP FORCE
CRC can alternatively be configured in an :ref:`oraaccess.xml <optclientfiles>`
or :ref:`sqlnet.ora <optnetfiles>` file on the Node.js host, see `Client
or :ref:`sqlnet.ora <optnetfiles>` file on the Python host, see `Client
Configuration Parameters
<https://www.oracle.com/pls/topic/lookup?ctx=dblatest&id=GUID-E63D75A1-FCAA-4A54-A3D2-B068442CE766>`__.

Expand Down

0 comments on commit aecf006

Please sign in to comment.