Skip to content

Commit

Permalink
update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
newville committed Jan 19, 2025
1 parent b788c49 commit 958c47f
Show file tree
Hide file tree
Showing 22 changed files with 351 additions and 331 deletions.
2 changes: 1 addition & 1 deletion _sources/api.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ Interpreter methods and attributes

An Interpreter instance has many methods, but most of them are
implementation details for how to handle particular AST nodes, and should
not be considered as part of the usable API. The methods described be low,
not be considered as part of the usable API. The methods described below,
and the examples elsewhere in this documentation should be used as the
stable API.

Expand Down
13 changes: 7 additions & 6 deletions _sources/installation.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Requirements
~~~~~~~~~~~~~~~

Asteval is a pure Python module. The latest stable version is |release|, which
supports Python 3.8 through 3.12.
supports Python 3.9 through 3.13.

Installing `asteval` requires `setuptools` and `setuptools_scm`. No other
libraries outside of the standard library are required. If `numpy`_ and
Expand All @@ -20,11 +20,12 @@ Running the test suite requires the `pytest`, `coverage`, and `pytest-cov`
modules, deployment uses `build` and `twine`, and building the documentation
requires `sphinx`.

Python 3.8 through 3.12 are tested on Windows, MacOS, and Linux, with and
without `numpy`_ installed. Older Python versions have generally been
supported by `asteval` until they are well past the end of security fixes. That
is, while `asteval` is no longer tested with Python 3.7, the latest release may
continue to work with that version.
Python 3.9 through 3.13 are tested on Windows, MacOS, and Linux, with
and without `numpy`_ installed. Older Python versions have generally
been supported by `asteval` until they are well past the end of
security fixes. While `asteval` may continue to work with Python 3.8
or even 3.7, these are not supported.


Support for new versions of the Python 3 series is not guaranteed until some
time after the official release of that version, as we may not start testing
Expand Down
141 changes: 82 additions & 59 deletions _sources/motivation.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -52,18 +52,24 @@ approach the speed of `eval` and the `numexpr` modules.
How Safe is asteval?
=======================

Asteval avoids all of the exploits we know about that make :py:func:`eval`
dangerous. For reference, see, `Eval is really dangerous
<https://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html>`_ and the
comments and links therein. From this discussion it is apparent that not only
is :py:func:`eval` unsafe, but that it is a difficult prospect to make any
program that takes user input perfectly safe. In particular, if a user can
cause Python to crash with a segmentation fault, safety cannot be guaranteed.
Asteval explicitly forbids the exploits described in the above link, and works
hard to prevent malicious code from crashing Python or accessing the
underlying operating system. That said, we cannot guarantee that asteval is
completely safe from malicious code. We claim only that it is safer than the
builtin :py:func:`eval`, and that you might find it useful.
Asteval avoids all of the exploits we know about that make
:py:func:`eval` dangerous. For reference, see, `Eval is really
dangerous
<https://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html>`_
and the comments and links therein. From this discussion it is
apparent that not only is :py:func:`eval` unsafe, but that it is a
difficult prospect to make any program that takes user input perfectly
safe. In particular, if a user can cause Python to crash with a
segmentation fault, safety cannot be guaranteed. Asteval explicitly
forbids the exploits described in the above link, and works hard to
prevent malicious code from crashing Python or accessing the
underlying operating system. That said, we cannot guarantee that
asteval is completely safe from malicious code. We claim only that it
is safer than the builtin :py:func:`eval`, and that you might find it
useful. We also note that several other Python libraries that
evaluate user-supplied expressions, including `numexpr` and `sympy`
use the builtin :py:func:`eval` as part of their processing.


Some of the things not allowed in the asteval interpreter for safety reasons include:

Expand All @@ -84,19 +90,22 @@ Some of the things not allowed in the asteval interpreter for safety reasons inc
In addition (and following the discussion in the link above), the following
attributes are blacklisted for all objects, and cannot be accessed:

``func_globals``, ``func_code``, ``func_closure``,
``im_class``, ``im_func``, ``im_self``,
``gi_code``, ``gi_frame``, ``f_locals``
``func_globals``, ``func_code``, ``func_closure``, ``im_class``,
``im_func``, ``im_self``, ``gi_code``, ``gi_frame``, ``f_locals``,
``__mro__``, ``_mro``

While this approach of making a blacklist cannot be guaranteed to be complete,
it does eliminate entire classes of attacks known to be able to seg-fault the
Python interpreter.
[Note: this list may be incomplete - there may be other disallowed
attributes]. While this approach of making a blacklist cannot be
guaranteed to be complete, it does eliminate entire classes of attacks
known to be able to seg-fault the Python interpreter or give access to
the operating system.

An important caveat is that asteval will typically expose numpy ``ufuncs`` from the
numpy module. Several of these can seg-fault Python without too much trouble.
If you are paranoid about safe user input that can never cause a segmentation
fault, you may want to consider disabling the use of numpy, or take extra care
to specify what can be used.
An important caveat is that a typical use of asteval will import and
expose numpy ``ufuncs`` from the numpy module. Several of these can
seg-fault Python without too much trouble. If you safety from user
input causing segmentation fault is a primary concern, you may want to
consider disabling the use of numpy, or take extra care to specify
what numpy functions can be used.

In 2024, an independent security audit of asteval done by Andrew Effenhauser,
Ayman Hammad, and Daniel Crowley in the X-Force Security Research division of
Expand All @@ -108,35 +117,46 @@ needed, these modules can be added to any Interpreter either using the
``user_symbols`` argument when creating it, or adding the needed symbols to the
symbol table after the Interpreter is created.

There are important categories of safety that asteval may attempt to address,
but cannot guarantee success. The most important of these is resource hogging,
which might be used for a denial-of-service attack. There is no guaranteed
timeout on any calculation, and so a reasonable looking calculation such as::
In 2025, William Khem Marquez demonstrated two vulnerabilities: one
from leaving some AST objects exposed within the interpreter for
user-defined functions ("Procedures"), and one with f-string
formatting. Both of these were fixed for version 1.0.6.

There are other categories of safety that asteval may attempt to
address, but cannot guarantee success. The most important of these is
resource hogging, which might be used for a denial-of-service attack.
There is no guaranteed timeout on any calculation, and so a reasonable
looking calculation such as::

from asteval import Interpreter
aeval = Interpreter()
txt = """nmax = 1e8
txt = """
nmax = 1e8
a = sqrt(arange(nmax)) # using numpy.sqrt() and numpy.arange()
"""
aeval.eval(txt)

can take a noticeable amount of CPU time - if it does not, increasing that
value of ``nmax`` almost certainly will, and can even crash the Python shell.

As another example, consider the expression ``x**y**z``. For values
``x=y=z=5``, the run time will be well under 0.001 seconds. For ``x=y=z=8``,
run time will still be under 1 sec. Changing to ``x=8, y=9, z=9``, will cause
the statement to take several seconds. With ``x=y=z=9``, executing that
statement may take more than 1 hour on some machines. It is not hard to come
up with short program that would run for hundreds of years, which probably
exceeds anyones threshold for an acceptable run-time. There simply is not a
good way to predict how long any code will take to run from the text of the
code itself: run time cannot be determined lexically.

To be clear, for the ``x**y**z`` exponentiation example, asteval will raise a
runtime error, telling you that an exponent > 10,000 is not allowed. Several
other attempts are made to prevent long-running operations or memory
exhaustion. These checks will prevent:
can take a noticeable amount of CPU time - if it does not, increasing
that value of ``nmax`` almost certainly will, and can even crash the
Python shell.

As another example, and an illustration of the fundamental problem,
consider the Python expression ``a = x**y**z``. For values
``x=y=z=5``, the run time will be well under 0.001 seconds. For
``x=y=z=8``, run time will still be under 1 sec. Changing to ``x=8,
y=9, z=9``, Python will ake several seconds (the value is :math:`\sim
10^{350,000,000}`) With ``x=y=z=9``, executing that statement may take
more than 1 hour on some machines. It is not hard to come up with
short program that would run for hundreds of years, which probably
exceeds everyones threshold for an acceptable run-time. The point
here is tha there simply is not a good way to predict how long any
code will take to run from the text of the code itself: run time
cannot be determined lexically.

To be clear, for the ``x**y**z`` exponentiation example, asteval will
raise a runtime error, telling you that an exponent > 10,000 is not
allowed. Several other attempts are also made to prevent long-running
operations or memory exhaustion. These checks will prevent:

* statements longer than 50,000 bytes.
* values of exponents (``p`` in ``x**p``) > 10,000.
Expand All @@ -145,11 +165,23 @@ exhaustion. These checks will prevent:
* more than 262144 open buffers
* opening a file with a mode other than ``'r'``, ``'rb'``, or ``'ru'``.

These checks happen at runtime, not by analyzing the text of the code. As with
the example above using ``numpy.arange``, very large arrays and lists can be
created that might approach memory limits. There are countless other "clever
ways" to have very long run times that cannot be readily predicted from the
text.
These checks happen at runtime, not by analyzing the text of the code.
As with the example above using ``numpy.arange``, very large arrays
and lists can be created that might approach memory limits. There are
countless other "clever ways" to have very long run times that cannot
be readily predicted from the text of the code.

By default, the list of supported functions does include Python's
``open()`` -- in read-only mode -- which will allow disk access to the
untrusted user. If ``numpy`` is supported, its ``load()`` and
``loadtxt()`` functions will also normally be supported. By itself,
including these functions does not elevate permissions, and access is
restricted to 'read-only mode'. Still, the user of the asteval
interpreter would be able to read files with the privileges of the
calling program. In some cases, this may not be desirable, and you
may want to remove some of these functions from the symbol table,
re-implement them, or ensure that your program cannot access
information on disk that should be kept private.

The exponential example also highlights the issue that there is not a good way
to check for a long-running calculation within a single Python process. That
Expand Down Expand Up @@ -182,15 +214,6 @@ executing expressions, with a code like this::
with limited_recursion(100):
Interpreter().eval(...)

A secondary security concern is that the default list of supported functions
does include Python's ``open()`` which will allow disk access to the untrusted
user. If ``numpy`` is supported, its ``load()`` and ``loadtxt()`` functions will
also normally be supported. Including these functions does not elevate
permissions, but it does allow the user of the asteval interpreter to read
files with the privileges of the calling program. In some cases, this may not
be desirable, and you may want to remove some of these functions from the
symbol table, re-implement them, or ensure that your program cannot access
information on disk that should be kept private.

In summary, while asteval attempts to be safe and is definitely safer than
using :py:func:`eval`, there may be ways that using asteval could lead to
Expand Down
15 changes: 2 additions & 13 deletions _static/basic.css
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
/*
* basic.css
* ~~~~~~~~~
*
* Sphinx stylesheet -- basic theme.
*
* :copyright: Copyright 2007-2024 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/

/* -- main layout ----------------------------------------------------------- */
Expand Down Expand Up @@ -115,15 +108,11 @@ img {
/* -- search page ----------------------------------------------------------- */

ul.search {
margin: 10px 0 0 20px;
padding: 0;
margin-top: 10px;
}

ul.search li {
padding: 5px 0 5px 20px;
background-image: url(file.png);
background-repeat: no-repeat;
background-position: 0 7px;
padding: 5px 0;
}

ul.search li a {
Expand Down
7 changes: 0 additions & 7 deletions _static/bizstyle.css
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
/*
* bizstyle.css_t
* ~~~~~~~~~~~~~~
*
* Sphinx stylesheet -- business style theme.
*
* :copyright: Copyright 2007-2024 by Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/

@import url("basic.css");
Expand Down
16 changes: 5 additions & 11 deletions _static/bizstyle.js
Original file line number Diff line number Diff line change
@@ -1,14 +1,8 @@
//
// bizstyle.js
// ~~~~~~~~~~~
//
// Sphinx javascript -- for bizstyle theme.
//
// This theme was created by referring to 'sphinxdoc'
//
// :copyright: Copyright 2007-2024 by Sphinx team, see AUTHORS.
// :license: BSD, see LICENSE for details.
//
/*
* Sphinx javascript -- for bizstyle theme.
*
* This theme was created by referring to 'sphinxdoc'
*/
const initialiseBizStyle = () => {
if (navigator.userAgent.indexOf("iPhone") > 0 || navigator.userAgent.indexOf("Android") > 0) {
document.querySelector("li.nav-item-0 a").innerText = "Top"
Expand Down
7 changes: 0 additions & 7 deletions _static/doctools.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
/*
* doctools.js
* ~~~~~~~~~~~
*
* Base JavaScript utilities for all Sphinx HTML documentation.
*
* :copyright: Copyright 2007-2024 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/
"use strict";

Expand Down
2 changes: 1 addition & 1 deletion _static/documentation_options.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
const DOCUMENTATION_OPTIONS = {
VERSION: '1.0.5',
VERSION: '1.0.6',
LANGUAGE: 'en',
COLLAPSE_INDEX: false,
BUILDER: 'html',
Expand Down
7 changes: 0 additions & 7 deletions _static/language_data.js
Original file line number Diff line number Diff line change
@@ -1,13 +1,6 @@
/*
* language_data.js
* ~~~~~~~~~~~~~~~~
*
* This script contains the language-specific data used by searchtools.js,
* namely the list of stopwords, stemmer, scorer and splitter.
*
* :copyright: Copyright 2007-2024 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/

var stopwords = ["a", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "near", "no", "not", "of", "on", "or", "such", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", "with"];
Expand Down
Loading

0 comments on commit 958c47f

Please sign in to comment.