Merge pull request #1358 from napalm-automation/develop

Release 3.3.0

.github/dependabot.yml

@@ -0,0 +1,19 @@
+version: 2
+updates:
+- package-ecosystem: pip
+  directory: "/"
+  schedule:
+    interval: daily
+  open-pull-requests-limit: 10
+  ignore:
+  - dependency-name: pytest
+    versions:
+    - ">= 6.a, < 7"
+  - dependency-name: black
+    versions:
+    - 21.4b0
+    - 21.4b1
+  - dependency-name: tox
+    versions:
+    - 3.21.3
+    - 3.22.0

.github/workflows/commit.yaml

@@ -0,0 +1,67 @@
+--- 
+name: build
+on: [push, pull_request]
+jobs:
+  std_tests:
+    runs-on: ubuntu-latest
+    strategy:
+      max-parallel: 4
+      matrix:
+        python-version: [3.6, 3.7, 3.8, 3.9]
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v2
+
+      - name: Setup Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e .
+          pip install -r requirements.txt
+          pip install -r requirements-dev.txt
+
+      - name: Run black
+        run: |
+          black --check .
+
+      - name: Run linter
+        run: |
+          pylama .
+
+      - name: Run Tests
+        run: |
+          py.test --cov=napalm --cov-report term-missing -vs --pylama
+        
+  build_docs:
+    needs: std_tests
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        python-version: [3.8]
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v2
+
+      - name: Setup Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e .
+          pip install -r requirements.txt
+          pip install -r requirements-dev.txt
+          python -m pip install -r docs/requirements.txt
+
+      - name: Doctests
+        run: |
+          make doctest

README.md

@@ -1,6 +1,5 @@
 [![PyPI](https://img.shields.io/pypi/v/napalm.svg)](https://pypi.python.org/pypi/napalm)
 [![PyPI versions](https://img.shields.io/pypi/pyversions/napalm.svg)](https://pypi.python.org/pypi/napalm)
-[![Build Status](https://travis-ci.org/napalm-automation/napalm.svg?branch=master)](https://travis-ci.org/napalm-automation/napalm)
 [![Coverage Status](https://coveralls.io/repos/github/napalm-automation/napalm/badge.svg)](https://coveralls.io/github/napalm-automation/napalm)
 [![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
 

docs/conf.py

@@ -54,15 +54,15 @@
 master_doc = "index"
 
 # General information about the project.
-project = u"NAPALM"
-copyright = u"2020, David Barroso"
+project = "NAPALM"
+copyright = "2021, David Barroso/Mircea Ulinic/Kirk Byers"
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
-version = "0"
+version = "3"
 # The full version, including alpha/beta/rc tags.
 release = "3"
 
@@ -284,6 +284,8 @@
     "cli",
     "close",
     "commit_config",
+    "confirm_commit",
+    "has_pending_commit",
     "compare_config",
     "discard_config",
     "load_merge_candidate",

docs/development/triage.rst

@@ -26,7 +26,7 @@ Driver labels
 -------------
 
 Each platform supported by NAPALM has associated a label, e.g., ``junos``, ``eos``,
-``ios``, ``iosxr``, ``vyos``, etc. It is mandatory that the maintainer to apply
+``ios``, ``iosxr_netconf``, ``iosxr``, ``vyos``, etc. It is mandatory that the maintainer to apply
 one or more of these labels.
 
 .. _triage-api-change-label:

docs/index.rst

@@ -34,6 +34,8 @@ You can select the driver you need by doing the following:
    >>> from napalm import get_network_driver
    >>> get_network_driver('eos')
    <class napalm.eos.eos.EOSDriver at 0x10ebad6d0>
+   >>> get_network_driver('iosxr_netconf')
+   <class napalm.iosxr_netconf.iosxr_netconf.IOSXRNETCONFDriver at 0x10ad170f0>
    >>> get_network_driver('iosxr')
    <class napalm.iosxr.iosxr.IOSXRDriver at 0x10ec90050>
    >>> get_network_driver('junos')

docs/support/index.rst

@@ -8,21 +8,22 @@ General support matrix
 
 
 
-  =====================   ==========  =============   ============ ============  ============ ============
-  _                       EOS         Junos           IOS-XR       NX-OS         NX-OS SSH    IOS
-  =====================   ==========  =============   ============ ============  ============ ============
-  **Driver Name**         eos         junos           iosxr        nxos          nxos_ssh     ios
-  **Structured data**     Yes         Yes             No           Yes           No           No
-  **Minimum version**     4.15.0F     12.1            5.1.0        6.1 [#g1]_                 12.4(20)T
-  **Backend library**     `pyeapi`_   `junos-eznc`_   `pyIOSXR`_   `pynxos`_     `netmiko`_   `netmiko`_
-  **Caveats**             :doc:`eos`                               :doc:`nxos`   :doc:`nxos`  :doc:`ios`
-  =====================   ==========  =============   ============ ============  ============ ============
+  =====================   ==========  =============  ====================  ==================  ============  ============  ============
+  _                       EOS         Junos          IOS-XR (NETCONF)      IOS-XR (XML-Agent)  NX-OS         NX-OS SSH     IOS
+  =====================   ==========  =============  ====================  ==================  ============  ============  ============
+  **Driver Name**         eos         junos          iosxr_netconf         iosxr               nxos          nxos_ssh      ios
+  **Structured data**     Yes         Yes            Yes                   No                  Yes           No            No
+  **Minimum version**     4.15.0F     12.1           7.0                   5.1.0               6.1 [#g1]_    12.4(20)T     6.3.2
+  **Backend library**     `pyeapi`_   `junos-eznc`_  `ncclient`_           `pyIOSXR`_          `pynxos`_     `netmiko`_    `netmiko`_
+  **Caveats**             :doc:`eos`                 :doc:`iosxr_netconf`                      :doc:`nxos`   :doc:`nxos`   :doc:`ios`
+  =====================   ==========  =============  ====================  ==================  ============  ============  ============
 
 .. _pyeapi: https://github.com/arista-eosplus/pyeapi
 .. _junos-eznc: https://github.com/Juniper/py-junos-eznc
 .. _pyIOSXR: https://github.com/fooelisa/pyiosxr
 .. _pynxos: https://github.com/networktocode/pynxos
 .. _netmiko: https://github.com/ktbyers/netmiko
+.. _ncclient: https://github.com/ncclient/ncclient
 
 .. [#g1] NX-API support on the Nexus 5k, 6k and 7k families was introduced in version 7.2
 
@@ -32,20 +33,22 @@ General support matrix
 Configuration support matrix
 ----------------------------
 
-=====================   ==========  =====   ==========  ==============  ==============
-_                       EOS         Junos   IOS-XR      NX-OS           IOS
-=====================   ==========  =====   ==========  ==============  ==============
-**Config. replace**     Yes         Yes     Yes         Yes             Yes
-**Config. merge**       Yes         Yes     Yes         Yes             Yes
-**Compare config**      Yes         Yes     Yes [#c1]_  Yes [#c4]_      Yes
-**Atomic Changes**      Yes         Yes     Yes         Yes/No [#c5]_   Yes/No [#c5]_
-**Rollback**            Yes [#c2]_  Yes     Yes         Yes/No [#c5]_   Yes
-=====================   ==========  =====   ==========  ==============  ==============
+=====================   ==========  =====   ================  ==================  ==============  ==============
+_                       EOS         Junos   IOS-XR (NETCONF)  IOS-XR (XML-Agent)  NX-OS           IOS
+=====================   ==========  =====   ================  ==================  ==============  ==============
+**Config. replace**     Yes         Yes     Yes               Yes                 Yes             Yes
+**Config. merge**       Yes         Yes     Yes               Yes                 Yes             Yes
+**Commit Confirm**      Yes         Yes     No                No                  No              No
+**Compare config**      Yes         Yes     Yes               Yes [#c1]_          Yes [#c4]_      Yes
+**Atomic Changes**      Yes         Yes     Yes               Yes                 Yes/No [#c5]_   Yes/No [#c5]_
+**Rollback**            Yes [#c2]_  Yes     Yes               Yes                 Yes/No [#c5]_   Yes
+=====================   ==========  =====   ================  ==================  ==============  ==============
 
 .. [#c1] Hand-crafted by the API as the device doesn't support the feature.
 .. [#c2] Not supported but emulated. Check caveats.
 .. [#c4] For merges, the diff is very simplistic. See caveats.
 .. [#c5] No for merges. See caveats.
+.. [#c6] NAPALM requires Junos OS >= 14.1 for commit-confirm functionality.
 
 .. warning:: Before building a workflow to deploy configuration it is important you understand what the table above means;
             what are atomic changes and which devices support it, what does replacing or merging configuration mean, etc.
@@ -65,13 +68,13 @@ Other methods
 .. |yes|   unicode:: U+02705 .. Yes
 .. |no|    unicode:: U+0274C .. No
 
-============================== =====  =====   ======  ======  =====
-_                              EOS    Junos   IOS-XR  NX-OS   IOS
-============================== =====  =====   ======  ======  =====
-**load_template**              |yes|  |yes|   |yes|   |yes|   |yes|
-**ping**                       |yes|  |yes|   |no|    |yes|   |yes|
-**traceroute**                 |yes|  |yes|   |yes|   |yes|   |yes|
-============================== =====  =====   ======  ======  =====
+============================== =====  =====   ===================  ======  ======  =====
+_                              EOS    Junos   IOS-XR (NETCONF)     IOS-XR  NX-OS   IOS
+============================== =====  =====   ===================  ======  ======  =====
+**load_template**              |yes|  |yes|   |yes|                |yes|   |yes|   |yes|
+**ping**                       |yes|  |yes|   |no|                 |no|    |yes|   |yes|
+**traceroute**                 |yes|  |yes|   |yes|                |yes|   |yes|   |yes|
+============================== =====  =====   ===================  ======  ======  =====
 
 Available configuration templates
 ---------------------------------
@@ -92,6 +95,7 @@ Caveats
    eos
    ios
    nxos
+   iosxr_netconf
 
 Optional arguments
 ------------------
@@ -114,7 +118,7 @@ ____________________________________
 * :code:`alt_key_file` (ios, iosxr, nxos_ssh) - SSH host key file to use (if ``alt_host_keys`` is ``True``).
 * :code:`auto_probe` (junos) - A timeout in seconds, for which to probe the device. Probing determines if the device accepts remote connections. If `auto_probe` is set to ``0``, no probing will be done. (default: ``0``).
 * :code:`auto_rollback_on_error` (ios) - Disable automatic rollback (certain versions of IOS support configure replace, but not rollback on error) (default: ``True``).
-* :code:`config_lock` (iosxr, junos) - Lock the config during open() (default: ``False``).
+* :code:`config_lock` (iosxr_netconf, iosxr, junos) - Lock the config during open() (default: ``False``).
 * :code:`lock_disable` (junos) - Disable all configuration locking for management by an external system (default: ``False``).
 * :code:`config_private` (junos) - Configure private, no DB locking (default: ``False``).
 * :code:`canonical_int` (ios) - Convert operational interface's returned name to canonical name (fully expanded name) (default: ``False``).
@@ -124,21 +128,18 @@ ____________________________________
 * :code:`global_delay_factor` (ios, nxos_ssh) - Allow for additional delay in command execution (default: ``1``).
 * :code:`ignore_warning` (junos) - Allows to set `ignore_warning` when loading configuration to avoid exceptions via junos-pyez. (default: ``False``).
 * :code:`keepalive` (iosxr, junos) - SSH keepalive interval, in seconds (default: ``30`` seconds).
-* :code:`key_file` (ios, iosxr, junos, nxos_ssh) - Path to a private key file. (default: ``False``).
-* :code:`port` (eos, ios, iosxr, junos, nxos, nxos_ssh) - Allows you to specify a port other than the default.
+* :code:`key_file` (ios, iosxr_netconf, iosxr, junos, nxos_ssh) - Path to a private key file. (default: ``False``).
+* :code:`port` (eos, ios, iosxr_netconf, iosxr, junos, nxos, nxos_ssh) - Allows you to specify a port other than the default.
 * :code:`secret` (ios, nxos_ssh) - Password required to enter privileged exec (enable) (default: ``''``).
 * :code:`ssh_config_file` (ios, iosxr, junos, nxos_ssh) - File name of OpenSSH configuration file.
 * :code:`ssh_strict` (ios, iosxr, nxos_ssh) - Automatically reject unknown SSH host keys (default: ``False``, which means unknown SSH host keys will be accepted).
 * :code:`ssl_verify` (nxos) - Requests argument, enable the SSL certificates verification. See requests ssl-cert-verification for valide values (default: ``None`` equivalent to ``False``).
 * :code:`transport` (eos, ios, nxos) - Protocol to connect with (see `The transport argument`_ for more information).
 * :code:`use_keys` (ios, iosxr, nxos_ssh) - Paramiko argument, enable searching for discoverable private key files in ``~/.ssh/`` (default: ``False``).
 * :code:`eos_autoComplete` (eos) - Allows to set `autoComplete` when running commands. (default: ``None`` equivalent to ``False``)
-* :code:`eos_fn0039_config` (eos) - Transform old style configuration to the new 
-  style, available beginning with EOS release 4.23.0, as per FN 0039. Beware 
-  that enabling this option will change the configuration you're loading 
-  through NAPALM. Default: ``False`` (won't change your configuration 
-  commands).
-
+* :code:`config_encoding` (iosxr_netconf) - Set encoding to either ``"xml"`` or ``"cli"`` for configuration load methods. (default: ``"cli"``)
+* :code:`eos_fn0039_config` (eos) - Transform old style configuration to the new style, available beginning with EOS release 4.23.0, as per FN 0039. Beware 
+  that enabling this option will change the configuration you're loading through NAPALM. Default: ``False`` (won't change your configuration commands).
   .. versionadded:: 3.0.1
 
 The transport argument

docs/support/iosxr_netconf.rst

@@ -0,0 +1,22 @@
+IOS-XR (NETCONF)
+----------------
+
+
+Minimum IOS-XR OS Version
+~~~~~~~~~~~~~~~~~~~~~~~~~
+Only devices running IOS-XR 7.0 or later are supported by NAPALM and the IOS-XR NETCONF driver.
+
+
+Device management using XML Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Using ``iosxr_netconf`` and a ``config_encoding="xml"`` for NAPALM configuration operations is entirely experimental. There is a very good chance XML configurations will not work properly and that only small subsections of the configuration will be configurable using merge operations.
+
+
+Device management using CLI Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+All configuration methods (``load_merge_candidate``, ``load_replace_candidate``, ``get_config``, ``compare_config``) support configuration encoded in XML and CLI (unstructured) format. This can be specified by using the ``config_encoding`` optional_args argument and setting it to either ``cli`` or ``xml`` (``cli`` is the default value).
+
+
+Retrieving device environment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+In IOS-XR 64-bit devices that support an administration mode, the proper operation of ``get_environment`` requires that the ``iosxr_netconf`` driver session is authenticated against a username defined in that administration mode.