Merge branch 'feature' into 14731-plugins-catalog

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -26,7 +26,7 @@ body:
     attributes:
       label: NetBox Version
       description: What version of NetBox are you currently running?
-      placeholder: v4.0.6
+      placeholder: v4.0.7
     validations:
       required: true
   - type: dropdown

.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -14,7 +14,7 @@ body:
     attributes:
       label: NetBox version
       description: What version of NetBox are you currently running?
-      placeholder: v4.0.6
+      placeholder: v4.0.7
     validations:
       required: true
   - type: dropdown

docs/configuration/error-reporting.md

@@ -31,6 +31,17 @@ The sampling rate for errors. Must be a value between 0 (disabled) and 1.0 (repo
 
 ---
 
+## SENTRY_SEND_DEFAULT_PII
+
+Default: False
+
+Maps to the Sentry SDK's [`send_default_pii`](https://docs.sentry.io/platforms/python/configuration/options/#send-default-pii) parameter. If enabled, certain personally identifiable information (PII) is added.
+
+!!! warning "Sensitive data"
+    If you enable this option, be aware that sensitive data such as cookies and authentication tokens will be logged.
+
+---
+
 ## SENTRY_TAGS
 
 An optional dictionary of tag names and values to apply to Sentry error reports.For example:

docs/configuration/system.md

@@ -177,7 +177,7 @@ The dotted path to the desired search backend class. `CachedValueSearchBackend`
 
 Default: None (local storage)
 
-The backend storage engine for handling uploaded files (e.g. image attachments). NetBox supports integration with the [`django-storages`](https://django-storages.readthedocs.io/en/stable/) package, which provides backends for several popular file storage services. If not configured, local filesystem storage will be used.
+The backend storage engine for handling uploaded files (e.g. image attachments). NetBox supports integration with the [`django-storages`](https://django-storages.readthedocs.io/en/stable/) and [`django-storage-swift`](https://github.com/dennisv/django-storage-swift) packages, which provide backends for several popular file storage services. If not configured, local filesystem storage will be used.
 
 The configuration parameters for the specified storage backend are defined under the `STORAGE_CONFIG` setting.
 
@@ -187,7 +187,7 @@ The configuration parameters for the specified storage backend are defined under
 
 Default: Empty
 
-A dictionary of configuration parameters for the storage backend configured as `STORAGE_BACKEND`. The specific parameters to be used here are specific to each backend; see the [`django-storages` documentation](https://django-storages.readthedocs.io/en/stable/) for more detail.
+A dictionary of configuration parameters for the storage backend configured as `STORAGE_BACKEND`. The specific parameters to be used here are specific to each backend; see the documentation for your selected backend ([`django-storages`](https://django-storages.readthedocs.io/en/stable/) or [`django-storage-swift`](https://github.com/dennisv/django-storage-swift)) for more detail.
 
 If `STORAGE_BACKEND` is not defined, this setting will be ignored.
 

docs/development/release-checklist.md

@@ -135,4 +135,6 @@ First, run the `build-site` action, by navigating to Actions > build-site > Run
 
 Once the documentation files have been compiled, they must be published by running the `deploy-kinsta` action. Select the desired deployment environment (staging or production) and specify `latest` as the deploy tag.
 
+Clear the CDN cache from the [Kinsta](https://my.kinsta.com/) portal. Navigate to _Sites_ / _NetBox Labs_ / _Live_, select _CDN_ in the left-nav, click the _Clear CDN cache_ button, and confirm the clear operation.
+
 Finally, verify that the documentation at <https://netboxlabs.com/docs/netbox/en/stable/> has been updated.

docs/plugins/removal.md

@@ -70,3 +70,19 @@ DROP TABLE
 netbox=> DROP TABLE pluginname_bar;
 DROP TABLE
 ```
+
+### Remove the Django Migration Records
+
+After removing the tables created by a plugin, the migrations that created the tables need to be removed from Django's migration history as well. This is necessary to make it possible to reinstall the plugin at a later time. If the migration history were left in place, Django would skip all migrations that were executed in the course of a previous installation, which would cause the plugin to fail after reinstallation.
+
+```no-highlight
+netbox=> SELECT * FROM django_migrations WHERE app='pluginname';
+ id  |    app     |          name          |            applied
+-----+------------+------------------------+-------------------------------
+ 492 | pluginname | 0001_initial           | 2023-12-21 11:59:59.325995+00
+ 493 | pluginname | 0002_add_foo           | 2023-12-21 11:59:59.330026+00
+netbox=> DELETE FROM django_migrations WHERE app='pluginname';
+```
+
+!!! warning
+    Exercise extreme caution when altering Django system tables. Users are strongly encouraged to perform a backup of their database immediately before taking these actions.

docs/release-notes/version-4.0.md

@@ -1,6 +1,42 @@
 # NetBox v4.0
 
-## v4.0.7 (FUTURE)
+## v4.0.8 (FUTURE)
+
+---
+
+## v4.0.7 (2024-07-09)
+
+### Enhancements
+
+* [#14554](https://github.com/netbox-community/netbox/issues/14554) - Add support for [django-storage-swift](https://github.com/dennisv/django-storage-swift) storage backend
+* [#16424](https://github.com/netbox-community/netbox/issues/16424) - Enable filtering of devices by cluster and cluster group
+* [#16716](https://github.com/netbox-community/netbox/issues/16716) - Display NAT address (if any) for OOB IP address under device view
+* [#16725](https://github.com/netbox-community/netbox/issues/16725) - Always position the admin section last in the navigation menu
+* [#16791](https://github.com/netbox-community/netbox/issues/16791) - Add 200 & 400 Gbps selections for circuit termination port speed
+* [#16802](https://github.com/netbox-community/netbox/issues/16802) - Introduce `SENTRY_SEND_DEFAULT_PII` configuration parameter and disable PII export by default
+* [#16817](https://github.com/netbox-community/netbox/issues/16817) - Add 200 & 400 Gbps selections for circuit commit rate
+
+### Bug Fixes
+
+* [#16523](https://github.com/netbox-community/netbox/issues/16523) - Restore highlighting of current device in virtual chassis members panel
+* [#16654](https://github.com/netbox-community/netbox/issues/16654) - Fix parent item assignment for inventory item bulk import
+* [#16657](https://github.com/netbox-community/netbox/issues/16657) - Fix translation of object types in global search
+* [#16679](https://github.com/netbox-community/netbox/issues/16679) - Avoid overwriting custom JSON fields during bulk edit
+* [#16689](https://github.com/netbox-community/netbox/issues/16689) - System configuration view should reflect static parameters when no config revisions exist
+* [#16714](https://github.com/netbox-community/netbox/issues/16714) - Fix cloning of device types with 0U height
+* [#16721](https://github.com/netbox-community/netbox/issues/16721) - Fix errant API request after deselecting a rack in device edit form
+* [#16723](https://github.com/netbox-community/netbox/issues/16723) - Fix escaping of path to virtual environment in `upgrade.sh`
+* [#16735](https://github.com/netbox-community/netbox/issues/16735) - Object list "results" tab should show a count of zero when empty
+* [#16747](https://github.com/netbox-community/netbox/issues/16747) - Avoid clearing entire search cache when manually reindexing specific apps/models
+* [#16758](https://github.com/netbox-community/netbox/issues/16758) - Ensure manually selected lagnuage persists across browser sessions
+* [#16779](https://github.com/netbox-community/netbox/issues/16779) - Fix saved filter selection for child object lists
+* [#16780](https://github.com/netbox-community/netbox/issues/16780) - IKE proposal created via REST API should not require authentication_algorithm
+* [#16796](https://github.com/netbox-community/netbox/issues/16796) - Allow assignment of VM with no site to a cluster with a site
+* [#16806](https://github.com/netbox-community/netbox/issues/16806) - Fix redirect URL when creating contact assignments with "add another" button
+* [#16807](https://github.com/netbox-community/netbox/issues/16807) - Fix layout of VLAN edit form when custom fields are present
+* [#16808](https://github.com/netbox-community/netbox/issues/16808) - Fix event rule triggering in scenario where objects are updated immediately prior to deletion
+* [#16813](https://github.com/netbox-community/netbox/issues/16813) - Fix AttributeError exception when filtering bookmarks in dashboard widget by object type
+* [#16843](https://github.com/netbox-community/netbox/issues/16843) - Permit creation of IKE policies via REST API without specifying an IKE mode
 
 ---
 

netbox/account/views.py

@@ -113,7 +113,7 @@ def post(self, request):
 
             # Set the user's preferred language (if any)
             if language := request.user.config.get('locale.language'):
-                response.set_cookie(settings.LANGUAGE_COOKIE_NAME, language)
+                response.set_cookie(settings.LANGUAGE_COOKIE_NAME, language, max_age=request.session.get_expiry_age())
 
             return response
 
@@ -208,7 +208,7 @@ def post(self, request):
 
             # Set/clear language cookie
             if language := form.cleaned_data['locale.language']:
-                response.set_cookie(settings.LANGUAGE_COOKIE_NAME, language)
+                response.set_cookie(settings.LANGUAGE_COOKIE_NAME, language, max_age=request.session.get_expiry_age())
             else:
                 response.delete_cookie(settings.LANGUAGE_COOKIE_NAME)
 

netbox/circuits/choices.py

@@ -38,6 +38,8 @@ class CircuitCommitRateChoices(ChoiceSet):
         (25000000, '25 Gbps'),
         (40000000, '40 Gbps'),
         (100000000, '100 Gbps'),
+        (200000000, '200 Gbps'),
+        (400000000, '400 Gbps'),
         (1544, 'T1 (1.544 Mbps)'),
         (2048, 'E1 (2.048 Mbps)'),
     ]
@@ -69,6 +71,8 @@ class CircuitTerminationPortSpeedChoices(ChoiceSet):
         (25000000, '25 Gbps'),
         (40000000, '40 Gbps'),
         (100000000, '100 Gbps'),
+        (200000000, '200 Gbps'),
+        (400000000, '400 Gbps'),
         (1544, 'T1 (1.544 Mbps)'),
         (2048, 'E1 (2.048 Mbps)'),
     ]

netbox/circuits/forms/bulk_import.py

@@ -66,9 +66,6 @@ class CircuitTypeImportForm(NetBoxModelImportForm):
     class Meta:
         model = CircuitType
         fields = ('name', 'slug', 'color', 'description', 'tags')
-        help_texts = {
-            'color': mark_safe(_('RGB color in hexadecimal. Example:') + ' <code>00ff00</code>'),
-        }
 
 
 class CircuitImportForm(NetBoxModelImportForm):

netbox/core/views.py

@@ -632,7 +632,7 @@ def get(self, request):
             config = ConfigRevision.objects.get(pk=cache.get('config_version'))
         except ConfigRevision.DoesNotExist:
             # Fall back to using the active config data if no record is found
-            config = ConfigRevision(data=get_config().defaults)
+            config = get_config()
 
         # Raw data export
         if 'export' in request.GET:

netbox/dcim/filtersets.py

@@ -20,7 +20,7 @@
     ContentTypeFilter, MultiValueCharFilter, MultiValueMACAddressFilter, MultiValueNumberFilter, MultiValueWWNFilter,
     NumericArrayFilter, TreeNodeMultipleChoiceFilter,
 )
-from virtualization.models import Cluster
+from virtualization.models import Cluster, ClusterGroup
 from vpn.models import L2VPN
 from wireless.choices import WirelessRoleChoices, WirelessChannelChoices
 from wireless.models import WirelessLAN, WirelessLink
@@ -1012,6 +1012,17 @@ class DeviceFilterSet(
         queryset=Cluster.objects.all(),
         label=_('VM cluster (ID)'),
     )
+    cluster_group = django_filters.ModelMultipleChoiceFilter(
+        field_name='cluster__group__slug',
+        queryset=ClusterGroup.objects.all(),
+        to_field_name='slug',
+        label=_('Cluster group (slug)'),
+    )
+    cluster_group_id = django_filters.ModelMultipleChoiceFilter(
+        field_name='cluster__group',
+        queryset=ClusterGroup.objects.all(),
+        label=_('Cluster group (ID)'),
+    )
     model = django_filters.ModelMultipleChoiceFilter(
         field_name='device_type__slug',
         queryset=DeviceType.objects.all(),

netbox/dcim/forms/bulk_import.py

@@ -174,9 +174,6 @@ class RackRoleImportForm(NetBoxModelImportForm):
     class Meta:
         model = RackRole
         fields = ('name', 'slug', 'color', 'description', 'tags')
-        help_texts = {
-            'color': mark_safe(_('RGB color in hexadecimal. Example:') + ' <code>00ff00</code>'),
-        }
 
 
 class RackImportForm(NetBoxModelImportForm):
@@ -384,9 +381,6 @@ class DeviceRoleImportForm(NetBoxModelImportForm):
     class Meta:
         model = DeviceRole
         fields = ('name', 'slug', 'color', 'vm_role', 'config_template', 'description', 'tags')
-        help_texts = {
-            'color': mark_safe(_('RGB color in hexadecimal. Example:') + ' <code>00ff00</code>'),
-        }
 
 
 class PlatformImportForm(NetBoxModelImportForm):
@@ -1052,7 +1046,7 @@ class InventoryItemImportForm(NetBoxModelImportForm):
     class Meta:
         model = InventoryItem
         fields = (
-            'device', 'name', 'label', 'role', 'manufacturer', 'part_id', 'serial', 'asset_tag', 'discovered',
+            'device', 'name', 'label', 'role', 'manufacturer', 'parent', 'part_id', 'serial', 'asset_tag', 'discovered',
             'description', 'tags', 'component_type', 'component_name',
         )
 
@@ -1104,9 +1098,6 @@ class InventoryItemRoleImportForm(NetBoxModelImportForm):
     class Meta:
         model = InventoryItemRole
         fields = ('name', 'slug', 'color', 'description')
-        help_texts = {
-            'color': mark_safe(_('RGB color in hexadecimal. Example:') + ' <code>00ff00</code>'),
-        }
 
 
 #
@@ -1183,9 +1174,6 @@ class Meta:
             'side_a_device', 'side_a_type', 'side_a_name', 'side_b_device', 'side_b_type', 'side_b_name', 'type',
             'status', 'tenant', 'label', 'color', 'length', 'length_unit', 'description', 'comments', 'tags',
         ]
-        help_texts = {
-            'color': mark_safe(_('RGB color in hexadecimal. Example:') + ' <code>00ff00</code>'),
-        }
 
     def _clean_side(self, side):
         """

netbox/dcim/forms/filtersets.py

@@ -14,6 +14,7 @@
 from utilities.forms.fields import ColorField, DynamicModelMultipleChoiceField, TagFilterField
 from utilities.forms.rendering import FieldSet
 from utilities.forms.widgets import NumberWithOptions
+from virtualization.models import Cluster, ClusterGroup
 from vpn.models import L2VPN
 from wireless.choices import *
 
@@ -655,6 +656,7 @@ class DeviceFilterForm(
             'console_ports', 'console_server_ports', 'power_ports', 'power_outlets', 'interfaces', 'pass_through_ports',
             name=_('Components')
         ),
+        FieldSet('cluster_group_id', 'cluster_id', name=_('Cluster')),
         FieldSet(
             'has_primary_ip', 'has_oob_ip', 'virtual_chassis_member', 'config_template_id', 'local_context_data',
             'has_virtual_device_context',
@@ -821,6 +823,16 @@ class DeviceFilterForm(
             choices=BOOLEAN_WITH_BLANK_CHOICES
         )
     )
+    cluster_id = DynamicModelMultipleChoiceField(
+        queryset=Cluster.objects.all(),
+        required=False,
+        label=_('Cluster')
+    )
+    cluster_group_id = DynamicModelMultipleChoiceField(
+        queryset=ClusterGroup.objects.all(),
+        required=False,
+        label=_('Cluster group')
+    )
     tag = TagFilterField(model)
 
 

netbox/dcim/tests/test_filtersets.py

@@ -9,7 +9,7 @@
 from netbox.choices import ColorChoices
 from tenancy.models import Tenant, TenantGroup
 from utilities.testing import ChangeLoggedFilterSetTests, create_test_device
-from virtualization.models import Cluster, ClusterType
+from virtualization.models import Cluster, ClusterType, ClusterGroup
 from wireless.choices import WirelessChannelChoices, WirelessRoleChoices
 
 User = get_user_model()
@@ -1959,10 +1959,16 @@ def setUpTestData(cls):
         Rack.objects.bulk_create(racks)
 
         cluster_type = ClusterType.objects.create(name='Cluster Type 1', slug='cluster-type-1')
+        cluster_groups = (
+            ClusterGroup(name='Cluster Group 1', slug='cluster-group-1'),
+            ClusterGroup(name='Cluster Group 2', slug='cluster-group-2'),
+            ClusterGroup(name='Cluster Group 3', slug='cluster-group-3'),
+        )
+        ClusterGroup.objects.bulk_create(cluster_groups)
         clusters = (
-            Cluster(name='Cluster 1', type=cluster_type),
-            Cluster(name='Cluster 2', type=cluster_type),
-            Cluster(name='Cluster 3', type=cluster_type),
+            Cluster(name='Cluster 1', type=cluster_type, group=cluster_groups[0]),
+            Cluster(name='Cluster 2', type=cluster_type, group=cluster_groups[1]),
+            Cluster(name='Cluster 3', type=cluster_type, group=cluster_groups[2]),
         )
         Cluster.objects.bulk_create(clusters)
 
@@ -2213,6 +2219,13 @@ def test_cluster(self):
         params = {'cluster_id': [clusters[0].pk, clusters[1].pk]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
 
+    def test_cluster_group(self):
+        cluster_groups = ClusterGroup.objects.all()[:2]
+        params = {'cluster_group_id': [cluster_groups[0].pk, cluster_groups[1].pk]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+        params = {'cluster_group': [cluster_groups[0].slug, cluster_groups[1].slug]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
     def test_model(self):
         params = {'model': ['model-1', 'model-2']}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)