Product Changes | PDF Download Button #104

z-priest · 2023-04-12T21:21:41Z

This PR contains the working prototype implementation of the following features:

Issue #70

vendor-contact Product table - the ability to add/edit/delete product names and their respective critical infrastructure sectors from within a vendor contact
Case vendor drop down when adding affected products to case CVE - Only vendors attached to the case can have affected products in a case CVE
Auto-suggest product name based on selected vendor when adding affected products to case CVE - allowing to click and auto-fill the product name from the database or enter in new text to add a new product to the vendor
Banner above CVE displaying when there is unpublished CVE information (for example: new or changed vendors and products) - banner disappears after publishing or republishing the CVE
Additional notes have been added to the README in the localstackdev directory

Issue #67

PDF Button for Vul Notes - With accompanying headers/footers and a watermark on the front page

Development merge for LocalStack support

sei-vsarvepalli · 2023-04-14T17:49:56Z

@z-priest

Just had a little time to take a quick look at the code itself before I write some tests.

How is the vince.models.Sector managed? There seems to be no API calls for managing Sector I can see.

+    re_path(r'^ajax_calls/prods/(?P<org_id>.*)/$', views.autocomplete_prods),
+    re_path(r'^contact/add/product/(?P<pk>[0-9]+)/$', views.AddProductToContact.as_view(), name='addproduct'),
+    re_path(r'^contact/edit/product/(?P<pk>[0-9]+)/$', views.EditProduct.as_view(), name='editproduct'),
+    re_path('contact/remove/product/(?P<pk>[0-9]+)/(?P<product>[0-9]+)/$', views.RemoveProductFromContact.as_view(), name='rmproduct'),

The autocomplete_prods uses a request.GET['term'] to find the relevant along with an org_id but when term is empty the org_id seems to be dropped? Can you check to see if it actually makes sense.
Looks like you can do all of these under one ajax_call for all DML operations on Product. Combine them all to Asynch calls and avoid server requests (preferred but not required)
On logging. It will be better to log also the method that is being called

logger.debug(self.request.POST)

should be more like

logger.debug(f"{self.__class__.__name__} post: {self.request.POST}")

sei-vsarvepalli · 2023-04-14T17:59:50Z

vince/views.py

+    else:
+        prods = list(VendorProduct.objects.all().values_list('name', flat=True))
+
+    print('prods: ', prods)


Don't use debug prints use logger always.

mstrad · 2023-04-17T19:54:22Z

@z-priest

Just had a little time to take a quick look at the code itself before I write some tests.

How is the vince.models.Sector managed? There seems to be no API calls for managing Sector I can see.
+    re_path(r'^ajax_calls/prods/(?P<org_id>.*)/$', views.autocomplete_prods),
+    re_path(r'^contact/add/product/(?P<pk>[0-9]+)/$', views.AddProductToContact.as_view(), name='addproduct'),
+    re_path(r'^contact/edit/product/(?P<pk>[0-9]+)/$', views.EditProduct.as_view(), name='editproduct'),
+    re_path('contact/remove/product/(?P<pk>[0-9]+)/(?P<product>[0-9]+)/$', views.RemoveProductFromContact.as_view(), name='rmproduct'),
The autocomplete_prods uses a request.GET['term'] to find the relevant along with an org_id but when term is empty the org_id seems to be dropped? Can you check to see if it actually makes sense.

Looks like you can do all of these under one ajax_call for all DML operations on Product. Combine them all to Asynch calls and avoid server requests (preferred but not required)

On logging. It will be better to log also the method that is being called
logger.debug(self.request.POST)
should be more like
logger.debug(f"{self.__class__.__name__} post: {self.request.POST}")

@sei-vsarvepalli Do you have an example in the code currently reflecting your suggestion to question (3)? So far the database calls for other views/models appear to be done similarly to what we've written. I worry about our timeline here in pushing these features into production by May 1st.

Another concern I have is data migration testing of these product features when they are put into production. How will currently saved products in existing CVE details be handled when our changes are implemented? Will you and your team be able to handle that?

Will follow up via email as well.

sei-vsarvepalli · 2023-04-17T20:45:09Z

@z-priest
Just had a little time to take a quick look at the code itself before I write some tests.

How is the vince.models.Sector managed? There seems to be no API calls for managing Sector I can see.
+    re_path(r'^ajax_calls/prods/(?P<org_id>.*)/$', views.autocomplete_prods),
+    re_path(r'^contact/add/product/(?P<pk>[0-9]+)/$', views.AddProductToContact.as_view(), name='addproduct'),
+    re_path(r'^contact/edit/product/(?P<pk>[0-9]+)/$', views.EditProduct.as_view(), name='editproduct'),
+    re_path('contact/remove/product/(?P<pk>[0-9]+)/(?P<product>[0-9]+)/$', views.RemoveProductFromContact.as_view(), name='rmproduct'),
The autocomplete_prods uses a request.GET['term'] to find the relevant along with an org_id but when term is empty the org_id seems to be dropped? Can you check to see if it actually makes sense.

Looks like you can do all of these under one ajax_call for all DML operations on Product. Combine them all to Asynch calls and avoid server requests (preferred but not required)

On logging. It will be better to log also the method that is being called
logger.debug(self.request.POST)
should be more like
logger.debug(f"{self.__class__.__name__} post: {self.request.POST}")
@sei-vsarvepalli Do you have an example in the code currently reflecting your suggestion to question (3)? So far the database calls for other views/models appear to be done similarly to what we've written. I worry about our timeline here in pushing these features into production by May 1st.

Another concern I have is data migration testing of these product features when they are put into production. How will currently saved products in existing CVE details be handled when our changes are implemented? Will you and your team be able to handle that?

Will follow up via email as well.

For simplifying Products management

An example could be the the method vinny.views.casetracking In this code, a GET method is DQL to query for a record and it gets sent back in JSON format - you can use JavaScript to render/use it. A POST method is DML for Insert/Update. We can easily do a Delete DML with the POST as well. It is not exactly REST (with Django Serializers headache) but can be used access/manage data asynchronously. It will make separating presentation from data easier, no need to create/change forms.py but just HTML/Javascript based forms, that can be dynamically built on the client.

VINCE/vinny/views.py

Line 474 in e6d1088

@user_passes_test(is_not_pending, login_url="vinny:login")

BTW - it is not in critical path for code implementation itself.

_On current data migration for Products _

I am assuming it will just be ignored from a quick view of your code. They will not be in the Products DB until, an edit and change of the Product. I have not tested it.

- Use multi-line JavaScript comments for minify script - Remove URLs as hard-coded and use HTML data-attributes - Send dictionary from ajax_calls, avoid array/list to allow for extensions - Check to see if GET parameter is useable

sei-vsarvepalli · 2023-04-19T03:14:22Z

Running some Tests against the database. Some UI issues found in click tester from Chrome. The Products already added cannot be modified. May consider "onfocus=blur" on those rows. Same can be applied for the CWE rows to reduce confusion on editing already added CWE's with new autocomplete entries.

mstrad · 2023-04-19T15:52:12Z

Running some Tests against the database. Some UI issues found in click tester from Chrome. The Products already added cannot be modified. May consider "onfocus=blur" on those rows. Same can be applied for the CWE rows to reduce confusion on editing already added CWE's with new autocomplete entries.

@sei-vsarvepalli Could you clarify the issue that you are seeing? On our end everytime we seek to edit a product we are able to on our end. And that should be allowed as far as we understand, because products entered in, a coordinator or even a vendor could make a mistake that needs to be fixed. Or affected products of a CVE may need to be updated.

sei-vsarvepalli · 2023-04-20T02:11:06Z

Running some Tests against the database. Some UI issues found in click tester from Chrome. The Products already added cannot be modified. May consider "onfocus=blur" on those rows. Same can be applied for the CWE rows to reduce confusion on editing already added CWE's with new autocomplete entries.

@sei-vsarvepalli Could you clarify the issue that you are seeing? On our end everytime we seek to edit a product we are able to on our end. And that should be allowed as far as we understand, because products entered in, a coordinator or even a vendor could make a mistake that needs to be fixed. Or affected products of a CVE may need to be updated.

It looks like there are a few problems with org_auto function. May need to just rewrite this whole function. The item variable in the function is overloaded (string or jQuery object) and likely to be confused and cause issues. The preloaded values in a cve-product-$i when coming from a pre-existing value triggers org_auto(null, true) which activates the /* new form */ and will not enable autocomplete on these input items.

VINCE/vince/static/vince/js/cve.js

Lines 78 to 109 in 70806f9

    
           function org_auto(item, init) { 
        
               const regex = /[0-9]+/g; 
        
               let item_found = '' 
        
               let prod_input = '' 
        
               let orgid = '' 
        
               if(item){ 
        
           	orgid = $(item).val(); 
        
               } 
        
               if (item && init == true) { 
        
           	/* new row in form */ 
        
           	prod_input = $("#id_product-"+item[0].id.match(regex)[0]+"-cve_affected_product") 
        
           	item_found = $("#id_product-"+item[0].id.match(regex)[0]+"-organization") 
        
               } else if (item && init == false){ 
        
           	/* came back to form after submit or after clearing an organization selection  */ 
        
           	let row_id =  $(item)[0].id.match(regex)[0] 
        
           	prod_input = $("#id_product-"+item.match(regex)[0]+"-cve_affected_product") 
        
           	item_found = $("#id_product-"+row_id+"-organization") 
        
           	complete_prod(orgid, $(item)[0].id.match(regex)[0], prod_input, item_found)		 
        
               } else { 
        
           	/* new form */ 
        
           	prod_input = $('#id_product-0-cve_affected_product') 
        
           	item_found = $('#id_product-0-organization') 
        
               } 
        
               const row_id = item_found[0].id.match(regex)[0] 
        
               prod_input.parent().append('<div id=newprod_indicator_'+row_id+' style="color:red;font-size:14px"></div>') 
        
               $(item_found).change(function() { 
        
           	if (item_found.val()){ 
        
           	    complete_prod(orgid, row_id, prod_input, item_found) 
        
           	} 
        
               }); 
        
           }

Will have to rewrite the item variable to jQuery selector and use forEach or each iterators for applying triggers to all the selected input items.

On HTML elements inline Javascript will fail on the AWS side as the CSP headers will block these too. Don't use onchange/onclick event handlers inside the HTML elements. These should all be triggered from document.ready() with bind or jQuery.on event handlers.

VINCE/vince/forms.py

Line 2367 in 70806f9

    
           widget = forms.Select(attrs={'class': 'form-control', 'onChange': "change_org(this)"}),

I will update in a couple of days with new code.

sei-vsarvepalli · 2023-04-20T17:10:22Z

vince/models.py

+
+    sector = ArrayField( models.CharField( max_length = 50 ), blank = True, null = True )
+
+    uuid = models.UUIDField(default=uuid.uuid4, editable=False, unique=True)


Shouldn't this have META - unique_together @mstrad @z-priest

class Meta: unique_together = (('name', 'organization'),)

I was not aware that unique_together was a thing, but that makes sense. In that case, would we need to change this to be UUIDv5? I believe UUIDv5 is deterministically calculated based on the combination of organization and name or does the UUID library somehow do this with UUIDv5?

No need to move to UUIDv5 - just the uniqueness can be enforced. Should the uniqueness be also be case insensitive? I can take care of the updates in the code.

for example "Vendor - newProduct" and "Vendor - NewProduct" should not be allowed - correct?

Verify from your ops to ensure that this is correct.

Thank You,

@mstrad I believe this would be correct.

Yes I believe that's correct

sei-vsarvepalli · 2023-04-20T19:11:02Z

vince/views.py

+                                                    organization=contact[0].vendor_name))
+
+                    #check if vendorProduct exists, if not create it.
+                    checkprod = VendorProduct.objects.filter(name = cd['cve_affected_product'])


Shouldn't this check for organization match as well? like below

checkprod = VendorProduct.objects.filter(name = cd['cve_affected_product'], organization=contact[0])

Preferably
checkprod = VendorProduct.objects.filter(name__iexact = cd['cve_affected_product'], organization=contact[0])

Few bugs found when running further Test Cases against VendorProduct with multi vendors. A duplicate product name under two vendors causes Null-Property reference exception. On delete the cascade will also fail.

Shouldn't this check for organization match as well? like below

checkprod = VendorProduct.objects.filter(name = cd['cve_affected_product'], organization=contact[0])

Preferably checkprod = VendorProduct.objects.filter(name__iexact = cd['cve_affected_product'], organization=contact[0])

Yes, it should be!

Few bugs found when running further Test Cases against VendorProduct with multi vendors. A duplicate product name under two vendors causes Null-Property reference exception. On delete the cascade will also fail.

I guess it makes sense why it is failing, because there is a single VendorProduct table with reference to contact_fk and no uniqueness is built into that table property ['name']
I wonder how to solve this? Accessing UUID instead of name fields? Prepending the contact['name'] to a vendorproduct['name'] so ACME_Product1 when saving?

Then when displaying or modifying the name break the string on contact['name']_ ?

Or add a different another property as this above and keeping the ['name'] field?

Based on your feedback, I can spend some time working on a patch tomorrow for both this and what I can come up with for org_auto above.

@z-priest Don't bother more development, Part of the testCases suggest some patches. I will apply and run the test cases again and push updates later. This way we can avoid duplicate work as it is hard to push test outputs to you in our current setup.

- Add unique_together for name+organization under VendorProduct - Check uniqueness of VendorProduct caseInsensitive - Convert org_auto and cwe_auto to Factory pattern - Merge <label> fields for multiple-row fields with disitinct id's to support accessibility - Bugs in CWE lookup - Added Cached lookup for CWE and Products with same filters.

sei-vsarvepalli · 2023-04-21T14:43:33Z

After the 6e12387 push one problem remains, on change trigger to clear and repopulate autocomplete.

sei-vsarvepalli · 2023-04-21T20:10:52Z

It looks like the problem is related to redirect issue in the server side vince/views.py test cases did not catch this!

VINCE/vince/views.py

Line 10519 in 6138197

return HttpResponseRedirect(reverse('vince:contact', args=orgid) + '#products')

Fix is

+       return HttpResponseRedirect(reverse('vince:contact', args=[orgid]) + '#products')
-        return HttpResponseRedirect(reverse('vince:contact', args=orgid) + '#products')

File "/var/app/current/vince/views.py", line 10533, in post return HttpResponseRedirect(reverse('vince:contact', args=orgid) + '#products') File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/django/urls/base.py", line 86, in reverse return resolver._reverse_with_prefix(view, prefix, *args, **kwargs) File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/django/urls/resolvers.py", line 698, in _reverse_with_prefix raise NoReverseMatch(msg)django.urls.exceptions.NoReverseMatch: Reverse for 'contact' with arguments '('2', '0')' not found. 1 pattern(s) tried: ['vince/contact/(?P[0-9]+)

…iews.py

sei-vsarvepalli · 2023-04-21T20:41:33Z

This code is now running as v2.0.8 in AWS for testing. Chrome click testing in progress.

sei-vsarvepalli · 2023-05-01T16:15:19Z

Current gaps observed by users, coordinators/operators and vendor-meeting feedback

The product tree cannot be translated to CSAF JSON and will not be visible in the advisory - only in the CVE JSON for the CVE Program.
Vendors will not have access to the Product and Version information and be able to provide Status appropriately or update any of the information.
Only Affected status is saved by the Operator, if a Product or system is unaffected or unknown it is not captured/available
The defaultStatus is not collected or not used from the CVE5 JSON schema
No current way to link CSAF nested or CSAF related advisory links for a product status

Related urls from CVE Program and CSAF issues:
https://cveproject.github.io/cve-schema/schema/v5.0/docs/mindmap.html
CVEProject/cve-schema#215
oasis-tcs/csaf#584

sei-vsarvepalli · 2023-05-03T16:11:38Z

Just came out yesterday from oss-security pointed out by @laurie-tyz - May consider this with 2.1.1 release candidate for Monday May 8th

https://www.djangoproject.com/weblog/2023/may/03/security-releases/

sei-vsarvepalli and others added 2 commits March 28, 2023 14:01

Merge pull request CERTCC#102 from CERTCC/development

e6d1088

Development merge for LocalStack support

Product Additions CERTCC#70, CERTCC#67

d287d8f

z-priest changed the title ~~Pr 70~~ Product Changes | PDF Download Button Apr 13, 2023

sei-vsarvepalli reviewed Apr 14, 2023

View reviewed changes

sei-vsarvepalli self-assigned this Apr 14, 2023

sei-vsarvepalli requested a review from sei-jsteele April 14, 2023 22:20

sei-vsarvepalli added bug Something isn't working enhancement New feature or request labels Apr 14, 2023

sei-vsarvepalli self-requested a review April 14, 2023 22:21

sei-jsteele approved these changes Apr 17, 2023

View reviewed changes

z-priest and others added 2 commits April 18, 2023 09:52

Fixes based on feedback

2dd86be

Updates from Vijay

70806f9

- Use multi-line JavaScript comments for minify script - Remove URLs as hard-coded and use HTML data-attributes - Send dictionary from ajax_calls, avoid array/list to allow for extensions - Check to see if GET parameter is useable

sei-vsarvepalli force-pushed the pr_70 branch from baea7ae to 70806f9 Compare April 19, 2023 03:09

sei-vsarvepalli reviewed Apr 20, 2023

View reviewed changes

Auto-merge with AWS updates and bug fix for orgid redirect in vince/v…

3c90e20

…iews.py

sei-vsarvepalli force-pushed the pr_70 branch from 6138197 to 3c90e20 Compare April 21, 2023 20:25

Fix bug cvss_translator GH issue CERTCC#105

304d830

Updates to UAR-Flow

6330274

Updates to CHANGELOG.md

0029990

sei-vsarvepalli added the security Affects security label May 3, 2023

Some updates to logging on v2.1.1

7224a63

sei-vsarvepalli merged commit 3bba8b4 into CERTCC:development May 8, 2023

This was referenced May 16, 2023

VINCE Storage of Vendor products and their affected versions #70

Closed

Implement PDF Button to VulNotes #67

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Product Changes | PDF Download Button #104

Product Changes | PDF Download Button #104

z-priest commented Apr 12, 2023

sei-vsarvepalli commented Apr 14, 2023

sei-vsarvepalli Apr 14, 2023

mstrad commented Apr 17, 2023

sei-vsarvepalli commented Apr 17, 2023

sei-vsarvepalli commented Apr 19, 2023 •

edited

Loading

mstrad commented Apr 19, 2023

sei-vsarvepalli commented Apr 20, 2023

sei-vsarvepalli Apr 20, 2023 •

edited

Loading

z-priest Apr 20, 2023

sei-vsarvepalli Apr 20, 2023

z-priest Apr 20, 2023

mstrad Apr 20, 2023 •

edited

Loading

sei-vsarvepalli Apr 20, 2023

sei-vsarvepalli Apr 20, 2023

z-priest Apr 20, 2023

z-priest Apr 20, 2023 •

edited

Loading

sei-vsarvepalli Apr 21, 2023

sei-vsarvepalli commented Apr 21, 2023

sei-vsarvepalli commented Apr 21, 2023

sei-vsarvepalli commented Apr 21, 2023

sei-vsarvepalli commented May 1, 2023

sei-vsarvepalli commented May 3, 2023


		sector = ArrayField( models.CharField( max_length = 50 ), blank = True, null = True )

		uuid = models.UUIDField(default=uuid.uuid4, editable=False, unique=True)

Product Changes | PDF Download Button #104

Product Changes | PDF Download Button #104

Conversation

z-priest commented Apr 12, 2023

Issue #70

Issue #67

sei-vsarvepalli commented Apr 14, 2023

Choose a reason for hiding this comment

mstrad commented Apr 17, 2023

sei-vsarvepalli commented Apr 17, 2023

sei-vsarvepalli commented Apr 19, 2023 • edited Loading

mstrad commented Apr 19, 2023

sei-vsarvepalli commented Apr 20, 2023

sei-vsarvepalli Apr 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mstrad Apr 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

z-priest Apr 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sei-vsarvepalli commented Apr 21, 2023

sei-vsarvepalli commented Apr 21, 2023

sei-vsarvepalli commented Apr 21, 2023

sei-vsarvepalli commented May 1, 2023

sei-vsarvepalli commented May 3, 2023

sei-vsarvepalli commented Apr 19, 2023 •

edited

Loading

sei-vsarvepalli Apr 20, 2023 •

edited

Loading

mstrad Apr 20, 2023 •

edited

Loading

z-priest Apr 20, 2023 •

edited

Loading