Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AppService] functionapp create: Check if storage account is network restricted #30605

Open
wants to merge 7 commits into
base: dev
Choose a base branch
from

Conversation

kamperiadis
Copy link
Contributor

Related command
az functionapp create

Description
We need to throw exceptions during the function app creation process if the storage account is network restricted

Testing Guide

  1. Create a network restricted storage account by either disabling the public network access or setting the default network rule action to 'deny'
  2. Create a function app with the storage account from the previous step

History Notes


This checklist is used to make sure that common guidelines for a pull request are followed.

Copy link

azure-client-tools-bot-prd bot commented Jan 2, 2025

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.9
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.12
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.12
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.12
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9

Copy link

Hi @kamperiadis,
Since the current milestone time is less than 7 days, this pr will be reviewed in the next milestone.

Copy link

azure-client-tools-bot-prd bot commented Jan 2, 2025

⚠️AzureCLI-BreakingChangeTest
⚠️appservice
rule cmd_name rule_message suggest_message
⚠️ 1006 - ParaAdd functionapp create cmd functionapp create added parameter configure_networking_later

Copy link

github-actions bot commented Jan 2, 2025

⚠️Your changes in this PR will be released on Jan 14, 2025 due to CCOA (extend to Jan 6, 2025)

@yonzhan
Copy link
Collaborator

yonzhan commented Jan 2, 2025

Thank you for your contribution! We will review the pull request and get back to you soon.

@kamperiadis
Copy link
Contributor Author

I notice that the assertion for tests fail if there are resources that we create on customers' behalf. Does it mean we can only run those tests in live mode instead of relying on recordings? @zhoxing-ms

@zhoxing-ms
Copy link
Contributor

I notice that the assertion for tests fail if there are resources that we create on customers' behalf. Does it mean we can only run those tests in live mode instead of relying on recordings?

Why can't these resources be dynamically created in the tests?

@kamperiadis
Copy link
Contributor Author

I notice that the assertion for tests fail if there are resources that we create on customers' behalf. Does it mean we can only run those tests in live mode instead of relying on recordings?

Why can't these resources be dynamically created in the tests?

The intention is for the command to create those resources in behalf of the customer, as opposed to the customer (or tests) to create them separately.

@zhoxing-ms
Copy link
Contributor

If the resource is dynamically created through CLI commands in the test code, the create request will also be recorded in the recording file, so it supports execution in replay mode

@kamperiadis
Copy link
Contributor Author

@zhoxing-ms The resources that are internally created by the command (not in the test code but in the command itself), the tool has trouble determining that it is the same call.

For instance, in the changes I made, I am creating a new file share resource (e.g. fileServices/default/shares/functionapp000002092f77bfe918) when the az functionapp create command is run. And the tool has trouble with that file share resource that got created internally on behalf of the customer by the az functionapp create command:

except CannotOverwriteExistingCassetteException as ex:

      raise AssertionError(ex)

E AssertionError: Can't overwrite existing cassette ('/mnt/vss/_work/1/s/src/azure-cli/azure/cli/command_modules/appservice/tests/latest/recordings/test_functionapp_elastic_premium_restricted_public_network_access_storage_vnet.yaml') in your current record mode ('once').
E No match for the request (<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003/fileServices/default/shares/functionapp000002092f77bfe918?api-version=2023-05-01>) was found.
E Found 2 similar requests with 1 different matcher(s) :
E
E 1 - (<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003?api-version=2023-05-01>)..)
E Matchers succeeded : ['method', 'scheme', 'host', 'port', '_custom_request_query_matcher']
E Matchers failed :
E path - assertion failure :
E /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003/fileServices/default/shares/functionapp000002092f77bfe918 != /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003
E
E 2 - (<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003/fileServices/default/shares/functionapp00000215a2c24cb746?api-version=2023-05-01>)..)
E Matchers succeeded : ['method', 'scheme', 'host', 'port', '_custom_request_query_matcher']
E Matchers failed :
E path - assertion failure :
E /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003/fileServices/default/shares/functionapp000002092f77bfe918 != /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003/fileServices/default/shares/functionapp00000215a2c24cb746

@zhoxing-ms
Copy link
Contributor

The resources that are internally created by the command (not in the test code but in the command itself), the tool has trouble determining that it is the same call.

Could you please show me the related code link?

@kamperiadis
Copy link
Contributor Author

@zhoxing-ms This is where we create a file share when the customer runs az functionapp create with a network restricted storage account:

create_file_share(cmd.cli_ctx, resource_group_name, storage_account, content_share_name)

@kamperiadis kamperiadis marked this pull request as ready for review January 10, 2025 20:00
@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Jan 13, 2025

if (plan_info is not None and is_plan_elastic_premium(cmd, plan_info)) or consumption_plan_location is not None:
site_config.app_settings.append(NameValuePair(name='WEBSITE_CONTENTAZUREFILECONNECTIONSTRING',
value=con_string))

I have encountered similar issues before, and I suspect it is due to the CI pipeline not meeting this if condition during execution the test. Therefore, this resulted in the failure to execute create_file_share

@kamperiadis
Copy link
Contributor Author

@zhoxing-ms I do not think that if condition is the problem. As you can see in the exception I included above, it cannot find a match in the file share created:

No match for the request (<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003/fileServices/default/shares/functionapp000002092f77bfe918?api-version=2023-05-01>) was found

Found 2 similar requests with 1 different matcher(s) :
...
<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.Storage/storageAccounts/funcstorage000003/fileServices/default/shares/functionapp00000215a2c24cb746?api-version=2023-05-01>

When the test was run again and was compared against the recording, there were two different file share names (functionapp000002092f77bfe918 and functionapp00000215a2c24cb746).

However, if I run the test in live mode, there are no exceptions thrown. Should I just make the tests run in live mode only?

@kamperiadis kamperiadis force-pushed the networkrestrictedstorage branch from 17efde4 to c99c070 Compare January 14, 2025 20:01
@kamperiadis kamperiadis force-pushed the networkrestrictedstorage branch from c99c070 to 9156908 Compare January 22, 2025 21:53
@zhoxing-ms
Copy link
Contributor

zhoxing-ms commented Jan 24, 2025

@kamperiadis Could you confirm that there are no other commands in the test to create file share? If not, is it possible that the file share was created twice during local testing, but only once in the CI pipeline?

@@ -5128,7 +5144,10 @@ def create_functionapp(cmd, resource_group_name, name, storage_account, plan=Non
if (plan_info is not None and is_plan_elastic_premium(cmd, plan_info)) or consumption_plan_location is not None:
site_config.app_settings.append(NameValuePair(name='WEBSITE_CONTENTAZUREFILECONNECTIONSTRING',
value=con_string))
site_config.app_settings.append(NameValuePair(name='WEBSITE_CONTENTSHARE', value=_get_content_share_name(name)))
content_share_name = _get_content_share_name(name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function may cause the dismatch between resource name and the recording file.

def _get_content_share_name(app_name):
# content share name should be up to 63 characters long, lowercase letter and digits, and random
# so take the first 50 characters of the app name and add the last 12 digits of a random uuid
share_name = app_name[0:50]
suffix = str(uuid.uuid4()).rsplit('-', maxsplit=1)[-1]
return share_name.lower() + suffix

Copy link
Contributor Author

@kamperiadis kamperiadis Jan 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @yanzhudd We actually do intend to create a new content share resource with a new name on behalf of the customer. Does this mean we should just make the tests run live? Or what would you require for this PR to be merged?

Copy link
Contributor

@yanzhudd yanzhudd Jan 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, you can mark the tests as live only, and please remove the related recording files as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @yanzhudd Those tests are already marked as live_only and there are no recording files for them.

@yanzhudd
Copy link
Contributor

Please let me know if this PR is ready to review/merge.

@kamperiadis
Copy link
Contributor Author

kamperiadis commented Jan 27, 2025

@yanzhudd @zhoxing-ms The PR is ready to be merged. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot Functions az functionapp Network az network vnet/lb/nic/dns/etc... Storage az storage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants