Azure support #126
Comments
|
@hh don't have credits for Azure on hand but I can try and get us some, meanwhile you get $200 for free at first I believe. |
|
Gene, can you please give them admin access to the Azure account we setup.
…On Tue, Feb 28, 2017 at 10:40 AM, Eugene ***@***.***> wrote:
@hh <https://github.com/hh> don't have credits for Azure on hand but I
can try and get us some. You get $200 for free at first I believe.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#126 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AC8MBpHnU1I1-0lDinrqP_BBmhhg78Itks5rhD_tgaJpZM4KXHVt>
.
--
Dan Kohn <mailto:[email protected]>
Executive Director, Cloud Native Computing Foundation <https://cncf.io/>
tel:+1-415-233-1000
|
|
@dankohn I can't, only ever used my personal free account (I think), not an admin on the main one - at least that's what it says when I go to the "Active Directory". |
|
Working in branch https://github.com/cncf/demo/tree/azure |
|
We had to migrate to a pay-as-you-go to get around as the free limits on cores is too low for quick-iterations on dev work. We will refactor for smaller instances later to allow free-tier to work after everything is functioning. |
|
We seem to have triggered block on our API access from NZ. We moved to an IP in the states for now. Is there someone at Microsoft that might be interested in supporting / unblocking us as we move along? |
|
I'm hopeful @brendandburns might know someone who could send CNCF a few Azure credits. |
|
Can you send me the subscription ID (bburns [at] microsoft [dot] com) and I'll see what we can do on this side. --brendan |
|
@brendandburns done and thanks! |
|
We started getting kubelet panics when we started using Took a while to come across: kubernetes/kubernetes#42576 Now we're off to generate |
|
Oh boy, on AWS all the stuff it wants in azure.json was simply inferred by Kubernetes (you'd just tag those resources with the cluster name and pass just that to it). This makes it much more convoluted. This is an unfortunate discrepancy. As far as I know Azure also has the concept of tags. |
|
Looks like we need to populate this manually: https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/azure/azure.go#L38 // Config holds the configuration parsed from the --cloud-config flag
// All fields are required unless otherwise specified
type Config struct {
// The cloud environment identifier. Takes values from https://github.com/Azure/go-autorest/blob/ec5f4903f77ed9927ac95b19ab8e44ada64c1356/autorest/azure/environments.go#L13
Cloud string `json:"cloud" yaml:"cloud"`
// The AAD Tenant ID for the Subscription that the cluster is deployed in
TenantID string `json:"tenantId" yaml:"tenantId"`
// The ID of the Azure Subscription that the cluster is deployed in
SubscriptionID string `json:"subscriptionId" yaml:"subscriptionId"`
// The name of the resource group that the cluster is deployed in
ResourceGroup string `json:"resourceGroup" yaml:"resourceGroup"`
// The location of the resource group that the cluster is deployed in
Location string `json:"location" yaml:"location"`
// The name of the VNet that the cluster is deployed in
VnetName string `json:"vnetName" yaml:"vnetName"`
// The name of the subnet that the cluster is deployed in
SubnetName string `json:"subnetName" yaml:"subnetName"`
// The name of the security group attached to the cluster's subnet
SecurityGroupName string `json:"securityGroupName" yaml:"securityGroupName"`
// (Optional in 1.6) The name of the route table attached to the subnet that the cluster is deployed in
RouteTableName string `json:"routeTableName" yaml:"routeTableName"`
// (Optional) The name of the availability set that should be used as the load balancer backend
// If this is set, the Azure cloudprovider will only add nodes from that availability set to the load
// balancer backend pool. If this is not set, and multiple agent pools (availability sets) are used, then
// the cloudprovider will try to add all nodes to a single backend pool which is forbidden.
// In other words, if you use multiple agent pools (availability sets), you MUST set this field.
PrimaryAvailabilitySetName string `json:"primaryAvailabilitySetName" yaml:"primaryAvailabilitySetName"`
// The ClientID for an AAD application with RBAC access to talk to Azure RM APIs
AADClientID string `json:"aadClientId" yaml:"aadClientId"`
// The ClientSecret for an AAD application with RBAC access to talk to Azure RM APIs
AADClientSecret string `json:"aadClientSecret" yaml:"aadClientSecret"`
} |
|
Yea, would be nice to have this for Azure: https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws/tags.go#L54 // ClusterID is our cluster identifier: we tag AWS resources with this value,
// and thus we can run two independent clusters in the same VPC or subnets.
// This gives us similar functionality to GCE projects.
ClusterID string |
|
Yes, you have no choice -- and that is not how the other providers are implemented: Could be an interesting thing to suggest upstream. Edit: Jinx. :) I haven't read the azure provider, do you think it would be messy to add this to it? |
|
@hh Please see https://github.com/Azure/acs-engine I think you will find it a much more pleasant way to turn up an Azure kubernetes cluster. I'll work on the subcription stuff. |
|
We have been looking pretty heavily at the individual config generation parts at https://github.com/Azure/acs-engine/tree/master/parts They've been useful in integrating the acs-engine approach. |
|
Just starting multiple build/deploys at once on Azure, had to increase default core quota from 10 to 100. Should be able to do least ten concurrent builds soon. |
|
RE: Azure relying on hostname == nodeName, this is due to lack of metadata service, so we make assumptions: https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/azure/azure_instances.go#L98 |
|
Great to meet you in Berlin @colemickens ! We really are cutting ourselves on some of the bleeding edges of kubernetes + azure + terraform on this one. :) Let me know if you want some help in creating those upstream issues for azure / kubernetes we talked about here: https://github.com/cncf/demo/blob/fd21acc7655e849a8cdda9faf0c547fa2916a0dc/azure/readme.org#notable-issues I opened issues for terraform azurerm_dns_srv_record list and azure-sdk-for-go NetworkInterfaceDnsSettings.dnsServers resolution We'll look into refactoring this sometime soon after another cloud or two and likely add a second Azure approach using acs-engine provisioning of the kubernetes cluster. |
Finally, I think you might have mentioned another issue where the Azure DNS nameservers weren't forwarding requests? Again, if you can file an Issue against cncf/demo and tag me in it, I will share all three with the right folks internally and get the discussion going. |
|
@hh, I've got an internal mail drafted, just waiting for cncf/demo issues for the other one (or two) issues listed the previous post. Then I can start getting the right folks to chime in. Thanks! |
|
Seeing that private DNS zones are not yet supported, I suspect that's why CNAME record resolution is broken. |
|
I've been spinning up AWS and Azure side by side, and I can confirm that the reason we needed a work around for CNAME was due to Azure not yet providing support for private zones Our work around was to use multiple A records. We have a working Azure deploy for now, even if we are abusing the public DNS service records. :) |
|
@colemickens I just ran into panic-when-optional-config-keys-missing issue. Did you fill an issue for that? Can't find it. |
|
@discordianfish I just filed: kubernetes/kubernetes#47543 |
No description provided.
The text was updated successfully, but these errors were encountered: