Specify Zones

[rubydotexe]

Hi! I have multiple zones in my Cloud DNS, and I was wondering if there could be a variable that allows me to specify which domain zone(s) I am using when calling the Google Cloud API. I am using Caddy with the Google Cloud DNS plugin, and abstractly, being unable to specify between zones means I get this error: unable to find Google managaged zone for domain xyz.

An adjustment here would also mean that the github.com/caddy-dns/googleclouddns would need to be adjusted too, but I would make a new issue there if my request is approved.

[brockwood]

Hello @rubydotexe and apologies for the delay in replying.

Could you please give an example of when this occurs? Currently, when the googleclouddns client starts up, it will list all the zones for the specified project and then match up the DNS zone name passed in from Caddy with the zone in Google Cloud. The only catch is that the zone has to be a public zone; if it is private, the client will ignore it.

Any additional information you could provide would be appreciated, thanks!

[rubydotexe]

Hello @rubydotexe and apologies for the delay in replying.

Could you please give an example of when this occurs? Currently, when the googleclouddns client starts up, it will list all the zones for the specified project and then match up the DNS zone name passed in from Caddy with the zone in Google Cloud. The only catch is that the zone has to be a public zone; if it is private, the client will ignore it.

Any additional information you could provide would be appreciated, thanks!

Sure, here's the full log of what I'm experiencing within Caddy. I want to stress that my zones are configured correctly in google cloud, in fact, I currently only have one.

caddy  | {"level":"info","ts":1702069271.746822,"logger":"tls.obtain","msg":"acquiring lock","identifier":"*.4tress.xyz"}
caddy  | {"level":"info","ts":1702069271.7479944,"logger":"tls.obtain","msg":"lock acquired","identifier":"*.4tress.xyz"}
caddy  | {"level":"info","ts":1702069271.7481263,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"*.4tress.xyz"}
caddy  | {"level":"info","ts":1702069271.7502823,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["*.4tress.xyz"],"ca":"https://acme-staging-v02.api.letsencrypt.org/directory","account":"[email protected]"}
caddy  | {"level":"info","ts":1702069271.7502942,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["*.4tress.xyz"],"ca":"https://acme-staging-v02.api.letsencrypt.org/directory","account":"[email protected]"}
caddy  | {"level":"info","ts":1702069271.7555842,"logger":"tls","msg":"finished cleaning storage units"}
caddy  | {"level":"info","ts":1702069272.4134185,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"*.4tress.xyz","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
caddy  | {"level":"error","ts":1702069272.8652723,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"*.4tress.xyz","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.4tress.xyz\" (usually OK if presenting also failed)"}
caddy  | {"level":"error","ts":1702069272.9322636,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"*.4tress.xyz","issuer":"acme-staging-v02.api.letsencrypt.org-directory","error":"[*.4tress.xyz] solving challenges: presenting for challenge: adding temporary record for zone \"xyz.\": unable to find Google managaged zone for domain xyz. (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/127499404/12841077594) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}
caddy  | {"level":"info","ts":1702069272.9324965,"logger":"tls.issuance.zerossl","msg":"waiting on internal rate limiter","identifiers":["*.4tress.xyz"],"ca":"https://acme-staging-v02.api.letsencrypt.org/directory","account":"[email protected]"}
caddy  | {"level":"info","ts":1702069272.9325206,"logger":"tls.issuance.zerossl","msg":"done waiting on internal rate limiter","identifiers":["*.4tress.xyz"],"ca":"https://acme-staging-v02.api.letsencrypt.org/directory","account":"[email protected]"}
caddy  | {"level":"info","ts":1702069273.2803829,"logger":"tls.issuance.zerossl.acme_client","msg":"trying to solve challenge","identifier":"*.4tress.xyz","challenge_type":"dns-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"}
caddy  | {"level":"error","ts":1702069273.4599297,"logger":"tls.issuance.zerossl.acme_client","msg":"cleaning up solver","identifier":"*.4tress.xyz","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.4tress.xyz\" (usually OK if presenting also failed)"}
caddy  | {"level":"error","ts":1702069273.5224414,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"*.4tress.xyz","issuer":"acme-staging-v02.api.letsencrypt.org-directory","error":"[*.4tress.xyz] solving challenges: presenting for challenge: adding temporary record for zone \"xyz.\": unable to find Google managaged zone for domain xyz. (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/127499404/12841077884) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}
caddy  | {"level":"error","ts":1702069273.5226629,"logger":"tls.obtain","msg":"will retry","error":"[*.4tress.xyz] Obtain: [*.4tress.xyz] solving challenges: presenting for challenge: adding temporary record for zone \"xyz.\": unable to find Google managaged zone for domain xyz. (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/127499404/12841077884) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":1.774656135,"max_duration":2592000}
caddy  | {"level":"info","ts":1702069333.5244484,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"*.4tress.xyz"}

[rubydotexe]

My A records are pointed to the correct IP, I've verified rather obsessively, but I can check my configuration again upon suggestion.

[brockwood]

Hey @rubydotexe. thank you for the additional context. I've been trying to reproduce your issue and I've not had any luck. I have a GCP project with a public zone wxgl.us. and a wildcard entry pointing at 127.0.0.1 for testing purposes:

I am using the JSON file from a service account with DNS admin rights in the project and I have the following environment variables defined:

• GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json
• GCP_PROJECT=project-name

I then am using the following Caddyfile:

*.wxgl.us {
	bind 127.0.0.1
	tls <email address> {
		dns googleclouddns {
			gcp_project {$GCP_PROJECT}
		}
	}
	respond "Hello there! I'm a Caddy!"
}

I'm able to generate a cert in this configuration:

2023/12/20 17:42:26.419 INFO    tls.obtain      acquiring lock  {"identifier": "*.wxgl.us"}
2023/12/20 17:42:26.425 INFO    tls     cleaning storage unit   {"storage": "FileStorage:/home/caddy/.local/share/caddy"}
2023/12/20 17:42:26.426 INFO    tls     finished cleaning storage units
2023/12/20 17:42:26.430 INFO    tls.obtain      lock acquired   {"identifier": "*.wxgl.us"}
2023/12/20 17:42:26.431 INFO    tls.obtain      obtaining certificate   {"identifier": "*.wxgl.us"}
2023/12/20 17:42:26.720 INFO    tls.issuance.acme       waiting on internal rate limiter        {"identifiers": ["*.wxgl.us"], "ca": "https://acme-v02.api.letsencrypt.org/directory", "account": "[email protected]"}
2023/12/20 17:42:26.720 INFO    tls.issuance.acme       done waiting on internal rate limiter   {"identifiers": ["*.wxgl.us"], "ca": "https://acme-v02.api.letsencrypt.org/directory", "account": "user@@home.com"}
2023/12/20 17:42:26.944 INFO    tls.issuance.acme.acme_client   trying to solve challenge       {"identifier": "*.wxgl.us", "challenge_type": "dns-01", "ca": "https://acme-v02.api.letsencrypt.org/directory"}
2023/12/20 17:42:30.499 INFO    tls.issuance.acme.acme_client   authorization finalized {"identifier": "*.wxgl.us", "authz_status": "valid"}
2023/12/20 17:42:30.499 INFO    tls.issuance.acme.acme_client   validations succeeded; finalizing order {"order": "https://acme-v02.api.letsencrypt.org/acme/order/<order number>/<redacted>"}
2023/12/20 17:42:31.391 INFO    tls.issuance.acme.acme_client   successfully downloaded available certificate chains    {"count": 2, "first_url": "https://acme-v02.api.letsencrypt.org/acme/cert/<redacted>"}
2023/12/20 17:42:31.392 INFO    tls.obtain      certificate obtained successfully       {"identifier": "*.wxgl.us"}
2023/12/20 17:42:31.392 INFO    tls.obtain      releasing lock  {"identifier": "*.wxgl.us"}

I'm hoping you can share how this setup differs with your own to see if we can figure out what's going on. Thanks!

[rubydotexe]

Thanks for continuing to troubleshoot with me, @brockwood. I am using a dockerized version of Caddy. I thought I linked this originally, but I didn't, so I apologize for not providing full context earlier. You can learn more about my Caddy set up here. I started with Google Cloud, then went to namecheap, and then went to Google-cloud again. Currrently, this is my set-up. I decided to mimic yours.

{
	acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}

(tls_config) {
	tls {
		dns googleclouddns {
			gcp_project {env.GCP_PROJECT}
			gcp_application_default {env.GCP_APPLICATION_DEFAULT}
		}
	}
}

*.4tress.xyz {
	import tls_config
	header Access-Control-Allow-Origin "*"
	header Access-Control-Allow-Methods "POST, GET, OPTIONS"
	respond "Hello there! I'm a Caddy!"
}

Edit:

Here's my new build file:

FROM caddy:builder AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/[email protected]

FROM caddy:latest

RUN apk add --no-cache bash
RUN apk add --no-cache vim

COPY credentials.json /path/
COPY .bash_aliases /root/
COPY .bashrc /root/

ENV GCP_PROJECT="projectname"
ENV GCP_APPLICATION_DEFAULT="/path/credentials.json"

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

[rubydotexe]

Update: Frustratingly, there is no issue when I replicate my issue on WSL2 Ubuntu. Something seems to just be uniquely broken on my Debian server.

Edit: Disregard, the problem has been reproduced successfully.