← All writing

Automate AWS Route53 records for Kubernetes Ingress using External DNS

While exposing your application by Kubernetes Ingress with a load balancer, managing DNS records could be a headache because of the dynamic…

First published on Medium

While exposing your application by Kubernetes Ingress with a load balancer, managing DNS records could be a headache because of the dynamic DNS name. If you need to re-create your resources, a manual update is required to point your domain to a new DNS name. Your application must handle them correctly and on time to avoid downtime.

In this article, I will show you how to automate the DNS record creation using External DNS on AWS Route53 according to the rules in a Kubernetes Ingress object.

Requirements

  1. AWS Console access
  2. A Route53 host zone of your domain
  3. An EKS cluster with AWS Load Balancer Controller
  4. aws-cli installed in your machine, and configure your credentials on the executing machine like aws configure
  5. kubectl, helm, eksctl installed in your machine

Let’s get started!

This is the official repository of External DNS, and we will install it using the official Helm chart.

GitHub - kubernetes-sigs/external-dns: Configure external DNS servers (AWS Route53, Google CloudDNS…Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services … github.com external-dns/charts/external-dns at master · kubernetes-sigs/external-dnsConfigure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services … github.com

Create a namespace

We need a separate namespace for all the resources of External DNS. I will use external-dns as the example.

kubectl create ns external-dns

Create an IAM policy

It will need the following permssions. Replace <your hostzone id> with your domain hostzone ID.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "route53:ChangeResourceRecordSets"
            ],
            "Resource": [
                "arn:aws:route53:::hostedzone/<your hostzone id>"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "route53:ListHostedZones",
                "route53:ListResourceRecordSets",
                "route53:ListTagsForResource"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Create a service account

We need a service account that attached with the IAM policy just created in the previous step, which will be consumed by the External DNS pods for the Route53 records.

Replace the <cluster name> with your EKS cluster name, and IAM policy ARN with the ARN of the IAM policy just created.

eksctl create iamserviceaccount --name external-dns --namespace external-dns --cluster <cluster name> --attach-policy-arn <IAM policy ARN> --approve

Deployment

  1. Add the Helm repository of External DNS.
    helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
  2. Install the External DNS chart. Prepare your own values.yaml in advance.
    helm upgrade --install external-dns external-dns/external-dns -f values.yaml --version 1.15.0

This is my values.yaml as an example.

# Default values for external-dns.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

image:
  # -- Image repository for the `external-dns` container.
  repository: registry.k8s.io/external-dns/external-dns
  # -- (string) Image tag for the `external-dns` container, this will default to `.Chart.AppVersion` if not set.
  tag:
  # -- Image pull policy for the `external-dns` container.
  pullPolicy: IfNotPresent

# -- Image pull secrets.
imagePullSecrets: []

# -- (string) Override the name of the chart.
nameOverride:

# -- (string) Override the full name of the chart.
fullnameOverride:

# -- Labels to add to all chart resources.
commonLabels: {}

serviceAccount:
  # -- If `true`, create a new `ServiceAccount`.
  create: false
  # -- Labels to add to the service account.
  labels: {}
  # -- Annotations to add to the service account.
  annotations: {}
  # -- (string) If this is set and `serviceAccount.create` is `true` this will be used for the created `ServiceAccount` name, if set and `serviceAccount.create` is `false` then this will define an existing `ServiceAccount` to use.
  name: external-dns
  # -- Set this to `false` to [opt out of API credential automounting](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#opt-out-of-api-credential-automounting) for the `ServiceAccount`.
  automountServiceAccountToken:

service:
  # -- Service annotations.
  annotations: {}
  # -- Service HTTP port.
  port: 7979
  # -- Service IP families.
  ipFamilies: []
  # -- (string) Service IP family policy.
  ipFamilyPolicy:

rbac:
  # -- If `true`, create a `ClusterRole` & `ClusterRoleBinding` with access to the Kubernetes API.
  create: true
  # -- Additional rules to add to the `ClusterRole`.
  additionalPermissions: []

# -- Annotations to add to the `Deployment`.
deploymentAnnotations: {}

# -- [Deployment Strategy](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy).
deploymentStrategy:
  type: Recreate

# -- (int) Specify the number of old `ReplicaSets` to retain to allow rollback of the `Deployment``.
revisionHistoryLimit:

# -- Labels to add to the `Pod`.
podLabels: {}

# -- Annotations to add to the `Pod`.
podAnnotations: {}

# -- (bool) Set this to `false` to [opt out of API credential automounting](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#opt-out-of-api-credential-automounting) for the `Pod`.
automountServiceAccountToken:

# -- If `true`, the `Pod` will have [process namespace sharing](https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/) enabled.
shareProcessNamespace: false

# -- [Pod security context](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#podsecuritycontext-v1-core), this supports full customisation.
# @default -- See _values.yaml_
podSecurityContext:
  runAsNonRoot: true
  fsGroup: 65534
  seccompProfile:
    type: RuntimeDefault

# -- (string) Priority class name for the `Pod`.
priorityClassName:

# -- (int) Termination grace period for the `Pod` in seconds.
terminationGracePeriodSeconds:

# -- (string) [DNS policy](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy) for the pod, if not set the default will be used.
dnsPolicy:

# -- (object) [DNS config](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-config) for the pod, if not set the default will be used.
dnsConfig:

# -- [Init containers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) to add to the `Pod` definition.
initContainers: []

# -- [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) for the `external-dns` container.
# @default -- See _values.yaml_
securityContext:
  privileged: false
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 65532
  runAsGroup: 65532
  capabilities:
    drop: ["ALL"]

# -- [Environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) for the `external-dns` container.
env: []

# -- [Liveness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container.
# @default -- See _values.yaml_
livenessProbe:
  httpGet:
    path: /healthz
    port: http
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 2
  successThreshold: 1

# -- [Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container.
# @default -- See _values.yaml_
readinessProbe:
  httpGet:
    path: /healthz
    port: http
  initialDelaySeconds: 5
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 6
  successThreshold: 1

# -- Extra [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) for the `Pod`.
extraVolumes: []

# -- Extra [volume mounts](https://kubernetes.io/docs/concepts/storage/volumes/) for the `external-dns` container.
extraVolumeMounts: []

# -- [Resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for the `external-dns` container.
resources: {}

# -- Node labels to match for `Pod` [scheduling](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/).
nodeSelector: {}

# -- Affinity settings for `Pod` [scheduling](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/). If an explicit label selector is not provided for pod affinity or pod anti-affinity one will be created from the pod selector labels.
affinity: {}

# -- Topology spread constraints for `Pod` [scheduling](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/). If an explicit label selector is not provided one will be created from the pod selector labels.
topologySpreadConstraints: []

# -- Node taints which will be tolerated for `Pod` [scheduling](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/).
tolerations: []

serviceMonitor:
  # -- If `true`, create a `ServiceMonitor` resource to support the _Prometheus Operator_.
  enabled: false
  # -- Additional labels for the `ServiceMonitor`.
  additionalLabels: {}
  # -- Annotations to add to the `ServiceMonitor`.
  annotations: {}
  # -- (string) If set create the `ServiceMonitor` in an alternate namespace.
  namespace:
  # -- (string) If set override the _Prometheus_ default interval.
  interval:
  # -- (string) If set override the _Prometheus_ default scrape timeout.
  scrapeTimeout:
  # -- (string) If set overrides the _Prometheus_ default scheme.
  scheme:
  # -- Configure the `ServiceMonitor` [TLS config](https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#tlsconfig).
  tlsConfig: {}
  # -- (string) Provide a bearer token file for the `ServiceMonitor`.
  bearerTokenFile:
  # -- [Relabel configs](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) to apply to samples before ingestion.
  relabelings: []
  # -- [Metric relabel configs](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs) to apply to samples before ingestion.
  metricRelabelings: []
  # -- Provide target labels for the `ServiceMonitor`.
  targetLabels: []

# -- Log level.
logLevel: info

# -- Log format.
logFormat: text

# -- Interval for DNS updates.
interval: 1m

# -- If `true`, triggers run loop on create/update/delete events in addition of regular interval.
triggerLoopOnEvent: false

# -- if `true`, _ExternalDNS_ will run in a namespaced scope (`Role`` and `Rolebinding`` will be namespaced too).
namespaced: false

# -- _Kubernetes_ resources to monitor for DNS entries.
sources:
  - ingress

# -- How DNS records are synchronized between sources and providers; available values are `sync` & `upsert-only`.
policy: sync

# -- Specify the registry for storing ownership and labels.
# Valid values are `txt`, `aws-sd`, `dynamodb` & `noop`.
registry: txt
# -- (string) Specify an identifier for this instance of _ExternalDNS_ wWhen using a registry other than `noop`.
txtOwnerId: demo-external-dns
# -- (string) Specify a prefix for the domain names of TXT records created for the `txt` registry.
# Mutually exclusive with `txtSuffix`.
txtPrefix:
# -- (string) Specify a suffix for the domain names of TXT records created for the `txt` registry.
# Mutually exclusive with `txtPrefix`.
txtSuffix:

## - Limit possible target zones by domain suffixes.
domainFilters:
  - example.com

provider:
  # -- _ExternalDNS_ provider name; for the available providers and how to configure them see [README](https://github.com/kubernetes-sigs/external-dns/blob/master/charts/external-dns/README.md#providers).
  name: aws
  webhook:
    image:
      # -- (string) Image repository for the `webhook` container.
      repository:
      # -- (string) Image tag for the `webhook` container.
      tag:
      # -- Image pull policy for the `webhook` container.
      pullPolicy: IfNotPresent
    # -- [Environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) for the `webhook` container.
    env: []
    # -- Extra arguments to provide for the `webhook` container.
    args: []
    # -- Extra [volume mounts](https://kubernetes.io/docs/concepts/storage/volumes/) for the `webhook` container.
    extraVolumeMounts: []
    # -- [Resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for the `webhook` container.
    resources: {}
    # -- [Pod security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) for the `webhook` container.
    # @default -- See _values.yaml_
    securityContext: {}
    # -- [Liveness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container.
    # @default -- See _values.yaml_
    livenessProbe:
      httpGet:
        path: /healthz
        port: http-webhook
      initialDelaySeconds: 10
      periodSeconds: 10
      timeoutSeconds: 5
      failureThreshold: 2
      successThreshold: 1
    # -- [Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `webhook` container.
    # @default -- See _values.yaml_
    readinessProbe:
      httpGet:
        path: /healthz
        port: http-webhook
      initialDelaySeconds: 5
      periodSeconds: 10
      timeoutSeconds: 5
      failureThreshold: 6
      successThreshold: 1
    # -- Optional [Service Monitor](https://prometheus-operator.dev/docs/operator/design/#servicemonitor) configuration for the `webhook` container.
    # @default -- See _values.yaml_
    serviceMonitor:
      interval:
      scheme:
      tlsConfig: {}
      bearerTokenFile:
      scrapeTimeout:
      metricRelabelings: []
      relabelings: []

# -- Extra arguments to provide to _ExternalDNS_.
extraArgs: []

secretConfiguration:
  # -- If `true`, create a `Secret` to store sensitive provider configuration (**DEPRECATED**).
  enabled: false
  # -- Mount path for the `Secret`, this can be templated.
  mountPath:
  # -- Sub-path for mounting the `Secret`, this can be templated.
  subPath:
  # -- `Secret` data.
  data: {}

Deploy an Ingress Object

Now, you can deploy an ingress object to test if the DNS records can be added automatically to your Route hostzone.

This is an example Ingress definition to use.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/load-balancer-name: demo
spec:
  ingressClassName: alb
  rules:
  - host: demo.example.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: demo
            port: 
              number: 80

Deploy it using kubectl apply -f ingress.yaml.

Result

Check the log of the External DNS pod which you should see some similar outputs. Also, check your AWS Route53 console to see the newly created records!

time="2024-11-11T01:28:10Z" level=info msg="Applying provider record filter for domains: [example.com. .example.com.]"
time="2024-11-11T01:28:10Z" level=info msg="Desired change: CREATE cname-demo.example.com TXT" profile=default zoneID=/hostedzone/<hostzone id> zoneName=example.com.
time="2024-11-11T01:28:10Z" level=info msg="Desired change: CREATE demo.example.com A" profile=default zoneID=/hostedzone/<hostzone id> zoneName=example.com.
time="2024-11-11T01:28:10Z" level=info msg="Desired change: CREATE demo.example.com TXT" profile=default zoneID=/hostedzone/<hostzone id> zoneName=example.com.
time="2024-11-11T01:28:10Z" level=info msg="3 record(s) were successfully updated" profile=default zoneID=/hostedzone/<hostzone id> zoneName=example.com.

If you try to delete the Ingress object now, the DNS records will also be deleted. Try it yourself!

Example

GitHub - yukccy/terraform-externaldns-eks: Sample repository for my Medium article, storing the…Sample repository for my Medium article, storing the Terraform scripts to create necessary AWS resources. … github.com

I have prepared a GitHub repository with Terraform scripts to create network resources, EKS cluster, load balancer controller on AWS. Unfortunately I didn’t include the IAM policy and the Helm install of External DNS due to limited time, but you can simply get them installed by following the provided steps.

In this repository you will also see the sample values.yaml for External DNS Helm chart, and definition of the Ingress object for testing.

Feel free to try it out, but don’t forget to update the files with your correct values. And make sure to destroy all the resources if you used my Terraform scripts.

Read it in a terminal$ tonychan writing automate-aws-route53-records-for-kubernetes-ingress-using-external-dns