Using CoreDNS and MetalLB on bare-metal Kubernetes clusters

If you build a bare-metal Kubernetes cluster, the first awkward question is usually not how to run a pod. It is how anything else on your LAN is supposed to find the services running inside the cluster.

An Ingress controller such as Traefik can route HTTP traffic once it reaches the cluster. MetalLB can give that Ingress controller a stable LAN address. But your laptop, phone or another machine on the network still needs DNS to turn a name such as dashboard.cluster.home.arpa into that address.

This is a refreshed version of a 2019 home-lab recipe. The shape is still useful: run a small CoreDNS instance inside the cluster, expose it through MetalLB on a fixed LAN IP, and configure your router’s DHCP server to hand out that IP as the DNS server for clients on the network.

The details have changed since 2019. Kubernetes APIs moved on, MetalLB changed its annotations, and CoreDNS no longer needs the old proxy plugin example. The goal here is to keep the idea, but make the manifests safe to read in a current cluster.

The Shape of the Setup

The setup has three moving parts:

  • MetalLB advertises a LAN IP for the CoreDNS service.
  • CoreDNS answers for an internal wildcard zone, for example *.cluster.home.arpa.
  • Your router gives that CoreDNS IP to clients through DHCP.

The request path looks like this:

DNS request flow: DHCP gives clients the CoreDNS IP, MetalLB answers ARP for that IP, and CoreDNS resolves the internal wildcard domain to the Traefik load-balancer IP.

A client receives 192.168.88.50 as its DNS server from DHCP. When it asks for dashboard.cluster.home.arpa, MetalLB answers ARP for 192.168.88.50 and sends the packet to the node currently advertising the CoreDNS service. CoreDNS then returns the address of the Ingress controller, for example 192.168.88.100, and Traefik handles the HTTP routing from there.

This avoids editing /etc/hosts on every machine. It also avoids depending on dnsmasq support in a home router. If your router can host the DNS records cleanly, that is often simpler. If it cannot, this cluster-hosted resolver is a useful compromise.

Before You Start

This recipe assumes you already have:

  • a working Kubernetes cluster
  • MetalLB installed
  • an Ingress controller such as Traefik exposed with type: LoadBalancer
  • a LAN address range reserved for MetalLB
  • access to the router or DHCP server for your subnet

The examples use:

  • 192.168.88.50 for the LAN-facing CoreDNS service
  • 192.168.88.100 for the Traefik service
  • cluster.home.arpa as the internal DNS zone

The home.arpa zone is reserved for non-unique home-network names. If this is not a home network, use a subdomain you control, such as lab.example.com. Avoid inventing a public-looking domain you do not own.

There is also an operational tradeoff. If every client on your LAN uses DNS running inside the Kubernetes cluster, cluster problems can become DNS problems. For a home lab that may be acceptable. For something more serious, prefer DNS outside the cluster or configure your existing DNS server to delegate only the internal Kubernetes zone to CoreDNS.

Keep this resolver on a trusted LAN. Do not expose a recursive DNS service like this to the public internet.

Configure MetalLB’s Address Pool

Modern MetalLB does not start assigning addresses until you define an address pool and an advertisement. For a simple layer 2 home-lab setup, the configuration can look like this:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: lan
  namespace: metallb-system
spec:
  addresses:
    - 192.168.88.50-192.168.88.100
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: lan
  namespace: metallb-system
spec:
  ipAddressPools:
    - lan

Make sure that range does not overlap with normal DHCP leases. You want the router to hand out the CoreDNS IP as a DNS server, not to lease the same address to somebody’s laptop during a moment of creative chaos.

Create the CoreDNS Configuration

This is a separate CoreDNS instance for LAN clients. It is not the same service Kubernetes uses internally for cluster DNS.

The zone file below resolves both cluster.home.arpa and wildcard names under it to the Traefik address. That means dashboard.cluster.home.arpa, grafana.cluster.home.arpa and whoami.cluster.home.arpa can all point at the Ingress controller. Traefik can then route requests by host name.

apiVersion: v1
kind: Namespace
metadata:
  name: external-dns
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: external-dns
  namespace: external-dns
data:
  Corefile: |
    .:53 {
        errors
        health :8080
        ready :8181
        file /etc/coredns/zones/cluster.home.arpa cluster.home.arpa
        prometheus :9153
        forward . 192.168.88.1
        cache 30
        loop
        reload
        loadbalance
    }

  cluster.home.arpa: |
    $TTL 60
    $ORIGIN cluster.home.arpa.
    @       IN SOA ns.cluster.home.arpa. hostmaster.cluster.home.arpa. (
                2026062901 ; serial
                7200       ; refresh
                3600       ; retry
                1209600    ; expire
                60         ; minimum
            )
    @       IN NS    ns.cluster.home.arpa.
    ns      IN A     192.168.88.50
    @       IN A     192.168.88.100
    *       IN A     192.168.88.100

In this example, unknown names are forwarded to the router at 192.168.88.1. Use whatever upstream resolver makes sense on your network, but do not point it back to the same CoreDNS service.

If you want CoreDNS to use DNS-over-TLS through Cloudflare instead, replace the forward . 192.168.88.1 line with:

        forward . tls://1.1.1.1 tls://1.0.0.1 {
            tls_servername cloudflare-dns.com
            health_check 5s
        }

That is DNS-over-TLS, not DNS-over-HTTPS. The distinction matters because CoreDNS is opening a TLS connection to a DNS resolver, not sending DNS queries over HTTP.

Deploy CoreDNS

The original 2019 manifest used extensions/v1beta1 for the Deployment and coredns/coredns:1.3.1. That was normal at the time, but it is historical now. Use apps/v1 and pin a CoreDNS image version you have tested.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
  namespace: external-dns
  labels:
    app.kubernetes.io/name: external-dns
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: external-dns
  template:
    metadata:
      labels:
        app.kubernetes.io/name: external-dns
    spec:
      containers:
        - name: coredns
          image: coredns/coredns:1.14.2
          imagePullPolicy: IfNotPresent
          args:
            - -conf
            - /etc/coredns/Corefile
          ports:
            - name: dns-udp
              containerPort: 53
              protocol: UDP
            - name: dns-tcp
              containerPort: 53
              protocol: TCP
            - name: metrics
              containerPort: 9153
              protocol: TCP
            - name: health
              containerPort: 8080
              protocol: TCP
            - name: ready
              containerPort: 8181
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /health
              port: health
            initialDelaySeconds: 10
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /ready
              port: ready
            initialDelaySeconds: 5
            periodSeconds: 10
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              add:
                - NET_BIND_SERVICE
              drop:
                - ALL
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              memory: 128Mi
          volumeMounts:
            - name: config
              mountPath: /etc/coredns
              readOnly: true
      volumes:
        - name: config
          configMap:
            name: external-dns
            items:
              - key: Corefile
                path: Corefile
              - key: cluster.home.arpa
                path: zones/cluster.home.arpa

Expose DNS Through MetalLB

Current Kubernetes supports a LoadBalancer service with both TCP and UDP ports through the MixedProtocolLBService feature. That means you can expose DNS with one service instead of the two-service shared-IP workaround that was common in older examples.

apiVersion: v1
kind: Service
metadata:
  name: external-dns
  namespace: external-dns
  labels:
    app.kubernetes.io/name: external-dns
  annotations:
    metallb.io/loadBalancerIPs: "192.168.88.50"
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: external-dns
  ports:
    - name: dns-udp
      port: 53
      targetPort: dns-udp
      protocol: UDP
    - name: dns-tcp
      port: 53
      targetPort: dns-tcp
      protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: external-dns-metrics
  namespace: external-dns
  labels:
    app.kubernetes.io/name: external-dns
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "9153"
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: external-dns
  ports:
    - name: metrics
      port: 9153
      targetPort: metrics
      protocol: TCP

If you are on an older Kubernetes version that cannot use mixed protocols in one LoadBalancer service, create separate TCP and UDP services and give them the same address with MetalLB’s current shared-IP annotation:

metadata:
  annotations:
    metallb.io/allow-shared-ip: "external-dns"
    metallb.io/loadBalancerIPs: "192.168.88.50"

The old annotation prefix was metallb.universe.tf. Use metallb.io for current MetalLB.

Hand Out the DNS Server Through DHCP

Now configure your router or DHCP server to give clients 192.168.88.50 as their DNS server.

On a MikroTik router, that means updating the DNS server in the DHCP network settings for the subnet. On other routers it may be called “DNS server”, “name server” or “DHCP option 6”.

Reconnect a client to the network, renew its DHCP lease, or restart its network interface. Then check which resolver it received.

On Linux or macOS:

dig dashboard.cluster.home.arpa

You should see:

dashboard.cluster.home.arpa. 60 IN A 192.168.88.100

You can also query CoreDNS directly:

dig @192.168.88.50 dashboard.cluster.home.arpa
dig @192.168.88.50 kubernetes.io

The first command tests the local wildcard zone. The second command tests forwarding to the upstream resolver.

After that, create an Ingress for dashboard.cluster.home.arpa, point it at a service, and open http://dashboard.cluster.home.arpa from a machine on the LAN. DNS should resolve the name to Traefik, and Traefik should route the request inside the cluster.

What Aged Since 2019

The original version of this recipe depended on a few details that are no longer current:

  • extensions/v1beta1 Deployments are gone from current Kubernetes clusters.
  • .spec.loadBalancerIP was deprecated in Kubernetes v1.24 because its behavior varies across load balancer implementations.
  • MetalLB now uses the metallb.io annotation prefix.
  • MetalLB requires explicit IPAddressPool and advertisement resources.
  • CoreDNS examples should use forward, not the old proxy plugin.
  • The Cloudflare example is DNS-over-TLS, not DNS-over-HTTPS.
  • The MetalLB shared-IP issue mentioned in the original post was fixed in 2019.

The useful idea survived those changes. For a bare-metal home lab, CoreDNS plus MetalLB is still a tidy way to give the whole LAN stable names for services in the cluster. Just keep the DNS responsibility clear: CoreDNS is authoritative for your internal zone, and everything else should forward somewhere that will not loop back into itself.

References

Share

You may also like...

9 Responses

  1. amir says:

    Thank you so much, this is a great article and it helped so much.
    Can I know how I can support more than one domain like example2.com, example3.com… ?
    Thanks!

    • Sergey Anisimov says:

      Hi Amir, thank you for your feedback.
      In my example above I’m resolving all sub-domains like foo.example.com, bar.example.com etc. to the same Traefik’s IP – 192.168.88.100. Traefik knows how to route traffic for different sub-domains to different services. If you need to support different high level domains (e.g. example2.com, example3.com), you just need to extend your CoreDNS configuration with additional records, similar to the one above. You can resolve additional domains to the same Traefik and it can handle the rest. I hope it make sense 🙂

  2. amir says:

    Thanks for fast reply! Actually I have tried with CoreDNS config but not successful. Is there any reference or example?
    By the way after applying new configuration even I rollback to the configuration with one address the server doesn’t work . I have to change the server IP for external-dns LoadBalancer to another IP to make it work!!

    Thanks again!

  3. Jose says:

    I have been trying to implement this with k3s on some raspberry pis but I somehow keep getting coredns errors with “read udp .. i/o timeout”. Have you had to update this since creation?

    • Sergey Anisimov says:

      Hi Jose,

      The last time I had this solution running, that was three weeks ago, it was working fine. But I haven’t updated k8s or any of the services for quite some time. I’m planning to update shortly and see if I get any issues.

  4. Joe says:

    Hi
    I’m just trying to experience with metallb and bgp using my mikrotik hap ac as well!
    However, Im suffering from routing issues and I don’t understand why the routing itself won’t work.
    I configured the correct AS values both in metallb and setup the peer in mikrotik, and I see it connection established.
    I even expose a service over some range, and I see my mikrotik creates the route (in ip routes) with distance 20.
    When I try to navigate to the service it ranges from being able to load it (happened at the beginning for once or twice) but took around 5seconds to load, to not being able to load at all. Im not sure what is the problem or how to debug it at this point honestly.

    Can yo u share your experience and configs related to this specific area?

  5. ll says:

    thank you for this article. It works. Although I had to change the apiVersion from extensions/v1beta1 to apps/v1 for the deployment of coredns: https://github.com/moikot/bare-metal-k8s/blob/master/coredns/deployment.yaml

    Rest works flawlessly

Leave a Reply

Your email address will not be published. Required fields are marked *