Traefik & Kubernetes

The Kubernetes Ingress Controller, The Custom Resource Way.

Configuration Examples

Configuring KubernetesCRD and Deploying/Exposing Services
# All resources definition must be declared
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
      - ingressroutes
      - traefikservices
      - ingressroutetcps
      - tlsoptions
    verbs:
      - get
      - list
      - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: default
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik
  labels:
    app: traefik

spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      containers:
        - name: traefik
          image: traefik:v2.1
          args:
            - --log.level=DEBUG
            - --api
            - --api.insecure
            - --entrypoints.web.address=:80
            - --providers.kubernetescrd
          ports:
            - name: web
              containerPort: 80
            - name: admin
              containerPort: 8080

---
apiVersion: v1
kind: Service
metadata:
  name: traefik
spec:
  type: LoadBalancer
  selector:
    app: traefik
  ports:
    - protocol: TCP
      port: 80
      name: web
      targetPort: 80
    - protocol: TCP
      port: 8080
      name: admin
      targetPort: 8080
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: myingressroute
  namespace: default

spec:
  entryPoints:
    - web

  routes:
  - match: Host(`foo`) && PathPrefix(`/bar`)
    kind: Rule
    services:
    - name: whoami
      port: 80
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: whoami
  namespace: default
  labels:
    app: containous
    name: whoami

spec:
  replicas: 2
  selector:
    matchLabels:
      app: containous
      task: whoami
  template:
    metadata:
      labels:
        app: containous
        task: whoami
    spec:
      containers:
        - name: containouswhoami
          image: containous/whoami
          ports:
            - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: default

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: whoami

Routing Configuration

Custom Resource Definition (CRD)

  • You can find an exhaustive list, generated from Traefik's source code, of the custom resources and their attributes in the reference page.
  • Validate that the prerequisites are fulfilled before using the Traefik custom resources.
  • Traefik CRDs are building blocks that you can assemble according to your needs.

You can find an excerpt of the available custom resources in the table below:

Kind Purpose Concept Behind
IngressRoute HTTP Routing HTTP router
Middleware Tweaks the HTTP requests before they are sent to your service HTTP Middlewares
TraefikService Abstraction for HTTP loadbalancing/mirroring HTTP service
IngressRouteTCP TCP Routing TCP router
TLSOptions Allows to configure some parameters of the TLS connection TLSOptions

Kind: IngressRoute

IngressRoute is the CRD implementation of a Traefik HTTP router.

Register the IngressRoute kind in the Kubernetes cluster before creating IngressRoute objects.

IngressRoute Attributes

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: foo
  namespace: bar
spec:
  entryPoints:                      # [1]
    - foo
  routes:                           # [2]
  - kind: Rule
    match: Host(`test.domain.com`)  # [3]
    priority: 10                    # [4]
    middlewares:                    # [5]
    - name: middleware1             # [6]
      namespace: default            # [7]
    services:                       # [8]
    - kind: Service
      name: foo
      namespace: default
      passHostHeader: true
      port: 80
      responseForwarding:
        flushInterval: 1ms
      scheme: https
      sticky:
        cookie:
          httpOnly: true
          name: cookie
          secure: true
      strategy: RoundRobin
      weight: 10
  tls:                              # [9]
    secretName: supersecret         # [10]
    options:                        # [11]
      name: opt                     # [12]
      namespace: default            # [13]
    certResolver: foo               # [14]
    domains:                        # [15]
    - main: foo.com                 # [16]
      sans:                         # [17]
      - a.foo.com
      - b.foo.com
Ref Attribute Purpose
[1] entryPoints List of entry points name
[2] routes List of route
[3] routes[n].match Defines the rule corresponding to an underlying router.
[4] routes[n].priority Disambiguate rules of the same length, for route matching
[5] routes[n].middlewares List of reference to Middleware
[6] middlewares[n].name Defines the Middleware name
[7] middlewares[n].namespace Defines the Middleware namespace
[8] routes[n].services List of any combination of TraefikService and reference to a Kubernetes service
[9] tls Defines TLS certificate configuration
[10] tls.secretName Defines the secret name used to store the certificate (in the IngressRoute namespace)
[11] tls.options Defines the reference to a TLSOption
[12] options.name Defines the TLSOption name
[13] options.namespace Defines the TLSOption namespace
[14] tls.certResolver Defines the reference to a CertResolver
[15] tls.domains List of domains
[16] domains[n].main Defines the main domain name
[17] domains[n].sans List of SANs (alternative domains)
Declaring an IngressRoute
# All resources definition must be declared
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: testName
  namespace: default
spec:
  entryPoints:
    - web
  routes:
  - kind: Rule
    match: Host(`test.domain.com`)
    middlewares:
    - name: middleware1
      namespace: default
    priority: 10
    services:
    - kind: Service
      name: foo
      namespace: default
      passHostHeader: true
      port: 80
      responseForwarding:
        flushInterval: 1ms
      scheme: https
      sticky:
        cookie:
          httpOnly: true
          name: cookie
          secure: true
      strategy: RoundRobin
      weight: 10
  tls:
    certResolver: foo
    domains:
    - main: foo.com
      sans:
      - a.foo.com
      - b.foo.com
    options:
      name: opt
      namespace: default
    secretName: supersecret
# All resources definition must be declared
# Prefixing with /foo
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: middleware1
  namespace: default
spec:
  addPrefix:
    prefix: /foo
apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
  name: opt
  namespace: default

spec:
  minVersion: VersionTLS12
apiVersion: v1
kind: Secret
metadata:
  name: supersecret

data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0=

Kind: Middleware

Middleware is the CRD implementation of a Traefik middleware.

Register the Middleware kind in the Kubernetes cluster before creating Middleware objects or referencing middlewares in the IngressRoute objects.

Declaring and Referencing a Middleware
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: stripprefix
  namespace: foo

spec:
  stripPrefix:
    prefixes:
      - /stripit
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutebar

spec:
  entryPoints:
    - web
  routes:
  - match: Host(`bar.com`) && PathPrefix(`/stripit`)
    kind: Rule
    services:
    - name: whoami
      port: 80
    middlewares:
    - name: stripprefix
      namespace: foo

Cross-provider namespace

As Kubernetes also has its own notion of namespace, one should not confuse the kubernetes namespace of a resource (in the reference to the middleware) with the provider namespace, when the definition of the middleware comes from another provider. In this context, specifying a namespace when referring to the resource does not make any sense, and will be ignored.

More information about available middlewares in the dedicated middlewares section.

Kind: TraefikService

TraefikService is the CRD implementation of a "Traefik Service".

Register the TraefikService kind in the Kubernetes cluster before creating TraefikService objects, referencing services in the IngressRoute/IngressRouteTCP objects or recursively in others TraefikService objects.

Disambiguate Traefik and Kubernetes Services

As the field name can reference different types of objects, use the field kind to avoid any ambiguity.

The field kind allows the following values:

TraefikService object allows to use any (valid) combinations of:

Server Load Balancing

More information in the dedicated server load balancing section.

Declaring and Using Server Load Balancing
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutebar
  namespace: default

spec:
  entryPoints:
    - web
  routes:
  - match: Host(`bar.com`) && PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: svc1
      namespace: default
    - name: svc2
      namespace: default
apiVersion: v1
kind: Service
metadata:
  name: svc1
  namespace: default

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: app1
---
apiVersion: v1
kind: Service
metadata:
  name: svc2
  namespace: default

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: app2

Weighted Round Robin

More information in the dedicated Weighted Round Robin service load balancing section.

Declaring and Using Weighted Round Robin
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutebar
  namespace: default

spec:
  entryPoints:
    - web
  routes:
  - match: Host(`bar.com`) && PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: wrr1
      namespace: default
      kind: TraefikService
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: wrr1
  namespace: default

spec:
  weighted:
    services:
      - name: svc1
        port: 80
        weight: 1
      - name: wrr2
        kind: TraefikService
        weight: 1
      - name: mirror1
        kind: TraefikService
        weight: 1

---
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: wrr2
  namespace: default

spec:
  weighted:
    services:
      - name: svc2
        port: 80
        weight: 1
      - name: svc3
        port: 80
        weight: 1
apiVersion: v1
kind: Service
metadata:
  name: svc1
  namespace: default

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: app1
---
apiVersion: v1
kind: Service
metadata:
  name: svc2
  namespace: default

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: app2
---
apiVersion: v1
kind: Service
metadata:
  name: svc3
  namespace: default

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: app3

Mirroring

More information in the dedicated mirroring service section.

Declaring and Using Mirroring
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutebar
  namespace: default

spec:
  entryPoints:
    - web
  routes:
  - match: Host(`bar.com`) && PathPrefix(`/foo`)
    kind: Rule
    services:
    - name: mirror1
      namespace: default
      kind: TraefikService
# Mirroring from a k8s Service
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: mirror1
  namespace: default

spec:
  mirroring:
    name: svc1
    port: 80
    mirrors:
      - name: svc2
        port: 80
        percent: 20
      - name: svc3
        kind: TraefikService
        percent: 20
# Mirroring from a Traefik Service
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: mirror1
  namespace: default

spec:
  mirroring:
    name: wrr1
    kind: TraefikService
     mirrors:
       - name: svc2
         port: 80
         percent: 20
       - name: svc3
         kind: TraefikService
         percent: 20
apiVersion: v1
kind: Service
metadata:
  name: svc1
  namespace: default

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: app1
---
apiVersion: v1
kind: Service
metadata:
  name: svc2
  namespace: default

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: containous
    task: app2

References and namespaces

If the optional namespace attribute is not set, the configuration will be applied with the namespace of the current resource.

Additionally, when the definition of the TraefikService is from another provider, the cross-provider syntax ([email protected]) should be used to refer to the TraefikService, just as in the middleware case.

Specifying a namespace attribute in this case would not make any sense, and will be ignored (except if the provider is kubernetescrd).

Kind IngressRouteTCP

IngressRouteTCP is the CRD implementation of a Traefik TCP router.

Register the IngressRouteTCP kind in the Kubernetes cluster before creating IngressRouteTCP objects.

IngressRouteTCP Attributes

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: ingressroutetcpfoo

spec:
  entryPoints:                  # [1]
    - footcp
  routes:                       # [2]
  - match: HostSNI(`*`)         # [3]
    services:                   # [4]
    - name: foo                 # [5]
      port: 8080                # [6]
      weight: 10                # [7]
      TerminationDelay: 400     # [8]
  tls:                          # [9]
    secretName: supersecret     # [10]
    options:                    # [11]
      name: opt                 # [12]
      namespace: default        # [13]
    certResolver: foo           # [14]
    domains:                    # [15]
    - main: foo.com             # [16]
      sans:                     # [17]
      - a.foo.com
      - b.foo.com
    passthrough: false          # [18]
Ref Attribute Purpose
[1] entryPoints List of entrypoints name
[2] routes List of route
[3] routes[n].match Defines the rule corresponding to an underlying router.
[4] routes[n].services List of any combination of TraefikService and reference to a Kubernetes service
[5] services[n].name Defines the name of a Kubernetes service
[6] services[n].port Defines the port of a Kubernetes service
[7] services[n].weight Defines the weight to apply to the server load balancing
[8] services[n].TerminationDelay corresponds to the deadline that the proxy sets, after one of its connected peers indicates it has closed the writing capability of its connection, to close the reading capability as well, hence fully terminating the connection.
It is a duration in milliseconds, defaulting to 100. A negative value means an infinite deadline (i.e. the reading capability is never closed).
[9] tls Defines TLS certificate configuration
[10] tls.secretName Defines the secret name used to store the certificate (in the IngressRoute namespace)
[11] tls.options Defines the reference to a TLSOption
[12] options.name Defines the TLSOption name
[13] options.namespace Defines the TLSOption namespace
[14] tls.certResolver Defines the reference to a CertResolver
[15] tls.domains List of domains
[16] domains[n].main Defines the main domain name
[17] domains[n].sans List of SANs (alternative domains)
[18] tls.passthrough If true, delegates the TLS termination to the backend
Declaring an IngressRouteTCP
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: ingressroutetcpfoo

spec:
  entryPoints:
    - footcp
  routes:
  # Match is the rule corresponding to an underlying router.
  - match: HostSNI(`*`)
    services:
    - name: foo
      port: 8080
      TerminationDelay: 400
      weight: 10
    - name: bar
      port: 8081
      TerminationDelay: 500
      weight: 10
  tls:
    certResolver: foo
    domains:
    - main: foo.com
      sans:
      - a.foo.com
      - b.foo.com
    options:
      name: opt
      namespace: default
    secretName: supersecret
    passthrough: false
apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
  name: opt
  namespace: default

spec:
  minVersion: VersionTLS12
apiVersion: v1
kind: Secret
metadata:
  name: supersecret

data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0=

Kind: TLSOption

TLSOption is the CRD implementation of a Traefik "TLS Option".

Register the TLSOption kind in the Kubernetes cluster before creating TLSOption objects or referencing TLS options in the IngressRoute / IngressRouteTCP objects.

TLSOption Attributes

apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
  name: mytlsoption
  namespace: default

spec:
  minVersion: VersionTLS12                      # [1]
  maxVersion: VersionTLS13                      # [1]
  curvePreferences:                             # [3]
    - CurveP521
    - CurveP384
  cipherSuites:                                 # [4]
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_RSA_WITH_AES_256_GCM_SHA384
  clientAuth:                                   # [5]
    secretNames:                                # [6]
      - secretCA1
      - secretCA2
    clientAuthType: VerifyClientCertIfGiven     # [7]
  sniStrict: true                               # [8]
Ref Attribute Purpose
[1] minVersion Defines the minimum TLS version that is acceptable
[2] maxVersion Defines the maximum TLS version that is acceptable
[3] cipherSuites list of supported cipher suites for TLS versions up to TLS 1.2
[4] curvePreferences List of the elliptic curves references that will be used in an ECDHE handshake, in preference order
[5] clientAuth determines the server's policy for TLS Client Authentication
[6] clientAuth.secretNames list of names of the referenced Kubernetes Secrets (in TLSOption namespace)
[7] clientAuth.clientAuthType defines the client authentication type to apply. The available values are: NoClientCert, RequestClientCert, VerifyClientCertIfGiven and RequireAndVerifyClientCert
[8] sniStrict if true, Traefik won't allow connections from clients connections that do not specify a server_name extension
Declaring and referencing a TLSOption
apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
  name: mytlsoption
  namespace: default

spec:
  minVersion: VersionTLS12
  sniStrict: true
  cipherSuites:
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_RSA_WITH_AES_256_GCM_SHA384
  clientAuth:
    secretNames:
      - secretCA1
      - secretCA2
    clientAuthType: VerifyClientCertIfGiven
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutebar

spec:
  entryPoints:
    - web
  routes:
  - match: Host(`bar.com`) && PathPrefix(`/stripit`)
    kind: Rule
    services:
    - name: whoami
      port: 80
  tls:
    options: 
      name: mytlsoption
      namespace: default
apiVersion: v1
kind: Secret
metadata:
  name: secretCA1
  namespace: default

data:
  tls.ca: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=

---
apiVersion: v1
kind: Secret
metadata:
  name: secretCA2
  namespace: default

data:
  tls.ca: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=

References and namespaces

If the optional namespace attribute is not set, the configuration will be applied with the namespace of the IngressRoute.

Additionally, when the definition of the TLS option is from another provider, the cross-provider syntax ([email protected]) should be used to refer to the TLS option, just as in the middleware case. Specifying a namespace attribute in this case would not make any sense, and will be ignored.

Further

Also see the full example with Let's Encrypt.