From 9bfc11f90ef7a9157cdb3b5cf9f199fed5103b8c Mon Sep 17 00:00:00 2001
From: k8s-ci-robot The Example Please check the auth example. NGINX supports load balancing by client-server mapping based on consistent hashing for a given key. The key can contain text, variables or any combination thereof. This feature allows for request stickiness other than client IP or cookies. The ketama consistent hashing method will be used which ensures only a few keys would be remapped to different servers on upstream group changes. There is a special mode of upstream hashing called subset. In this mode, upstream servers are grouped into subsets, and stickiness works by mapping keys to a subset instead of individual upstream servers. Specific server is chosen uniformly at random from the selected sticky subset. It provides a balance between stickiness and load distribution. To enable consistent hashing for a backend: "subset" hashing can be enabled setting Please check the chashsubset example. This is similar to Note that This configuration setting allows you to control the value for host in the following statement: It is possible to enable Client Certificate Authentication using additional annotations in Ingress Rule. Client Certificate Authentication is applied per host and it is not possible to specify rules that differ for individual paths. To enable, add the annotation You can further customize client certificate authentication and behavior with these annotations: The following headers are sent to the upstream service according to the Example Please check the client-certs example. Attention TLS with Client Authentication is not possible in Cloudflare and might result in unexpected behavior. Cloudflare only allows Authenticated Origin Pulls and is required to use their own certificate: https://blog.cloudflare.com/protecting-the-origin-with-tls-authenticated-origin-pulls/ Only Authenticated Origin Pulls are allowed and can be configured by following their tutorial: https://support.cloudflare.com/hc/en-us/articles/204494148-Setting-up-NGINX-to-use-TLS-Authenticated-Origin-Pulls It is possible to authenticate to a proxied HTTPS backend with certificate using additional annotations in Ingress Rule. Using this annotation you can add additional configuration to the NGINX location. For example: Since version 1.9.0 this annotation is disabled by default and has to be explicitly enabled, see allow-snippet-annotations. Enabling it can be dangerous in multi-tenant clusters, as it can lead to people with otherwise limited permissions being able to retrieve all secrets on the cluster. See CVE-2021-25742 and the related issue on github for more information. Like the Example usage: Since version 1.9.0 this annotation is disabled by default and has to be explicitly enabled, see allow-snippet-annotations. Enabling it can be dangerous in multi-tenant clusters, as it can lead to people with otherwise limited permissions being able to retrieve all secrets on the cluster. See CVE-2021-25742 and the related issue on github for more information. Like the Example usage: auth-secret
can have two forms:
auth-file
- default, an htpasswd file in the key auth
within the secretauth-map
- the keys of the secret are the usernames, and the values are the hashed passwordsnginx.ingress.kubernetes.io/auth-realm: "realm string"
Custom NGINX upstream hashing ¶
nginx.ingress.kubernetes.io/upstream-hash-by
: the nginx variable, text value or any combination thereof to use for consistent hashing. For example: nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri"
or nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri$host"
or nginx.ingress.kubernetes.io/upstream-hash-by: "${request_uri}-text-value"
to consistently hash upstream requests by the current request URI.nginx.ingress.kubernetes.io/upstream-hash-by-subset
: "true". This maps requests to subset of nodes instead of a single one. nginx.ingress.kubernetes.io/upstream-hash-by-subset-size
determines the size of each subset (default 3).Custom NGINX load balancing ¶
load-balance
in ConfigMap, but configures load balancing algorithm per ingress.
nginx.ingress.kubernetes.io/upstream-hash-by
takes preference over this. If this and nginx.ingress.kubernetes.io/upstream-hash-by
are not set then we fallback to using globally configured load balancing algorithm.Custom NGINX upstream vhost ¶
proxy_set_header Host $host
, which forms part of the location block. This is useful if you need to call the upstream server by something other than $host
.Client Certificate Authentication ¶
nginx.ingress.kubernetes.io/auth-tls-secret: namespace/secretName
. This secret must have a file named ca.crt
containing the full Certificate Authority chain ca.crt
that is enabled to authenticate against this Ingress.
nginx.ingress.kubernetes.io/auth-tls-verify-depth
: The validation depth between the provided client certificate and the Certification Authority chain. (default: 1)nginx.ingress.kubernetes.io/auth-tls-verify-client
: Enables verification of client certificates. Possible values are:
on
: Request a client certificate that must be signed by a certificate that is included in the secret key ca.crt
of the secret specified by nginx.ingress.kubernetes.io/auth-tls-secret: namespace/secretName
. Failed certificate verification will result in a status code 400 (Bad Request) (default)off
: Don't request client certificates and don't do client certificate verification.optional
: Do optional client certificate validation against the CAs from auth-tls-secret
. The request fails with status code 400 (Bad Request) when a certificate is provided that is not signed by the CA. When no or an otherwise invalid certificate is provided, the request does not fail, but instead the verification result is sent to the upstream service.optional_no_ca
: Do optional client certificate validation, but do not fail the request when the client certificate is not signed by the CAs from auth-tls-secret
. Certificate verification result is sent to the upstream service.nginx.ingress.kubernetes.io/auth-tls-error-page
: The URL/Page that user should be redirected in case of a Certificate Authentication Errornginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream
: Indicates if the received certificates should be passed or not to the upstream server in the header ssl-client-cert
. Possible values are "true" or "false" (default).nginx.ingress.kubernetes.io/auth-tls-match-cn
: Adds a sanity check for the CN of the client certificate that is sent over using a string / regex starting with "CN=", example: "CN=myvalidclient"
. If the certificate CN sent during mTLS does not match your string / regex it will fail with status code 403. Another way of using this is by adding multiple options in your regex, example: "CN=(option1|option2|myvalidclient)"
. In this case, as long as one of the options in the brackets matches the certificate CN then you will receive a 200 status code. auth-tls-*
annotations:
ssl-client-issuer-dn
: The issuer information of the client certificate. Example: "CN=My CA"ssl-client-subject-dn
: The subject information of the client certificate. Example: "CN=My Client"ssl-client-verify
: The result of the client verification. Possible values: "SUCCESS", "FAILED: ssl-client-cert
: The full client certificate in PEM format. Will only be sent when nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream
is set to "true". Example: -----BEGIN%20CERTIFICATE-----%0A...---END%20CERTIFICATE-----%0A
Backend Certificate Authentication ¶
nginx.ingress.kubernetes.io/proxy-ssl-secret: secretName
: Specifies a Secret with the certificate tls.crt
, key tls.key
in PEM format used for authentication to a proxied HTTPS server. It should also contain trusted CA certificates ca.crt
in PEM format used to verify the certificate of the proxied HTTPS server. This annotation expects the Secret name in the form "namespace/secretName".nginx.ingress.kubernetes.io/proxy-ssl-verify
: Enables or disables verification of the proxied HTTPS server certificate. (default: off)nginx.ingress.kubernetes.io/proxy-ssl-verify-depth
: Sets the verification depth in the proxied HTTPS server certificates chain. (default: 1)nginx.ingress.kubernetes.io/proxy-ssl-ciphers
: Specifies the enabled ciphers for requests to a proxied HTTPS server. The ciphers are specified in the format understood by the OpenSSL library.nginx.ingress.kubernetes.io/proxy-ssl-name
: Allows to set proxy_ssl_name. This allows overriding the server name used to verify the certificate of the proxied HTTPS server. This value is also passed through SNI when a connection is established to the proxied HTTPS server.nginx.ingress.kubernetes.io/proxy-ssl-protocols
: Enables the specified protocols for requests to a proxied HTTPS server.nginx.ingress.kubernetes.io/proxy-ssl-server-name
: Enables passing of the server name through TLS Server Name Indication extension (SNI, RFC 6066) when establishing a connection with the proxied HTTPS server.Configuration snippet ¶
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Request-Id: $req_id";
-
Custom HTTP Errors ¶
custom-http-errors
value in the ConfigMap, this annotation will set NGINX proxy-intercept-errors
, but only for the NGINX location associated with this ingress. If a default backend annotation is specified on the ingress, the errors will be routed to that annotation's default backend service (instead of the global default backend). Different ingresses can specify different sets of error codes. Even if multiple ingress objects share the same hostname, this annotation can be used to intercept different error codes for each ingress (for example, different error codes to be intercepted for different paths on the same hostname, if each path is on a different ingress). If custom-http-errors
is also specified globally, the error values specified in this annotation will override the global value for the given ingress' hostname and path.nginx.ingress.kubernetes.io/custom-http-errors: "404,415"
+
Custom HTTP Errors ¶
custom-http-errors
value in the ConfigMap, this annotation will set NGINX proxy-intercept-errors
, but only for the NGINX location associated with this ingress. If a default backend annotation is specified on the ingress, the errors will be routed to that annotation's default backend service (instead of the global default backend). Different ingresses can specify different sets of error codes. Even if multiple ingress objects share the same hostname, this annotation can be used to intercept different error codes for each ingress (for example, different error codes to be intercepted for different paths on the same hostname, if each path is on a different ingress). If custom-http-errors
is also specified globally, the error values specified in this annotation will override the global value for the given ingress' hostname and path.nginx.ingress.kubernetes.io/custom-http-errors: "404,415"
This annotation is of the form nginx.ingress.kubernetes.io/default-backend: <svc name>
to specify a custom default backend. This <svc name>
is a reference to a service inside of the same namespace in which you are applying this annotation. This annotation overrides the global default backend. In case the service has multiple ports, the first one is the one which will receive the backend traffic.
This service will be used to handle the response when the configured service in the Ingress rule does not have any active endpoints. It will also be used to handle the error responses if both this annotation and the custom-http-errors annotation are set.
To enable Cross-Origin Resource Sharing (CORS) in an Ingress rule, add the annotation nginx.ingress.kubernetes.io/enable-cors: "true"
. This will add a section in the server location enabling this functionality.
CORS can be controlled with the following annotations:
nginx.ingress.kubernetes.io/cors-allow-methods
: Controls which methods are accepted.
This is a multi-valued field, separated by ',' and accepts only letters (upper and lower case).
GET, PUT, POST, DELETE, PATCH, OPTIONS
nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-headers
: Controls which headers are accepted.
This is a multi-valued field, separated by ',' and accepts letters, numbers, _ and -.
DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization
nginx.ingress.kubernetes.io/cors-allow-headers: "X-Forwarded-For, X-app123-XPTO"
nginx.ingress.kubernetes.io/cors-expose-headers
: Controls which headers are exposed to response.
This is a multi-valued field, separated by ',' and accepts letters, numbers, _, - and *.
nginx.ingress.kubernetes.io/cors-expose-headers: "*, X-CustomResponseHeader"
nginx.ingress.kubernetes.io/cors-allow-origin
: Controls what's the accepted Origin for CORS.
This is a multi-valued field, separated by ','. It must follow this format: http(s)://origin-site.com
or http(s)://origin-site.com:port
*
nginx.ingress.kubernetes.io/cors-allow-origin: "https://origin-site.com:4443, http://origin-site.com, https://example.org:1199"
It also supports single level wildcard subdomains and follows this format: http(s)://*.foo.bar
, http(s)://*.bar.foo:8080
or http(s)://*.abc.bar.foo:9000
- Example: nginx.ingress.kubernetes.io/cors-allow-origin: "https://*.origin-site.com:4443, http://*.origin-site.com, https://example.org:1199"
nginx.ingress.kubernetes.io/cors-allow-credentials
: Controls if credentials can be passed during CORS operations.
true
nginx.ingress.kubernetes.io/cors-allow-credentials: "false"
nginx.ingress.kubernetes.io/cors-max-age
: Controls how long preflight requests can be cached.
1728000
nginx.ingress.kubernetes.io/cors-max-age: 600
Note
For more information please see https://enable-cors.org
Enables automatic conversion of preload links specified in the “Link” response header fields into push requests.
Example
nginx.ingress.kubernetes.io/http2-push-preload: "true"
Allows the definition of one or more aliases in the server definition of the NGINX configuration using the annotation nginx.ingress.kubernetes.io/server-alias: "<alias 1>,<alias 2>"
. This will create a server with the same configuration, but adding new values to the server_name
directive.
Note
A server-alias name cannot conflict with the hostname of an existing server. If it does, the server-alias annotation will be ignored. If a server-alias is created and later a new server with the same hostname is created, the new server configuration will take place over the alias configuration.
For more information please see the server_name
documentation.
Using the annotation nginx.ingress.kubernetes.io/server-snippet
it is possible to add custom configuration in the server configuration block.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata: