diff --git a/README.md b/README.md index fcdc59579..dc5027711 100644 --- a/README.md +++ b/README.md @@ -22,158 +22,6 @@ The Ingress resource embodies this idea, and an Ingress controller is meant to h An Ingress Controller is a daemon, deployed as a Kubernetes Pod, that watches the apiserver's `/ingresses` endpoint for updates to the [Ingress resource](https://kubernetes.io/docs/concepts/services-networking/ingress/). Its job is to satisfy requests for Ingresses. -## Contents +## Documentation -- [Conventions](#conventions) -- [Requirements](#requirements) -- [Deployment](deploy/README.md) -- [Command line arguments](docs/user-guide/cli-arguments.md) -- [Contribute](CONTRIBUTING.md) -- [TLS](docs/user-guide/tls.md) -- [Annotation ingress.class](#annotation-ingressclass) -- [Customizing NGINX](#customizing-nginx) - - [Custom NGINX configuration](docs/user-guide/configmap.md) - - [Annotations](docs/user-guide/annotations.md) -- [Source IP address](#source-ip-address) -- [Exposing TCP and UDP Services](docs/user-guide/exposing-tcp-udp-services.md) -- [Proxy Protocol](#proxy-protocol) -- [ModSecurity Web Application Firewall](docs/user-guide/modsecurity.md) -- [OpenTracing](docs/user-guide/opentracing.md) -- [VTS and Prometheus metrics](docs/examples/customization/custom-vts-metrics-prometheus/README.md) -- [Custom errors](docs/user-guide/custom-errors.md) -- [NGINX status page](docs/user-guide/nginx-status-page.md) -- [Running multiple ingress controllers](#running-multiple-ingress-controllers) -- [Disabling NGINX ingress controller](#disabling-nginx-ingress-controller) -- [Retries in non-idempotent methods](#retries-in-non-idempotent-methods) -- [Log format](docs/user-guide/log-format.md) -- [Websockets](#websockets) -- [Optimizing TLS Time To First Byte (TTTFB)](#optimizing-tls-time-to-first-byte-tttfb) -- [Debug & Troubleshooting](docs/troubleshooting.md) -- [Limitations](#limitations) -- [Why endpoints and not services?](#why-endpoints-and-not-services) -- [External Articles](docs/user-guide/external-articles.md) - -## Conventions - -Anytime we reference a tls secret, we mean (x509, pem encoded, RSA 2048, etc). You can generate such a certificate with: -`openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ${KEY_FILE} -out ${CERT_FILE} -subj "/CN=${HOST}/O=${HOST}"` -and create the secret via `kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}` - -## Requirements - -The default backend is a service which handles all url paths and hosts the nginx controller doesn't understand (i.e., all the requests that are not mapped with an Ingress). -Basically a default backend exposes two URLs: - -- `/healthz` that returns 200 -- `/` that returns 404 - -The sub-directory [`/images/404-server`](https://github.com/kubernetes/ingress-nginx/tree/master/images/404-server) provides a service which satisfies the requirements for a default backend. The sub-directory [`/images/custom-error-pages`](https://github.com/kubernetes/ingress-nginx/tree/master/images/custom-error-pages) provides an additional service for the purpose of customizing the error pages served via the default backend. - -## Annotation ingress.class - -If you have multiple Ingress controllers in a single cluster, you can pick one by specifying the `ingress.class` -annotation, eg creating an Ingress with an annotation like - -```yaml -metadata: - name: foo - annotations: - kubernetes.io/ingress.class: "gce" -``` - -will target the GCE controller, forcing the nginx controller to ignore it, while an annotation like - -```yaml -metadata: - name: foo - annotations: - kubernetes.io/ingress.class: "nginx" -``` - -will target the nginx controller, forcing the GCE controller to ignore it. - -__Note__: Deploying multiple ingress controller and not specifying the annotation will result in both controllers fighting to satisfy the Ingress. - -### Customizing NGINX - -There are three ways to customize NGINX: - -1. [ConfigMap](docs/user-guide/configmap.md): using a Configmap to set global configurations in NGINX. -2. [Annotations](docs/user-guide/annotations.md): use this if you want a specific configuration for a particular Ingress rule. -3. [Custom template](docs/user-guide/custom-template.md): when more specific settings are required, like [open_file_cache](http://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache), adjust [listen](http://nginx.org/en/docs/http/ngx_http_core_module.html#listen) options as `rcvbuf` or when is not possible to change the configuration through the ConfigMap. - -## Source IP address - -By default NGINX uses the content of the header `X-Forwarded-For` as the source of truth to get information about the client IP address. This works without issues in L7 **if we configure the setting `proxy-real-ip-cidr`** with the correct information of the IP/network address of trusted external load balancer. - -If the ingress controller is running in AWS we need to use the VPC IPv4 CIDR. - -Another option is to enable proxy protocol using `use-proxy-protocol: "true"`. - -In this mode NGINX does not use the content of the header to get the source IP address of the connection. - -## Proxy Protocol - -If you are using a L4 proxy to forward the traffic to the NGINX pods and terminate HTTP/HTTPS there, you will lose the remote endpoint's IP address. To prevent this you could use the [Proxy Protocol](http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt) for forwarding traffic, this will send the connection details before forwarding the actual TCP connection itself. - -Amongst others [ELBs in AWS](http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html) and [HAProxy](http://www.haproxy.org/) support Proxy Protocol. - -### Running multiple ingress controllers - -If you're running multiple ingress controllers, or running on a cloud provider that natively handles ingress, you need to specify the annotation `kubernetes.io/ingress.class: "nginx"` in all ingresses that you would like this controller to claim. This mechanism also provides users the ability to run _multiple_ NGINX ingress controllers (e.g. one which serves public traffic, one which serves "internal" traffic). When utilizing this functionality the option `--ingress-class` should be changed to a value unique for the cluster within the definition of the replication controller. Here is a partial example: - -``` -spec: - template: - spec: - containers: - - name: nginx-ingress-internal-controller - args: - - /nginx-ingress-controller - - '--default-backend-service=ingress/nginx-ingress-default-backend' - - '--election-id=ingress-controller-leader-internal' - - '--ingress-class=nginx-internal' - - '--configmap=ingress/nginx-ingress-internal-controller' -``` - -Not specifying the annotation will lead to multiple ingress controllers claiming the same ingress. Specifying a value which does not match the class of any existing ingress controllers will result in all ingress controllers ignoring the ingress. - -The use of multiple ingress controllers in a single cluster is supported in Kubernetes versions >= 1.3. - -### Websockets - -Support for websockets is provided by NGINX out of the box. No special configuration required. - -The only requirement to avoid the close of connections is the increase of the values of `proxy-read-timeout` and `proxy-send-timeout`. - -The default value of this settings is `60 seconds`. - -A more adequate value to support websockets is a value higher than one hour (`3600`). - -**Important:** If the NGINX ingress controller is exposed with a service `type=LoadBalancer` make sure the protocol between the loadbalancer and NGINX is TCP. - -### Optimizing TLS Time To First Byte (TTTFB) - -NGINX provides the configuration option [ssl_buffer_size](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_buffer_size) to allow the optimization of the TLS record size. - -This improves the [TLS Time To First Byte](https://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-to-first-byte/) (TTTFB). -The default value in the Ingress controller is `4k` (NGINX default is `16k`). - -### Retries in non-idempotent methods - -Since 1.9.13 NGINX will not retry non-idempotent requests (POST, LOCK, PATCH) in case of an error. -The previous behavior can be restored using `retry-non-idempotent=true` in the configuration ConfigMap. - -### Disabling NGINX ingress controller - -Setting the annotation `kubernetes.io/ingress.class` to any other value which does not match a valid ingress class will force the NGINX Ingress controller to ignore your Ingress. If you are only running a single NGINX ingress controller, this can be achieved by setting this to any value except "nginx" or an empty string. - -Do this if you wish to use one of the other Ingress controllers at the same time as the NGINX controller. - -### Limitations - -- Ingress rules for TLS require the definition of the field `host` - -### Why endpoints and not services - -The NGINX ingress controller does not use [Services](http://kubernetes.io/docs/user-guide/services) to route traffic to the pods. Instead it uses the Endpoints API in order to bypass [kube-proxy](http://kubernetes.io/docs/admin/kube-proxy/) to allow NGINX features like session affinity and custom load balancing algorithms. It also removes some overhead, such as conntrack entries for iptables DNAT. +See [docs/index.md](docs/index.md) for detailed documentation. diff --git a/deploy/README.md b/deploy/README.md index 01c97bdfd..8048daa46 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -1,321 +1,3 @@ -# Installation Guide +# Deployment documentation moved! -## Contents - -- [Mandatory commands](#mandatory-commands) -- [Install without RBAC roles](#install-without-rbac-roles) -- [Install with RBAC roles](#install-with-rbac-roles) -- [Custom Provider](#custom-provider) - - [Docker for Mac](#docker-for-mac) - - [minikube](#minikube) - - [AWS](#aws) - - [GCE - GKE](#gce---gke) - - [Azure](#azure) - - [Baremetal](#baremetal) -- [Using Helm](#using-helm) -- [Verify installation](#verify-installation) -- [Detect installed version](#detect-installed-version) -- [Deploying the config-map](#deploying-the-config-map) - -## Generic Deployment - -The following resources are required for a generic deployment. - -### Mandatory commands - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/namespace.yaml \ - | kubectl apply -f - - -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml \ - | kubectl apply -f - - -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \ - | kubectl apply -f - - -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/tcp-services-configmap.yaml \ - | kubectl apply -f - - -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/udp-services-configmap.yaml \ - | kubectl apply -f - -``` - -### Install without RBAC roles - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/without-rbac.yaml \ - | kubectl apply -f - -``` - -### Install with RBAC roles - -Please check the [RBAC](rbac.md) document. - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/rbac.yaml \ - | kubectl apply -f - - -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/with-rbac.yaml \ - | kubectl apply -f - -``` - -## Custom Service Provider Deployment - -There are cloud provider specific yaml files. - -### Docker for Mac - -Kubernetes is available for Docker for Mac's Edge channel. Switch to the [Edge -channel][edge] and [enable Kubernetes][enable]. - -[edge]: https://docs.docker.com/docker-for-mac/install/ -[enable]: https://docs.docker.com/docker-for-mac/#kubernetes - -Patch the nginx ingress controller deployment to add the flag `--publish-service` - -```console -kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type='json' \ - --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/publish-service-patch.yaml)" -``` - -Create a service - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/docker-for-mac/service.yaml \ - | kubectl apply -f - -``` - -### minikube - -For standard usage: - -```console -minikube addons enable ingress -``` - -For development: - -1. Disable the ingress addon: - -```console -$ minikube addons disable ingress -``` - -2. Use the [docker daemon](https://github.com/kubernetes/minikube/blob/master/docs/reusing_the_docker_daemon.md) -3. [Build the image](../docs/development.md) -4. Perform [Mandatory commands](#mandatory-commands) -5. Install the `nginx-ingress-controller` deployment [without RBAC roles](#install-without-rbac-roles) or [with RBAC roles](#install-with-rbac-roles) -6. Edit the `nginx-ingress-controller` deployment to use your custom image. Local images can be seen by performing `docker images`. - -```console -$ kubectl edit deployment nginx-ingress-controller -n ingress-nginx -``` - -edit the following section: - -```yaml -image: : -imagePullPolicy: IfNotPresent -name: nginx-ingress-controller -``` - -7. Confirm the `nginx-ingress-controller` deployment exists: - -```console -$ kubectl get pods -n ingress-nginx -NAME READY STATUS RESTARTS AGE -default-http-backend-66b447d9cf-rrlf9 1/1 Running 0 12s -nginx-ingress-controller-fdcdcd6dd-vvpgs 1/1 Running 0 11s -``` - -### AWS - -In AWS we use an Elastic Load Balancer (ELB) to expose the NGINX Ingress controller behind a Service of `Type=LoadBalancer`. -Since Kubernetes v1.9.0 it is possible to use a classic load balancer (ELB) or network load balancer (NLB) -Please check the [elastic load balancing AWS details page](https://aws.amazon.com/es/elasticloadbalancing/details/) - -#### Elastic Load Balancer - ELB - -This setup requires to choose in which layer (L4 or L7) we want to configure the ELB: - -- [Layer 4](https://en.wikipedia.org/wiki/OSI_model#Layer_4:_Transport_Layer): use TCP as the listener protocol for ports 80 and 443. -- [Layer 7](https://en.wikipedia.org/wiki/OSI_model#Layer_7:_Application_Layer): use HTTP as the listener protocol for port 80 and terminate TLS in the ELB - -Patch the nginx ingress controller deployment to add the flag `--publish-service` - -```console -kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type='json' \ - --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/publish-service-patch.yaml)" -``` - -For L4: - -```console -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/service-l4.yaml -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/patch-configmap-l4.yaml -``` - -For L7: - -Change line of the file `provider/aws/service-l7.yaml` replacing the dummy id with a valid one `"arn:aws:acm:us-west-2:XXXXXXXX:certificate/XXXXXX-XXXXXXX-XXXXXXX-XXXXXXXX"` -Then execute: - -```console -kubectl apply -f provider/aws/service-l7.yaml -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/patch-configmap-l7.yaml -``` - -This example creates an ELB with just two listeners, one in port 80 and another in port 443 - -![Listeners](../docs/images/elb-l7-listener.png) - -If the ingress controller uses RBAC run: - -```console -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-with-rbac.yaml -``` - -If not run: - -```console -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-without-rbac.yaml -``` - -#### Network Load Balancer (NLB) - -This type of load balancer is supported since v1.10.0 as an ALPHA feature. - -```console -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/service-nlb.yaml -``` - -If the ingress controller uses RBAC run: - -```console -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-with-rbac.yaml -``` - -If not run: - -```console -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-without-rbac.yaml -``` - -### GCE - GKE - -Patch the nginx ingress controller deployment to add the flag `--publish-service` - -```console -kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type='json' \ - --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/publish-service-patch.yaml)" -``` - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/gce-gke/service.yaml \ - | kubectl apply -f - -``` - -If the ingress controller uses RBAC run: - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-with-rbac.yaml | kubectl apply -f - -``` - -If not run: - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-without-rbac.yaml | kubectl apply -f - -``` - -**Important Note:** proxy protocol is not supported in GCE/GKE - -### Azure - -Patch the nginx ingress controller deployment to add the flag `--publish-service` - -```console -kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type='json' \ - --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/publish-service-patch.yaml)" -``` - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/azure/service.yaml \ - | kubectl apply -f - -``` - -If the ingress controller uses RBAC run: - -```console -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-with-rbac.yaml -``` - -If not run: - -```console -kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-without-rbac.yaml -``` - -**Important Note:** proxy protocol is not supported in GCE/GKE - -### Baremetal - -Using [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport): - -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml \ - | kubectl apply -f - -``` - -## Using Helm - -NGINX Ingress controller can be installed via [Helm](https://helm.sh/) using the chart [stable/nginx](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) from the official charts repository. -To install the chart with the release name `my-nginx`: - -```console -helm install stable/nginx-ingress --name my-nginx -``` - -If the kubernetes cluster has RBAC enabled, then run: - -```console -helm install stable/nginx-ingress --name my-nginx --set rbac.create=true -``` - -## Verify installation - -To check if the ingress controller pods have started, run the following command: - -```console -kubectl get pods --all-namespaces -l app=ingress-nginx --watch -``` - -Once the operator pods are running, you can cancel the above command by typing `Ctrl+C`. -Now, you are ready to create your first ingress. - -## Detect installed version - -To detect which version of the ingress controller is running, exec into the pod and run `nginx-ingress-controller version` command. - -```console -POD_NAMESPACE=ingress-nginx -POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app=ingress-nginx -o jsonpath={.items[0].metadata.name}) -kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version -``` - -## Deploying the config-map - -A config map can be used to configure system components for the nginx-controller. In order to begin using a config-map -make sure it has been created and is being used in the deployment. - -It is created as seen in the [Mandatory Commands](#mandatory-commands) section above. -```console -curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \ - | kubectl apply -f - -``` - -and is setup to be used in the deployment [without-rbac](without-rbac.yaml) or [with-rbac](with-rbac.yaml) with the following line: -```yaml -- --configmap=$(POD_NAMESPACE)/nginx-configuration -``` - -For information on using the config-map, see its [user-guide](../docs/user-guide/configmap.md). +See (/docs/deploy.md)[../docs/deploy.md]. \ No newline at end of file diff --git a/docs/deploy.md b/docs/deploy.md new file mode 100644 index 000000000..284f990eb --- /dev/null +++ b/docs/deploy.md @@ -0,0 +1,321 @@ +# Installation Guide + +## Contents + +- [Mandatory commands](#mandatory-commands) +- [Install without RBAC roles](#install-without-rbac-roles) +- [Install with RBAC roles](#install-with-rbac-roles) +- [Custom Provider](#custom-provider) + - [Docker for Mac](#docker-for-mac) + - [minikube](#minikube) + - [AWS](#aws) + - [GCE - GKE](#gce---gke) + - [Azure](#azure) + - [Baremetal](#baremetal) +- [Using Helm](#using-helm) +- [Verify installation](#verify-installation) +- [Detect installed version](#detect-installed-version) +- [Deploying the config-map](#deploying-the-config-map) + +## Generic Deployment + +The following resources are required for a generic deployment. + +### Mandatory commands + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/namespace.yaml \ + | kubectl apply -f - + +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml \ + | kubectl apply -f - + +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \ + | kubectl apply -f - + +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/tcp-services-configmap.yaml \ + | kubectl apply -f - + +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/udp-services-configmap.yaml \ + | kubectl apply -f - +``` + +### Install without RBAC roles + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/without-rbac.yaml \ + | kubectl apply -f - +``` + +### Install with RBAC roles + +Please check the [RBAC](rbac.md) document. + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/rbac.yaml \ + | kubectl apply -f - + +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/with-rbac.yaml \ + | kubectl apply -f - +``` + +## Custom Service Provider Deployment + +There are cloud provider specific yaml files. + +### Docker for Mac + +Kubernetes is available for Docker for Mac's Edge channel. Switch to the [Edge +channel][edge] and [enable Kubernetes][enable]. + +[edge]: https://docs.docker.com/docker-for-mac/install/ +[enable]: https://docs.docker.com/docker-for-mac/#kubernetes + +Patch the nginx ingress controller deployment to add the flag `--publish-service` + +```console +kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type='json' \ + --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/publish-service-patch.yaml)" +``` + +Create a service + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/docker-for-mac/service.yaml \ + | kubectl apply -f - +``` + +### minikube + +For standard usage: + +```console +minikube addons enable ingress +``` + +For development: + +1. Disable the ingress addon: + +```console +$ minikube addons disable ingress +``` + +2. Use the [docker daemon](https://github.com/kubernetes/minikube/blob/master/docs/reusing_the_docker_daemon.md) +3. [Build the image](./development.md) +4. Perform [Mandatory commands](#mandatory-commands) +5. Install the `nginx-ingress-controller` deployment [without RBAC roles](#install-without-rbac-roles) or [with RBAC roles](#install-with-rbac-roles) +6. Edit the `nginx-ingress-controller` deployment to use your custom image. Local images can be seen by performing `docker images`. + +```console +$ kubectl edit deployment nginx-ingress-controller -n ingress-nginx +``` + +edit the following section: + +```yaml +image: : +imagePullPolicy: IfNotPresent +name: nginx-ingress-controller +``` + +7. Confirm the `nginx-ingress-controller` deployment exists: + +```console +$ kubectl get pods -n ingress-nginx +NAME READY STATUS RESTARTS AGE +default-http-backend-66b447d9cf-rrlf9 1/1 Running 0 12s +nginx-ingress-controller-fdcdcd6dd-vvpgs 1/1 Running 0 11s +``` + +### AWS + +In AWS we use an Elastic Load Balancer (ELB) to expose the NGINX Ingress controller behind a Service of `Type=LoadBalancer`. +Since Kubernetes v1.9.0 it is possible to use a classic load balancer (ELB) or network load balancer (NLB) +Please check the [elastic load balancing AWS details page](https://aws.amazon.com/es/elasticloadbalancing/details/) + +#### Elastic Load Balancer - ELB + +This setup requires to choose in which layer (L4 or L7) we want to configure the ELB: + +- [Layer 4](https://en.wikipedia.org/wiki/OSI_model#Layer_4:_Transport_Layer): use TCP as the listener protocol for ports 80 and 443. +- [Layer 7](https://en.wikipedia.org/wiki/OSI_model#Layer_7:_Application_Layer): use HTTP as the listener protocol for port 80 and terminate TLS in the ELB + +Patch the nginx ingress controller deployment to add the flag `--publish-service` + +```console +kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type='json' \ + --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/publish-service-patch.yaml)" +``` + +For L4: + +```console +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/service-l4.yaml +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/patch-configmap-l4.yaml +``` + +For L7: + +Change line of the file `provider/aws/service-l7.yaml` replacing the dummy id with a valid one `"arn:aws:acm:us-west-2:XXXXXXXX:certificate/XXXXXX-XXXXXXX-XXXXXXX-XXXXXXXX"` +Then execute: + +```console +kubectl apply -f provider/aws/service-l7.yaml +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/patch-configmap-l7.yaml +``` + +This example creates an ELB with just two listeners, one in port 80 and another in port 443 + +![Listeners](../docs/images/elb-l7-listener.png) + +If the ingress controller uses RBAC run: + +```console +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-with-rbac.yaml +``` + +If not run: + +```console +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-without-rbac.yaml +``` + +#### Network Load Balancer (NLB) + +This type of load balancer is supported since v1.10.0 as an ALPHA feature. + +```console +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/aws/service-nlb.yaml +``` + +If the ingress controller uses RBAC run: + +```console +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-with-rbac.yaml +``` + +If not run: + +```console +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-without-rbac.yaml +``` + +### GCE - GKE + +Patch the nginx ingress controller deployment to add the flag `--publish-service` + +```console +kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type='json' \ + --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/publish-service-patch.yaml)" +``` + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/gce-gke/service.yaml \ + | kubectl apply -f - +``` + +If the ingress controller uses RBAC run: + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-with-rbac.yaml | kubectl apply -f - +``` + +If not run: + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-without-rbac.yaml | kubectl apply -f - +``` + +**Important Note:** proxy protocol is not supported in GCE/GKE + +### Azure + +Patch the nginx ingress controller deployment to add the flag `--publish-service` + +```console +kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type='json' \ + --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/publish-service-patch.yaml)" +``` + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/azure/service.yaml \ + | kubectl apply -f - +``` + +If the ingress controller uses RBAC run: + +```console +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-with-rbac.yaml +``` + +If not run: + +```console +kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/patch-service-without-rbac.yaml +``` + +**Important Note:** proxy protocol is not supported in GCE/GKE + +### Baremetal + +Using [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport): + +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml \ + | kubectl apply -f - +``` + +## Using Helm + +NGINX Ingress controller can be installed via [Helm](https://helm.sh/) using the chart [stable/nginx](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) from the official charts repository. +To install the chart with the release name `my-nginx`: + +```console +helm install stable/nginx-ingress --name my-nginx +``` + +If the kubernetes cluster has RBAC enabled, then run: + +```console +helm install stable/nginx-ingress --name my-nginx --set rbac.create=true +``` + +## Verify installation + +To check if the ingress controller pods have started, run the following command: + +```console +kubectl get pods --all-namespaces -l app=ingress-nginx --watch +``` + +Once the operator pods are running, you can cancel the above command by typing `Ctrl+C`. +Now, you are ready to create your first ingress. + +## Detect installed version + +To detect which version of the ingress controller is running, exec into the pod and run `nginx-ingress-controller version` command. + +```console +POD_NAMESPACE=ingress-nginx +POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app=ingress-nginx -o jsonpath={.items[0].metadata.name}) +kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version +``` + +## Deploying the config-map + +A config map can be used to configure system components for the nginx-controller. In order to begin using a config-map +make sure it has been created and is being used in the deployment. + +It is created as seen in the [Mandatory Commands](#mandatory-commands) section above. +```console +curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \ + | kubectl apply -f - +``` + +and is setup to be used in the deployment [without-rbac](../deploy/without-rbac.yaml) or [with-rbac](../deploy/with-rbac.yaml) with the following line: +```yaml +- --configmap=$(POD_NAMESPACE)/nginx-configuration +``` + +For information on using the config-map, see its [user-guide](./user-guide/configmap.md). diff --git a/docs/development.md b/docs/development.md index c351e3953..a9a7e4deb 100644 --- a/docs/development.md +++ b/docs/development.md @@ -107,7 +107,7 @@ $ TAG= REGISTRY=$USER/ingress-controller make docker-push ## Deploying There are several ways to deploy the ingress controller onto a cluster. -Please check the [deployment guide](../deploy/README.md) +Please check the [deployment guide](./deploy.md) ## Testing diff --git a/docs/examples/rewrite/README.md b/docs/examples/rewrite/README.md index f67cd90d5..3ce607992 100644 --- a/docs/examples/rewrite/README.md +++ b/docs/examples/rewrite/README.md @@ -5,8 +5,8 @@ This example demonstrates how to use the Rewrite annotations ## Prerequisites You will need to make sure your Ingress targets exactly one Ingress -controller by specifying the [ingress.class annotation](/README.md#annotation-ingressclass), -and that you have an ingress controller [running](/deploy/README.md) in your cluster. +controller by specifying the [ingress.class annotation](../../index.md#annotation-ingressclass), +and that you have an ingress controller [running](../../deploy.md) in your cluster. ## Deployment diff --git a/docs/examples/static-ip/README.md b/docs/examples/static-ip/README.md index 130a2f88e..348997fc4 100644 --- a/docs/examples/static-ip/README.md +++ b/docs/examples/static-ip/README.md @@ -4,10 +4,10 @@ This example demonstrates how to assign a static-ip to an Ingress on through the ## Prerequisites -You need a [TLS cert](/docs/examples/PREREQUISITES.md#tls-certificates) and a [test HTTP service](/docs/examples/PREREQUISITES.md#test-http-service) for this example. +You need a [TLS cert](../PREREQUISITES.md#tls-certificates) and a [test HTTP service](../PREREQUISITES.md#test-http-service) for this example. You will also need to make sure your Ingress targets exactly one Ingress -controller by specifying the [ingress.class annotation](/README.md#annotation-ingressclass), -and that you have an ingress controller [running](/deploy/README.md) in your cluster. +controller by specifying the [ingress.class annotation](../../index.md#annotation-ingressclass), +and that you have an ingress controller [running](../../deploy.md) in your cluster. ## Acquiring an IP diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 000000000..5e313e587 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,154 @@ +## Contents + +- [Conventions](#conventions) +- [Requirements](#requirements) +- [Deployment](./deploy.md) +- [Command line arguments](./user-guide/cli-arguments.md) +- [TLS](./user-guide/tls.md) +- [Annotation ingress.class](#annotation-ingressclass) +- [Customizing NGINX](#customizing-nginx) + - [Custom NGINX configuration](./user-guide/configmap.md) + - [Annotations](./user-guide/annotations.md) +- [Source IP address](#source-ip-address) +- [Exposing TCP and UDP Services](./user-guide/exposing-tcp-udp-services.md) +- [Proxy Protocol](#proxy-protocol) +- [ModSecurity Web Application Firewall](./user-guide/modsecurity.md) +- [OpenTracing](./user-guide/opentracing.md) +- [VTS and Prometheus metrics](./examples/customization/custom-vts-metrics-prometheus/README.md) +- [Custom errors](./user-guide/custom-errors.md) +- [NGINX status page](./user-guide/nginx-status-page.md) +- [Running multiple ingress controllers](#running-multiple-ingress-controllers) +- [Disabling NGINX ingress controller](#disabling-nginx-ingress-controller) +- [Retries in non-idempotent methods](#retries-in-non-idempotent-methods) +- [Log format](./user-guide/log-format.md) +- [Websockets](#websockets) +- [Optimizing TLS Time To First Byte (TTTFB)](#optimizing-tls-time-to-first-byte-tttfb) +- [Debug & Troubleshooting](./troubleshooting.md) +- [Limitations](#limitations) +- [Why endpoints and not services?](#why-endpoints-and-not-services) +- [External Articles](./user-guide/external-articles.md) + +## Conventions + +Anytime we reference a tls secret, we mean (x509, pem encoded, RSA 2048, etc). You can generate such a certificate with: +`openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ${KEY_FILE} -out ${CERT_FILE} -subj "/CN=${HOST}/O=${HOST}"` +and create the secret via `kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}` + +## Requirements + +The default backend is a service which handles all url paths and hosts the nginx controller doesn't understand (i.e., all the requests that are not mapped with an Ingress). +Basically a default backend exposes two URLs: + +- `/healthz` that returns 200 +- `/` that returns 404 + +The sub-directory [`/images/404-server`](https://github.com/kubernetes/ingress-nginx/tree/master/images/404-server) provides a service which satisfies the requirements for a default backend. The sub-directory [`/images/custom-error-pages`](https://github.com/kubernetes/ingress-nginx/tree/master/images/custom-error-pages) provides an additional service for the purpose of customizing the error pages served via the default backend. + +## Annotation ingress.class + +If you have multiple Ingress controllers in a single cluster, you can pick one by specifying the `ingress.class` +annotation, eg creating an Ingress with an annotation like + +```yaml +metadata: + name: foo + annotations: + kubernetes.io/ingress.class: "gce" +``` + +will target the GCE controller, forcing the nginx controller to ignore it, while an annotation like + +```yaml +metadata: + name: foo + annotations: + kubernetes.io/ingress.class: "nginx" +``` + +will target the nginx controller, forcing the GCE controller to ignore it. + +__Note__: Deploying multiple ingress controller and not specifying the annotation will result in both controllers fighting to satisfy the Ingress. + +### Customizing NGINX + +There are three ways to customize NGINX: + +1. [ConfigMap](./user-guide/configmap.md): using a Configmap to set global configurations in NGINX. +2. [Annotations](./user-guide/annotations.md): use this if you want a specific configuration for a particular Ingress rule. +3. [Custom template](./user-guide/custom-template.md): when more specific settings are required, like [open_file_cache](http://nginx.org/en/./http/ngx_http_core_module.html#open_file_cache), adjust [listen](http://nginx.org/en/./http/ngx_http_core_module.html#listen) options as `rcvbuf` or when is not possible to change the configuration through the ConfigMap. + +## Source IP address + +By default NGINX uses the content of the header `X-Forwarded-For` as the source of truth to get information about the client IP address. This works without issues in L7 **if we configure the setting `proxy-real-ip-cidr`** with the correct information of the IP/network address of trusted external load balancer. + +If the ingress controller is running in AWS we need to use the VPC IPv4 CIDR. + +Another option is to enable proxy protocol using `use-proxy-protocol: "true"`. + +In this mode NGINX does not use the content of the header to get the source IP address of the connection. + +## Proxy Protocol + +If you are using a L4 proxy to forward the traffic to the NGINX pods and terminate HTTP/HTTPS there, you will lose the remote endpoint's IP address. To prevent this you could use the [Proxy Protocol](http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt) for forwarding traffic, this will send the connection details before forwarding the actual TCP connection itself. + +Amongst others [ELBs in AWS](http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html) and [HAProxy](http://www.haproxy.org/) support Proxy Protocol. + +### Running multiple ingress controllers + +If you're running multiple ingress controllers, or running on a cloud provider that natively handles ingress, you need to specify the annotation `kubernetes.io/ingress.class: "nginx"` in all ingresses that you would like this controller to claim. This mechanism also provides users the ability to run _multiple_ NGINX ingress controllers (e.g. one which serves public traffic, one which serves "internal" traffic). When utilizing this functionality the option `--ingress-class` should be changed to a value unique for the cluster within the definition of the replication controller. Here is a partial example: + +``` +spec: + template: + spec: + containers: + - name: nginx-ingress-internal-controller + args: + - /nginx-ingress-controller + - '--default-backend-service=ingress/nginx-ingress-default-backend' + - '--election-id=ingress-controller-leader-internal' + - '--ingress-class=nginx-internal' + - '--configmap=ingress/nginx-ingress-internal-controller' +``` + +Not specifying the annotation will lead to multiple ingress controllers claiming the same ingress. Specifying a value which does not match the class of any existing ingress controllers will result in all ingress controllers ignoring the ingress. + +The use of multiple ingress controllers in a single cluster is supported in Kubernetes versions >= 1.3. + +### Websockets + +Support for websockets is provided by NGINX out of the box. No special configuration required. + +The only requirement to avoid the close of connections is the increase of the values of `proxy-read-timeout` and `proxy-send-timeout`. + +The default value of this settings is `60 seconds`. + +A more adequate value to support websockets is a value higher than one hour (`3600`). + +**Important:** If the NGINX ingress controller is exposed with a service `type=LoadBalancer` make sure the protocol between the loadbalancer and NGINX is TCP. + +### Optimizing TLS Time To First Byte (TTTFB) + +NGINX provides the configuration option [ssl_buffer_size](http://nginx.org/en/./http/ngx_http_ssl_module.html#ssl_buffer_size) to allow the optimization of the TLS record size. + +This improves the [TLS Time To First Byte](https://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-to-first-byte/) (TTTFB). +The default value in the Ingress controller is `4k` (NGINX default is `16k`). + +### Retries in non-idempotent methods + +Since 1.9.13 NGINX will not retry non-idempotent requests (POST, LOCK, PATCH) in case of an error. +The previous behavior can be restored using `retry-non-idempotent=true` in the configuration ConfigMap. + +### Disabling NGINX ingress controller + +Setting the annotation `kubernetes.io/ingress.class` to any other value which does not match a valid ingress class will force the NGINX Ingress controller to ignore your Ingress. If you are only running a single NGINX ingress controller, this can be achieved by setting this to any value except "nginx" or an empty string. + +Do this if you wish to use one of the other Ingress controllers at the same time as the NGINX controller. + +### Limitations + +- Ingress rules for TLS require the definition of the field `host` + +### Why endpoints and not services + +The NGINX ingress controller does not use [Services](http://kubernetes.io/./user-guide/services) to route traffic to the pods. Instead it uses the Endpoints API in order to bypass [kube-proxy](http://kubernetes.io/./admin/kube-proxy/) to allow NGINX features like session affinity and custom load balancing algorithms. It also removes some overhead, such as conntrack entries for iptables DNAT. diff --git a/deploy/rbac.md b/docs/rbac.md similarity index 100% rename from deploy/rbac.md rename to docs/rbac.md