DevFW-CICD/ingress-nginx-helm
14
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge remote-tracking branch 'master/master' into refactor-template-headers

Browse source
This commit is contained in:
Manuel de Brito Fontes 2017-09-29 09:03:57 -03:00
commit 3ed6019f9f
33 changed files with 576 additions and 104 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

62
controllers/nginx/README.md
View file

@ -17,6 +17,7 @@ This is an nginx Ingress controller that uses [ConfigMap](https://kubernetes.io/
* [TCP Services](#exposing-tcp-services)
* [UDP Services](#exposing-udp-services)
* [Proxy Protocol](#proxy-protocol)
* [Opentracing](#opentracing)
* [NGINX customization](configuration.md)
* [Custom errors](#custom-errors)
* [NGINX status page](#nginx-status-page)
@ -334,8 +335,8 @@ version to fully support Kube-Lego is nginx Ingress controller 0.8.
## Exposing TCP services
Ingress does not support TCP services (yet). For this reason this Ingress controller uses the flag `--tcp-services-configmap` to point to an existing config map where the key is the external port to use and the value is `<namespace/service name>:<service port>:[PROXY]`
It is possible to use a number or the name of the port. The last field is optional. Adding `PROXY` in the last field we can enable Proxy Protocol in a TCP service.
Ingress does not support TCP services (yet). For this reason this Ingress controller uses the flag `--tcp-services-configmap` to point to an existing config map where the key is the external port to use and the value is `<namespace/service name>:<service port>:[PROXY]:[PROXY]`
It is possible to use a number or the name of the port. The two last fields are optional. Adding `PROXY` in either or both of the two last fields we can use Proxy Protocol decoding (listen) and/or encoding (proxy_pass) in a TCP service (https://www.nginx.com/resources/admin-guide/proxy-protocol/).
The next example shows how to expose the service `example-go` running in the namespace `default` in the port `8080` using the port `9000`
```
@ -378,14 +379,63 @@ Amongst others [ELBs in AWS](http://docs.aws.amazon.com/ElasticLoadBalancing/lat
Please check the [proxy-protocol](examples/proxy-protocol/) example
### Opentracing
Using the third party module [rnburn/nginx-opentracing](https://github.com/rnburn/nginx-opentracing) the NGINX ingress controller can configure NGINX to enable [OpenTracing](http://opentracing.io) instrumentation.
By default this feature is disabled.
To enable the instrumentation we just need to enable the instrumentation in the configuration configmap and set the host where we should send the traces.
In the [aledbf/zipkin-js-example](https://github.com/aledbf/zipkin-js-example) github repository is possible to see a dockerized version of zipkin-js-example with the required Kubernetes descriptors.
To install the example and the zipkin collector we just need to run:
```
$ kubectl create -f https://raw.githubusercontent.com/aledbf/zipkin-js-example/kubernetes/kubernetes/zipkin.yaml
$ kubectl create -f https://raw.githubusercontent.com/aledbf/zipkin-js-example/kubernetes/kubernetes/deployment.yaml
```
Also we need to configure the NGINX controller configmap with the required values:
```
apiVersion: v1
data:
enable-opentracing: "true"
zipkin-collector-host: zipkin.default.svc.cluster.local
kind: ConfigMap
metadata:
labels:
k8s-app: nginx-ingress-controller
name: nginx-custom-configuration
```
Using curl we can generate some traces:
```
$ curl -v http://$(minikube ip)/api -H 'Host: zipkin-js-example'
$ curl -v http://$(minikube ip)/api -H 'Host: zipkin-js-example'
```
In the zipkin inteface we can see the details:
![zipkin screenshot](docs/images/zipkin-demo.png "zipkin collector screenshot")
### Custom errors
In case of an error in a request the body of the response is obtained from the `default backend`. Each request to the default backend includes two headers:
- `X-Code` indicates the HTTP code
- `X-Format` the value of the `Accept` header
In case of an error in a request the body of the response is obtained from the `default backend`.
Each request to the default backend includes two headers:
Using this two headers is possible to use a custom backend service like [this one](https://github.com/aledbf/contrib/tree/nginx-debug-server/Ingress/images/nginx-error-server) that inspect each request and returns a custom error page with the format expected by the client. Please check the example [custom-errors](examples/custom-errors/README.md)
- `X-Code` indicates the HTTP code to be returned to the client.
- `X-Format` the value of the `Accept` header.
**Important:** the custom backend must return the correct HTTP status code to be returned. NGINX do not changes the reponse from the custom default backend.
Using this two headers is possible to use a custom backend service like [this one](https://github.com/kubernetes/ingress/tree/master/examples/customization/custom-errors/nginx) that inspect each request and returns a custom error page with the format expected by the client. Please check the example [custom-errors](examples/customization/custom-errors/nginx/README.md)
NGINX sends aditional headers that can be used to build custom response:
- X-Original-URI
- X-Namespace
- X-Ingress-Name
- X-Service-Name
### NGINX status page

15
controllers/nginx/configuration.md
View file

@ -135,14 +135,14 @@ Please check the [auth](/examples/auth/basic/nginx/README.md) example.
### Certificate Authentication
It's possible to enable Certificate based authentication using additional annotations in Ingress Rule.
It's possible to enable Certificate-Based Authentication (Mutual Authentication) using additional annotations in Ingress Rule.
The annotations are:
```
ingress.kubernetes.io/auth-tls-secret: secretName
```
The name of the secret that contains the full Certificate Authority chain that is enabled to authenticate against this ingress. It's composed of namespace/secretName
The name of the secret that contains the full Certificate Authority chain `ca.crt` that is enabled to authenticate against this ingress. It's composed of namespace/secretName.
```
ingress.kubernetes.io/auth-tls-verify-depth
@ -487,6 +487,17 @@ The default mime type list to compress is: `application/atom+xml application/jav
**bind-address:** Sets the addresses on which the server will accept requests instead of *. It should be noted that these addresses must exist in the runtime environment or the controller will crash loop.
**enable-opentracing:** enables the nginx Opentracing extension https://github.com/rnburn/nginx-opentracing
Default is "false"
**zipkin-collector-host:** specifies the host to use when uploading traces. It must be a valid URL
**zipkin-collector-port:** specifies the port to use when uploading traces
Default: 9411
**zipkin-service-name:** specifies the service name to use for any traces created
Default: nginx
### Default configuration options
The following table shows the options, the default value and a description.

BIN
controllers/nginx/docs/images/zipkin-demo.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 18 KiB

48
controllers/nginx/pkg/cmd/controller/nginx.go
View file

@ -35,6 +35,7 @@ import (
"github.com/spf13/pflag"
proxyproto "github.com/armon/go-proxyproto"
"github.com/ncabatoff/process-exporter/proc"
apiv1 "k8s.io/api/core/v1"
extensions "k8s.io/api/extensions/v1beta1"
@ -62,7 +63,7 @@ const (
var (
tmplPath = "/etc/nginx/template/nginx.tmpl"
cfgPath = "/etc/nginx/nginx.conf"
binary = "/usr/sbin/nginx"
nginxBinary = "/usr/sbin/nginx"
defIngressClass = "nginx"
)
@ -72,7 +73,7 @@ var (
func newNGINXController() *NGINXController {
ngx := os.Getenv("NGINX_BINARY")
if ngx == "" {
ngx = binary
ngx = nginxBinary
}
h, err := dns.GetSystemNameServers()
@ -200,7 +201,26 @@ NGINX master process died (%v): %v
break
}
conn.Close()
time.Sleep(1 * time.Second)
// kill nginx worker processes
fs, err := proc.NewFS("/proc")
procs, _ := fs.FS.AllProcs()
for _, p := range procs {
pn, err := p.Comm()
if err != nil {
glog.Errorf("unexpected error obtaining process information: %v", err)
continue
}
if pn == "nginx" {
osp, err := os.FindProcess(p.PID)
if err != nil {
glog.Errorf("unexpected error obtaining process information: %v", err)
continue
}
osp.Signal(syscall.SIGQUIT)
}
}
time.Sleep(100 * time.Millisecond)
}
// restart a new nginx master process if the controller
// is not being stopped
@ -710,6 +730,28 @@ func (n NGINXController) Check(_ *http.Request) error {
if res.StatusCode != 200 {
return fmt.Errorf("ingress controller is not healthy")
}
// check the nginx master process is running
fs, err := proc.NewFS("/proc")
if err != nil {
glog.Errorf("%v", err)
return err
}
f, err := ioutil.ReadFile("/run/nginx.pid")
if err != nil {
glog.Errorf("%v", err)
return err
}
pid, err := strconv.Atoi(strings.TrimRight(string(f), "\r\n"))
if err != nil {
return err
}
_, err = fs.NewProc(int(pid))
if err != nil {
glog.Errorf("%v", err)
return err
}
return nil
}

18
controllers/nginx/pkg/template/template.go
View file

@ -22,7 +22,6 @@ import (
"encoding/json"
"fmt"
"net"
"net/url"
"os"
"os/exec"
"strconv"
@ -151,7 +150,6 @@ var (
"serverConfig": func(all config.TemplateConfig, server *ingress.Server) interface{} {
return struct{ First, Second interface{} }{all, server}
},
"buildAuthSignURL": buildAuthSignURL,
"isValidClientBodyBufferSize": isValidClientBodyBufferSize,
"buildForwardedFor": buildForwardedFor,
"trustHTTPHeaders": trustHTTPHeaders,
@ -570,22 +568,6 @@ func buildNextUpstream(input interface{}) string {
return strings.Join(nextUpstreamCodes, " ")
}
func buildAuthSignURL(input interface{}) string {
s, ok := input.(string)
if !ok {
glog.Errorf("expected an 'string' type but %T was returned", input)
return ""
}
u, _ := url.Parse(s)
q := u.Query()
if len(q) == 0 {
return fmt.Sprintf("%v?rd=$request_uri", s)
}
return fmt.Sprintf("%v&rd=$request_uri", s)
}
// buildRandomUUID return a random string to be used in the template
func buildRandomUUID() string {
s := uuid.New()

18
controllers/nginx/pkg/template/template_test.go
View file

@ -310,24 +310,6 @@ func TestBuildResolvers(t *testing.T) {
}
}
func TestBuildAuthSignURL(t *testing.T) {
urlOne := "http://google.com"
validUrlOne := "http://google.com?rd=$request_uri"
urlTwo := "http://google.com?cat"
validUrlTwo := "http://google.com?cat&rd=$request_uri"
authSignURLOne := buildAuthSignURL(urlOne)
if authSignURLOne != validUrlOne {
t.Errorf("Expected '%v' but returned '%v'", validUrlOne, authSignURLOne)
}
authSignURLTwo := buildAuthSignURL(urlTwo)
if authSignURLTwo != validUrlTwo {
t.Errorf("Expected '%v' but returned '%v'", validUrlTwo, authSignURLTwo)
}
}
func TestBuildNextUpstream(t *testing.T) {
nextUpstream := "timeout http_500 http_502 non_idempotent"
validNextUpstream := "timeout http_500 http_502"

23
controllers/nginx/rootfs/etc/nginx/template/nginx.tmpl
View file

@ -461,19 +461,22 @@ stream {
}
server {
{{ range $address := $all.Cfg.BindAddressIpv4 }}
listen {{ $address }}:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.UseProxyProtocol }} proxy_protocol{{ end }};
listen {{ $address }}:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.ProxyProtocol.Decode }} proxy_protocol{{ end }};
{{ else }}
listen {{ $tcpServer.Port }}{{ if $tcpServer.Backend.UseProxyProtocol }} proxy_protocol{{ end }};
listen {{ $tcpServer.Port }}{{ if $tcpServer.Backend.ProxyProtocol.Decode }} proxy_protocol{{ end }};
{{ end }}
{{ if $IsIPV6Enabled }}
{{ range $address := $all.Cfg.BindAddressIpv6 }}
listen {{ $address }}:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.UseProxyProtocol }} proxy_protocol{{ end }};
listen {{ $address }}:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.ProxyProtocol.Decode }} proxy_protocol{{ end }};
{{ else }}
listen [::]:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.UseProxyProtocol }} proxy_protocol{{ end }};
listen [::]:{{ $tcpServer.Port }}{{ if $tcpServer.Backend.ProxyProtocol.Decode }} proxy_protocol{{ end }};
{{ end }}
{{ end }}
proxy_timeout {{ $cfg.ProxyStreamTimeout }};
proxy_pass tcp-{{ $tcpServer.Port }}-{{ $tcpServer.Backend.Namespace }}-{{ $tcpServer.Backend.Name }}-{{ $tcpServer.Backend.Port }};
{{ if $tcpServer.Backend.ProxyProtocol.Encode }}
proxy_protocol on;
{{ end }}
}
{{ end }}
@ -514,6 +517,8 @@ stream {
location @custom_{{ $errCode }} {
internal;
proxy_intercept_errors off;
proxy_set_header X-Code {{ $errCode }};
proxy_set_header X-Format $http_accept;
proxy_set_header X-Original-URI $request_uri;
@ -521,6 +526,7 @@ stream {
proxy_set_header X-Ingress-Name $ingress_name;
proxy_set_header X-Service-Name $service_name;
rewrite (.*) / break;
proxy_pass http://upstream-default-backend;
}
{{ end }}
@ -626,6 +632,10 @@ stream {
{{ end }}
{{ end }}
{{ if not (empty $server.ServerSnippet) }}
{{ $server.ServerSnippet }}
{{ end }}
{{ range $location := $server.Locations }}
{{ $path := buildLocation $location }}
{{ $authPath := buildAuthLocation $location }}
@ -704,7 +714,7 @@ stream {
{{ end }}
{{ if not (empty $location.ExternalAuth.SigninURL) }}
error_page 401 = {{ buildAuthSignURL $location.ExternalAuth.SigninURL }};
error_page 401 = $location.ExternalAuth.SigninURL;
{{ end }}
{{/* if the location contains a rate limit annotation, create one */}}
@ -763,6 +773,9 @@ stream {
proxy_set_header X-Original-URI $request_uri;
proxy_set_header X-Scheme $pass_access_scheme;
{{/* This header is used for external authentication */}}
proxy_set_header X-Auth-Request-Redirect $request_uri;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";

42
core/pkg/ingress/annotations/serversnippet/main.go Normal file
View file

@ -0,0 +1,42 @@
/*
Copyright 2016 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package serversnippet
import (
extensions "k8s.io/api/extensions/v1beta1"
"k8s.io/ingress/core/pkg/ingress/annotations/parser"
)
const (
annotation = "ingress.kubernetes.io/server-snippet"
)
type serverSnippet struct {
}
// NewParser creates a new server snippet annotation parser
func NewParser() parser.IngressAnnotation {
return serverSnippet{}
}
// Parse parses the annotations contained in the ingress rule
// used to indicate if the location/s contains a fragment of
// configuration to be included inside the paths of the rules
func (a serverSnippet) Parse(ing *extensions.Ingress) (interface{}, error) {
return parser.GetStringAnnotation(annotation, ing)
}

58
core/pkg/ingress/annotations/serversnippet/main_test.go Normal file
View file

@ -0,0 +1,58 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package serversnippet
import (
"testing"
api "k8s.io/api/core/v1"
extensions "k8s.io/api/extensions/v1beta1"
meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func TestParse(t *testing.T) {
ap := NewParser()
if ap == nil {
t.Fatalf("expected a parser.IngressAnnotation but returned nil")
}
testCases := []struct {
annotations map[string]string
expected string
}{
{map[string]string{annotation: "more_headers"}, "more_headers"},
{map[string]string{annotation: "false"}, "false"},
{map[string]string{}, ""},
{nil, ""},
}
ing := &extensions.Ingress{
ObjectMeta: meta_v1.ObjectMeta{
Name: "foo",
Namespace: api.NamespaceDefault,
},
Spec: extensions.IngressSpec{},
}
for _, testCase := range testCases {
ing.SetAnnotations(testCase.annotations)
result, _ := ap.Parse(ing)
if result != testCase.expected {
t.Errorf("expected %v but returned %v, annotations: %s", testCase.expected, result, testCase.annotations)
}
}
}

8
core/pkg/ingress/controller/annotations.go
View file

@ -36,6 +36,7 @@ import (
"k8s.io/ingress/core/pkg/ingress/annotations/redirect"
"k8s.io/ingress/core/pkg/ingress/annotations/rewrite"
"k8s.io/ingress/core/pkg/ingress/annotations/secureupstream"
"k8s.io/ingress/core/pkg/ingress/annotations/serversnippet"
"k8s.io/ingress/core/pkg/ingress/annotations/serviceupstream"
"k8s.io/ingress/core/pkg/ingress/annotations/sessionaffinity"
"k8s.io/ingress/core/pkg/ingress/annotations/snippet"
@ -83,6 +84,7 @@ func newAnnotationExtractor(cfg extractorConfig) annotationExtractor {
"DefaultBackend": defaultbackend.NewParser(cfg),
"UpstreamVhost": upstreamvhost.NewParser(),
"VtsFilterKey": vtsfilterkey.NewParser(),
"ServerSnippet": serversnippet.NewParser(),
},
}
}
@ -128,6 +130,7 @@ const (
serverAlias = "Alias"
clientBodyBufferSize = "ClientBodyBufferSize"
certificateAuth = "CertificateAuth"
serverSnippet = "ServerSnippet"
)
func (e *annotationExtractor) ServiceUpstream(ing *extensions.Ingress) bool {
@ -181,3 +184,8 @@ func (e *annotationExtractor) CertificateAuth(ing *extensions.Ingress) *authtls.
secure := val.(*authtls.AuthSSLConfig)
return secure
}
func (e *annotationExtractor) ServerSnippet(ing *extensions.Ingress) string {
val, _ := e.annotations[serverSnippet].Parse(ing)
return val.(string)
}

34
core/pkg/ingress/controller/backend_ssl.go
View file

@ -74,35 +74,47 @@ func (ic *GenericController) getPemCertificate(secretName string) (*ingress.SSLC
cert, okcert := secret.Data[apiv1.TLSCertKey]
key, okkey := secret.Data[apiv1.TLSPrivateKeyKey]
ca := secret.Data["ca.crt"]
// namespace/secretName -> namespace-secretName
nsSecName := strings.Replace(secretName, "/", "-", -1)
var s *ingress.SSLCert
if okcert && okkey {
if cert == nil || key == nil {
return nil, fmt.Errorf("error retrieving cert or key from secret %v: %v", secretName, err)
if cert == nil {
return nil, fmt.Errorf("secret %v has no 'tls.crt'", secretName)
}
if key == nil {
return nil, fmt.Errorf("secret %v has no 'tls.key'", secretName)
}
// If 'ca.crt' is also present, it will allow this secret to be used in the
// 'ingress.kubernetes.io/auth-tls-secret' annotation
s, err = ssl.AddOrUpdateCertAndKey(nsSecName, cert, key, ca)
if err != nil {
return nil, fmt.Errorf("unexpected error creating pem file %v", err)
return nil, fmt.Errorf("unexpected error creating pem file: %v", err)
}
glog.V(3).Infof("found certificate and private key, configuring %v as a TLS Secret (CN: %v)", secretName, s.CN)
glog.V(3).Infof("found 'tls.crt' and 'tls.key', configuring %v as a TLS Secret (CN: %v)", secretName, s.CN)
if ca != nil {
glog.V(3).Infof("found 'ca.crt', secret %v can also be used for Certificate Authentication", secretName)
}
} else if ca != nil {
glog.V(3).Infof("found only ca.crt, configuring %v as an Certificate Authentication secret", secretName)
s, err = ssl.AddCertAuth(nsSecName, ca)
if err != nil {
return nil, fmt.Errorf("unexpected error creating pem file %v", err)
return nil, fmt.Errorf("unexpected error creating pem file: %v", err)
}
// makes this secret in 'syncSecret' to be used for Certificate Authentication
// this does not enable Certificate Authentication
glog.V(3).Infof("found only 'ca.crt', configuring %v as an Certificate Authentication Secret", secretName)
} else {
return nil, fmt.Errorf("no keypair or CA cert could be found in %v", secretName)
}
if err != nil {
return nil, err
}
s.Name = secret.Name
s.Namespace = secret.Namespace
return s, nil

63
core/pkg/ingress/controller/controller.go
View file

@ -238,17 +238,19 @@ func (ic GenericController) GetService(name string) (*apiv1.Service, error) {
// sync collects all the pieces required to assemble the configuration file and
// then sends the content to the backend (OnUpdate) receiving the populated
// template as response reloading the backend if is required.
func (ic *GenericController) syncIngress(key interface{}) error {
func (ic *GenericController) syncIngress(item interface{}) error {
ic.syncRateLimiter.Accept()
if ic.syncQueue.IsShuttingDown() {
return nil
}
if name, ok := key.(string); ok {
if obj, exists, _ := ic.listers.Ingress.GetByKey(name); exists {
ing := obj.(*extensions.Ingress)
ic.readSecrets(ing)
if element, ok := item.(task.Element); ok {
if name, ok := element.Key.(string); ok {
if obj, exists, _ := ic.listers.Ingress.GetByKey(name); exists {
ing := obj.(*extensions.Ingress)
ic.readSecrets(ing)
}
}
}
@ -346,6 +348,7 @@ func (ic *GenericController) getStreamServices(configmapName string, proto apiv1
}
var svcs []ingress.L4Service
var svcProxyProtocol ingress.ProxyProtocol
// k -> port to expose
// v -> <namespace>/<service name>:<port from service to be used>
for k, v := range configmap.Data {
@ -363,18 +366,22 @@ func (ic *GenericController) getStreamServices(configmapName string, proto apiv1
nsSvcPort := strings.Split(v, ":")
if len(nsSvcPort) < 2 {
glog.Warningf("invalid format (namespace/name:port:[PROXY]) '%v'", k)
glog.Warningf("invalid format (namespace/name:port:[PROXY]:[PROXY]) '%v'", k)
continue
}
nsName := nsSvcPort[0]
svcPort := nsSvcPort[1]
useProxyProtocol := false
svcProxyProtocol.Decode = false
svcProxyProtocol.Encode = false
// Proxy protocol is possible if the service is TCP
if len(nsSvcPort) == 3 && proto == apiv1.ProtocolTCP {
if strings.ToUpper(nsSvcPort[2]) == "PROXY" {
useProxyProtocol = true
if len(nsSvcPort) >= 3 && proto == apiv1.ProtocolTCP {
if len(nsSvcPort) >= 3 && strings.ToUpper(nsSvcPort[2]) == "PROXY" {
svcProxyProtocol.Decode = true
}
if len(nsSvcPort) == 4 && strings.ToUpper(nsSvcPort[3]) == "PROXY" {
svcProxyProtocol.Encode = true
}
}
@ -432,11 +439,11 @@ func (ic *GenericController) getStreamServices(configmapName string, proto apiv1
svcs = append(svcs, ingress.L4Service{
Port: externalPort,
Backend: ingress.L4Backend{
Name: svcName,
Namespace: svcNs,
Port: intstr.FromString(svcPort),
Protocol: proto,
UseProxyProtocol: useProxyProtocol,
Name: svcName,
Namespace: svcNs,
Port: intstr.FromString(svcPort),
Protocol: proto,
ProxyProtocol: svcProxyProtocol,
},
Endpoints: endps,
})
@ -509,9 +516,13 @@ func (ic *GenericController) getBackendServers(ingresses []*extensions.Ingress)
ca := ic.annotations.CertificateAuth(ing)
if ca != nil {
server.CertificateAuth = *ca
// It is possible that no CAFileName is found in the secret
if server.CertificateAuth.CAFileName == "" {
glog.V(3).Infof("secret %v does not contain 'ca.crt', mutual authentication not enabled - ingress rule %v/%v.", server.CertificateAuth.Secret, ing.Namespace, ing.Name)
}
}
} else {
glog.V(3).Infof("server %v already contains a muthual autentication configuration - ingress rule %v/%v", server.Hostname, ing.Namespace, ing.Name)
glog.V(3).Infof("server %v already contains a mutual authentication configuration - ingress rule %v/%v", server.Hostname, ing.Namespace, ing.Name)
}
for _, path := range rule.HTTP.Paths {
@ -671,7 +682,8 @@ func (ic *GenericController) getBackendServers(ingresses []*extensions.Ingress)
return aUpstreams, aServers
}
// GetAuthCertificate ...
// GetAuthCertificate is used by the auth-tls annotations to get a cert from a secret
func (ic GenericController) GetAuthCertificate(secretName string) (*resolver.AuthSSLCert, error) {
if _, exists := ic.sslCertTracker.Get(secretName); !exists {
ic.syncSecret(secretName)
@ -894,10 +906,12 @@ func (ic *GenericController) createServers(data []*extensions.Ingress,
RequestBuffering: bdef.ProxyRequestBuffering,
}
// generated on Start() with createDefaultSSLCertificate()
defaultPemFileName := fakeCertificatePath
defaultPemSHA := fakeCertificateSHA
// Tries to fetch the default Certificate. If it does not exists, generate a new self signed one.
// Tries to fetch the default Certificate from nginx configuration.
// If it does not exists, use the ones generated on Start()
defaultCertificate, err := ic.getPemCertificate(ic.cfg.DefaultSSLCertificate)
if err == nil {
defaultPemFileName = defaultCertificate.PemFileName
@ -976,6 +990,7 @@ func (ic *GenericController) createServers(data []*extensions.Ingress,
for _, ing := range data {
// setup server-alias based on annotations
aliasAnnotation := ic.annotations.Alias(ing)
srvsnippet := ic.annotations.ServerSnippet(ing)
for _, rule := range ing.Spec.Rules {
host := rule.Host
@ -991,6 +1006,17 @@ func (ic *GenericController) createServers(data []*extensions.Ingress,
}
}
//notifying the user that it has already been configured.
if servers[host].ServerSnippet != "" && srvsnippet != "" {
glog.Warningf("ingress %v/%v for host %v contains a Server Snippet section that it has already been configured.",
ing.Namespace, ing.Name, host)
}
// only add a server snippet if the server does not have one previously configured
if servers[host].ServerSnippet == "" && srvsnippet != "" {
servers[host].ServerSnippet = srvsnippet
}
// only add a certificate if the server does not have one previously configured
if servers[host].SSLCertificate != "" {
continue
@ -1054,6 +1080,7 @@ func (ic *GenericController) createServers(data []*extensions.Ingress,
servers[host].Alias = ""
}
}
return servers
}

2
core/pkg/ingress/resolver/main.go
View file

@ -54,7 +54,7 @@ type AuthSSLCert struct {
Secret string `json:"secret"`
// CAFileName contains the path to the secrets 'ca.crt'
CAFileName string `json:"caFilename"`
// PemSHA contains the SHA1 hash of the 'tls.crt' value
// PemSHA contains the SHA1 hash of the 'ca.crt' or combinations of (tls.crt, tls.key, tls.crt) depending on certs in secret
PemSHA string `json:"pemSha"`
}

12
core/pkg/ingress/types.go
View file

@ -236,6 +236,10 @@ type Server struct {
// CertificateAuth indicates the this server requires mutual authentication
// +optional
CertificateAuth authtls.AuthSSLConfig `json:"certificateAuth"`
// ServerSnippet returns the snippet of server
// +optional
ServerSnippet string `json:"serverSnippet"`
}
// Location describes an URI inside a server.
@ -359,5 +363,11 @@ type L4Backend struct {
Namespace string `json:"namespace"`
Protocol apiv1.Protocol `json:"protocol"`
// +optional
UseProxyProtocol bool `json:"useProxyProtocol"`
ProxyProtocol ProxyProtocol `json:"proxyProtocol"`
}
// ProxyProtocol describes the proxy protocol configuration
type ProxyProtocol struct {
Decode bool `json:"decode"`
Encode bool `json:"encode"`
}

2
core/pkg/net/ssl/ssl.go
View file

@ -271,7 +271,7 @@ func AddCertAuth(name string, ca []byte) (*ingress.SSLCert, error) {
return nil, fmt.Errorf("could not write CA file %v: %v", caFileName, err)
}
glog.V(3).Infof("Created CA Certificate for authentication: %v", caFileName)
glog.V(3).Infof("Created CA Certificate for Authentication: %v", caFileName)
return &ingress.SSLCert{
CAFileName: caFileName,
PemFileName: caFileName,

9
core/pkg/task/queue.go
View file

@ -48,7 +48,8 @@ type Queue struct {
lastSync int64
}
type element struct {
// Element represents one item of the queue
type Element struct {
Key interface{}
Timestamp int64
}
@ -72,7 +73,7 @@ func (t *Queue) Enqueue(obj interface{}) {
glog.Errorf("%v", err)
return
}
t.queue.Add(element{
t.queue.Add(Element{
Key: key,
Timestamp: ts,
})
@ -99,7 +100,7 @@ func (t *Queue) worker() {
}
ts := time.Now().UnixNano()
item := key.(element)
item := key.(Element)
if t.lastSync > item.Timestamp {
glog.V(3).Infof("skipping %v sync (%v > %v)", item.Key, t.lastSync, item.Timestamp)
t.queue.Forget(key)
@ -110,7 +111,7 @@ func (t *Queue) worker() {
glog.V(3).Infof("syncing %v", item.Key)
if err := t.sync(key); err != nil {
glog.Warningf("requeuing %v, err %v", item.Key, err)
t.queue.AddRateLimited(element{
t.queue.AddRateLimited(Element{
Key: item.Key,
Timestamp: time.Now().UnixNano(),
})

7
examples/PREREQUISITES.md
View file

@ -132,7 +132,12 @@ The final step is to create a secret with the content of this file. This secret
the TLS Auth directive:
```console
$ kubectl create secret generic caingress --namespace=default --from-file=ca.crt
$ kubectl create secret generic caingress --namespace=default --from-file=ca.crt=<ca.crt>
```
Note: You can also generate the CA Authentication Secret along with the TLS Secret by using:
```console
$ kubectl create secret generic caingress --namespace=default --from-file=ca.crt=<ca.crt> --from-file=tls.crt=<tls.crt> --from-file=tls.key=<tls.key>
```
## Test HTTP Service

13
examples/auth/client-certs/nginx/README.md
View file

@ -16,13 +16,12 @@ the child, except for the root, which has Issuer == Subject.
* Client Cert: Certificate used by the clients to authenticate themselves with the loadbalancer/backends.
## Prerequisites
You need a valid CA File, composed of a group of valid enabled CAs. This MUST be in PEM Format.
The instructions are described [here](../../../PREREQUISITES.md#ca-authentication)
The instructions are described [here](../../../PREREQUISITES.md)
Also your ingress must be configured as a HTTPs/TLS Ingress.
Also your ingress must be configured as a HTTPS/TLS Ingress.
## Deployment
@ -51,8 +50,7 @@ Name: nginx-test
Namespace: default
Address: 104.198.183.6
Default backend: default-http-backend:80 (10.180.0.4:8080,10.240.0.2:8080)
TLS:
tls-secret terminates ingress.test.com
TLS: tls-secret terminates ingress.test.com
Rules:
Host Path Backends
---- ---- --------
@ -79,13 +77,12 @@ Server: nginx/1.11.9
$ curl -I -k --key ~/user.key --cert ~/user.cer https://ingress.test.com
HTTP/1.1 200 OK
Server: nginx/1.11.9
```
You must use the full DNS name while testing, as NGINX relies on the Server Name (SNI) to select the correct Ingress to be used.
The curl version used here was ``curl 7.47.0``
## Which certificate was used for authentication?
In your backend application you might want to know which certificate was used for authentication. For this purpose, we pass the full certificate in PEM format to the backend in the `ssl-client-cert` header.
In your backend application you might want to know which certificate was used for authentication.
For this purpose, we pass the full certificate in PEM format to the backend in the `ssl-client-cert` header.

3
examples/auth/client-certs/nginx/nginx-tls-auth.yaml
View file

@ -21,6 +21,5 @@ spec:
tls:
- hosts:
- ingress.test.com
# Create this cert as described in 'multi-tls' example
secretName: cert
secretName: tls-secret

4
examples/customization/custom-errors/nginx/README.md
View file

@ -1,4 +1,4 @@
This example shows how is possible to use a custom backend to render custom error pages. The code of this example is located here [nginx-debug-server](https://github.com/aledbf/contrib/tree/nginx-debug-server)
This example shows how is possible to use a custom backend to render custom error pages. The code of this example is located here [custom-error-pages](https://github.com/kubernetes/ingress/tree/master/examples/customization/custom-errors/nginx)
The idea is to use the headers `X-Code` and `X-Format` that NGINX pass to the backend in case of an error to find out the best existent representation of the response to be returned. i.e. if the request contains an `Accept` header of type `json` the error should be in that format and not in `html` (the default in NGINX).
@ -78,5 +78,3 @@ $ curl -v http://172.17.4.99/ -H 'Accept: application/json'
* Connection #0 to host 172.17.4.99 left intact
```
By default the Ingress controller provides support for `html`, `json` and `XML`.

110
images/custom-error-pages/Makefile Normal file
View file

@ -0,0 +1,110 @@
all: push
BUILDTAGS=
# Use the 0.0 tag for testing, it shouldn't clobber any release builds
TAG?=0.1
REGISTRY?=aledbf
GOOS?=linux
DOCKER?=gcloud docker --
SED_I?=sed -i
GOHOSTOS ?= $(shell go env GOHOSTOS)
PKG=k8s.io/ingress/images/custom-error-pages
ifeq ($(GOHOSTOS),darwin)
SED_I=sed -i ''
endif
REPO_INFO=$(shell git config --get remote.origin.url)
ifndef COMMIT
COMMIT := git-$(shell git rev-parse --short HEAD)
endif
ARCH ?= $(shell go env GOARCH)
GOARCH = ${ARCH}
DUMB_ARCH = ${ARCH}
BASEIMAGE?=alpine:3.6
ALL_ARCH = amd64 arm arm64 ppc64le
QEMUVERSION=v2.9.1
IMGNAME = custom-error-pages
IMAGE = $(REGISTRY)/$(IMGNAME)
MULTI_ARCH_IMG = $(IMAGE)-$(ARCH)
ifeq ($(ARCH),arm)
QEMUARCH=arm
GOARCH=arm
endif
ifeq ($(ARCH),arm64)
QEMUARCH=aarch64
endif
ifeq ($(ARCH),ppc64le)
QEMUARCH=ppc64le
GOARCH=ppc64le
endif
#ifeq ($(ARCH),s390x)
# QEMUARCH=s390x
#endif
TEMP_DIR := $(shell mktemp -d)
DOCKERFILE := $(TEMP_DIR)/rootfs/Dockerfile
all: all-container
sub-container-%:
$(MAKE) ARCH=$* build container
sub-push-%:
$(MAKE) ARCH=$* push
all-container: $(addprefix sub-container-,$(ALL_ARCH))
all-push: $(addprefix sub-push-,$(ALL_ARCH))
container: .container-$(ARCH)
.container-$(ARCH):
cp -r ./* $(TEMP_DIR)
$(SED_I) 's|BASEIMAGE|$(BASEIMAGE)|g' $(DOCKERFILE)
$(SED_I) "s|QEMUARCH|$(QEMUARCH)|g" $(DOCKERFILE)
ifeq ($(ARCH),amd64)
# When building "normally" for amd64, remove the whole line, it has no part in the amd64 image
$(SED_I) "/CROSS_BUILD_/d" $(DOCKERFILE)
else
# When cross-building, only the placeholder "CROSS_BUILD_" should be removed
# Register /usr/bin/qemu-ARCH-static as the handler for ARM binaries in the kernel
$(DOCKER) run --rm --privileged multiarch/qemu-user-static:register --reset
curl -sSL https://github.com/multiarch/qemu-user-static/releases/download/$(QEMUVERSION)/x86_64_qemu-$(QEMUARCH)-static.tar.gz | tar -xz -C $(TEMP_DIR)/rootfs
$(SED_I) "s/CROSS_BUILD_//g" $(DOCKERFILE)
endif
$(DOCKER) build -t $(MULTI_ARCH_IMG):$(TAG) $(TEMP_DIR)/rootfs
ifeq ($(ARCH), amd64)
# This is for to maintain the backward compatibility
$(DOCKER) tag $(MULTI_ARCH_IMG):$(TAG) $(IMAGE):$(TAG)
endif
push: .push-$(ARCH)
.push-$(ARCH):
$(DOCKER) push $(MULTI_ARCH_IMG):$(TAG)
ifeq ($(ARCH), amd64)
$(DOCKER) push $(IMAGE):$(TAG)
endif
clean:
$(DOCKER) rmi -f $(MULTI_ARCH_IMG):$(TAG) || true
build: clean
CGO_ENABLED=0 GOOS=${GOOS} GOARCH=${GOARCH} go build -a -installsuffix cgo \
-ldflags "-s -w" \
-o ${TEMP_DIR}/rootfs/custom-error-pages ${PKG}/...
release: all-container all-push
echo "done"

2
images/custom-error-pages/README.md Normal file
View file

@ -0,0 +1,2 @@
Example of Custom error pages for the NGINX Ingress controller

94
images/custom-error-pages/main.go Normal file
View file

@ -0,0 +1,94 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"io"
"log"
"mime"
"net/http"
"os"
"strconv"
)
const (
FormatHeader = "X-Format"
CodeHeader = "X-Code"
ContentType = "Content-Type"
)
func main() {
path := "/www"
if os.Getenv("PATH") != "" {
path = os.Getenv("PATH")
}
http.HandleFunc("/", errorHandler(path))
http.ListenAndServe(fmt.Sprintf(":8080"), nil)
}
func errorHandler(path string) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
ext := "html"
format := r.Header.Get(FormatHeader)
if format == "" {
format = "text/html"
log.Printf("forma not specified. Using %v\n", format)
}
mediaType, _, _ := mime.ParseMediaType(format)
cext, err := mime.ExtensionsByType(mediaType)
if err != nil {
log.Printf("unexpected error reading media type extension: %v. Using %v\n", err, ext)
} else {
ext = cext[0]
}
w.Header().Set(ContentType, format)
errCode := r.Header.Get(CodeHeader)
code, err := strconv.Atoi(errCode)
if err != nil {
code = 404
log.Printf("unexpected error reading return code: %v. Using %v\n", err, code)
}
w.WriteHeader(code)
file := fmt.Sprintf("%v/%v%v", path, code, ext)
f, err := os.Open(file)
if err != nil {
log.Printf("unexpected error opening file: %v\n", err)
scode := strconv.Itoa(code)
file := fmt.Sprintf("%v/%cxx%v", path, scode[0], ext)
f, err := os.Open(file)
if err != nil {
log.Printf("unexpected error opening file: %v\n", err)
http.NotFound(w, r)
return
}
defer f.Close()
log.Printf("serving custom error response for code %v and format %v from file %v\n", code, format, file)
io.Copy(w, f)
return
}
defer f.Close()
log.Printf("serving custom error response for code %v and format %v from file %v\n", code, format, file)
io.Copy(w, f)
}
}

21
images/custom-error-pages/rootfs/Dockerfile Executable file
View file

@ -0,0 +1,21 @@
# Copyright 2017 The Kubernetes Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM BASEIMAGE
CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
COPY . /
CMD ["/custom-error-pages"]

1
images/custom-error-pages/rootfs/www/404.html Normal file
View file

@ -0,0 +1 @@
<span>The page you're looking for could not be found.</span>

1
images/custom-error-pages/rootfs/www/404.json Normal file
View file

@ -0,0 +1 @@
{ "message": "The page you're looking for could not be found" }

1
images/custom-error-pages/rootfs/www/4xx.html Normal file
View file

@ -0,0 +1 @@
4xx html

1
images/custom-error-pages/rootfs/www/4xx.json Normal file
View file

@ -0,0 +1 @@
4xx json

1
images/custom-error-pages/rootfs/www/500.html Normal file
View file

@ -0,0 +1 @@
500 html

1
images/custom-error-pages/rootfs/www/500.json Normal file
View file

@ -0,0 +1 @@
500 json

1
images/custom-error-pages/rootfs/www/5xx.html Normal file
View file

@ -0,0 +1 @@
5xx html

1
images/custom-error-pages/rootfs/www/5xx.json Normal file
View file

@ -0,0 +1 @@
5xx json

4
images/nginx-slim/Makefile
View file

@ -13,8 +13,8 @@
# limitations under the License.
# 0.0.0 shouldn't clobber any released builds
TAG = 0.25
REGISTRY = gcr.io/google_containers
TAG ?= 0.25
REGISTRY ?= gcr.io/google_containers
ARCH ?= $(shell go env GOARCH)
ALL_ARCH = amd64 arm arm64 ppc64le