Merge pull request #2307 from pcj/grpc-example

Documentation: grpc example
This commit is contained in:
k8s-ci-robot 2018-05-10 09:26:53 -07:00 committed by GitHub
commit c80365dcfa
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
16 changed files with 698 additions and 1 deletions

View file

@ -172,7 +172,7 @@ e2e-test:
.PHONY: cover
cover:
@rm -rf coverage.txt
@for d in `go list ./... | grep -v vendor | grep -v '/test/e2e'`; do \
@for d in `go list ./... | grep -v vendor | grep -v '/test/e2e' | grep -v 'images/grpc-fortune-teller'`; do \
t=$$(date +%s); \
go test -coverprofile=cover.out -covermode=atomic $$d || exit 1; \
echo "Coverage test $$d took $$(($$(date +%s)-t)) seconds"; \

View file

@ -0,0 +1,105 @@
# gRPC
This example demonstrates how to route traffic to a gRPC service through the
nginx controller.
## Prerequisites
1. You have a kubernetes cluster running.
2. You have a domain name such as `example.com` that is configured to route
traffic to the ingress controller. Replace references to
`fortune-teller.stack.build` (the domain name used in this example) to your
own domain name (you're also responsible for provisioning an SSL certificate
for the ingress).
3. You have the nginx-ingress controller installed in typical fashion (must be
at least
[quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.13.0](https://quay.io/kubernetes-ingress-controller/nginx-ingress-controller)
for grpc support.
4. You have a backend application running a gRPC server and listening for TCP
traffic. If you prefer, you can use the
[fortune-teller](https://github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller)
application provided here as an example.
### Step 1: kubernetes `Deployment`
```sh
$ kubectl create -f app.yaml
```
This is a standard kubernetes deployment object. It is running a grpc service
listening on port `50051`.
The sample application
[fortune-teller-app](https://github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller)
is a grpc server implemented in go. Here's the stripped-down implementation:
```go
func main() {
grpcServer := grpc.NewServer()
fortune.RegisterFortuneTellerServer(grpcServer, &FortuneTeller{})
lis, _ := net.Listen("tcp", ":50051")
grpcServer.Serve(lis)
}
```
The takeaway is that we are not doing any TLS configuration on the server (as we
are terminating TLS at the ingress level, grpc traffic will travel unencrypted
inside the cluster and arrive "insecure").
For your own application you may or may not want to do this. If you prefer to
forward encrypted traffic to your POD and terminate TLS at the gRPC server
itself, add the ingress annotation `nginx.ingress.kubernetes.io/secure-backends:
"true"`.
### Step 2: the kubernetes `Service`
```sh
$ kubectl create -f svc.yaml
```
Here we have a typical service. Nothing special, just routing traffic to the
backend application on port `50051`.
### Step 3: the kubernetes `Ingress`
```sh
$ kubectl create -f ingress.yaml
```
A few things to note:
1. We've tagged the ingress with the annotation
`nginx.ingress.kubernetes.io/grpc-backend: "true"`. This is the magic
ingredient that sets up the appropriate nginx configuration to route http/2
traffic to our service.
1. We're terminating TLS at the ingress and have configured an SSL certificate
`fortune-teller.stack.build`. The ingress matches traffic arriving as
`https://fortune-teller.stack.build:443` and routes unencrypted messages to
our kubernetes service.
### Step 4: test the connection
Once we've applied our configuration to kubernetes, it's time to test that we
can actually talk to the backend. To do this, we'll use the
[grpcurl](https://github.com/fullstorydev/grpcurl) utility:
```sh
$ grpcurl fortune-teller.stack.build:443 build.stack.fortune.FortuneTeller/Predict
{
"message": "Let us endeavor so to live that when we come to die even the undertaker will be sorry.\n\t\t-- Mark Twain, \"Pudd'nhead Wilson's Calendar\""
}
```
### Debugging Hints
1. Obviously, watch the logs on your app.
2. Watch the logs for the nginx-ingress-controller (increasing verbosity as
needed).
3. Double-check your address and ports.
4. Set the `GODEBUG=http2debug=2` environment variable to get detailed http/2
logging on the client and/or server.
5. Study RFC 7540 (http/2) <https://tools.ietf.org/html/rfc7540>.
> If you are developing public gRPC endpoints, check out
> https://proto.stack.build, a protocol buffer / gRPC build service that can use
> to help make it easier for your users to consume your API.

View file

@ -0,0 +1,20 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: fortune-teller-app
labels:
k8s-app: fortune-teller-app
namespace: default
spec:
replicas: 1
template:
metadata:
labels:
k8s-app: fortune-teller-app
spec:
containers:
- name: fortune-teller-app
image: quay.io/kubernetes-ingress-controller/grpc-fortune-teller:0.1
ports:
- containerPort: 50051
name: grpc

View file

@ -0,0 +1,7 @@
apiVersion: "stable.k8s.psg.io/v1"
kind: "Certificate"
metadata:
name: fortune-teller.stack.build
namespace: default
spec:
domain: "fortune-teller.stack.build"

View file

@ -0,0 +1,21 @@
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/grpc-backend: "true"
name: fortune-ingress
namespace: default
spec:
rules:
- host: fortune-teller.stack.build
http:
paths:
- backend:
serviceName: fortune-teller-service
servicePort: grpc
tls:
- secretName: fortune-teller.stack.build
hosts:
- fortune-teller.stack.build

View file

@ -0,0 +1,12 @@
apiVersion: v1
kind: Service
metadata:
name: fortune-teller-service
namespace: default
spec:
selector:
k8s-app: fortune-teller-app
ports:
- port: 50051
targetPort: 50051
name: grpc

View file

@ -0,0 +1,15 @@
load("@io_bazel_rules_go//go:def.bzl", "gazelle", "go_prefix")
gazelle(
name = "gazelle",
args = [
"-build_file_name",
"BUILD,BUILD.bazel",
],
command = "fix",
prefix = "github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller",
)
go_prefix(
prefix = "github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller",
)

104
images/grpc-fortune-teller/Gopkg.lock generated Normal file
View file

@ -0,0 +1,104 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
name = "github.com/golang/protobuf"
packages = [
"proto",
"protoc-gen-go/descriptor",
"ptypes",
"ptypes/any",
"ptypes/duration",
"ptypes/timestamp"
]
revision = "925541529c1fa6821df4e44ce2723319eb2be768"
version = "v1.0.0"
[[projects]]
name = "github.com/vromero/gofortune"
packages = [
"lib",
"lib/fortune"
]
revision = "8a3c485287c0d3d3e4f8f0d9e0058f6cdd29740d"
version = "v0.0.1"
[[projects]]
branch = "master"
name = "golang.org/x/net"
packages = [
"context",
"http2",
"http2/hpack",
"idna",
"internal/timeseries",
"lex/httplex",
"trace"
]
revision = "b3c676e531a6dc479fa1b35ac961c13f5e2b4d2e"
[[projects]]
name = "golang.org/x/text"
packages = [
"collate",
"collate/build",
"internal/colltab",
"internal/gen",
"internal/tag",
"internal/triegen",
"internal/ucd",
"language",
"secure/bidirule",
"transform",
"unicode/bidi",
"unicode/cldr",
"unicode/norm",
"unicode/rangetable"
]
revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
version = "v0.3.0"
[[projects]]
branch = "master"
name = "google.golang.org/genproto"
packages = ["googleapis/rpc/status"]
revision = "35de2414665fc36f56b72d982c5af480d86de5ab"
[[projects]]
name = "google.golang.org/grpc"
packages = [
".",
"balancer",
"balancer/base",
"balancer/roundrobin",
"codes",
"connectivity",
"credentials",
"encoding",
"encoding/proto",
"grpclb/grpc_lb_v1/messages",
"grpclog",
"internal",
"keepalive",
"metadata",
"naming",
"peer",
"reflection",
"reflection/grpc_reflection_v1alpha",
"resolver",
"resolver/dns",
"resolver/passthrough",
"stats",
"status",
"tap",
"transport"
]
revision = "d89cded64628466c4ab532d1f0ba5c220459ebe8"
version = "v1.11.2"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "fcd36504ba120cfbbe161eac2ddd439c884fcef61ce318b5c0a544d27bc47043"
solver-name = "gps-cdcl"
solver-version = 1

View file

@ -0,0 +1,30 @@
# Gopkg.toml example
#
# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
# name = "github.com/user/project"
# version = "1.0.0"
#
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
# [[override]]
# name = "github.com/x/y"
# version = "2.4.0"
#
# [prune]
# non-go = false
# go-tests = true
# unused-packages = true
[prune]
go-tests = true
unused-packages = true

View file

@ -0,0 +1,59 @@
# grpc-fortune-teller
This is the grpc server application for the nginx-ingress grpc example. Bazel
0.12 is used for the building and container management.
## Build
Builds a statically compiled go binary cross-compiled for linux:
```
$ bazel build //app:fortune
Target //app:fortune up-to-date:
bazel-bin/app/linux_amd64_static_pure_stripped/fortune
```
> To build for your host system, comment out the `goos` and `goarch` attributes
> in the `go_binary` rule.
## Run
Builds a minimal docker image that wraps the go_binary, loads it into your local
docker image repository, and runs it:
```sh
$ bazel run //app:image
Loaded image ID: sha256:aa597c897c873116fcbfccafecf5ab0f6f4178a05e4a00c8e79de91ac0d2e9e7
Tagging aa597c897c873116fcbfccafecf5ab0f6f4178a05e4a00c8e79de91ac0d2e9e7 as bazel/app:image
2018/05/01 02:13:43 Restored /tmp/fortune-teller/usr/share/games/fortunes/fortunes.dat
2018/05/01 02:13:43 Restored /tmp/fortune-teller/usr/share/games/fortunes/literature
2018/05/01 02:13:43 Restored /tmp/fortune-teller/usr/share/games/fortunes/literature.dat
2018/05/01 02:13:43 Restored /tmp/fortune-teller/usr/share/games/fortunes/riddles
2018/05/01 02:13:43 Restored /tmp/fortune-teller/usr/share/games/fortunes/riddles.dat
2018/05/01 02:13:43 Restored /tmp/fortune-teller/usr/share/games/fortunes/fortunes
2018/05/01 02:13:43 Assets restored to /tmp/fortune-teller
2018/05/01 02:13:43 Listening for gRPC requests at 50051
```
Or run it via docker:
```sh
$ docker run bazel/app:image
```
Build image and push to the container registry specified in the `container_push`
rule:
```sh
$ bazel run //app:push
```
## Invoke
```sh
$ grpcurl -plaintext localhost:50051 build.stack.fortune.FortuneTeller/Predict
{
"message": "Whenever the literary German dives into a sentence, that is the last\nyou are going to see of him until he emerges on the other side of his\nAtlantic with his verb in his mouth.\n\t\t-- Mark Twain \"A Connecticut Yankee in King Arthur's Court\""
}
```

View file

@ -0,0 +1,62 @@
workspace(name = "com_github_kubernetes_ingress_nginx_images_grpc_fortune_teller")
#####################################################################
# RULES_GO
#####################################################################
git_repository(
name = "io_bazel_rules_go",
remote = "https://github.com/bazelbuild/rules_go.git",
commit = "161c91485b007c6bf51c0e81808cf4ee2ded299d",
)
http_archive(
name = "com_github_scele_rules_go_dep",
urls = ["https://github.com/scele/rules_go_dep/archive/49a5e4ca9f6a16c9b4c930a51ce3a537498bb4e1.tar.gz"],
strip_prefix = "rules_go_dep-49a5e4ca9f6a16c9b4c930a51ce3a537498bb4e1",
sha256 = "f170d3d6f55e216f1493f975cde6c489d7070da2a8a41fd4de9812d96f4fb38b",
)
load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains")
load("@com_github_scele_rules_go_dep//dep:dep.bzl", "dep_import")
go_register_toolchains(go_version = "1.9")
go_rules_dependencies()
dep_import(
name = "godeps",
prefix = "github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller",
gopkg_lock = "//:Gopkg.lock",
)
load("@godeps//:Gopkg.bzl", "go_deps")
go_deps()
#############################################################
# RULES_DOCKER
#############################################################
RULES_DOCKER_VERSION = "553d5506bb7325185950f91533b967da8f5bc536"
http_archive(
name = "io_bazel_rules_docker",
url = "https://github.com/bazelbuild/rules_docker/archive/%s.zip" % RULES_DOCKER_VERSION,
strip_prefix = "rules_docker-" + RULES_DOCKER_VERSION,
sha256 = "e0b3d966f2a5c0fe921b6294df7c823afa63b4c439f0a7f3b9da3ed6534bab83",
)
load(
"@io_bazel_rules_docker//container:container.bzl",
container_repositories = "repositories",
)
container_repositories()
load(
"@io_bazel_rules_docker//go:image.bzl",
go_image_repositories = "repositories",
)
go_image_repositories()

View file

@ -0,0 +1,69 @@
load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library", "go_embed_data")
load("@io_bazel_rules_docker//go:image.bzl", "go_image")
load("@io_bazel_rules_docker//container:push.bzl", "container_push")
# Concatenates the fortune databases to a single bundle.
# May need to adjust paths for your system (built on ubuntu 16.04).
# $ apt-get install fortune
genrule(
name = "tar",
outs = ["fortune.tar"],
cmd = " && ".join([
"OUT=$$(pwd)/$@",
"tar -cvf $$OUT /usr/share/games/fortunes",
]),
)
# Generates a .go source file with the tarball content in
# the fortuneFiles variable.
go_embed_data(
name = "fortune_assets",
srcs = [
":tar",
],
package = "main",
unpack = True,
var = "fortuneFiles",
)
go_library(
name = "go_default_library",
srcs = [
"main.go",
":fortune_assets", # keep
],
importpath = "github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller/app",
visibility = ["//visibility:private"],
deps = [
"//proto/fortune:go_default_library",
"@com_github_vromero_gofortune//lib/fortune:go_default_library",
"@org_golang_google_grpc//:go_default_library",
"@org_golang_google_grpc//codes:go_default_library",
"@org_golang_google_grpc//reflection:go_default_library",
],
)
go_binary(
name = "fortune",
embed = [":go_default_library"],
goarch = "amd64",
goos = "linux",
pure = "on",
static = "on",
visibility = ["//visibility:public"],
)
go_image(
name = "image",
binary = ":fortune",
visibility = ["//visibility:public"],
)
container_push(
name = "push",
format = "Docker",
image = ":image",
registry = "quay.io",
repository = "kubernetes-ingress-controller/grpc-fortune-teller",
tag = "0.1",
)

View file

@ -0,0 +1,137 @@
/*
Copyright 2018 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"context"
"fmt"
"io/ioutil"
"log"
"net"
"os"
"path"
proto "github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller/proto/fortune"
"github.com/vromero/gofortune/lib/fortune"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/reflection"
)
const (
grpcPort = 50051
)
func main() {
baseDir := "/tmp/fortune-teller"
mustMkdirAll(baseDir)
opts := []grpc.ServerOption{
grpc.MaxConcurrentStreams(200),
}
grpcServer := grpc.NewServer(opts...)
fortuneTeller := &FortuneTeller{
fs: createFortuneFilesystemNodeDescriptor(baseDir),
}
proto.RegisterFortuneTellerServer(grpcServer, fortuneTeller)
reflection.Register(grpcServer)
lis, err := net.Listen("tcp", fmt.Sprintf(":%d", grpcPort))
if err != nil {
log.Fatalf("Error while starting grpc server: %v\n", err)
}
log.Printf("Listening for gRPC requests at %d\n", grpcPort)
grpcServer.Serve(lis)
}
// FortuneTeller - struct that will implement the grpc service interface.
type FortuneTeller struct {
fs *fortune.FileSystemNodeDescriptor
}
// Predict - implementation for the grpc unary request method.
func (f *FortuneTeller) Predict(ctx context.Context, r *proto.PredictionRequest) (*proto.PredictionResponse, error) {
_, data, err := fortune.GetRandomFortune(*f.fs)
if err != nil {
return nil, grpc.Errorf(codes.Internal, "Unable to render fortune: %v", err)
}
return &proto.PredictionResponse{
Message: data,
}, nil
}
func createFortuneFilesystemNodeDescriptor(baseDir string) *fortune.FileSystemNodeDescriptor {
// Restore the packed fortune data
fortuneDir := path.Join(baseDir, "usr/share/games/fortunes")
mustRestore(baseDir, fortuneFiles, nil)
// init gofortune fs
fs, err := fortune.LoadPaths([]fortune.ProbabilityPath{
{Path: fortuneDir},
})
if err != nil {
log.Fatalf("Unable to load fortune paths: %v", err)
}
fortune.SetProbabilities(&fs, true) // consider all equal probabilities
return &fs
}
// mustRestore - Restore assets.
func mustRestore(baseDir string, assets map[string][]byte, mappings map[string]string) {
// unpack variable is provided by the go_embed data and is a
// map[string][]byte such as {"/usr/share/games/fortune/literature.dat":
// bytes... }
for basename, bytes := range assets {
if mappings != nil {
replacement := mappings[basename]
if replacement != "" {
basename = replacement
}
}
filename := path.Join(baseDir, basename)
dirname := path.Dir(filename)
//log.Printf("file %s, dir %s, rel %d, abs %s, absdir: %s", file, dir, rel, abs, absdir)
if err := os.MkdirAll(dirname, os.ModePerm); err != nil {
log.Fatalf("Failed to create asset dir %s: %v", dirname, err)
}
if err := ioutil.WriteFile(filename, bytes, os.ModePerm); err != nil {
log.Fatalf("Failed to write asset %s: %v", filename, err)
}
log.Printf("Restored %s", filename)
}
log.Printf("Assets restored to %s", baseDir)
}
// mustMkdirAll - make all dirs and panic if fail
func mustMkdirAll(dirs ...string) {
for _, dir := range dirs {
err := os.MkdirAll(dir, os.ModePerm)
if err != nil {
panic(fmt.Sprintf("Failed mkdir %s: %v", dir, err))
}
}
}

View file

@ -0,0 +1,24 @@
load("@io_bazel_rules_go//go:def.bzl", "go_library")
load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
go_library(
name = "go_default_library",
srcs = ["doc.go"],
embed = [":build_stack_fortune_go_proto"], # keep
importpath = "github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller/proto/fortune",
visibility = ["//visibility:public"],
)
proto_library(
name = "build_stack_fortune_proto",
srcs = ["fortune.proto"],
visibility = ["//visibility:public"],
)
go_proto_library(
name = "build_stack_fortune_go_proto",
compilers = ["@io_bazel_rules_go//proto:go_grpc"],
importpath = "github.com/kubernetes/ingress-nginx/images/grpc-fortune-teller/proto/fortune",
proto = ":build_stack_fortune_proto",
visibility = ["//visibility:public"],
)

View file

@ -0,0 +1,18 @@
/*
Copyright 2018 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Package fortune acts as a placeholder f for generated *.pb.go files.
package fortune

View file

@ -0,0 +1,14 @@
syntax = "proto3";
package build.stack.fortune;
message PredictionRequest {
}
message PredictionResponse {
string message = 1;
}
service FortuneTeller {
rpc Predict(PredictionRequest) returns (PredictionResponse);
}