From Zero To Microservice [1]

How to Design API, Server & Client With Ease?

Previous article:

Microservices 101

Next article:

From Zero to Microservice [2]

What is gRPC (Again)?

Fig 1: A Cloud-Native application using gRPC for communication, Source: https://learn.microsoft.com/de-de/dotnet/architecture/cloud-native/grpc

gRPC is a modern open source high performance Remote Procedure Call (RPC) framework that can run in any environment. [1]

It lets you:

define the service interface and the structure of the payload messages using Protobuf (Protocol Buffers) in a Protobuf file named e.g. <service>.proto, then

generate the server and client stubs automatically using <service>.proto in a variety of languages and platforms, but also

integrate bi-directional streaming and authentication features of HTTP/2

Planning

Fig 2: A DB consisting of key-value pairs, Source: https://www.scylladb.com/glossary/key-value-database/

Let’s start with the following user story:

As an end user, I want to perform read/write operations on person data.

So we want to design a microservice which looks up person by a selected identifier and/or updates person data.

I choose Go as the programming language due to its simplicity and high speed.

Instead of creating HTTP server / client from scratch, I prefer to use gRPC for code generation.

Setup

Setup folder structure (I named it person):

Fig 3: Folder structure

Initialize Git: git init

Create a Go package: go mod init github.com/<your-grpc>

Install dependencies:

• go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2
• go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28

Save installed programs into PATH: export PATH="$PATH:$(go env GOPATH)/bin"

API Design

Fig 4: Protocol Buffers used for API design, Source: https://www.xenonstack.com/insights/google-protocol-buffer/

Rephrasing User Story: The server / client should be able to perform get / set operations on a DB consisting of personal data.

Fig 5: Proto file header

I set the name of package: person (see 7, later in the code generation: a Go package person will be created). The path of Go package should be specified as: <parent>/person (see 3).

Using the documentation [2], I defined a brand new service “PersonQuery” as follows.

Fig 6: A service definition

Service definition is what a client sees from the outside. It specifies all methods which can be called remotely using stubs.

Each remote procedure contains exactly one input and one output message. If no input / output is specified, an empty message can be used instead (see 24).

An input / output message can also be a stream message, which means it blocks the procedure and reads / writes data until termination condition reached (see 15, 18 and 21).

Fig 7: A message with a single field

Fig 8: A message with simple fields

A message specifies the payload message which can be used as input / output. Message fields have a type (e.g. string, float) and the order inside message, starting with 1.

Fig 9: An enum defined in message

Enum is defined similarly as a message type, but its order is also equal to its values. Zero is the default value of an enum (see 42).

If a field is removed from a message in a new version of the service, the reserved field numbers (see 50) should never be reused.

Fig 10: An inner message extending its parent message by new fields

Similarly how aggregation works [3], the messages can extend a message type by including its payload (see 60).

Fig 11: Optional fields

A message can contain also optional fields, which are set to their default value, if not set.

Type-specific default values are: zero, empty string, false, or a null pointer otherwise. The exact implementation of null pointer depends on the language (NULL in C++, None in Python, nil in Golang, …).

Fig 12: More complex fields

Union concept is not widely accepted in all languages: You have to use exactly one of the fields in a message. If that’s the case (see 84), you can use oneof <union> { ... } to guarantee that exactly one element inside a message is used at once.

If the requirements of a fields are not fully understood, or you have to iterate over its fields, you can use map<<key_type>, <value_type>> as a new field type. Similarly, map concept is not implemented across all languages, too.

Using repeated at the beginning of a simple field, an 1-dimensional array of same type can be defined.

Code Generation

Enter the following command in terminal to generate the client and server stub:

protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative <your-stub>/<your-stub>.proto

Here are the instant results:

Fig 13: service PersonQuery -> interface PersonQueryClient

Fig 14: service PersonQuery -> interface PersonQueryServer

Fig 15: message Person -> struct Person (with all getter/setter methods)

Implement the service

The API is successfully created, it’s time to implement the server and the client.

server/main.go:

Fig 16: Server implementation

Line 23: Server is defined as a data structure with server payload and optional fields you wish to use.

Lines 28, 44: Server struct implements methods GetPerson and SetPerson. Since other methods won’t be used, they are implemented as empty functions.

Lines 29, 46: Message type Person has built-in getter methods for each field x, named as GetX.

Line 45: Empty message type is initialized.

Fig 17: Server main function

Line 79: Initialize listener which will accept client connections on specified TCP port

Lines 89–90: Initialize a new gRPC server, which copies the attributes and methods of PersonQueryServerStruct

Line 93: Accept client connections and perform operations

Implement the client

Now that gRPC server works as expected, we can define the front-end of the application.

It translates the requests made by end user to valid Remote Procedure Calls on the server, as follows:

client/main.go:

Fig 18: Client main function

Lines 34–35: Connection to the server is established with transport credentials

Line 51: Command execution takes place (more in the next image)

Fig 19: Client-side command execution

Line 59: Since json cannot deal with more complex data structures, protojson has to be used instead.

Lines 62, 69: Here are two examples of Remote Procedure Calls. Methods are called remotely from PersonQueryClient and the call is forwarded to PersonQueryServer.

Build & Run the Application

Open a new terminal for server

Build server: go build -o <your-server> ./server

Run server: go run ./server <args>

Then, open a new terminal for client

Run client: go run client/main.go <args>

To be Continued …

Well, the core functionality is done.

In the following article, I dockerize the application and deploy it to a local Kubernetes cluster. Stay tuned!

From Zero to Microservice [2]

References

[1] Quick Start (gRPC): https://grpc.io/docs/languages/go/quickstart/
[2] Language Guide (proto3): https://developers.google.com/protocol-buffers/docs/proto3
[3] Inheritance vs. Aggregation: https://stackoverflow.com/questions/269496/inheritance-vs-aggregation