zen/README.md

Zen

In house Library to solve the biggest problem in the micro services world, Observability.

To setup proper Observability, we have to fulfill the three pillars of Observability, which are:

1. Logging
2. Metrics
3. Tracing

There are surprising big requirements and code setup to take the maximum advantage of these pillars, which are:

1. Structured and Standardized Logging format.
2. Metrics collection that have proper labels and contexts.
3. Tracing that should be able to cross the service boundaries.

Lot of knowledge and boilerplate code is required to setup these pillars, which is not only time consuming but also error prone.

This library is an attempt to solve these problems by providing a simple and easy to use API to setup Observability in your services.

Some API are just re-exports, especially those that comes from OpenTelemetry, but the usage of these API is already simplified and made more user friendly.

Status

Still in development. Not ready even for basic usage.

Features

1. Layered Config

The library provides a layered configuration system, which allows the user to provide overrides at different levels.

The library provides the following levels of configuration with the latter overriding the former:

1. Default Config: The default configuration that comes with the library. This only contains configuration for zen library itself.
2. Vault Config: The configuration that comes from the Vault.
3. Config File: zconfig.json and zconfig.yaml at the same cwd as where the command is run. Ignored if not found. zconfig.json is looked first then zconfig.yaml. The usage of Config File is actually discouraged because they contains secrets and must be inside .gitignore for security purposes. But it's there if you want easy configuration override on local development.
4. Environment Variables: The most interesting and probably the most eye-catching one because it can easily override-able inline by DevOps team or in CI steps. Zen will lookup .env file on cwd on startup and load up any environment variables that DOES NOT EXIST before. So .env loading will not override any previous set variables. See Environment Variables documentation for more details on how to use this.
5. Command Line Arguments: This is one of those putting the cart before the horse kind of situation. Popular CLI parser libraries requires flags to be defined first and this library does not know those prior values at the start of the application other than default values inside the library. Thus, configuring config from cli flags are limited to only configuring zen library itself.

2. OpenTelemetry Integration

OpenTelemetry is chosen because it's vendor agnostic nature and are supported by big players in Observability world like Grafana Stack, Elastic Stack (v2 and up), and OpenObserve.

Migrating vendors is just as easy as changing the Collector endpoint in the config.

Unfortunately, there are always costs to pay for this flexibility. The cost is the complexity of the API and the boilerplate code required to setup the OpenTelemetry itself.

This library will simplify the setup of OpenTelemetry by providing a simple API to setup the Tracer, Meter, and Logger at the startup of the application.

This library also has the goal to further integrate Bareksa's infrastructure with OpenTelemetry by providing an API to setup Clients for these infrastructures instrumented with OpenTelemetry out of the box.

3. Developer Friendly Logging

Structure Logging is good for machines to consume, but it's not very friendly for human read. This library provides the best of both worlds by detecting the environment and choose the best logging format for that environment.

On, interactive environment (TTY Detected), e.g. running via Terminal Emulator, the library will output the logs in a human readable format.

On, non-interactive environment, e.g. running in a container, the library will output the logs in a JSON structured format.

When running tests, by default the library will be silent, but you can easily enable the logs by setting the ZEN_LOG_LEVEL to debug using inline environment variables. Note that Environment Variables are not loaded from .env file when running tests.

4. Notification and Reporting System

Just like Tower lib, Zen also provides an API to send notifications. However, unlike Tower, Zen is a Client-Server architecture. The usage of this library acts as a client to the server. The server (see: serve.go) is also on this repository, but it's not designed to be used as a library and should not be imported in any other services.

This implementation allows non-Golang services to also send notifications to the server, and also allows governance to control the notification system centrally.

The server uses ConnectRPC as protocol, allowing to use grpc and http as transport, and allows code generation for other languages.

5. Instrumented Clients

The library provides an API to setup clients for Bareksa's infrastructure that are instrumented with OpenTelemetry out of the box.

Clients:

1. Redis
2. Any database that supports database/sql interface, e.g. MySQL, PostgreSQL, etc.
3. HTTP Server/Client
4. GRPC Server/Client
5. Kafka (Sarama)

Documentation

WIP