docs: add README.md
This commit is contained in:
parent
d2112053b2
commit
2a80be3a6b
104
README.md
Normal file
104
README.md
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
# Zen
|
||||
|
||||
In house Library to solve the biggest problem in the micro services world, **Observability**.
|
||||
|
||||
To setup proper Observability, we have to fulfill the three pillars of Observability, which are:
|
||||
|
||||
1. **Logging**
|
||||
2. **Metrics**
|
||||
3. **Tracing**
|
||||
|
||||
There are surprising big requirements and code setup to take the maximum advantage of these pillars, which are:
|
||||
|
||||
1. Structured and Standardized Logging format.
|
||||
2. Metrics collection that have proper labels and contexts.
|
||||
3. Tracing that should be able to cross the service boundaries.
|
||||
|
||||
Lot of knowledge and boilerplate code is required to setup these pillars, which is not only time consuming but also error prone.
|
||||
|
||||
This library is an attempt to solve these problems by providing a simple and easy to use API to setup Observability in your services.
|
||||
|
||||
Some API are just re-exports, especially those that comes from [OpenTelemetry], but the usage of these API is already simplified and made more user friendly.
|
||||
|
||||
## Status
|
||||
|
||||
Still in development. Not ready even for basic usage.
|
||||
|
||||
## Features
|
||||
|
||||
### 1. Layered Config
|
||||
|
||||
The library provides a layered configuration system, which allows the user to provide overrides at different levels.
|
||||
|
||||
The library provides the following levels of configuration with the latter overriding the former:
|
||||
|
||||
1. **Default Config**: The default configuration that comes with the library. This only contains configuration for `zen` library itself.
|
||||
2. **Vault Config**: The configuration that comes from the [Vault].
|
||||
3. **Config File**: `zconfig.json` and `zconfig.yaml` at the same `cwd` as where the command is run. Ignored if not found. `zconfig.json` is looked first then `zconfig.yaml`.
|
||||
The usage of `Config File` is actually discouraged because they contains secrets and must be inside `.gitignore` for security purposes. But it's there
|
||||
if you want easy configuration override on local development.
|
||||
4. **Environment Variables**: The most interesting and probably the most eye-catching one because it can easily override-able inline by `DevOps` team or in `CI` steps.
|
||||
`Zen` will lookup `.env` file on `cwd` on startup and load up any `environment variables` that **DOES NOT EXIST** before. So `.env` loading will not override
|
||||
any previous set variables. See [Environment Variables](#environment-variables) documentation for more details on how to use this.
|
||||
5. **Command Line Arguments**: This is one of those _putting the cart before the horse_ kind of situation. Popular CLI parser libraries requires flags
|
||||
to be defined first and this library does not know those prior values at the start of the application other than default values inside the library. Thus,
|
||||
configuring config from `cli flags` are limited to only configuring `zen` library itself.
|
||||
|
||||
### 2. [OpenTelemetry] Integration
|
||||
|
||||
[OpenTelemetry] is chosen because it's vendor agnostic nature and are supported by big players in Observability world like [Grafana Stack][Grafana], [Elastic Stack (v2 and up)][Elastic], and [OpenObserve].
|
||||
|
||||
Migrating vendors is just as easy as changing the Collector endpoint in the config.
|
||||
|
||||
Unfortunately, there are always costs to pay for this flexibility. The cost is the complexity of the API and the boilerplate code required to [setup the OpenTelemetry itself][OpenTelemetry Setup].
|
||||
|
||||
This library will simplify the setup of [OpenTelemetry] by providing a simple API to setup the Tracer, Meter, and Logger at the startup of the application.
|
||||
|
||||
This library also has the goal to further integrate Bareksa's infrastructure with [OpenTelemetry] by providing an API to setup `Clients` for these infrastructures instrumented with [OpenTelemetry] out of the box.
|
||||
|
||||
### 3. Developer Friendly Logging
|
||||
|
||||
Structure Logging is good for machines to consume, but it's not very friendly for human read.
|
||||
This library provides the best of both worlds by detecting the environment and choose the best logging format for that environment.
|
||||
|
||||
On, interactive environment (TTY Detected), e.g. running via Terminal Emulator, the library will output the logs in a human readable format.
|
||||
|
||||
On, non-interactive environment, e.g. running in a container, the library will output the logs in a JSON structured format.
|
||||
|
||||
When running tests, by default the library will be silent, but you can easily enable the logs by setting the `ZEN_LOG_LEVEL` to `debug` using inline environment variables.
|
||||
Note that Environment Variables are not loaded from `.env` file when running tests.
|
||||
|
||||
### 4. Notification and Reporting System
|
||||
|
||||
Just like [Tower] lib, `Zen` also provides an API to send notifications. However, unlike [Tower], `Zen` is a Client-Server architecture. The usage of this library
|
||||
acts as a client to the server. The server (see: [serve.go](./cmd/zen/serve/serve.go>)) is also on this repository, but it's not designed to be used as a library and
|
||||
should not be imported in any other services.
|
||||
|
||||
This implementation allows non-Golang services to also send notifications to the server, and also allows governance to control the notification system centrally.
|
||||
|
||||
The server uses [ConnectRPC] as protocol, allowing to use `grpc` and `http` as transport, and allows code generation for other languages.
|
||||
|
||||
### 5. Instrumented Clients
|
||||
|
||||
The library provides an API to setup clients for Bareksa's infrastructure that are instrumented with [OpenTelemetry] out of the box.
|
||||
|
||||
Clients:
|
||||
|
||||
1. Redis
|
||||
2. Any database that supports `database/sql` interface, e.g. MySQL, PostgreSQL, etc.
|
||||
3. HTTP Server/Client
|
||||
4. GRPC Server/Client
|
||||
5. Kafka (Sarama)
|
||||
|
||||
## Documentation
|
||||
|
||||
`WIP`
|
||||
|
||||
[OpenTelemetry]: https://opentelemetry.io
|
||||
[OpenTelemetry Setup]: https://opentelemetry.io/docs/languages/go/getting-started
|
||||
[Vault]: https://vault-ha.bareksa.dev
|
||||
[Grafana]: https://grafana.com
|
||||
[Elastic]: https://www.elastic.co
|
||||
[OpenObserve]: https://openobserve.ai
|
||||
[Tower]: https://gitlab.bareksa.com/backend/tower
|
||||
[ConnectRPC]: https://connectrpc.com
|
||||
Loading…
Reference in a new issue