docs: add README.md

README.md

@@ -0,0 +1,104 @@
+# Zen
+
+In house Library to solve the biggest problem in the micro services world, **Observability**.
+
+To setup proper Observability, we have to fulfill the three pillars of Observability, which are:
+
+1. **Logging**
+2. **Metrics**
+3. **Tracing**
+
+There are surprising big requirements and code setup to take the maximum advantage of these pillars, which are:
+
+1. Structured and Standardized Logging format.
+2. Metrics collection that have proper labels and contexts.
+3. Tracing that should be able to cross the service boundaries.
+
+Lot of knowledge and boilerplate code is required to setup these pillars, which is not only time consuming but also error prone.
+
+This library is an attempt to solve these problems by providing a simple and easy to use API to setup Observability in your services.
+
+Some API are just re-exports, especially those that comes from [OpenTelemetry], but the usage of these API is already simplified and made more user friendly.
+
+## Status
+
+Still in development. Not ready even for basic usage.
+
+## Features
+
+### 1. Layered Config
+
+The library provides a layered configuration system, which allows the user to provide overrides at different levels.
+
+The library provides the following levels of configuration with the latter overriding the former:
+
+1. **Default Config**: The default configuration that comes with the library. This only contains configuration for `zen` library itself.
+2. **Vault Config**: The configuration that comes from the [Vault].
+3. **Config File**: `zconfig.json` and `zconfig.yaml` at the same `cwd` as where the command is run. Ignored if not found. `zconfig.json` is looked first then `zconfig.yaml`.
+   The usage of `Config File` is actually discouraged because they contains secrets and must be inside `.gitignore` for security purposes. But it's there
+   if you want easy configuration override on local development.
+4. **Environment Variables**: The most interesting and probably the most eye-catching one because it can easily override-able inline by `DevOps` team or in `CI` steps.
+   `Zen` will lookup `.env` file on `cwd` on startup and load up any `environment variables` that **DOES NOT EXIST** before. So `.env` loading will not override
+   any previous set variables. See [Environment Variables](#environment-variables) documentation for more details on how to use this.
+5. **Command Line Arguments**: This is one of those _putting the cart before the horse_ kind of situation. Popular CLI parser libraries requires flags
+   to be defined first and this library does not know those prior values at the start of the application other than default values inside the library. Thus,
+   configuring config from `cli flags` are limited to only configuring `zen` library itself.
+
+### 2. [OpenTelemetry] Integration
+
+[OpenTelemetry] is chosen because it's vendor agnostic nature and are supported by big players in Observability world like [Grafana Stack][Grafana], [Elastic Stack (v2 and up)][Elastic], and [OpenObserve].
+
+Migrating vendors is just as easy as changing the Collector endpoint in the config.
+
+Unfortunately, there are always costs to pay for this flexibility. The cost is the complexity of the API and the boilerplate code required to [setup the OpenTelemetry itself][OpenTelemetry Setup].
+
+This library will simplify the setup of [OpenTelemetry] by providing a simple API to setup the Tracer, Meter, and Logger at the startup of the application.
+
+This library also has the goal to further integrate Bareksa's infrastructure with [OpenTelemetry] by providing an API to setup `Clients` for these infrastructures instrumented with [OpenTelemetry] out of the box.
+
+### 3. Developer Friendly Logging
+
+Structure Logging is good for machines to consume, but it's not very friendly for human read.
+This library provides the best of both worlds by detecting the environment and choose the best logging format for that environment.
+
+On, interactive environment (TTY Detected), e.g. running via Terminal Emulator, the library will output the logs in a human readable format.
+
+On, non-interactive environment, e.g. running in a container, the library will output the logs in a JSON structured format.
+
+When running tests, by default the library will be silent, but you can easily enable the logs by setting the `ZEN_LOG_LEVEL` to `debug` using inline environment variables.
+Note that Environment Variables are not loaded from `.env` file when running tests.
+
+### 4. Notification and Reporting System
+
+Just like [Tower] lib, `Zen` also provides an API to send notifications. However, unlike [Tower], `Zen` is a Client-Server architecture. The usage of this library
+acts as a client to the server. The server (see: [serve.go](./cmd/zen/serve/serve.go>)) is also on this repository, but it's not designed to be used as a library and
+should not be imported in any other services.
+
+This implementation allows non-Golang services to also send notifications to the server, and also allows governance to control the notification system centrally.
+
+The server uses [ConnectRPC] as protocol, allowing to use `grpc` and `http` as transport, and allows code generation for other languages.
+
+### 5. Instrumented Clients
+
+The library provides an API to setup clients for Bareksa's infrastructure that are instrumented with [OpenTelemetry] out of the box.
+
+Clients:
+
+1. Redis
+2. Any database that supports `database/sql` interface, e.g. MySQL, PostgreSQL, etc.
+3. HTTP Server/Client
+4. GRPC Server/Client
+5. Kafka (Sarama)
+
+## Documentation
+
+`WIP`
+
+[OpenTelemetry]: https://opentelemetry.io
+[OpenTelemetry Setup]: https://opentelemetry.io/docs/languages/go/getting-started
+[Vault]: https://vault-ha.bareksa.dev
+[Grafana]: https://grafana.com
+[Elastic]: https://www.elastic.co
+[OpenObserve]: https://openobserve.ai
+[Tower]: https://gitlab.bareksa.com/backend/tower
+[ConnectRPC]: https://connectrpc.com