Installing and configuring Tengri

Operating Systems

Installation from DEB packages is supported for the following operating systems:

  • Debian 12 (Bookworm)

  • Debian 13 (Trixie) — recommended

  • Ubuntu 22.04 LTS (Jammy Jellyfish)

  • Ubuntu 24.04 LTS (Noble Numbat)

Open ports

Ports must be open for installation:

  • 5433 — client connections (ODBC/JDBC, PostgreSQL)

  • 3000 — web UI

  • 3001 — WebSocket connections

  • 3005 — Iceberg REST API

  • 9002 — S3 API

  • 9591 — OpenTelemetry metrics

For cluster configuration only:

  • 2379 — client connections to etcd

  • 2380 — traffic between etcd nodes

  • 57776 — RPC (inter-node communication)

The full list of ports of all services and firewall configuration guidance are on the Network ports page.

Tengri installation

To install Tengri in single-machine configuration:

  1. Get the archive with the necessary files and unpack it:

    tar xf example_client.tar.gz
    cd example_client
    example_client/
    ├── client.crt
    ├── client.key
    ├── install.sh
    └── license.txt
  2. Run the script to install the certificate, configuration file, and APT repository:

    sudo ./install.sh
  3. Install the packages:

    1. Only Tengri

      sudo apt install tengri
    2. Tengri with local S3 (MinIO)

      sudo apt install tengri tengri-minio
  • Once installed, Tengri will be ready to run on the 3000 port.
    The default login and password are admin / admin.

  • The tengri-minio package will install local MinIO. The password of its built-in minioadmin user is generated during the installation and written to /etc/tengri/secrets.env, where Tengri reads it from; see Local MinIO storage for details. Other settings are described in the MinIO documentation.

Tengri installation in cluster configuration

Tengri uses a distributed architecture with two main components:

  • Endpoints — SQL-server instances that clients connect to

  • Agents — compute wokers that execute queries

All nodes automatically discover each other via etcd — a distributed coordination service.

Preliminary requirements

  • Minimum 3 servers (on Debian 13 Trixie or Ubuntu 22.04 LTS Jammy Jellyfish)

  • Network connectivity between all servers

  • Open ports

  • A shared metadata catalog — a single PostgreSQL instance reachable from every server

Step 1: Install Tengri on each server

On each server in your cluster, run:

apt install tengri

This command will install all the required components, including the tengri-etcd package with the etcd service. Right after the installation, etcd and both Tengri services are already running in a single-node configuration — the steps below turn them into a cluster.

Step 2: Configuring etcd

The tengri-etcd package installs the /etc/etcd/etcd.conf.yml configuration file. Describe the nodes in the file, then point the service at that file.

The initial-* parameters only take effect when the cluster is bootstrapped. If etcd has already been started in a single-node configuration, stop the service on each server and clear the data directory before forming the cluster:

systemctl stop etcd
rm -rf /var/lib/etcd/default/*
  1. Describe the cluster nodes in /etc/etcd/etcd.conf.yml. An example for the first server:

    name: "tengri-etcd-1"
    data-dir: "/var/lib/etcd/default"
    listen-client-urls: "http://0.0.0.0:2379"
    listen-peer-urls: "http://0.0.0.0:2380"
    advertise-client-urls: "http://<THIS_SERVER_IP>:2379"
    initial-advertise-peer-urls: "http://<THIS_SERVER_IP>:2380"
    initial-cluster: "tengri-etcd-1=http://<SERVER_1_IP>:2380,tengri-etcd-2=http://<SERVER_2_IP>:2380,tengri-etcd-3=http://<SERVER_3_IP>:2380"
    initial-cluster-token: "tengri-cluster"
    initial-cluster-state: "new"

    On each server, name — the unique node name — and both *-advertise-* addresses, which point to that server, are different. The initial-cluster, initial-cluster-token and initial-cluster-state parameters are the same on all servers.

  2. Point the service at the configuration file: in /usr/lib/systemd/system/etcd.service, set the value of DAEMON_ARGS in the [Service] section:

    Environment="DAEMON_ARGS=--config-file=/etc/etcd/etcd.conf.yml"
    Values from the configuration file take precedence over the ETCD_* variables set in the unit.
    To keep the change across package upgrades, run systemctl edit etcd instead of editing the unit and add the same line to the [Service] section of the drop-in file.
  3. Restart etcd on each server:

    systemctl daemon-reload
    systemctl restart etcd
  4. Check that the cluster has formed:

    etcdctl --endpoints=http://<SERVER_1_IP>:2379,http://<SERVER_2_IP>:2379,http://<SERVER_3_IP>:2379 member list -w table
    etcdctl --endpoints=http://<SERVER_1_IP>:2379,http://<SERVER_2_IP>:2379,http://<SERVER_3_IP>:2379 endpoint health

Step 3: Configuring each Tengri node

Tengri settings are specified in the /etc/tengri/tengri.conf file (by default, there is no file — create it). The services read it as a systemd environment file: lines of the form PARAMETER=value, without the export prefix.

On each server:

TNGRI_ADVERTISE_RPC_HOST=<THIS_SERVER_IP>
TNGRI_ETCD_HOST="<SERVER_1_IP>:2379 <SERVER_2_IP>:2379 <SERVER_3_IP>:2379"
TNGRI_CATALOG_URI=postgresql://prostore:prostore@<CATALOG_IP>:5432/prostore
  • TNGRI_ADVERTISE_RPC_HOST — the address other cluster nodes use to reach this server’s RPC. The default value (localhost) does not work in a cluster.

  • TNGRI_ETCD_HOST — the addresses of the etcd nodes, separated by spaces. The same on all servers.

  • TNGRI_CATALOG_URI — the connection string of the metadata catalog. The same on all servers: cluster nodes work with a shared catalog rather than with the local one configured at installation.

The remaining parameters are listed in the Configuration settings section.

Step 4: Launching Tengri services

The tengri (endpoint) and tengri-agent (agent) services are enabled and started when the package is installed. After changing the configuration, restart them on each server:

systemctl restart tengri tengri-agent

If the services have been stopped, enable and start them:

systemctl enable --now tengri tengri-agent

Once started, the nodes automatically:

  1. Register with etcd

  2. Discover other endpoints and agents

  3. Establish RPC connections for distributed query execution

Step 5: Access the admin panel

Open a browser and navigate to the admin interface of any endpoint:

http://<ANY_ENDPOINT_IP>:3000/admin

Cluster tab

For each server, create its description in the cluster:

  • Click "New Endpoint"

  • Enter the parameters: address and RPC port, if different

  • Click "Done"

After that, the indicators of all servers should turn green, which means that the cluster is up and running.

In the Cluster tab you can view:

  • All registered endpoints with their RPC addresses

  • All registered agents with their RAM allocation

  • Status of connections between nodes


Done! Your Tengri cluster is now running on multiple servers with automatic load balancing.

Troubleshooting

Nodes are not detecting each other:

  • Check that etcd is running: etcdctl endpoint health.

  • Check the firewall rules for ports 2379, 2380 and 57776

  • Check that TNGRI_ADVERTISE_RPC_HOST points to an available IP (not localhost)

View cluster status directly:

etcdctl get --prefix /service/tengri/

Configuration settings

  • Additional configuration settings for Tengri can be specified in the /etc/tengri/tengri.conf file (by default, there is no file).

    Full list of possible settings in the tengri.conf file
    # TNGRi configuration parameters
    
    # =========
    # Licencing
    # =========
    #
    TNGRI_LICENSE_FILE=/opt/tengri/license.txt
    
    # ===========================
    # S3 configuration parameters
    # ===========================
    #
    #
    # Access key id and secret key of user that will connect to S3
    # rename SECRET_ACCESS_KEY to SECRET_KEY
    TNGRI_S3_ACCESS_KEY_ID=minioadmin
    TNGRI_S3_SECRET_ACCESS_KEY=minioadmin
    #
    # Region to send to S3 server. Generally you don't need to change that, just make
    # sure that target S3 has this same region
    TNGRI_S3_REGION=eu-central-1
    TNGRI_S3_DEFAULT_REGION=eu-central-1
    #
    # API endpoint URL for target S3. Must be in FQDN form
    #     TNGRI_S3_ENDPOINT_URL=http://example.com
    TNGRI_S3_ENDPOINT_URL=http://127.0.0.1:9002
    #
    # Bucket name to use. Must exist before start, so make sure that it exists and
    # has full access rights for this bucket. Must be in full form, i.e.
    #     TNGRI_S3_BUCKET_NAME=s3://example:9000
    TNGRI_S3_BUCKET_NAME=prostore
    #
    TNGRI_S3_BROWSER_DEFAULT_PREFIX=Stage/
    
    # =======================
    # TNGRi server parameters
    # =======================
    #
    #
    # Iceberg catlog path in S3 bucket
    TNGRI_WAREHOUSE_PATH=s3://prostore/iceberg/
    #
    # Name for the default schema in TNGRi.
    TNGRI_DEFAULT_SCHEMA=public
    #
    # URI for catalog endpoint. Must be either PostgreSQL or SQLite connection string, i.e.
    #     TNGRI_CATALOG_URI=postgresql://user:password@example.com/database
    TNGRI_CATALOG_URI=postgresql://prostore:prostore@127.0.0.1:5432/prostore
    
    # Path to writable directory where cached files will be placed.
    #
    # WARNING: Should be placed on SSD storage. Tengri uses aggressive caching strategies
    # to reduce S3 load
    #
    TNGRI_CACHE_DIR=/var/lib/tengri/cache
    #
    # Path to writable directory where python kernels will be stored: the jupyter
    # kernel specs and the connection files of the running kernels.
    TNGRI_KERNEL_DIR=/var/lib/tengri/kernels
    #
    # Path to *.ipynb examples shipped with tengri
    TNGRI_EXAMPLES_DIR=/usr/share/tengri/examples/
    #
    # Path to local duckdb extensions (download from duckdb.org if not set)
    TNGRI_EXTENSION_DIR=/var/lib/tengri/duckdb-extensions
    #
    # If set to false then no agent is required to process queries.
    # Must be true for production environments
    TNGRI_REMOTE_SESSIONS=true
    #
    # Set to true for more log output
    TNGRI_DEBUG=false
    #
    # Host to bind main server to
    TNGRI_HOST=0.0.0.0
    #
    # Port to bind PostgreSQL protocol implementation to
    TNGRI_PORT=5433
    #
    # Host to bind RPC server for communication between agent and server
    TNGRI_RPC_HOST=0.0.0.0
    #
    # Port to bind RPC server for communication between agent and server
    TNGRI_RPC_PORT=57776
    #
    # The name the rest of the installation reaches this node by. Agents dial the
    # endpoint's RPC by it, the endpoint dials each worker by it, and the endpoint
    # reaches its own SQL port by it. Name every node on a multi-node installation.
    # TNGRI_INTERNAL_HOST=localhost
    #
    # Host part of the RPC address this node publishes for its peers, when it has to
    # differ from TNGRI_INTERNAL_HOST.
    # TNGRI_ADVERTISE_RPC_HOST=
    
    # =============================
    # Web application configuration
    # =============================
    #
    #
    # SSL configuration. If set then Web communication will be done over SSL, otherwise
    # it will be plaintext.
    #
    # Path to SSL key.
    # TNGRI_SSL_KEY=
    #
    # Path to SSL certificate file.
    # TNGRI_SSL_CERT=
    #
    # Enables TNGRi web app.
    TNGRI_ENABLE_WEBSOCKET=true
    #
    # Path to static files for web app.
    TNGRI_WEBAPP_DIR=/usr/share/tengri-web
    #
    # Host to bind HTTP server to
    TNGRI_WEBAPP_HOST=0.0.0.0
    #
    # Port to bind HTTP server to
    TNGRI_WEBAPP_PORT=3000
    #
    # Host to bind Websocket server to. It is used extensively by webapp and python clients
    TNGRI_WEBSOCKET_HOST=0.0.0.0
    #
    # Port to bind Websocket server to
    TNGRI_WEBSOCKET_PORT=3001
    
    # ============================
    # TNGRi services configuration
    # ============================
    #
    # Enable TNGRi's Iceberg REST catalog API provider
    TNGRI_ENABLE_REST_CATALOG=true
    
    # Host to bind REST catalog server to
    TNGRI_REST_CATALOG_HOST=0.0.0.0
    
    # Port to bind REST catalog to
    TNGRI_REST_CATALOG_PORT=3005
    
    # Enable compaction daemon
    TNGRI_ENABLE_COMPACTION=true
    
    # Enable notebook CRON daemon
    TNGRI_ENABLE_CRON=true
    
    # Enable the S3 access proxy: staging traffic goes through it and the control
    # plane it reads credentials and rulings from runs here. The proxy endpoint
    # below is ignored while this is off.
    TNGRI_S3_ACCESS_ENABLE_CONTROL=true
    
    # Where the S3 access proxy listens, as this server reaches it
    TNGRI_S3_ACCESS_PROXY_ENDPOINT_URL=http://127.0.0.1:9003
    
    # =============================
    # Prometheus metrics exporter
    # =============================
    #
    # Enable Prometheus metrics HTTP server
    # TNGRI_ENABLE_PROMETHEUS=true
    #
    # Host to bind Prometheus metrics server to
    # TNGRI_PROMETHEUS_HOST=127.0.0.1
    #
    # Port for Prometheus metrics
    # TNGRI_PROMETHEUS_PORT=9591
    #
    # URI path for metrics endpoint
    # TNGRI_PROMETHEUS_URI=/metrics
    #
    # Metrics namespace (prefix for metric names, e.g. tengri_up, tengri_login_total)
    # TNGRI_PROMETHEUS_NAMESPACE=tengri
    
    # ============================
    # TNGRi deployment information
    # ============================
    #
    # Needed for external clients to connect to TNGRi services. For example, python clients
    # connecting from user's local machines. Set these to the values that point to this
    # installation, for example, if using reverse proxy for HTTP or Postgres protocol.
    #
    # The name this installation answers to. Every address below is built from it unless
    # set explicitly, and notebook kernels reach the web app by it, so with TNGRI_SSL_KEY
    # in use it must be the name the certificate was issued for.
    # TNGRI_ENDPOINT_HOST=localhost
    #
    # Change the name of site deployment
    # TNGRI_SITE_NAME=tngri
    #
    # Web app address to connect to. Notebook kernels reach it too, i.e.
    #     TNGRI_SITE_HTTP_ADDR=https://example.com:3000
    # TNGRI_SITE_HTTP_ADDR=
    #
    # Websocket address to connect to. Must be in FQDN form, i.e.
    #     TNGRI_SITE_WS_ADDR=ws://example.com:3001
    # TNGRI_SITE_WS_ADDR=
    #
    # Postgres protocol address to connect to, i.e.
    #     TNGRI_SITE_PSQL_ADDR=example.com:5433
    # TNGRI_SITE_PSQL_ADDR=
    #
    # S3 address to connect to. Must be in FQDN form, i.e.
    #     TNGRI_SITE_S3_ADDR=http://example.com:9000
    # TNGRI_SITE_S3_ADDR=
    #
    # S3 access key for public usage (Stage only)
    # TNGRI_SITE_PUBLIC_LOADER_ACCESS_KEY=public_loader
    #
    # S3 secret key for public usage (Stage only)
    # TNGRI_SITE_PUBLIC_LOADER_SECRET_KEY=public_loader
    #
    # =============================
    # Thirdparty resources configuration
    # =============================
    #
    #
    # Pypi repository to be used in python environments within tengri (default https://pypi.org/simple/)
    #
    # TNGRI_PIP_INDEX_URL=
    #
    # Where browser notebooks fetch Pyodide packages that are not bundled with the
    # instance (default https://cdn.jsdelivr.net/pyodide/v{version}/full/, where
    # {version} is the Pyodide the frontend was built against). An empty value
    # limits browser notebooks to the bundled packages.
    #
    # TNGRI_PYODIDE_CDN_URL=
    #
    # PyPI repository for browser notebooks, searched after the bundled wheels.
    # Follows TNGRI_PIP_INDEX_URL unless set; an empty value keeps browser
    # notebooks off any public index.
    #
    # TNGRI_PYODIDE_PIP_INDEX_URL=
    #
    # npm registry to be used in node environments within tengri (default
    # https://registry.npmjs.org). Point it at an internal mirror to keep dashboard
    # builds off the public internet.
    #
    # TNGRI_NPM_REGISTRY_URL=
    #
    # Python container image to run python code in (default python-env)
    #
    # TNGRI_CONTAINER_IMAGE_PYTHON=
    #
    # Shipped defaults for the Tengri S3 access proxy. Each setting is also a
    # command-line flag: the variable name is the flag in upper case with dashes
    # as underscores.
    #
    # The object store the proxy sits in front of. The credential it holds on
    # every client's behalf is TENGRI_S3_PROXY_UPSTREAM_ACCESS_KEY and
    # TENGRI_S3_PROXY_UPSTREAM_SECRET_KEY, set in secrets.env.
    # TENGRI_S3_PROXY_UPSTREAM=http://127.0.0.1:9002
    
    # Where clients reach the proxy. The admin and status surface stays on
    # loopback.
    # TENGRI_S3_PROXY_LISTEN=:9003
    # TENGRI_S3_PROXY_ADMIN_LISTEN=127.0.0.1:9004
    
    # The Tengri server's control plane: who an access key speaks for, and
    # whether that principal may reach a key.
    TENGRI_S3_PROXY_CREDENTIAL_ENDPOINT=http://127.0.0.1:3006/internal/s3/credentials
    TENGRI_S3_PROXY_DECISION_ENDPOINT=http://127.0.0.1:3006/internal/s3/decision
    
    # The prefixes ruled on per principal. Everything else is forwarded on
    # authentication alone, so widening this list is how the rollout advances.
    # /Stage is the staging root: where the browser is rooted and where an account's
    # own prefix, its folder and the shared one all live.
    TENGRI_S3_PROXY_GOVERNED_PREFIXES=/Stage,-/iceberg,!/iceberg/public
  • Additional MinIO configuration settings can be specified in the /etc/default/tengri-minio file according to MinIO documentation.

    After updating the MinIO configuration file, it is necessary to execute:

    sudo systemctl daemon-reload
    sudo systemctl restart minio

    If MINIO_ROOT_USER or MINIO_ROOT_PASSWORD were changed in the file, run sudo tengri-minio-sync-credentials before the restart and restart tengri as well, see Changing the password.

Resetting the administrator password

In case you have lost access to the administrator password, you can reset it.

To do this, run the command on the server where the package was installed:

sudo -u tengri bash -c 'export $(egrep -v "^#|^$|.*#" /var/lib/tengri/tengri.defaults | xargs); prostore reset-admin-password'

After that the current administrator password will be reset and the new automatically generated password will be shown in the output log.