Local MinIO storage

The tengri-minio package installs a local S3-compatible MinIO object storage on the same server and configures it for Tengri. This page describes where the storage credentials come from, how to view and change them, what happens when the package is upgraded or removed, and how to troubleshoot common problems.

Overview

The package installs the minio server, the minio-client tool and the minio.service unit, which serves the S3 API on port 9002 and the web console on port 9001. The installation creates the prostore bucket (Tengri data) and the mimir bucket (metrics), the prostore policy and the public_loader service account, which can only access the Stage/ and iceberg/public/ paths of the prostore bucket.

Tengri accesses the storage as the built-in MinIO user minioadmin. The password of this user is generated when the package is installed: a random string of 30–40 Latin letters and digits. The password never appears in any package file or in the installation output.

Where the credentials are kept

File Purpose

/etc/tengri/secrets.env

The source of the credentials for the Tengri services: TNGRI_S3_ACCESS_KEY_ID and TNGRI_S3_SECRET_ACCESS_KEY. The file is owned by root, group tengri, mode 0640; it is maintained by the packages and does not need manual editing.

/etc/default/tengri-minio

MinIO startup parameters: MINIO_VOLUMES, MINIO_OPTS and, if the administrator set them, MINIO_ROOT_USER and MINIO_ROOT_PASSWORD. The package never modifies this file.

/etc/tengri/tengri.conf

Tengri settings overriding secrets.env. For MinIO this file is not a source of credentials: the TNGRI_S3_* values in it apply to the Tengri services only.

minio.service starts through a wrapper that assembles the credentials from the two files: a value in /etc/default/tengri-minio takes precedence, what is missing there comes from secrets.env. The user and the password are looked up independently; when the user is set nowhere, minioadmin is used. Differences between the files are reported to the journal without revealing the values.

To view the current credentials:

sudo grep '^TNGRI_S3_' /etc/tengri/secrets.env

TNGRI_S3_ACCESS_KEY_ID and TNGRI_S3_SECRET_ACCESS_KEY in /etc/tengri/tengri.conf are the regular way to give the Tengri services other credentials: a dedicated MinIO account instead of minioadmin, for example, or an external S3. MinIO does not read this file, so the service wrapper and the tengri-minio-sync-credentials command only report the difference from secrets.env without changing tengri.conf. Such a warning means no more than that Tengri and MinIO use different credentials; if that is intended, it can be ignored.

Changing the password

  1. Set the new values in /etc/default/tengri-minio (the minimum MinIO password length is 8 characters):

    MINIO_ROOT_USER=minioadmin
    MINIO_ROOT_PASSWORD=<new password>
  2. Copy them to secrets.env for the Tengri services:

    sudo tengri-minio-sync-credentials

    The command only copies the values from /etc/default/tengri-minio to /etc/tengri/secrets.env; it generates nothing and restarts nothing. If tengri.conf holds different values, it warns about them.

  3. Restart the services:

    sudo systemctl restart minio tengri

The password may stay in /etc/default/tengri-minio afterwards or be removed from it: secrets.env already holds a copy, and the wrapper takes the password from there. The public_loader account is not affected by a password change.

tengri-mimir keeps its own S3 credentials in /etc/mimir/config.yaml (access_key_id, secret_access_key); after the password is generated or changed, update them to the values from secrets.env and restart mimir.

Upgrading from previous versions

On upgrade the package checks whether it knows the password of the running server and reports the outcome in the installation output:

  • No password was set before — a new one is generated and written to secrets.env, and the MinIO and Tengri services are restarted. Nothing needs to be done.

  • A password is set in /etc/default/tengri-minio — it is copied to secrets.env, nothing is generated. The output shows NOTICE: THE MINIO ROOT CREDENTIALS …​ WERE COPIED.

  • The password is unknown: the server runs with a password that neither file holds — the package is installed, but minio.service will not start. The output shows WARNING: THE MINIO ROOT PASSWORD IS NOT KNOWN TO THE PACKAGES.

In the last case there are two ways forward:

  • If the administrator knows the password — set it in /etc/default/tengri-minio and follow Changing the password.

  • If the stored data is not needed — stop the service, empty the data directory and reinstall the package; the reinstallation generates a new password:

    sudo systemctl stop minio
    sudo rm -rf /var/lib/minio/*
    sudo apt install --reinstall tengri-minio

Troubleshooting

The wrapper records every reason minio.service refuses to start in the service journal:

sudo journalctl -u minio -n 50

A configuration error exits the service with code 78, and systemd does not restart it automatically: after fixing the files, start the service by hand with sudo systemctl restart minio.

Journal message What to do

no root password: MINIO_ROOT_PASSWORD is not set in /etc/default/tengri-minio and TNGRI_S3_SECRET_ACCESS_KEY is not set in /etc/tengri/secrets.env

No file holds a password. Set it in /etc/default/tengri-minio and run tengri-minio-sync-credentials, or reinstall the package to generate a new one.

…​ is set to an empty value in …​

The variable exists in the named file with an empty value. Set a value or remove the line.

MINIO_VOLUMES is not set in /etc/default/tengri-minio

Restore the MINIO_VOLUMES="/var/lib/minio" line in the file.

warning: …​ differs …​

The values in the files differ; the service starts. If /etc/default/tengri-minio and secrets.env differ, run tengri-minio-sync-credentials. If tengri.conf is the one that differs, make sure the override for Tengri is intended.

To verify that the Tengri services hold the same credentials as MinIO, run the client as the user from secrets.env:

sudo sh -c '. /etc/tengri/secrets.env; \
  MC_HOST_local="http://${TNGRI_S3_ACCESS_KEY_ID:-minioadmin}:${TNGRI_S3_SECRET_ACCESS_KEY}@127.0.0.1:9002" \
  minio-client ls local'

The expected output is the list of the mimir and prostore buckets.

Removing the package

Removing tengri-minio deletes the /var/lib/minio data directory with the contents of the prostore and mimir buckets and removes TNGRI_S3_ACCESS_KEY_ID and TNGRI_S3_SECRET_ACCESS_KEY from /etc/tengri/secrets.env. No backup is made.

tengri-minio is a dependency of tengri-mimir, which in turn is a dependency of tengri, so apt remove tengri-minio removes those as well. To switch to an external S3, set TNGRI_S3_ENDPOINT_URL and the credentials in /etc/tengri/tengri.conf without removing the package.