m3tal-core

README

M3TAL System Documentation

Overview

This document provides technical details and operational procedures for the M3TAL system.

Prerequisites

Docker Engine and Docker Compose V2 are strictly REQUIRED and must be installed prior to M3TAL installation and operation. M3TAL internally orchestrates Docker containers using Docker Engine and Docker Compose V2.

Installation

To install the M3TAL CLI and API daemon via APT, follow these steps:

# 1. Add the GPG signing key
curl -fsSL https://jakej985-rgb.github.io/m3tal-core/KEY.gpg | sudo gpg --dearmor -o /usr/share/keyrings/m3tal-archive-keyring.gpg

# 2. Add the APT repository
echo "deb [signed-by=/usr/share/keyrings/m3tal-archive-keyring.gpg] https://jakej985-rgb.github.io/m3tal-core stable main" | sudo tee /etc/apt/sources.list.d/m3tal.list

# 3. Install
sudo apt update && sudo apt install -y m3tal

Firewall Considerations

If you are using Traefik for public-facing access (i.e., DASHBOARD_EXPOSE_MODE=traefik or exposing other services via Traefik), ensure that host port 80 (and 443 if HTTPS is configured) is open in your firewall (e.g., ufw allow 80/tcp).

Filesystem Contract

The M3TAL system adheres to the following filesystem structure for critical components:

Path	Purpose
/etc/m3tal/.env	Primary configuration file. Managed by m3tal config wizard.
/var/lib/m3tal/state.db	SQLite state database. Auto-created by the API daemon.
/opt/m3tal/stack/	Canonical stack directory. Contains compose files and Traefik config.
/docker	Symlink → /opt/m3tal/stack/. This is the user-facing path for all stack operations.
/docker/users.json	Dashboard credential store. Managed by m3tal dashpass.

Core Components

The M3TAL system consists of several interconnected components:

• CLI binary (/usr/bin/m3tal): A Go binary serving as the unified entrypoint for all M3TAL operations. It is installed via APT.
• API daemon (m3tal-api.service): A Go binary running as a systemd service, listening on host-local port 8080. It manages Docker interactions, the internal state database, and exposes the M3TAL API routes.
• Dashboard container (m3tal-dashboard): A Python/Flask application container running internally on port 8082. It communicates with the API daemon at http://host.docker.internal:8080.
• Traefik gateway (routing-compose.yml): A reverse proxy container that exposes services by domain name, typically on host port 80. It utilizes a file provider for dynamic routing configurations.
• Cloudflared (routing-compose.yml): An optional Cloudflare tunnel container, facilitating zero-configuration internet access for exposed services.

Deployment Lifecycle

M3TAL orchestrates Docker containers using Docker Compose V2. The m3tal up command is a wrapper around docker compose that operates on all *-compose.yml files located within the /docker/ directory, effectively deploying each as an independent stack.

The /opt/m3tal/stack/ directory serves as the canonical source of truth where all core M3TAL stack files reside. The /docker directory is a symlink to /opt/m3tal/stack/ and acts as the user-facing alias for all stack operations, including adding new user-defined stacks.

Adding a New Stack

To deploy a new Docker Compose stack:

1. Place your Docker Compose file (e.g., my-stack-compose.yml) directly into the /docker/ directory. This file will be automatically included by m3tal up.
2. Ensure any required environment variables for your new stack are set in /etc/m3tal/.env (e.g., using m3tal config set KEY value).
3. Run m3tal up to start all defined stacks, including your newly added one.

Dashboard Access

The M3TAL Dashboard can be accessed in two distinct modes, controlled by the DASHBOARD_EXPOSE_MODE variable in /etc/m3tal/.env.

Local Mode (Default)

• Configuration: DASHBOARD_EXPOSE_MODE=local (This is the default setting for a new installation).
• Mechanism: M3TAL uses the m3tal-compose.local.yml override file, which adds a direct port binding to the dashboard container (${DASHBOARD_PORT:-8082}:8082).
• Access: Direct access via http://HOST_IP:8082 or http://localhost:8082.
• Requirements: No Traefik instance is required for this mode.
• Use Case: Ideal for LAN-only setups, initial configurations, first-time users, and local development or testing. A new user performing a default installation will access the dashboard directly via port 8082.

Traefik Mode

• Configuration: DASHBOARD_EXPOSE_MODE=traefik.
• Mechanism: M3TAL uses the m3tal-compose.traefik.yml override file, which applies Traefik labels to the dashboard service. Traefik, if running, interprets these labels to route dash.${DOMAIN} to the dashboard container's internal port 8082.
• Access: Via a configured domain, typically http://dash.DOMAIN.
• Requirements: A Traefik gateway must be running and correctly configured (e.g., via m3tal up).
• Use Case: Suited for domain-based environments where multiple services are exposed behind a single reverse proxy.

Traefik Gateway

Traefik automatically discovers and routes traffic to Docker services by interpreting Traefik labels defined within their Docker Compose service definitions. Crucially, services are not exposed by Traefik by default and require traefik.enable=true along with other relevant labels to be discoverable and routable.

Traefik is deployed as a container via the routing-compose.yml stack. It binds host port 80 as its primary HTTP entry point and loads dynamic configuration from files located in /docker/dynamic/ (which is mapped to /etc/traefik/dynamic inside the container), allowing for hot-reloading of routing rules.

For example, Traefik routes requests for api.DOMAIN to the M3TAL Go API daemon, which listens on the host-local port 8080. This is achieved through a dynamic configuration file (e.g., /docker/dynamic/api.yml) that defines a router and a service pointing to http://host.docker.internal:8080. Similarly, dash.DOMAIN is routed to the M3TAL dashboard container (when DASHBOARD_EXPOSE_MODE=traefik) using labels directly on the dashboard service.

Exposing a Custom User Service via Traefik

To expose your own Docker Compose service via Traefik, add appropriate labels to its service definition:

# /docker/my-app-compose.yml
services:
  my-app:
    image: nginx:alpine
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.myapp.rule=Host(`app.DOMAIN`)"
      - "traefik.http.services.myapp.loadbalancer.server.port=80"
      - "traefik.http.routers.myapp.entrypoints=web"
    networks:
      - proxy

networks:
  proxy:
    external: true

After placing this file in /docker/, run m3tal up to deploy your service and configure Traefik.

Service Management

The M3TAL API daemon runs as a systemd service. You can manage it using standard systemctl commands:

• Check status: systemctl status m3tal-api
• Restart service: systemctl restart m3tal-api
• View logs: journalctl -u m3tal-api -f

Quick Demo

To quickly get the M3TAL Dashboard up and running for initial setup or testing:

• Start the Dashboard specifically: Use m3tal dash up. This command fetches the latest dashboard compose files, configures them based on your DASHBOARD_EXPOSE_MODE setting, and starts only the m3tal-dashboard container.
• Deploy all M3TAL stacks and user-defined services: Use m3tal up. This command orchestrates and deploys all *-compose.yml files found within the /docker/ directory, including core M3TAL services (like Traefik if configured) and any user-defined Docker Compose files.

Port Map

The following table lists the primary network ports utilized by the M3TAL system:

Port	Service	Access	Description
80	Traefik HTTP entry point	Public	The public-facing HTTP port for services exposed via Traefik.
8080	M3TAL API daemon (Go)	Host-local	The internal port the M3TAL API daemon listens on.
8081	Traefik dashboard	Host-local only	The internal Traefik dashboard port, accessible only from the host machine.
8082	M3TAL Dashboard	Direct port (local mode) or via Traefik (traefik mode)	The port the M3TAL Dashboard container listens on internally. Access method depends on DASHBOARD_EXPOSE_MODE.

Note: These are the primary M3TAL-managed ports. User-added Docker Compose stacks may expose additional ports on their own, depending on their configurations.