m3tal-core

README

M3TAL System Documentation

Overview

This document provides technical details and operational procedures for the M3TAL system. It describes the core components, deployment methods, access mechanisms, and service management practices for M3TAL.

Prerequisites

Docker Engine and Docker Compose V2 are strictly REQUIRED and must be installed prior to M3TAL installation and operation. M3TAL internally orchestrates Docker containers using Docker Engine and Docker Compose V2.

Installation

The M3TAL CLI binary and API daemon are distributed via an APT repository. Follow these steps to install M3TAL:

# 1. Add the GPG signing key
curl -fsSL https://jakej985-rgb.github.io/m3tal-core/KEY.gpg | sudo gpg --dearmor -o /usr/share/keyrings/m3tal-archive-keyring.gpg

# 2. Add the APT repository
echo "deb [signed-by=/usr/share/keyrings/m3tal-archive-keyring.gpg] https://jakej985-rgb.github.io/m3tal-core stable main" | sudo tee /etc/apt/sources.list.d/m3tal.list

# 3. Install
sudo apt update && sudo apt install -y m3tal

Core Components

The M3TAL system comprises several interconnected components:

• CLI binary (/usr/bin/m3tal): A Go binary serving as the unified command-line interface for all M3TAL operations.
• API daemon (m3tal-api.service): A Go binary running as a systemd service, listening on host-local port 8080. It manages Docker interactions, the internal state database, and exposes API routes for M3TAL functionalities.
• Dashboard container (m3tal-dashboard): A Python/Flask application running within a Docker container, internally listening on port 8082. It communicates with the API daemon at http://host.docker.internal:8080.
• Traefik gateway (routing-compose.yml): A Docker container acting as a reverse proxy, exposing services via domain names on host port 80. It uses a file provider for dynamic routing configurations.
• Cloudflared (routing-compose.yml): An optional Docker container for establishing Cloudflare tunnels, facilitating secure public access without direct port exposure.

Filesystem Contract

The following paths define critical locations within the M3TAL filesystem:

Path	Purpose
/etc/m3tal/.env	Primary configuration file for environment variables. Managed by m3tal config wizard.
/var/lib/m3tal/state.db	SQLite state database for the API daemon. Automatically created upon API daemon startup.
/opt/m3tal/stack/	Canonical directory for all Docker Compose stack files and Traefik dynamic configuration.
/docker	Symlink to /opt/m3tal/stack/. This is the user-facing path for all Docker Compose stack operations.
/docker/users.json	Credential store for the M3TAL Dashboard. Managed by m3tal dashpass.

Deployment Lifecycle

M3TAL orchestrates Docker containers using Docker Compose V2. The m3tal up command is a wrapper around docker compose that operates on all *-compose.yml files located within the /docker/ directory, effectively deploying each as an independent stack.

The /opt/m3tal/stack/ directory is the canonical source of truth where all stack files reside. The /docker directory is a symlink to /opt/m3tal/stack/, serving as the user-facing alias for all stack operations.

Adding a New Stack

To deploy a new Docker Compose stack:

1. Place your Docker Compose file (e.g., my-stack-compose.yml) directly into the /docker/ directory. This file will be automatically included by m3tal up.
2. Ensure any required environment variables for your new stack are set in /etc/m3tal/.env (use m3tal config wizard or m3tal config set KEY value).
3. Run m3tal up to deploy all stacks, including your newly added service.

Firewall Considerations

If you are using Traefik for public-facing access (i.e., DASHBOARD_EXPOSE_MODE=traefik or exposing other services via Traefik), ensure that host port 80 (and 443 if HTTPS is configured) is open in your firewall (e.g., ufw allow 80/tcp).

Dashboard Access

The M3TAL Dashboard can be accessed in two distinct modes, controlled by the DASHBOARD_EXPOSE_MODE environment variable in /etc/m3tal/.env.

Local Mode (Default)

• Configuration: DASHBOARD_EXPOSE_MODE=local
• Access: http://HOST_IP:8082 or http://localhost:8082
• Mechanism: This mode directly binds host port 8082 to the dashboard container's internal port 8082 using an override file (m3tal-compose.local.yml). A new user performing a default installation will access the dashboard directly via port 8082. No Traefik configuration or domain is required.
• Use Case: Ideal for LAN-only setups, initial deployments, or local development environments.

Traefik Mode

• Configuration: DASHBOARD_EXPOSE_MODE=traefik
• Access: http://dash.DOMAIN (e.g., http://dash.example.com)
• Mechanism: This mode configures Traefik labels within an override file (m3tal-compose.traefik.yml) to route requests from dash.DOMAIN to the dashboard container's internal port 8082. This requires Traefik to be running via m3tal up.
• Use Case: Suitable for domain-based deployments where multiple services are exposed through a single reverse proxy.

Traefik Gateway

Traefik automatically discovers and routes traffic to Docker services by interpreting Traefik labels defined within their Docker Compose service definitions. Crucially, services are not exposed by Traefik by default and require traefik.enable=true along with other relevant labels to be discoverable and routable.

Traefik is deployed via the routing-compose.yml stack. It binds host port 80 as its primary HTTP entry point.

Traefik also utilizes dynamic configuration files, such as dynamic/api.yml located within the /docker/dynamic/ directory, to define routing for services not explicitly managed by Docker labels. For instance, dynamic/api.yml routes api.DOMAIN to the M3TAL Go API daemon, which listens on the host-local port 8080, specifically http://host.docker.internal:8080 from within the Docker network context. Similarly, when DASHBOARD_EXPOSE_MODE=traefik, dash.DOMAIN is routed to the dashboard container (listening on internal port 8082) via its Traefik labels.

Exposing a Custom Service via Traefik

To expose a custom user service through Traefik, add the necessary labels to its Docker Compose service definition:

# In /docker/my-app-compose.yml
services:
  my-app:
    image: nginx:alpine
    container_name: my-custom-web-app
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.myapp.rule=Host(`app.DOMAIN`)" # Replace DOMAIN with your actual domain
      - "traefik.http.services.myapp.loadbalancer.server.port=80" # Internal port of the service
      - "traefik.http.routers.myapp.entrypoints=web" # Use the 'web' entrypoint (port 80)
    networks:
      - proxy # Ensure your service is on the 'proxy' network for Traefik to discover it

networks:
  proxy:
    external: true # Assumes the 'proxy' network is external and created by M3TAL

After placing this file in /docker/ and running m3tal up, Traefik will automatically configure routing for app.DOMAIN to your my-app container.

Service Management

The M3TAL API daemon (m3tal-api.service) is managed by systemd. Standard systemctl commands apply:

• Check status: systemctl status m3tal-api
• Restart API daemon: systemctl restart m3tal-api
• View logs: journalctl -u m3tal-api -f

Quick Demo

The m3tal CLI provides specific commands for managing components:

• To start only the dashboard container:

  m3tal dash up
  

  This command will download the necessary compose files, read your DASHBOARD_EXPOSE_MODE setting, and start the dashboard with the appropriate configuration.
• To orchestrate and deploy all stacks defined in the /docker/ directory, including the dashboard and any user-defined compose files:

  m3tal up
  

  This command ensures all M3TAL-managed and user-defined Docker Compose stacks are running.

Port Map

The following table lists the primary network ports utilized by the M3TAL system:

| Port | Service | Access | Description Warm up your day, with a cup of joe that awakens your senses, in the midst of a world of flavor, it is worth trying . ```

Environment Variable Configuration

The M3TAL system relies on a set of environment variables for core configuration. These can be adjusted using m3tal config wizard or m3tal config set KEY value.

Key	Default Value	Description
DASHBOARD_PORT	8082	Internal port the M3TAL Dashboard container listens on.
DASHBOARD_EXPOSE_MODE	local	Mode for dashboard access: local (direct port) or traefik (domain).
HTTP_PORT	8080	Port the M3TAL API daemon (Go) listens on.
STATE_DIR	./state	Directory for the SQLite state database.
LOG_LEVEL	info	Logging verbosity level (debug, info, warn, error).
DASHBOARD_SECRET	change_me_immediately	Secret key for the dashboard for session management. CHANGE THIS.
API_TOKEN	change_me_api_token	Bearer token for API authentication. CHANGE THIS.
ADMIN_PASSWORD	admin_pass	Default administrator password for the dashboard.
NETWORK_NAME	m3tal	Name of the Docker network M3TAL services connect to.
LOCAL_IP	127.0.0.1	Local IP address M3TAL components might bind to.
DOMAIN	localhost	Base domain for Traefik routing (e.g., dash.DOMAIN).
VPN_USER	user	Username for VPN services if configured.
VPN_PASSWORD	password	Password for VPN services if configured.
BASE_STORAGE_PATH	./data	Base path for persistent storage volumes.
MEDIA_PATH	./data/media	Path for media storage.
CONFIG_PATH	./data/config	Path for configuration files.
DOWNLOADS_PATH	./data/downloads	Path for downloads storage.
PUID	1000	User ID for container permissions (e.g., for volumes).
PGID	1000	Group ID for container permissions (e.g., for volumes).
TZ	America/Denver	Timezone for containers and services.
TRAEFIK_WEB_PORT	80	Host port Traefik listens on for HTTP traffic.
TRAEFIK_WEBHTTPS_PORT	443	Host port Traefik listens on for HTTPS traffic.
TRAEFIK_DASHBOARD_PORT	8080	Internal Traefik dashboard port (host-local access only).
DEBUG_MODE	false	Enables debug logging and features if set to true.
METRICS_ENABLED	true	Enables or disables metrics collection.