WIUNIX ACADEMY
Prometheus and Grafana with Docker Compose: private monitoring and recovery
Page 3 of 6. Review the earlier steps and prerequisites before continuing.
YOUR SYSTEM. YOUR STEPS.
Where are you working?
Choose your OS and version. The prerequisites, commands and troubleshooting follow your environment.
USE YOUR OWN VALUES
Configure this tutorial
Set the example values for your environment; related explanations and copy panels update together. Never enter passwords, private keys or tokens. Values stay in this tab and are not sent to the server or share links. Check actual DNS, certificates and network policy before running commands.
Different IP addresses, domains or ports? Open the settings below, change the values and select “Apply to tutorial”, then copy the updated commands.
Set values for my environment11 editable values
LET’S BUILD
Open the scrape configuration
Create the next file; service names resolve on the Compose network.
Dedicated Linux host; Bash shell
sudo nano '/srv/academy-monitor/config/prometheus.yml'
Ready to save the next configuration.
Scrape without the Docker API
Prometheus scrapes itself and reads service targets from a directory bind. Atomic replacement of a file in this directory remains visible; binding the individual target file would risk an old inode after replacement. Fixed service DNS is enough for this one-instance Compose stack. Docker Swarm or scaled replicas need a distinct discovery/relabel design. Bounded sample and label limits reduce accidental cardinality explosions but do not replace a series budget.
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: prometheus
static_configs:
- targets: ["prometheus:9090"]
- job_name: compose-services
sample_limit: 20000
label_limit: 40
label_value_length_limit: 256
file_sd_configs:
- files: ["/etc/prometheus/targets/*.json"]
refresh_interval: 30s
LET’S BUILD
Create a file-discovery target
Write a non-secret target list. The metrics endpoint has no separate basic auth here because it is reachable only by trusted network peers and host administrators. Grafana login protects dashboards, not this internal metrics endpoint. For an exporter outside this bridge, design reachability, TLS/authentication and label limits separately before adding its address; do not use host loopback inside Prometheus.
Dedicated Linux host; Bash shell
sudo tee '/srv/academy-monitor/config/targets/services.json' >/dev/null <<'JSON'
[{"targets":["grafana:3000"],"labels":{"service":"grafana","stack":"academy-monitor"}}]
JSON
Valid JSON with one Grafana service target.
LET’S BUILD
Provision the Prometheus data source
The data-source URL is resolved by Grafana inside the bridge; it must not be the browser tunnel address. A stable UID connects the dashboard to this source. Provisioning is authoritative on startup; edit the file rather than making a temporary UI-only change. No credential is needed for the private Prometheus endpoint in this bounded lab.
Dedicated Linux host; Bash shell
sudo tee '/srv/academy-monitor/config/datasources/prometheus.yaml' >/dev/null <<'YAML'
apiVersion: 1
datasources:
- name: Academy Prometheus
uid: academy-prometheus
type: prometheus
access: proxy
url: http://prometheus:9090
isDefault: true
editable: false
jsonData:
httpMethod: POST
timeInterval: 15s
YAML
A source file with stable UID and service-DNS URL exists.
LET’S BUILD
Provision a dashboard folder
Use a source-managed dashboard. UI saves are disabled so a restart does not silently erase an edit. The read-only dashboard bind is separate from Grafana’s writable database directory.
Dedicated Linux host; Bash shell
sudo tee '/srv/academy-monitor/config/dashboards/academy.yaml' >/dev/null <<'YAML'
apiVersion: 1
providers:
- name: academy
orgId: 1
folder: Academy
type: file
disableDeletion: true
allowUiUpdates: false
updateIntervalSeconds: 30
options:
path: /var/lib/grafana/dashboards
YAML
Provider points at the mounted dashboard directory.
LET’S BUILD
Write the small reviewable dashboard
This file has two useful panels and no imported marketplace dependencies. up=1 means a successful scrape, not business availability; process memory is not total host/container memory. Container cgroups, host filesystems, remote checks and alert delivery remain separate instrumentation. Do not import a node_exporter dashboard and assume absent host metrics exist.
Dedicated Linux host; Bash shell
sudo tee '/srv/academy-monitor/dashboards/compose.json' >/dev/null <<'JSON'
{
"uid": "academy-compose",
"title": "Compose service health",
"schemaVersion": 39,
"version": 1,
"editable": false,
"timezone": "browser",
"refresh": "15s",
"time": {
"from": "now-30m",
"to": "now"
},
"panels": [
{
"id": 1,
"type": "timeseries",
"title": "Scrape health (1 up / 0 failed)",
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 0
},
"datasource": {
"type": "prometheus",
"uid": "academy-prometheus"
},
"targets": [
{
"refId": "A",
"expr": "up",
"legendFormat": "{{job}} / {{instance}}"
}
],
"fieldConfig": {
"defaults": {
"min": 0,
"max": 1
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom"
}
}
},
{
"id": 2,
"type": "timeseries",
"title": "Process resident memory \u2014 not host RAM",
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 8
},
"datasource": {
"type": "prometheus",
"uid": "academy-prometheus"
},
"targets": [
{
"refId": "A",
"expr": "process_resident_memory_bytes",
"legendFormat": "{{job}} / {{instance}}"
}
],
"fieldConfig": {
"defaults": {
"unit": "bytes"
},
"overrides": []
},
"options": {
"legend": {
"displayMode": "list",
"placement": "bottom"
}
}
}
]
}
JSON
Dashboard JSON references academy-prometheus and contains up/process memory queries.