Installing Loki Log Aggregation: A Step-by-Step Guide

Loki is a popular open-source log aggregation tool that, when paired with Grafana (Official site), provides a scalable and efficient way to handle log management and visualization. Whether you’re running applications on Kubernetes, microservices, or standalone servers, Loki is designed to simplify log collection and analysis without complicated setups like other logging solutions.

Prerequisites

A server or a system running a recent version of Linux (Ubuntu, CentOS, etc.).
Basic knowledge of terminal operations.
Docker and Docker Compose installed on your system. If you haven’t set up Docker yet, check our guide on Installing Fluentd.
An operational Grafana instance for visualization.

Step 1: Setting Up Docker Environment

First, ensure your Docker and Docker Compose installations are up to date. Update your system packages and Docker as follows:

sudo apt update
sudo apt upgrade
sudo systemctl restart docker

Confirm Docker is running without issues:

docker --version
docker-compose --version

Step 2: Create a Docker Compose File

Create a new directory for Loki. Inside this directory, create a docker-compose.yml file:

mkdir loki-promtail
echo "version: '2'

services:
  loki:
    image: grafana/loki:2.3.0
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    volumes:
      - ./loki:/etc/loki

  promtail:
    image: grafana/promtail:2.3.0
    volumes:
      - /var/log:/var/log
      - ./promtail:/etc/promtail
    command: -config.file=/etc/promtail/config.yaml
" > docker-compose.yml

Step 3: Loki Configuration

Create a local-config.yaml for Loki in the same configuration directory:

mkdir loki
echo "auth_enabled: false
server:
  http_listen_port: 3100
ingester:
  lifecycler:
    ring:
      kvstore:
        store: inmemory
    final_sleep: 0s
delete_requests_ignore_unknown_tenant: true
" > loki/local-config.yaml

Step 4: Promtail Configuration

Promtail is used to ship logs to Loki. Configure its settings:

mkdir promtail
echo "server:
  http_listen_port: 9080
  grpc_listen_port: 0
positions:
  filename: /tmp/positions.yaml
clients:
  - url: http://loki:3100/loki/api/v1/push
scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*log
" > promtail/config.yaml

Step 5: Running Loki and Promtail

Use Docker Compose to start Loki and Promtail:

docker-compose up -d

Loki and Promtail will start in detached mode. Verify their status:

docker-compose ps

Step 6: Configure Grafana to Use Loki

Now log in to Grafana and configure it to use Loki:

Go to Settings > Data Sources.
Click Add Data Source and select Loki from the list.
Enter http://localhost:3100 as the URL.
Click Save & Test to check if Grafana can reach Loki.

Once set up, create dashboards to visualize your logs effectively.

Troubleshooting Tips

If Loki or Promtail fails to start, check their logs using docker-compose logs loki/promtail.
Ensure ports 3100 (Loki) and 9080 (Promtail) are open on your firewall.
Check configuration files for any syntax errors.