Grafana Loki
Generated on 20 Mar 2023 from the Grafana Loki catalog page
Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus.
Compared to other log aggregation systems, Loki:
- Indexes a set of labels for each log stream, instead of indexing the context of logs.
- Indexes and groups log streams using the same labels as Prometheus.
- Automatically scrapes and indexes Kubernetes](https://kubernetes.io/) metadata, such as pod logs.
- Has native support in Grafana v6.0.
A Loki-based logging stack consists of 3 components:
- Promtail is the agent, responsible for gathering logs and sending them to Loki.
- Loki is the main server, responsible for storing logs and processing queries.
- Grafana is platform, responsible for querying and displaying the logs.
Notes:
- This stack requires a minimum configuration of two nodes at the $10/month plan (2GB memory / 1 vCPU).
- The Loki stack 1-Click App also includes a $1/month block storage for Loki time series database (starting at PVs of 5GB).
Software Included
Package | Version | License |
---|---|---|
Loki | 2.1.0 | Apache 2.0 |
Promtail | 2.1.0 | Apache 2.0 |
Creating an App using the Control Panel
Click the Deploy to DigitalOcean button to install a Kubernetes 1-Click Application. If you aren’t logged in, this link will prompt you to log in with your DigitalOcean account.
Creating an App using the API
In addition to creating Grafana Loki using the control panel, you can also use the DigitalOcean API. As an example, to create a 3 node DigitalOcean Kubernetes cluster made up of Basic Droplets in the SFO2 region, you can use the following doctl
command. You need to authenticate with doctl
with your API access token) and replace the $CLUSTER_NAME
variable with the chosen name for your cluster in the command below.
doctl kubernetes clusters create --size s-4vcpu-8gb $CLUSTER_NAME --1-clicks loki
Getting Started After Deploying Grafana Loki
Getting Started with DigitalOcean Kubernetes
You can connect to your DigitalOcean Kubernetes cluster by following our how-to guide.
For additional instructions on configuring a DigitalOcean Kubernetes cluster, see the following guides:
- How to Set Up a DigitalOcean Managed Kubernetes Cluster (DOKS)
- How to Set up DigitalOcean Container Registry
Confirming that Loki Stack is Running
First, verify that the Helm installation was successful by running following command:
helm ls -n loki-stack
If the installation was successful, the STATUS
column value in the output reads deployed
:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
loki loki-stack 1 2022-02-16 14:47:29.497728 +0200 EET deployed loki-stack-2.5.1 v2.1.0
Next, verify that the Loki pods are up and running with the following command:
kubectl get pods -n loki-stack
If they’re running, all pods listed in the output are in a READY
state and the STATUS
for each reads Running
:
NAME READY STATUS RESTARTS AGE
loki-0 1/1 Running 0 20h
loki-promtail-kvjxr 1/1 Running 0 20h
loki-promtail-nc7zg 1/1 Running 0 20h
loki-promtail-strvq 1/1 Running 0 20h
Configuring Grafana with Loki
First, expose the Grafana web interface on your local machine:
Note:
Grafana
isn’t installed by default when the Loki Stack
1-Click App is installed and needs to be installed. We recommend installing the Kubernetes Monitoring Stack 1-Click App, which includes Grafana
and its monitoring components.
To access the Grafana Web Panel, run the following command using the default credentials admin/prom-operator
:
kubectl port-forward svc/kube-prometheus-stack-grafana 3000:80 -n kube-prometheus-stack
Navigate to http://localhost:80/ and login with admin and the password (default credentials: admin/prom-operator). Then, follow the instructions for adding the Loki datasource, using the URL http://loki.loki-stack:3100.
For more details about deployment status and functionality, see the Loki tutorial.
Tweaking Helm Values
The loki-stack
has custom default Helm values. See the values file from the main GitHub repository.
To inspect the stack’s current values, run the following command:
helm show values grafana/loki-stack --version 2.5.1
To change these values, open the Helm values file values.yml
, change whatever values you want, save and exit the file, and apply the changes by running helm upgrade
command:
helm upgrade loki grafana/loki-stack --version 2.5.1 \
--namespace loki-stack \
--values values.yml
Upgrading the Loki Stack Chart
YYou can check what versions are available to upgrade by navigating to the loki-stack official releases page from GitHub. Alternatively, you can use ArtifactHUB.
To upgrade the stack to a newer version, run the following command, replacing the < >
placeholders with their corresponding information:
helm upgrade loki grafana/loki-stack \
--version <KUBE_Loki_STACK_NEW_VERSION> \
--namespace loki-stack \
--values <YOUR_HELM_VALUES_FILE>
See helm upgrade for command documentation.
Uninstalling
To uninstall Loki, you need to have Helm 3 installed. Once installed, run the following uninstall
command:
helm uninstall loki -n loki-stack
And then the following delete
command:
kubectl delete ns loki-stack