Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified docs/modules/demos/images/logging/logs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
38 changes: 29 additions & 9 deletions docs/modules/demos/pages/logging.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
:description: Deploy a logging stack with OpenSearch, Vector, and Zookeeper for log data analysis using OpenSearch Dashboards in Kubernetes.

:k8s-cpu: https://kubernetes.io/docs/tasks/debug/debug-cluster/resource-metrics-pipeline/#cpu
:opensearch-data-streams: https://docs.opensearch.org/latest/im-plugin/data-streams/

Install this demo on an existing Kubernetes cluster:

Expand Down Expand Up @@ -39,7 +40,9 @@ This demo will
** *OpenSearch*: A data store and search engine. This demo uses it to store and index the log data.
** *OpenSearch Dashboards*: A visualization and user interface. This demo uses it to make the log data easily accessible
to the user.
* Create a view in OpenSearch Dashboards to conveniently browse the log data.
* Create an index template and an Index State Management (ISM) policy in OpenSearch so that the log data are stored in
a data stream which is rolled over and deleted automatically.
* Create an index pattern and a view in OpenSearch Dashboards to conveniently browse the log data.

You can see the deployed products and their relationship in the following diagram:

Expand All @@ -61,28 +64,45 @@ $ stackablectl stacklet list -n logging
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ zookeeper ┆ simple-zk ┆ logging ┆ server-zk simple-zk-server.logging.svc.cluster.local:2282 ┆ Available, Reconciling, Running │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ opensearch-dashboards ┆ opensearch-dashboards ┆ logging ┆ http http://172.18.0.2:30206 ┆ │
│ ┆ ┆ ┆ metrics 172.18.0.2:31173 ┆ │
│ opensearch-dashboards ┆ opensearch-dashboards ┆ logging ┆ http http://10.104.213.49:31655 ┆ │
│ ┆ ┆ ┆ metrics 10.104.213.49:30865 ┆ │
└───────────────────────┴───────────────────────┴───────────┴────────────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────┘
----

include::partial$instance-hint.adoc[]

== Inspect the log data

You can have a look at the log data within the OpenSearch Dashboards web interface by running
`stackablectl stacklet list` and opening the URL in the opensearch-dashboard entry's info column. In this case, it is
http://172.18.0.2:30595.
You can have a look at the log data within the OpenSearch Dashboards web interface by running `stackablectl stacklet list -n logging` and opening the HTTP URL in the opensearch-dashboards entry's ENDPOINTS column.
In this case, it is http://10.104.213.49:31655.

image::logging/login.png[]

Log in with the username `admin` and password `adminadmin`.

NOTE: On first login, you will be presented with some options. Feel free to bypass them to get to the logs.
[NOTE]
====
On first login, you will be presented with some options.
Feel free to bypass them to get to the logs.
====

Click _Discover_ in the menu on the left to view the recent logs.
OpenSearch Dashboards opens the view _Logs_ which shows the most interesting fields of the recent log data.
If you do not see anything, increase the search window to greater than _Last 15 minutes_.

[NOTE]
====
If you click _Discover_ in the menu on the left, then a new search is started instead.
Open the view _Logs_ again with _Open_ in the upper right corner.
====

image::logging/logs.png[]

From here you can inspect the logs. You can select fields on the left side to limit the view to interesting information.
From here you can inspect the logs.
You can select fields on the left side to limit the view to interesting information.

[#data-stream]
== Log data stream

Vector writes all log data into the single {opensearch-data-streams}[data stream] `logs-generic-default`.
The data stream is created on the first write because the index template `logs` matches its name and contains a `data_stream` section.
The ISM policy `logs` rolls the backing indices of the data stream over once a day and deletes them after a week.
2 changes: 0 additions & 2 deletions stacks/_templates/opensearch-dashboards.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,3 @@ options:
# Use the ServiceAccount of OpenSearch because its permissions are already configured to work on
# OpenShift.
name: opensearch-serviceaccount
config:
opensearch_security.multitenancy.enabled: false
9 changes: 9 additions & 0 deletions stacks/_templates/truststore.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: TrustStore
metadata:
name: truststore-pem
spec:
secretClassName: tls
format: tls-pem
targetKind: ConfigMap
80 changes: 64 additions & 16 deletions stacks/_templates/vector-aggregator.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ options:
- vector
endpoints:
- $OPENSEARCH_HOSTS
mode: bulk
# Write to data streams instead of plain indices.
mode: data_stream
# The auto-detection of the API version does not work in Vector
# 0.41.1 for OpenSearch, so the version must be set explicitly
# (see https://github.com/vectordotdev/vector/issues/17690).
Expand All @@ -53,26 +54,73 @@ options:
auth:
strategy: basic
user: admin
password: "{{ openSearchAdminPassword }}"
password: $OPEN_SEARCH_ADMIN_PASSWORD
serviceAccount:
create: false
# Use the ServiceAccount of OpenSearch because its permissions are already configured to work
# on OpenShift.
name: opensearch-serviceaccount
initContainers:
# Wait for the index template which makes OpenSearch store the log data in data streams.
# If Vector wrote log events before this index template existed, then OpenSearch would
# create plain indices instead of data streams.
- name: wait-for-index-template
image: oci.stackable.tech/sdp/testing-tools:0.3.0-stackable0.0.0-dev
env:
- name: OPENSEARCH_HOSTS
valueFrom:
configMapKeyRef:
name: opensearch
key: OPENSEARCH_HOSTS
- name: OPEN_SEARCH_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: opensearch-user
key: password
volumeMounts:
- name: tls
mountPath: /certs
command:
- bash
- -euo
- pipefail
- -c
- |
#!/bin/bash
# The loop is bounded so that the pod ends up in CrashLoopBackOff if the job
# `setup-opensearch` failed permanently, instead of dropping all log events silently.
for _ in $(seq 120); do
if curl \
--fail \
--silent \
--show-error \
--cacert /certs/ca.crt \
--user "admin:${OPEN_SEARCH_ADMIN_PASSWORD}" \
--output /dev/null \
"${OPENSEARCH_HOSTS}/_index_template/logs"; then
exit 0
fi

echo 'Waiting for the index template "logs" to be created...'
sleep 5
done

echo 'The index template "logs" was not created within 10 minutes.'
echo 'Check the logs of the job "setup-opensearch".'
exit 1
env:
- name: OPEN_SEARCH_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: opensearch-user
key: password
envFrom:
- configMapRef:
name: opensearch
extraVolumeMounts:
# Use the certificate generated by the secret-operator
- name: tls
mountPath: /certs
extraVolumes:
- name: tls
ephemeral:
volumeClaimTemplate:
metadata:
annotations:
secrets.stackable.tech/class: tls
secrets.stackable.tech/scope: pod
spec:
storageClassName: secrets.stackable.tech
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1
configMap:
name: truststore-pem
9 changes: 0 additions & 9 deletions stacks/airflow/kafka.yaml
Comment thread
siegfriedweber marked this conversation as resolved.
Resolved
Original file line number Diff line number Diff line change
@@ -1,13 +1,4 @@
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: TrustStore
metadata:
name: truststore-pem
spec:
secretClassName: tls
format: tls-pem
targetKind: ConfigMap
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
Expand Down
Loading
Loading