From 2c5b24923e60b083d7e4ead15d59868cb03e87ba Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Tue, 7 Aug 2018 19:22:24 -0700 Subject: [PATCH] drafts for logging procedures --- .../rancher/v2.x/en/tools/logging/_index.md | 15 ++++++-- .../en/tools/logging/elasticsearch/_index.md | 28 ++++++++++----- .../logging/embedded-elasticsearch/_index.md | 8 ++--- .../v2.x/en/tools/logging/kafka/_index.md | 16 +++++++-- .../v2.x/en/tools/logging/splunk/_index.md | 33 +++++++++++------ .../v2.x/en/tools/logging/syslog/_index.md | 36 ++++++++++++++----- 6 files changed, 99 insertions(+), 37 deletions(-) diff --git a/content/rancher/v2.x/en/tools/logging/_index.md b/content/rancher/v2.x/en/tools/logging/_index.md index 143c56be407..d124366ae91 100644 --- a/content/rancher/v2.x/en/tools/logging/_index.md +++ b/content/rancher/v2.x/en/tools/logging/_index.md @@ -5,6 +5,15 @@ aliases: - /rancher/v2.x/en/tasks/logging/ --- -Rancher has the capability to push out cluster and project logs to an external -log monitoring tool such as Splunk or a Syslog server. This allows you to be -alerted of errors and warnings in your Kubernetes infrastructure. +To record cluster or project events, Rancher can integrate with a variety of popular logging services that exist outside of your clusters. Configuring logging services: + +- Alerts you to errors and warnings in your Kubernetes infrastructure in a live stream. +- Allows you to view data reports using your logging service's tools and identify trends in your clusters or projects. + +Rancher supports the following logging services: + +- [Embedded Elasticsearch]({{< baseurl >}}/rancher/v2.x/en/tools/logging/embedded-elasticsearch) +- [Elasticsearch]({{< baseurl >}}/rancher/v2.x/en/tools/logging/elasticsearch) +- [Splunk]({{< baseurl >}}/rancher/v2.x/en/tools/logging/splunk) +- [Kafka]({{< baseurl >}}/rancher/v2.x/en/tools/logging/kafka) +- [Syslog]({{< baseurl >}}/rancher/v2.x/en/tools/logging/syslog) \ No newline at end of file diff --git a/content/rancher/v2.x/en/tools/logging/elasticsearch/_index.md b/content/rancher/v2.x/en/tools/logging/elasticsearch/_index.md index 82d18e77577..c55f72d0253 100644 --- a/content/rancher/v2.x/en/tools/logging/elasticsearch/_index.md +++ b/content/rancher/v2.x/en/tools/logging/elasticsearch/_index.md @@ -3,40 +3,50 @@ title: Elasticsearch weight: 200 --- -If your organization uses Elasticsearch, you can configure it to log events from your cluster. Afterwards, you can use Elasticsearch to view data from cluster and containers. +If your organization uses [Elasticsearch](https://www.elastic.co/), either on premise or in the cloud, you can configure it to log events from your cluster. Afterwards, you can log into your Elasticsearch deployment to view logs for cluster or containers. -## Configuring Cluster Logging +## Configuring Elasticsearch Logging + +You can configure Elasticsearch to log events at both the cluster level and the project level. >**Prerequisites:** Configure an [Elasticsearch deployment](https://www.elastic.co/guide/en/cloud/saas-release/ec-create-deployment.html). +1. Browse to the cluster or project that you want to log. +{{% accordion id="cluster" label="To Configure Cluster Logging:" %}} 1. From the **Global** view, open the cluster that you want to configure logging for. 1. From the main menu, select **Tools > Logging**. +{{% /accordion %}} +{{% accordion id="project" label="To Configure Project Logging:" %}} +1. From the **Global** view, open the project that you want to configure logging for. + +1. From the main menu, select **Resources > Logging**. +{{% /accordion %}} 1. Select **Elasticsearch**. 1. Complete the **Elasticsearch Configuration** form. - 1. From the **Endpoint** field, enter the IP address and port for your Elasticsearch. You can copy this information from your the dashboard of your Elasticseach deployment. - + 1. From the **Endpoint** field, enter the IP address and port for your Elasticsearch. You can copy this information from your the dashboard of your Elasticsearch deployment. + 1. If you are using [X-Pack Security](https://www.elastic.co/guide/en/x-pack/current/xpack-introduction.html), enter your Elasticsearch **Username** and **Password** for authentication. 1. Enter an [Index Pattern](https://www.elastic.co/guide/en/kibana/current/index-patterns.html). -1. Complete the **SSL Configuration** form. +1. If your instance of Elasticsearch uses SSL, complete the **SSL Configuration** form. - 1. Enter a private key and client certificate. + 1. Enter a private key and client certificate. Either copy and paste them or browse to them using **Read from a file**. 1. Enter your private key password. - 1. If you want to verify connections to the Elasticsearch, select the **Enabled - Input trusted server certificate** option and then enter your truster server certificate. + 1. If you want to verify connections to Elasticsearch, select the **Enabled - Input trusted server certificate** option and then enter your trusted server certificate. 1. Complete the **Additional Logging Configuration** form. - 1. Use the **Add Field** button to add key value pairs used to filter log events. + 1. Use the **Add Field** button to add key value pairs that represent [custom log fields](http://docs.splunk.com/Documentation/AddOns/released/CiscoWSA/Configurew3clogfieldextractions) used to filter log events. 1. Enter a **Flush Interval**. This value determines how often the buffered logs are flushed. 1. Click **Save**. -**Result:** Rancher is now configured to send cluster and container events to Elasticsearch for logging. \ No newline at end of file +**Result:** Rancher is now configured to send cluster and container events to Elasticsearch for logging. Log into Elasticsearch or Kibana to view your cluster/project events. \ No newline at end of file diff --git a/content/rancher/v2.x/en/tools/logging/embedded-elasticsearch/_index.md b/content/rancher/v2.x/en/tools/logging/embedded-elasticsearch/_index.md index 6352697b8e5..3ff2b37a3a3 100644 --- a/content/rancher/v2.x/en/tools/logging/embedded-elasticsearch/_index.md +++ b/content/rancher/v2.x/en/tools/logging/embedded-elasticsearch/_index.md @@ -3,15 +3,15 @@ title: Embedded Elasticsearch weight: 100 --- -If your organization doesn't have any logging solutions, you can set one up that's embedded within the cluster. You don't have to configure an independent logging server. +If your organization doesn't have any logging solutions, you can use Rancher's Embedded Elasticsearch option to log record from your cluster. This option sets up an instance of [Elasticsearch](https://www.elastic.co/) within your cluster, and then uses cluster resources to run it. >**Notes:** > ->- Embedded Elasticsearch is experimental at this time. Therefore, persistent storage for Embedded Elasticsearch is unavailable.We plan to offer full support in a Rancher release that's yet to be determined. +>- Embedded Elasticsearch is experimental at this time. Therefore, persistent storage for Embedded Elasticsearch is unavailable. We plan to offer full support in a Rancher release that's yet to be determined. >- Embedded Elasticsearch is only available for clusters and not projects. >- Setting up Embedded Elasticsearch requires a well provisioned node within the cluster. See the prerequisites below for more details. -## Configuring Cluster Logging +## Configuring Embedded Elasticsearch Logging >**Prerequisites:** Your cluster must meet the following hardware requirements: > @@ -32,7 +32,7 @@ If your organization doesn't have any logging solutions, you can set one up that 1. Complete the **Additional Logging Configuration** form. - 1. Use the **Add Field** button to add key value pairs used to filter log events. + 1. Use the **Add Field** button to add key value pairs that represent [custom log fields](http://docs.splunk.com/Documentation/AddOns/released/CiscoWSA/Configurew3clogfieldextractions) used to filter log events. 1. Enter a **Flush Interval**. This value determines how often the buffered logs are flushed. diff --git a/content/rancher/v2.x/en/tools/logging/kafka/_index.md b/content/rancher/v2.x/en/tools/logging/kafka/_index.md index d993e575bac..fd3d4c6d268 100644 --- a/content/rancher/v2.x/en/tools/logging/kafka/_index.md +++ b/content/rancher/v2.x/en/tools/logging/kafka/_index.md @@ -5,11 +5,23 @@ weight: 400 You can configure a Kafka server to log events that occur in your Kubernetes cluster. +## Configuring Kafka Logging + +You can configure Kafka to log events at both the cluster level and the project level. + >**Prerequisite:** You must have a Kafka server configured. +1. Browse to the cluster or project that you want to log. +{{% accordion id="cluster" label="To Configure Cluster Logging:" %}} 1. From the **Global** view, open the cluster that you want to configure logging for. 1. From the main menu, select **Tools > Logging**. +{{% /accordion %}} +{{% accordion id="project" label="To Configure Project Logging:" %}} +1. From the **Global** view, open the project that you want to configure logging for. + +1. From the main menu, select **Resources > Logging**. +{{% /accordion %}} 1. Select **Kafka**. @@ -21,11 +33,11 @@ You can configure a Kafka server to log events that occur in your Kubernetes clu By default, Kafka uses port `9092`. - 1. From the Topic field, enter the name of a Kafka topic that your Kubernetes cluster submits logs to. + 1. From the **Topic** field, enter the name of a Kafka [topic](https://kafka.apache.org/documentation/#basic_ops_add_topic) that your Kubernetes cluster submits logs to. 1. Complete the **Additional Logging Configuration** form. - 1. Use the **Add Field** button to add key value pairs used to filter log events. + 1. Use the **Add Field** button to add key value pairs that represent [custom log fields](https://httpd.apache.org/docs/2.4/mod/mod_log_config.html) used to filter log events. 1. Enter a **Flush Interval**. This value determines how often the buffered logs are flushed. diff --git a/content/rancher/v2.x/en/tools/logging/splunk/_index.md b/content/rancher/v2.x/en/tools/logging/splunk/_index.md index add9f417f0d..38eda5b9342 100755 --- a/content/rancher/v2.x/en/tools/logging/splunk/_index.md +++ b/content/rancher/v2.x/en/tools/logging/splunk/_index.md @@ -1,19 +1,34 @@ --- title: Splunk -weight: 3725 +weight: 300 aliases: - /rancher/v2.x/en/tasks/logging/splunk/ --- -If your organization uses Splunk, you can configure it to log events from your cluster. Afterwards, you can use Splunk to view data from cluster and containers. +If your organization uses [Splunk](https://www.splunk.com/), you can configure it to log events from a cluster or project in Rancher. Afterwards events are logged, you can use Splunk to view data from cluster and containers. -## Configuring Cluster Logging +## Configuring Splunk Logging ->**Prerequisites:** Configure HTTP event collection for your type of Splunk server (Splunk Enterprise, Splunk Cloud, etc.). Enable all tokens, and then create a new token. For more information, see [Splunk Documentation](http://docs.splunk.com/Documentation/Splunk/7.1.2/Data/UsetheHTTPEventCollector#About_Event_Collector_tokens). +You can configure Splunk to log events at both the cluster level and the project level. +>**Prerequisites:** +> +>- Configure HTTP event collection for your Splunk Server (Splunk Enterprise or Splunk Cloud). +>- Enable all tokens, and then create a new token. +> +>For more information, see [Splunk Documentation](http://docs.splunk.com/Documentation/Splunk/7.1.2/Data/UsetheHTTPEventCollector#About_Event_Collector_tokens). + +1. Browse to the cluster or project that you want to log. +{{% accordion id="cluster" label="To Configure Cluster Logging:" %}} 1. From the **Global** view, open the cluster that you want to configure logging for. 1. From the main menu, select **Tools > Logging**. +{{% /accordion %}} +{{% accordion id="project" label="To Configure Project Logging:" %}} +1. From the **Global** view, open the project that you want to configure logging for. + +1. From the main menu, select **Resources > Logging**. +{{% /accordion %}} 1. Select **Splunk**. @@ -21,7 +36,7 @@ If your organization uses Splunk, you can configure it to log events from your c 1. From the **Endpoint** field, enter the IP address and port for you syslog server (i.e. `http://splunk-server:8088`) - If you're using Splunk Cloud, you'll need to work with [Splunk support](https://www.splunk.com/en_us/support-and-services.html) to get an endpoint URL. + If you're using Splunk Cloud, you'll need to work with [Splunk support](https://www.splunk.com/en_us/support-and-services.html) to get an endpoint URL. 1. Enter the **Token** you obtained while completing the prerequisites. @@ -31,7 +46,7 @@ If your organization uses Splunk, you can configure it to log events from your c 1. Complete the **Additional Logging Configuration** form. - 1. Use the **Add Field** button to add key value pairs used to filter log events. + 1. 1. Use the **Add Field** button to add key value pairs that represent [custom log fields](http://docs.splunk.com/Documentation/Splunk/7.1.2/Data/Configureindex-timefieldextraction) used to filter log events. 1. Enter a **Flush Interval**. This value determines how often the buffered logs are flushed. @@ -46,11 +61,9 @@ If your organization uses Splunk, you can configure it to log events from your c 1. Click on **Search & Reporting**. The number of **Indexed Events** listed should be increasing. 1. Click on Data Summary and select the Sources tab. - ![View Logs]({{< baseurl >}}/img/rancher/splunk/splunk4.jpg) 1. To view the actual logs, click on the source that you declared earlier. - ![View Logs]({{< baseurl >}}/img/rancher/splunk/splunk5.jpg) ## Troubleshooting @@ -63,7 +76,7 @@ $ curl http://splunk-server:8088/services/collector/event \ -d '{"event": "hello world"}' ``` -If Splunk is configured corretcly, you should **json** data returning `success code 0`. You should be able +If Splunk is configured correctly, you should receive **json** data returning `success code 0`. You should be able to send logging data to HEC. -If you received an error, check your configuration in Splunk & Rancher Cluster Logging. \ No newline at end of file +If you received an error, check your configuration in Splunk and Rancher. \ No newline at end of file diff --git a/content/rancher/v2.x/en/tools/logging/syslog/_index.md b/content/rancher/v2.x/en/tools/logging/syslog/_index.md index bddce4daec4..0af5b4fae17 100644 --- a/content/rancher/v2.x/en/tools/logging/syslog/_index.md +++ b/content/rancher/v2.x/en/tools/logging/syslog/_index.md @@ -3,31 +3,49 @@ title: Syslog weight: 500 --- -You can configure a syslog server to log events that occur in your Kubernetes cluster. +You can configure a [Syslog](https://tools.ietf.org/html/rfc5424) server to log events that occur in your Kubernetes cluster. ->**Prerequisite:** You must have a syslog server configured. +## Configuring Syslog + +You can configure Syslog to log events at both the cluster level and the project level. + +>**Prerequisite:** You must have a Syslog server configured. + +1. Browse to the cluster or project that you want to log. + +{{% accordion id="cluster" label="To Configure Cluster Logging:" %}} 1. From the **Global** view, open the cluster that you want to configure logging for. 1. From the main menu, select **Tools > Logging**. -1. Select **syslog**. +{{% /accordion %}} + +{{% accordion id="project" label="To Configure Project Logging:" %}} + +1. From the **Global** view, open the project that you want to configure logging for. + +1. From the main menu, select **Resources > Logging**. + +{{% /accordion %}} + +1. Select **Syslog**. 1. Complete the **Syslog Configuration** form. - 1. From the **Endpoint** field, enter the IP address and port for you syslog server. Additionally, select the protocol that your syslog server uses from the drop-down. + 1. From the **Endpoint** field, enter the IP address and port for you Syslog server. Additionally, select the protocol that your Syslog server uses from the drop-down. - 1. From the **Program** field, enter the name of the application logging events to your syslog server. + 1. From the **Program** field, enter the name of the application logging events to your Syslog server. - 1. Enter a **Token** that authenticates with your syslog server. + 1. Enter a **Token** that authenticates with your Syslog server. - 1. Select a **Log Severity** for events that are logged to the syslog server. For more information on each severity level, see the [syslog protocol documentation](https://tools.ietf.org/html/rfc5424#page-11). + 1. Select a **Log Severity** for events that are logged to the Syslog server. For more information on each severity level, see the [Syslog protocol documentation](https://tools.ietf.org/html/rfc5424#page-11). -1. If your syslog server uses **TCP** protocol, complete the **SSL Configuration** form. +1. If your Syslog server uses **TCP** protocol, complete the **SSL Configuration** form. 1. Enter a private key and client certificate. - 1. If you want to verify connections to the syslog service, select the **Enabled - Input trusted server certificate** option and then enter your truster server certificate. + 1. If you want to verify connections to the Syslog service, select the **Enabled - Input trusted server certificate** option and then enter your truster server certificate. 1. Complete the **Additional Logging Configuration** form.