From c9fd11654f297c05aa122f5479223d2c5bbdc9b3 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Mon, 30 Mar 2020 17:11:56 -0700 Subject: [PATCH 01/14] Update CIS scan doc --- .../v2.x/en/security/security-scan/_index.md | 27 ++++++++++++++----- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/content/rancher/v2.x/en/security/security-scan/_index.md b/content/rancher/v2.x/en/security/security-scan/_index.md index f23f3469a5d..2e9b25bb554 100644 --- a/content/rancher/v2.x/en/security/security-scan/_index.md +++ b/content/rancher/v2.x/en/security/security-scan/_index.md @@ -41,9 +41,7 @@ To check clusters for CIS Kubernetes Benchmark compliance, the security scan lev Each scan generates a report can be viewed in the Rancher UI and can be downloaded in CSV format. -As of Rancher v2.4, the CIS scan will use either the Benchmark v1.4 or v1.5, depending on the Kubernetes version. - -To determine which version of the [Benchmark](https://www.cisecurity.org/benchmark/kubernetes/) to use in the scan, Rancher chooses a version that is appropriate for the cluster's Kubernetes version. The Benchmark version is included in the generated report. +As of Rancher v2.4, the scan will use the CIS Benchmark v1.4. The Benchmark version is included in the generated report. The Benchmark provides recommendations of two types: Scored and Not Scored. Recommendations marked as Not Scored in the Benchmark are not included in the generated report. @@ -151,9 +149,10 @@ To enable recurring scans, edit the advanced options in the cluster configuratio To schedule scans for an existing cluster: 1. Go to the cluster view in Rancher. -1. Click **Ellipsis (...) > Edit.** -1. Go to the **Advanced Options** section. In the **CIS Scan Enabled** field, click **Yes.** -1. In the **CIS Scan Profile** field, choose a **Permissive** or **Hardened** profile. The corresponding CIS Benchmark version is included in the profile name. Any skipped tests [defined in a separate ConfigMap](#skipping-tests) will be skipped regardless of whether a **Permissive** or **Hardened** profile is selected. When selecting the the permissive profile, you should see which tests were skipped by Rancher (tests that are skipped by default for RKE clusters) and which tests were skipped by a Rancher user. In the hardened test profile, the only skipped tests will be skipped by users. +1. Click **Tools > CIS Scans.** +1. Click **Add Schedule.** This takes you to the section of the cluster editing page that is applicable to configuring a schedule for CIS scans. (This section can also be reached by going to the cluster view, clicking **Ellipsis (...) > Edit,** and going to the **Advanced Options.**) +1. In the **CIS Scan Enabled** field, click **Yes.** +1. In the **CIS Scan Profile** field, choose a **Permissive** or **Hardened** profile. The corresponding CIS Benchmark version is included in the profile name. Note: Any skipped tests [defined in a separate ConfigMap](#skipping-tests) will be skipped regardless of whether a **Permissive** or **Hardened** profile is selected. When selecting the the permissive profile, you should see which tests were skipped by Rancher (tests that are skipped by default for RKE clusters) and which tests were skipped by a Rancher user. In the hardened test profile, the only skipped tests will be skipped by users. 1. In the **CIS Scan Interval (cron)** job, enter a [cron expression](https://en.wikipedia.org/wiki/Cron#CRON_expression) to define how often the cluster will be scanned. 1. In the **CIS Scan Report Retention** field, enter the number of past reports that should be kept. @@ -213,7 +212,7 @@ Rancher provides a set of alerts for cluster scans. which are not configured to > **Prerequisite:** You need to configure a [notifier]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers/) before configuring, sending, or receiving alerts. -To activate an alert for a CIS scan result, +To activate an existing alert for a CIS scan result, 1. From the cluster view in Rancher, click **Tools > Alerts.** 1. Go to the section called **A set of alerts for cluster scans.** @@ -225,6 +224,20 @@ To activate an alert for a CIS scan result, **Result:** The notifications will be triggered when the a scan is run on a cluster and the active alerts have satisfied conditions. +To create a new alert, + +1. Go to the cluster view and click **Tools > CIS Scans.** +1. Click **Add Alert.** +1. Fill out the form. +1. Enter a name for the alert. +1. In the **Is** field, set the alert to be triggered when a scan is completed or when a scan has a failure. +1. In the **Send a** field, set the alert as a **Critical,** **Warning,** or **Info** alert level. +1. Choose a [notifier]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers/) for the alert. + +**Result:** The alert is created and activated. The notifications will be triggered when the a scan is run on a cluster and the active alerts have satisfied conditions. + +For more information about alerts, refer to [this page.]({{}}/rancher/v2.x/en/cluster-admin/tools/alerts/) + # Deleting a Report 1. From the cluster view in Rancher, click **Tools > CIS Scans.** From f9044bb3f3ddb84ade6ba22d9c2704ae497065de Mon Sep 17 00:00:00 2001 From: Robert Parker Date: Tue, 31 Mar 2020 10:33:49 -0700 Subject: [PATCH 02/14] move images --- .../img/rancher/new-app-registration-1.png | Bin .../img/rancher/new-app-registration-2.png | Bin .../img/rancher/select-client-secret.png | Bin .../img/rancher/select-required-permissions-1.png | Bin .../img/rancher/select-required-permissions-2.png | Bin 5 files changed, 0 insertions(+), 0 deletions(-) rename {src => static}/img/rancher/new-app-registration-1.png (100%) rename {src => static}/img/rancher/new-app-registration-2.png (100%) rename {src => static}/img/rancher/select-client-secret.png (100%) rename {src => static}/img/rancher/select-required-permissions-1.png (100%) rename {src => static}/img/rancher/select-required-permissions-2.png (100%) diff --git a/src/img/rancher/new-app-registration-1.png b/static/img/rancher/new-app-registration-1.png similarity index 100% rename from src/img/rancher/new-app-registration-1.png rename to static/img/rancher/new-app-registration-1.png diff --git a/src/img/rancher/new-app-registration-2.png b/static/img/rancher/new-app-registration-2.png similarity index 100% rename from src/img/rancher/new-app-registration-2.png rename to static/img/rancher/new-app-registration-2.png diff --git a/src/img/rancher/select-client-secret.png b/static/img/rancher/select-client-secret.png similarity index 100% rename from src/img/rancher/select-client-secret.png rename to static/img/rancher/select-client-secret.png diff --git a/src/img/rancher/select-required-permissions-1.png b/static/img/rancher/select-required-permissions-1.png similarity index 100% rename from src/img/rancher/select-required-permissions-1.png rename to static/img/rancher/select-required-permissions-1.png diff --git a/src/img/rancher/select-required-permissions-2.png b/static/img/rancher/select-required-permissions-2.png similarity index 100% rename from src/img/rancher/select-required-permissions-2.png rename to static/img/rancher/select-required-permissions-2.png From 257e68eeef1f624477e5470fdb9da6b7883f5544 Mon Sep 17 00:00:00 2001 From: Robert Parker Date: Tue, 31 Mar 2020 12:21:45 -0700 Subject: [PATCH 03/14] audit-39 --- .../cluster-access/kubectl/_index.md | 1 - .../v2.x/en/cluster-admin/nodes/_index.md | 2 - .../v2.x/en/faq/removing-rancher/_index.md | 3 +- .../air-gap-helm2/install-rancher/_index.md | 1 - .../helm2/rke-add-on/layer-4-lb/_index.md | 2 - .../air-gap/install-rancher/_index.md | 1 - content/rancher/v2.x/en/upgrades/_index.md | 2 - .../rollbacks/single-node-rollbacks/_index.md | 1 - nginx.conf | 43 +++++++++++++++++++ 9 files changed, 44 insertions(+), 12 deletions(-) diff --git a/content/rancher/v2.x/en/cluster-admin/cluster-access/kubectl/_index.md b/content/rancher/v2.x/en/cluster-admin/cluster-access/kubectl/_index.md index b6083597ff9..f8d6817e65f 100644 --- a/content/rancher/v2.x/en/cluster-admin/cluster-access/kubectl/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/cluster-access/kubectl/_index.md @@ -8,7 +8,6 @@ aliases: - /rancher/v2.x/en/concepts/clusters/kubeconfig-files/ - /rancher/v2.x/en/k8s-in-rancher/kubeconfig/ - /rancher/2.x/en/cluster-admin/kubeconfig - - /rancher/v2.x/en/cluster-admin/cluster-access/kubeconfig/ --- This section describes how to manipulate your downstream Kubernetes cluster with kubectl from the Rancher UI or from your workstation. diff --git a/content/rancher/v2.x/en/cluster-admin/nodes/_index.md b/content/rancher/v2.x/en/cluster-admin/nodes/_index.md index c90ae67303b..3290a860d9c 100644 --- a/content/rancher/v2.x/en/cluster-admin/nodes/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/nodes/_index.md @@ -1,8 +1,6 @@ --- title: Nodes and Node Pools weight: 2030 -aliases: - - /rancher/v2.x/en/k8s-in-rancher/nodes/ --- After you launch a Kubernetes cluster in Rancher, you can manage individual nodes from the cluster's **Node** tab. Depending on the [option used]({{}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) to provision the cluster, there are different node options available. diff --git a/content/rancher/v2.x/en/faq/removing-rancher/_index.md b/content/rancher/v2.x/en/faq/removing-rancher/_index.md index 01b53b46358..1cf26a9d8a0 100644 --- a/content/rancher/v2.x/en/faq/removing-rancher/_index.md +++ b/content/rancher/v2.x/en/faq/removing-rancher/_index.md @@ -6,7 +6,6 @@ aliases: - /rancher/v2.x/en/installation/removing-rancher/ - /rancher/v2.x/en/admin-settings/removing-rancher/ - /rancher/v2.x/en/admin-settings/removing-rancher/rancher-cluster-nodes/ - - /rancher/v2.x/en/removing-rancher/ --- This page is intended to answer questions about what happens if you don't want Rancher anymore, if you don't want a cluster to be managed by Rancher anymore, or if the Rancher server is deleted. @@ -55,4 +54,4 @@ At this time, there is no functionality to detach these clusters from Rancher. I The capability to manage these clusters without Rancher is being tracked in this [issue.](https://github.com/rancher/rancher/issues/25234) -For information about how to access clusters if the Rancher server is deleted, refer to [this section.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters) \ No newline at end of file +For information about how to access clusters if the Rancher server is deleted, refer to [this section.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters) diff --git a/content/rancher/v2.x/en/installation/options/air-gap-helm2/install-rancher/_index.md b/content/rancher/v2.x/en/installation/options/air-gap-helm2/install-rancher/_index.md index 619c44dcc14..f5895de9604 100644 --- a/content/rancher/v2.x/en/installation/options/air-gap-helm2/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/options/air-gap-helm2/install-rancher/_index.md @@ -5,7 +5,6 @@ aliases: - /rancher/v2.x/en/installation/air-gap-installation/install-rancher/ - /rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-system-charts/ - /rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/ - - /rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/ - /rancher/v2.x/en/installation/air-gap-single-node/install-rancher - /rancher/v2.x/en/installation/air-gap/install-rancher --- diff --git a/content/rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-4-lb/_index.md b/content/rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-4-lb/_index.md index fecaef3d2b3..f3c16cb9404 100644 --- a/content/rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-4-lb/_index.md +++ b/content/rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-4-lb/_index.md @@ -1,8 +1,6 @@ --- title: Kubernetes Install with External Load Balancer (TCP/Layer 4) weight: 275 -aliases: -- /rancher/v2.x/en/installation/k8s-install-server-install/ --- > #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** diff --git a/content/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher/_index.md b/content/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher/_index.md index c12809c0695..bb1b4ae209c 100644 --- a/content/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher/_index.md @@ -4,7 +4,6 @@ weight: 400 aliases: - /rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-system-charts/ - /rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/ - - /rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/ - /rancher/v2.x/en/installation/air-gap-single-node/install-rancher - /rancher/v2.x/en/installation/air-gap/install-rancher --- diff --git a/content/rancher/v2.x/en/upgrades/_index.md b/content/rancher/v2.x/en/upgrades/_index.md index 1d0ce9190a5..4debea3156e 100644 --- a/content/rancher/v2.x/en/upgrades/_index.md +++ b/content/rancher/v2.x/en/upgrades/_index.md @@ -1,8 +1,6 @@ --- title: Upgrades and Rollbacks weight: 150 -aliases: - - /rancher/v2.x/en/backups/rollbacks/ --- ### Upgrading Rancher diff --git a/content/rancher/v2.x/en/upgrades/rollbacks/single-node-rollbacks/_index.md b/content/rancher/v2.x/en/upgrades/rollbacks/single-node-rollbacks/_index.md index 3ad649798e0..4705d65d1d8 100644 --- a/content/rancher/v2.x/en/upgrades/rollbacks/single-node-rollbacks/_index.md +++ b/content/rancher/v2.x/en/upgrades/rollbacks/single-node-rollbacks/_index.md @@ -2,7 +2,6 @@ title: Docker Rollback weight: 1015 aliases: - - /rancher/v2.x/en/backups/rollbacks/single-node-rollbacks - /rancher/v2.x/en/upgrades/single-node-rollbacks --- diff --git a/nginx.conf b/nginx.conf index cbe1b9139c8..564352c0877 100644 --- a/nginx.conf +++ b/nginx.conf @@ -117,6 +117,49 @@ map $request_uri $redirect_uri { ~^/docs/rancher/v2.x/en/upgrades/upgrades/ha-server-upgrade-helm/?$ /docs/rancher/v2.x/en/upgrades/upgrades/ha/; ~^/docs/rancher/v2.x/en/upgrades/upgrades/single-node-upgrade/?$ /docs/rancher/v2.x/en/upgrades/upgrades/single-node/; ~^/docs/rke/latest/en/installation/os/?$ /docs/rke/latest/en/os/; + + ~^/docs/rancher/v2.x/en/k8s-in-rancher/nodes/?$ /docs/rancher/v2.x/en/cluster-admin/nodes/; + ~^/docs/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/?$ /docs/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher/; + ~^/docs/rancher/v2.x/en/installation/k8s-install-server-install/?$ /docs/rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-4-lb/; + ~^/docs/rancher/v1.0/en/infrastructure/hosts/?$ /docs/rancher/v1.0/en/rancher-ui/infrastructure/hosts/; + ~^/docs/rancher/v2.x/en/cluster-admin/cluster-access/kubeconfig/?$ /docs/rancher/v2.x/en/cluster-admin/cluster-access/kubectl/; + ~^/docs/os/v1.0/en/configuration/custom-console/?$ /docs/os/latest/en/configuration/switching-consoles/; + ~^/docs/os/latest/en/configuration/switching-consoles/?$ /docs/os/v1.x/en/configuration/switching-consoles/; + ~^/docs/os/v1.1/en/configuration/custom-console/?$ /docs/os/v1.1/en/configuration/switching-consoles/; + ~^/docs/os/v1.1/en/system-services/built-in-system-services/?$ /docs/os/v1.1/en/boot-process/built-in-system-services/; + ~^/docs/os/v1.2/en/configuration/custom-console/?$ /docs/os/v1.2/en/configuration/switching-consoles/; + ~^/docs/os/v1.2/en/system-services/built-in-system-services/?$ /docs/os/v1.2/en/boot-process/built-in-system-services/; + ~^/docs/rancher/v2.x/en/removing-rancher/?$ /docs/rancher/v2.x/en/faq/removing-rancher/; + ~^/docs/rancher/v2.x/en/installation/ha/?$ /docs/rancher/v2.x/en/installation/k8s-install/; + ~^/docs/rancher/v2.x/en/installation/ha/helm-rancher/?$ /docs/rancher/v2.x/en/installation/k8s-install/helm-rancher/; + ~^/docs/rancher/v2.x/en/installation/other-installation-methods/single-node/?$ /docs/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/; + ~^/docs/rancher/v2.x/en/installation/air-gap/install-rancher/?$ /docs/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher/; + ~^/docs/rancher/v1.0/en/api/v1/access-control/?$ /docs/rancher/v1.0/en/api/v1/api-keys/; + ~^/docs/os/latest/en/storage/additional-mounts/?$ /docs/os/v1.x/en/storage/additional-mounts/; + ~^/docs/os/v1.0/en/configuration/custom-rancheros-iso/?$ /docs/os/latest/custom-builds/custom-rancheros-iso/; + ~^/docs/os/v1.0/en/configuration/custom-kernels/?$ /docs/os/latest/custom-builds/custom-kernels/; + ~^/docs/rancher/v1.0/en/environments/?$ /docs/rancher/v1.0/en/configuration/environments/; + ~^/docs/os/v1.1/en/configuration/custom-kernels/?$ /docs/os/v1.1/en/custom-builds/custom-kernels/; + ~^/docs/os/v1.0/en/system-services/built-in-system-services/?$ /docs/os/latest/boot-process/built-in-system-services/; + ~^/docs/os/latest/custom-builds/custom-rancheros-iso/?$ /docs/os/v1.x/en/custom-builds/custom-rancheros-iso/; + ~^/docs/os/v1.0/en/system-services/?$ /docs/os/latest/en/system-services/adding-system-services/; + ~^/docs/os/v1.0/en/configuration/additional-mounts/?$ /docs/os/latest/en/storage/additional-mounts/; + ~^/docs/os/latest/custom-builds/custom-kernels/?$ /docs/os/v1.x/en/custom-builds/custom-kernels/; + ~^/docs/os/v1.1/en/system-services/?$ /docs/os/v1.1/en/system-services/adding-system-services/; + ~^/docs/os/v1.1/en/configuration/additional-mounts/?$ /docs/os/v1.1/en/storage/additional-mounts/; + ~^/docs/os/latest/boot-process/built-in-system-services/?$ /docs/os/v1.x/en/boot-process/built-in-system-services/; + ~^/docs/os/latest/en/system-services/adding-system-services/?$ /docs/os/v1.x/en/system-services/adding-system-services/; + ~^/docs/rancher/v1.0/en/cattle/rancher-compose/?$ /docs/rancher/v1.0/en/rancher-compose/; + ~^/docs/os/v1.1/en/configuration/custom-docker/?$ /docs/os/v1.1/en/configuration/switching-docker-versions/; + ~^/docs/os/v1.2/en/configuration/custom-kernels/?$ /docs/os/v1.x/en/custom-builds/custom-kernels/; + ~^/docs/os/v1.2/en/configuration/custom-rancheros-iso/?$ /docs/os/v1.x/en/custom-builds/custom-rancheros-iso/; + ~^/docs/os/v1.2/en/system-services/?$ /docs/os/v1.2/en/system-services/adding-system-services/; + ~^/docs/os/v1.2/en/configuration/additional-mounts/?$ /docs/os/v1.2/en/storage/additional-mounts/; + ~^/docs/rancher/v2.x/en/backups/rollbacks/?$ /docs/rancher/v2.x/en/upgrades/; + ~^/docs/rancher/v2.x/en/admin-settings/feature-flags/enable-not-default-storage-drivers/?$ /docs/rancher/v2.x/en/installation/options/feature-flags/enable-not-default-storage-drivers/; + ~^/docs/rancher/v2.x/en/installation/server-tags/?$ /docs/rancher/v2.x/en/installation/options/server-tags/; + ~^/rancher/v2.x/en/admin-settings/feature-flags/istio-virtual-service-ui/?$ /docs/rancher/v2.x/en/installation/options/feature-flags/istio-virtual-service-ui/; + ~^/docs/os/v1.1/en/configuration/custom-rancheros-iso/?$ /docs/os/v1.1/en/custom-builds/custom-rancheros-iso/; } server { From 028e3ceb490673a52a3232324c25ded20800c584 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 31 Mar 2020 13:54:17 -0700 Subject: [PATCH 04/14] Update installation-requirements networking section - Addresses comment around node-requirements page (now installation-requirements) in issue https://github.com/rancher/k3s/issues/1031 - Accompslishes this by adding a table to list ports, we actually took from /rancher/v2.x/en/installation/requirements/ports networking section so both guides are in alignment --- .../installation-requirements/_index.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/content/k3s/latest/en/installation/installation-requirements/_index.md b/content/k3s/latest/en/installation/installation-requirements/_index.md index 13da6c3d7ce..44e9bf5008f 100644 --- a/content/k3s/latest/en/installation/installation-requirements/_index.md +++ b/content/k3s/latest/en/installation/installation-requirements/_index.md @@ -37,12 +37,24 @@ K3s performance depends on the performance of the database. To ensure optimal sp ## Networking -The K3s server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach other nodes over UDP port 8472 (Flannel VXLAN). If you do not use Flannel and provide your own custom CNI, then port 8472 is not needed by K3s. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. +The K3s server needs port 6443 to be accessible by the nodes. -IMPORTANT: The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disabled access to port 8472. +The nodes need to be able to reach other nodes over UDP port 8472 when Flannel VXLAN is used. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. However, if you do not use Flannel and provide your own custom CNI, then port 8472 is not needed by K3s. If you wish to utilize the metrics server, you will need to open port 10250 on each node. +> **Important:** The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disables access to port 8472. + +
Inbound Rules for Rancher Server Nodes
+ +| Protocol | Port | Source | Description +|-----|-----|----------------|---| +| TCP | 6443 | K3s server nodes | Kubernetes API +| UDP | 8472 | K3s server and agent nodes | Required only for Flannel VXLAN +| TCP | 10250 | K3s server and agent nodes | kubelet + +Typically all outbound traffic is allowed. + ## Large Clusters Hardware requirements are based on the size of your K3s cluster. For production and large clusters, we recommend using a high-availability setup with an external database. The following options are recommended for the external database in production: From 2d431d73c9c2aa6d1fb3d0b33c7ce66e33cd89ff Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 31 Mar 2020 14:06:41 -0700 Subject: [PATCH 05/14] Address symlink issue on installation-requirements page - Resolves symlink issue in single-server section of https://github.com/rancher/k3s/issues/1031 - The INSTALL_K3S_SYMLINK env var now has a description that explains what binaries symlinks are created for --- content/k3s/latest/en/installation/install-options/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/installation/install-options/_index.md b/content/k3s/latest/en/installation/install-options/_index.md index e2bc4a2dbb2..d8c415cb96c 100644 --- a/content/k3s/latest/en/installation/install-options/_index.md +++ b/content/k3s/latest/en/installation/install-options/_index.md @@ -29,7 +29,7 @@ When using this method to install K3s, the following environment variables can b - `INSTALL_K3S_SYMLINK` - If set to 'skip' will not create symlinks, 'force' will overwrite, default will symlink if command does not exist in path. + By default will create symlinks for the kubectl, crictl, and ctr binaries if the commands do not already exist in path. If set to 'skip' will not create symlinks and 'force' will overwrite. - `INSTALL_K3S_SKIP_START` From 88079937e5b75b7def5eb81e88f22d5d31106ea9 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 31 Mar 2020 15:32:03 -0700 Subject: [PATCH 06/14] Tweaks to install-options page - Addresses additional issues outlined in https://github.com/rancher/k3s/issues/1031 single-server section --- content/k3s/latest/en/installation/install-options/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/installation/install-options/_index.md b/content/k3s/latest/en/installation/install-options/_index.md index d8c415cb96c..70b164b38aa 100644 --- a/content/k3s/latest/en/installation/install-options/_index.md +++ b/content/k3s/latest/en/installation/install-options/_index.md @@ -66,7 +66,7 @@ When using this method to install K3s, the following environment variables can b - `INSTALL_K3S_NAME` - Name of systemd service to create, will default from the K3s exec command if not specified. If specified the name will be prefixed with 'k3s-'. + Name of systemd service to create, will default to 'k3s' if running k3s as a server and 'k3s-agent' if running k3s as an agent. If specified the name will be prefixed with 'k3s-'. - `INSTALL_K3S_TYPE` From 7eb1c17dd1d8b0a929a59018e72c60252d95c9af Mon Sep 17 00:00:00 2001 From: Terdunov Vyacheslav Date: Wed, 1 Apr 2020 13:58:04 +0400 Subject: [PATCH 07/14] Fix How Upgrades Work link to cluster.yaml --- content/rke/latest/en/upgrades/how-upgrades-work/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rke/latest/en/upgrades/how-upgrades-work/_index.md b/content/rke/latest/en/upgrades/how-upgrades-work/_index.md index b5dd4af8c4b..99eba360c5f 100644 --- a/content/rke/latest/en/upgrades/how-upgrades-work/_index.md +++ b/content/rke/latest/en/upgrades/how-upgrades-work/_index.md @@ -62,7 +62,7 @@ Because RKE addons are necessary for allowing traffic into the cluster, they wil For more information on configuring the number of replicas for each addon, refer to [this section.](#replicas-for-rke-addons) -For an example showing how to configure the addons, refer to the [example cluster.yml.](#example-cluster-yml) +For an example showing how to configure the addons, refer to the [example cluster.yml.]({{}}rke/latest/en/upgrades/configuring-strategy/#example-cluster-yml) {{% /tab %}} {{% tab "RKE prior to v1.1.0" %}} @@ -87,4 +87,4 @@ Worker nodes are upgraded simultaneously, in batches of either 50 or the total n When a worker node is upgraded, it restarts several Docker processes, including the `kubelet` and `kube-proxy`. When `kube-proxy` comes up, it flushes `iptables`. When this happens, pods on this node can’t be accessed, resulting in downtime for the applications. {{% /tab %}} -{{% /tabs %}} \ No newline at end of file +{{% /tabs %}} From 12b9b48f51cc5c71a1491acb5df0052eaf97dce0 Mon Sep 17 00:00:00 2001 From: Sebastiaan van Steenis Date: Wed, 1 Apr 2020 14:15:59 +0200 Subject: [PATCH 08/14] Describe Rancher log levels --- .../rancher/v2.x/en/faq/technical/_index.md | 50 +------------------ .../rancher/v2.x/en/troubleshooting/_index.md | 5 ++ .../v2.x/en/troubleshooting/logging/_index.md | 48 ++++++++++++++++++ 3 files changed, 54 insertions(+), 49 deletions(-) create mode 100644 content/rancher/v2.x/en/troubleshooting/logging/_index.md diff --git a/content/rancher/v2.x/en/faq/technical/_index.md b/content/rancher/v2.x/en/faq/technical/_index.md index d5b22464771..1340d9623e3 100644 --- a/content/rancher/v2.x/en/faq/technical/_index.md +++ b/content/rancher/v2.x/en/faq/technical/_index.md @@ -56,55 +56,7 @@ New password for default admin user (user-xxxxx): ### How can I enable debug logging? -* Docker Install - * Enable -``` -$ docker exec -ti loglevel --set debug -OK -$ docker logs -f -``` - - * Disable -``` -$ docker exec -ti loglevel --set info -OK -``` - -* Kubernetes install (Helm) - * Enable -``` -$ KUBECONFIG=./kube_config_rancher-cluster.yml -$ kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | awk '{ print $1 }' | xargs -I{} kubectl --kubeconfig $KUBECONFIG -n cattle-system exec {} -- loglevel --set debug -OK -OK -OK -$ kubectl --kubeconfig $KUBECONFIG -n cattle-system logs -l app=rancher -``` - - * Disable -``` -$ KUBECONFIG=./kube_config_rancher-cluster.yml -$ kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | awk '{ print $1 }' | xargs -I{} kubectl --kubeconfig $KUBECONFIG -n cattle-system exec {} -- loglevel --set info -OK -OK -OK -``` - -* Kubernetes install (RKE add-on) - * Enable -``` -$ KUBECONFIG=./kube_config_rancher-cluster.yml -$ kubectl --kubeconfig $KUBECONFIG exec -n cattle-system $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name=="cattle-server") | .metadata.name') -- loglevel --set debug -OK -$ kubectl --kubeconfig $KUBECONFIG logs -n cattle-system -f $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name="cattle-server") | .metadata.name') -``` - - * Disable -``` -$ KUBECONFIG=./kube_config_rancher-cluster.yml -$ kubectl --kubeconfig $KUBECONFIG exec -n cattle-system $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name=="cattle-server") | .metadata.name') -- loglevel --set info -OK -``` +See [Troubleshooting: Logging]({{}}/rancher/v2.x/en/troubleshooting/logging/) ### My ClusterIP does not respond to ping diff --git a/content/rancher/v2.x/en/troubleshooting/_index.md b/content/rancher/v2.x/en/troubleshooting/_index.md index aa6c57afe66..edb5fb4f061 100644 --- a/content/rancher/v2.x/en/troubleshooting/_index.md +++ b/content/rancher/v2.x/en/troubleshooting/_index.md @@ -35,3 +35,8 @@ This section contains information to help you troubleshoot issues when using Ran - [Imported clusters]({{}}/rancher/v2.x/en/troubleshooting/imported-clusters/) If you experience issues when [Importing Kubernetes Clusters]({{}}/rancher/v2.x/en/cluster-provisioning/imported-clusters/) + +- [Logging]({{}}/rancher/v2.x/en/troubleshooting/logging/) + + Read more about what log levels can be configured and how to configure a log level. + diff --git a/content/rancher/v2.x/en/troubleshooting/logging/_index.md b/content/rancher/v2.x/en/troubleshooting/logging/_index.md new file mode 100644 index 00000000000..50024334901 --- /dev/null +++ b/content/rancher/v2.x/en/troubleshooting/logging/_index.md @@ -0,0 +1,48 @@ +--- +title: Logging +weight: 110 +--- + +The following log levels are used in Rancher: + +| Name | Description | +|---------|-------------| +| `info` | Logs informational messages. This is the default log level. | +| `debug` | Logs more detailed messages that can be used to debug. | +| `trace` | Logs very detailed messages on internal functions. This is very verbose and can contain sensitive information. | + +### How to configure a log level + +* Kubernetes install + * Configure debug log level +``` +$ KUBECONFIG=./kube_config_rancher-cluster.yml +$ kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | awk '{ print $1 }' | while read rancherpod; do kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $rancherpod -- loglevel --set debug; done +OK +OK +OK +$ kubectl --kubeconfig $KUBECONFIG -n cattle-system logs -l app=rancher +``` + + * Configure info log level +``` +$ KUBECONFIG=./kube_config_rancher-cluster.yml +$ kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | awk '{ print $1 }' | while read rancherpod; do kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $rancherpod -- loglevel --set info; done +OK +OK +OK +``` + +* Docker Install + * Configure debug log level +``` +$ docker exec -ti loglevel --set debug +OK +$ docker logs -f +``` + + * Configure info log level +``` +$ docker exec -ti loglevel --set info +OK +``` From f335afbad817a1fd1b5865580939033511f9a354 Mon Sep 17 00:00:00 2001 From: Sebastiaan van Steenis Date: Wed, 1 Apr 2020 14:32:35 +0200 Subject: [PATCH 09/14] Fix RDS tutorial link in Set up Infrastructure --- .../v2.x/en/installation/k8s-install/create-nodes-lb/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/installation/k8s-install/create-nodes-lb/_index.md b/content/rancher/v2.x/en/installation/k8s-install/create-nodes-lb/_index.md index 8fb2e197579..863ab0522eb 100644 --- a/content/rancher/v2.x/en/installation/k8s-install/create-nodes-lb/_index.md +++ b/content/rancher/v2.x/en/installation/k8s-install/create-nodes-lb/_index.md @@ -40,7 +40,7 @@ For a high-availability K3s installation, you will need to set up one of the fol When you install Kubernetes using the K3s installation script, you will pass in details for K3s to connect to the database. -For an example of one way to set up the database, refer to this [tutorial](./rds) for setting up a MySQL database on Amazon's RDS service. +For an example of one way to set up the database, refer to this [tutorial]({{}}/rancher/v2.x/en/installation/options/rds/) for setting up a MySQL database on Amazon's RDS service. For the complete list of options that are available for configuring a K3s cluster datastore, refer to the [K3s documentation.]({{}}/k3s/latest/en/installation/datastore/) From 7d7c5976e52bbb17926b4f35dd5c1ee1acff52ed Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 1 Apr 2020 13:00:37 -0700 Subject: [PATCH 10/14] Fix link --- content/rancher/v2.x/en/upgrades/upgrades/ha/helm2/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/upgrades/upgrades/ha/helm2/_index.md b/content/rancher/v2.x/en/upgrades/upgrades/ha/helm2/_index.md index f5d762cc5f5..2260c6988c5 100644 --- a/content/rancher/v2.x/en/upgrades/upgrades/ha/helm2/_index.md +++ b/content/rancher/v2.x/en/upgrades/upgrades/ha/helm2/_index.md @@ -3,7 +3,7 @@ title: Upgrading Rancher Installed on Kubernetes with Helm 2 weight: 1050 --- -> After Helm 3 was released, the [instructions for upgrading Rancher on a Kubernetes cluster](./ha) were updated to use Helm 3. +> After Helm 3 was released, the [instructions for upgrading Rancher on a Kubernetes cluster]({{}}/rancher/v2.x/en/upgrades/upgrades/ha) were updated to use Helm 3. > > If you are using Helm 2, we recommend [migrating to Helm 3](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) because it is simpler to use and more secure than Helm 2. > From a02c1d240f3bc4fbd40a5ab2a1348e99ee605dc7 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 1 Apr 2020 13:14:10 -0700 Subject: [PATCH 11/14] Remove reference to Rancher in K3s doc --- .../latest/en/installation/installation-requirements/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/installation/installation-requirements/_index.md b/content/k3s/latest/en/installation/installation-requirements/_index.md index 44e9bf5008f..ee053e09e0c 100644 --- a/content/k3s/latest/en/installation/installation-requirements/_index.md +++ b/content/k3s/latest/en/installation/installation-requirements/_index.md @@ -45,7 +45,7 @@ If you wish to utilize the metrics server, you will need to open port 10250 on e > **Important:** The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disables access to port 8472. -
Inbound Rules for Rancher Server Nodes
+
Inbound Rules for K3s Server Nodes
| Protocol | Port | Source | Description |-----|-----|----------------|---| From f698b97d3e4284e494efff4a767f22a88efcea20 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 1 Apr 2020 13:15:36 -0700 Subject: [PATCH 12/14] Revert "WIP: Misc changes and improvements to K3s docs" --- .../en/installation/install-options/_index.md | 4 ++-- .../installation-requirements/_index.md | 16 ++-------------- 2 files changed, 4 insertions(+), 16 deletions(-) diff --git a/content/k3s/latest/en/installation/install-options/_index.md b/content/k3s/latest/en/installation/install-options/_index.md index 70b164b38aa..e2bc4a2dbb2 100644 --- a/content/k3s/latest/en/installation/install-options/_index.md +++ b/content/k3s/latest/en/installation/install-options/_index.md @@ -29,7 +29,7 @@ When using this method to install K3s, the following environment variables can b - `INSTALL_K3S_SYMLINK` - By default will create symlinks for the kubectl, crictl, and ctr binaries if the commands do not already exist in path. If set to 'skip' will not create symlinks and 'force' will overwrite. + If set to 'skip' will not create symlinks, 'force' will overwrite, default will symlink if command does not exist in path. - `INSTALL_K3S_SKIP_START` @@ -66,7 +66,7 @@ When using this method to install K3s, the following environment variables can b - `INSTALL_K3S_NAME` - Name of systemd service to create, will default to 'k3s' if running k3s as a server and 'k3s-agent' if running k3s as an agent. If specified the name will be prefixed with 'k3s-'. + Name of systemd service to create, will default from the K3s exec command if not specified. If specified the name will be prefixed with 'k3s-'. - `INSTALL_K3S_TYPE` diff --git a/content/k3s/latest/en/installation/installation-requirements/_index.md b/content/k3s/latest/en/installation/installation-requirements/_index.md index ee053e09e0c..13da6c3d7ce 100644 --- a/content/k3s/latest/en/installation/installation-requirements/_index.md +++ b/content/k3s/latest/en/installation/installation-requirements/_index.md @@ -37,24 +37,12 @@ K3s performance depends on the performance of the database. To ensure optimal sp ## Networking -The K3s server needs port 6443 to be accessible by the nodes. +The K3s server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach other nodes over UDP port 8472 (Flannel VXLAN). If you do not use Flannel and provide your own custom CNI, then port 8472 is not needed by K3s. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. -The nodes need to be able to reach other nodes over UDP port 8472 when Flannel VXLAN is used. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. However, if you do not use Flannel and provide your own custom CNI, then port 8472 is not needed by K3s. +IMPORTANT: The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disabled access to port 8472. If you wish to utilize the metrics server, you will need to open port 10250 on each node. -> **Important:** The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disables access to port 8472. - -
Inbound Rules for K3s Server Nodes
- -| Protocol | Port | Source | Description -|-----|-----|----------------|---| -| TCP | 6443 | K3s server nodes | Kubernetes API -| UDP | 8472 | K3s server and agent nodes | Required only for Flannel VXLAN -| TCP | 10250 | K3s server and agent nodes | kubelet - -Typically all outbound traffic is allowed. - ## Large Clusters Hardware requirements are based on the size of your K3s cluster. For production and large clusters, we recommend using a high-availability setup with an external database. The following options are recommended for the external database in production: From b34a624e36a69ddbf9cb9245a64485309e49a595 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 1 Apr 2020 13:38:30 -0700 Subject: [PATCH 13/14] Fix pipeline link --- .../rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md index 0b756ed4de9..82e10d93ddf 100644 --- a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md @@ -7,7 +7,7 @@ aliases: Pipelines can be configured either through the UI or using a yaml file in the repository, i.e. `.rancher-pipeline.yml` or `.rancher-pipeline.yaml`. -In the [pipeline configuration docs](), we provide examples of each available feature within pipelines. Here is a full example for those who want to jump right in. +In the [pipeline configuration docs]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#pipeline-configuration), we provide examples of each available feature within pipelines. Here is a full example `rancher-pipeline.yml` for those who want to jump right in. ```yaml # example From b478606c09efed78b7c05500ec27b5d4b10e87d0 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 1 Apr 2020 17:32:16 -0700 Subject: [PATCH 14/14] Reorganize pipeline docs --- .../en/k8s-in-rancher/pipelines/_index.md | 857 ++++-------------- .../pipelines/concepts/_index.md | 36 + .../k8s-in-rancher/pipelines/config/_index.md | 658 ++++++++++++++ .../pipelines/docs-for-v2.0.x/_index.md | 1 + .../pipelines/example-repos/_index.md | 18 +- .../pipelines/example/_index.md | 4 +- .../pipelines/storage/_index.md | 103 +++ .../v2.x/en/project-admin/pipelines/_index.md | 346 +------ 8 files changed, 1003 insertions(+), 1020 deletions(-) create mode 100644 content/rancher/v2.x/en/k8s-in-rancher/pipelines/concepts/_index.md create mode 100644 content/rancher/v2.x/en/k8s-in-rancher/pipelines/config/_index.md rename content/rancher/v2.x/en/{project-admin => k8s-in-rancher}/pipelines/docs-for-v2.0.x/_index.md (98%) create mode 100644 content/rancher/v2.x/en/k8s-in-rancher/pipelines/storage/_index.md diff --git a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/_index.md index bfe10301d3f..767ecc6bbac 100644 --- a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/_index.md +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/_index.md @@ -6,36 +6,176 @@ aliases: --- +Rancher's pipeline provides a simple CI/CD experience. Use it to automatically checkout code, run builds or scripts, publish Docker images or catalog applications, and deploy the updated software to users. + +Setting up a pipeline can help developers deliver new software as quickly and efficiently as possible. Using Rancher, you can integrate with a GitHub repository to setup a continuous integration (CI) pipeline. + +After configuring Rancher and GitHub, you can deploy containers running Jenkins to automate a pipeline execution: + +- Build your application from code to image. +- Validate your builds. +- Deploy your build images to your cluster. +- Run unit tests. +- Run regression tests. + >**Notes:** > ->- Pipelines are new and improved for Rancher v2.1! Therefore, if you configured pipelines while using v2.0.x, you'll have to reconfigure them after upgrading to v2.1. ->- Still using v2.0.x? See the pipeline documentation for [previous versions]({{}}/rancher/v2.x/en/tools/pipelines/docs-for-v2.0.x). +>- Pipelines improved in Rancher v2.1. Therefore, if you configured pipelines while using v2.0.x, you'll have to reconfigure them after upgrading to v2.1. +>- Still using v2.0.x? See the pipeline documentation for [previous versions]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/docs-for-v2.0.x). +>- Rancher's pipeline provides a simple CI/CD experience, but it does not offer the full power and flexibility of and is not a replacement of enterprise-grade Jenkins or other CI tools your team uses. -Before setting up any pipelines, review the [pipeline overview]({{}}/rancher/v2.x/en/project-admin/pipelines/) and ensure that the project has [configured authentication to your version control provider]({{}}/rancher/v2.x/en/project-admin/pipelines/#version-control-providers), e.g. GitHub, GitLab, Bitbucket. If you haven't configured a version control provider, you can always use [Rancher's example repositories]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/example/) to view some common pipeline deployments. +This section covers the following topics: -If you can access a project, you can enable repositories to start building pipelines. Only an [administrator]({{}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/), [cluster owner or member]({{}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), or [project owner]({{}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) can authorize version control providers. +- [Concepts](#concepts) +- [How Pipelines Work](#how-pipelines-work) +- [Roles-based Access Control for Pipelines](#roles-based-access-control-for-pipelines) +- [Setting up Pipelines](#setting-up-pipelines) + - [Configure version control providers](#1-configure-version-control-providers) + - [Configure repositories](#2-configure-repositories) + - [Configure the pipeline](#3-configure-the-pipeline) +- [Pipeline Configuration Reference](#pipeline-configuration-reference) +- [Running your Pipelines](#running-your-pipelines) +- [Triggering a Pipeline](#triggering-a-pipeline) + - [Modifying the Event Triggers for the Repository](#modifying-the-event-triggers-for-the-repository) -## Concepts +# Concepts -When setting up a pipeline, it's helpful to know a few related terms. +For an explanation of concepts and terminology used in this section, refer to [this page.]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/concepts) -- **Pipeline:** +# How Pipelines Work - A pipeline consists of stages and steps. It is based on a specific repository. It defines the process to build, test, and deploy your code. Rancher uses the [pipeline as code](https://jenkins.io/doc/book/pipeline-as-code/) model. Pipeline configuration is represented as a pipeline file in the source code repository, using the file name `.rancher-pipeline.yml` or `.rancher-pipeline.yaml`. +After enabling the ability to use pipelines in a project, you can configure multiple pipelines in each project. Each pipeline is unique and can be configured independently. -- **Stages:** +A pipeline is configured off of a group of files that are checked into source code repositories. Users can configure their pipelines either through the Rancher UI or by adding a `.rancher-pipeline.yml` into the repository. - A pipeline stage consists of multiple steps. Stages are executed in the order defined in the pipeline file. The steps in a stage are executed concurrently. A stage starts when all steps in the former stage finish without failure. +Before pipelines can be configured, you will need to configure authentication to your version control provider, e.g. GitHub, GitLab, Bitbucket. If you haven't configured a version control provider, you can always use [Rancher's example repositories]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/) to view some common pipeline deployments. -- **Steps:** +When you configure a pipeline in one of your projects, a namespace specifically for the pipeline is automatically created. The following components are deployed to it: - A pipeline step is executed inside a specified stage. A step fails if it exits with a code other than `0`. If a step exits with this failure code, the entire pipeline fails and terminates. + - **Jenkins:** -- **Workspace:** + The pipeline's build engine. Because project users do not directly interact with Jenkins, it's managed and locked. - The workspace is the working directory shared by all pipeline steps. In the beginning of a pipeline, source code is checked out to the workspace. The command for every step bootstraps in the workspace. During a pipeline execution, the artifacts from a previous step will be available in future steps. The working directory is an ephemeral volume and will be cleaned out with the executor pod when a pipeline execution is finished. + >**Note:** There is no option to use existing Jenkins deployments as the pipeline engine. -## Configuring Repositories + - **Docker Registry:** + + Out-of-the-box, the default target for your build-publish step is an internal Docker Registry. However, you can make configurations to push to a remote registry instead. The internal Docker Registry is only accessible from cluster nodes and cannot be directly accessed by users. Images are not persisted beyond the lifetime of the pipeline and should only be used in pipeline runs. If you need to access your images outside of pipeline runs, please push to an external registry. + + - **Minio:** + + Minio storage is used to store the logs for pipeline executions. + + >**Note:** The managed Jenkins instance works statelessly, so don't worry about its data persistency. The Docker Registry and Minio instances use ephemeral volumes by default, which is fine for most use cases. If you want to make sure pipeline logs can survive node failures, you can configure persistent volumes for them, as described in [data persistency for pipeline components]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/storage). + +# Roles-based Access Control for Pipelines + +If you can access a project, you can enable repositories to start building pipelines. + +Only [administrators]({{}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/), [cluster owners or members]({{}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), or [project owners]({{}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) can configure version control providers and manage global pipeline execution settings. + +Project members can only configure repositories and pipelines. + +# Setting up Pipelines + +To set up pipelines, you will need to do the following: + +1. [Configure version control providers](#1-configure-version-control-providers) +2. [Configure repositories](#2-configure-repositories) +3. [Configure the pipeline](#3-configure-the-pipeline) + +### 1. Configure Version Control Providers + +Before you can start configuring a pipeline for your repository, you must configure and authorize a version control provider. + +| Provider | Available as of | +| --- | --- | +| GitHub | v2.0.0 | +| GitLab | v2.1.0 | +| Bitbucket | v2.2.0 | + +Select your provider's tab below and follow the directions. + +{{% tabs %}} +{{% tab "GitHub" %}} +1. From the **Global** view, navigate to the project that you want to configure pipelines. + +1. Select **Tools > Pipelines** in the navigation bar. In versions prior to v2.2.0, you can select **Resources > Pipelines**. + +1. Follow the directions displayed to **Setup a Github application**. Rancher redirects you to Github to setup an OAuth App in Github. + +1. From GitHub, copy the **Client ID** and **Client Secret**. Paste them into Rancher. + +1. If you're using GitHub for enterprise, select **Use a private github enterprise installation**. Enter the host address of your GitHub installation. + +1. Click **Authenticate**. + +{{% /tab %}} +{{% tab "GitLab" %}} + +_Available as of v2.1.0_ + +1. From the **Global** view, navigate to the project that you want to configure pipelines. + +1. Select **Tools > Pipelines** in the navigation bar. In versions prior to v2.2.0, you can select **Resources > Pipelines**. + +1. Follow the directions displayed to **Setup a GitLab application**. Rancher redirects you to GitLab. + +1. From GitLab, copy the **Application ID** and **Secret**. Paste them into Rancher. + +1. If you're using GitLab for enterprise setup, select **Use a private gitlab enterprise installation**. Enter the host address of your GitLab installation. + +1. Click **Authenticate**. + +>**Note:** +> 1. Pipeline uses Gitlab [v4 API](https://docs.gitlab.com/ee/api/v3_to_v4.html) and the supported Gitlab version is 9.0+. +> 2. If you use GitLab 10.7+ and your Rancher setup is in a local network, enable the **Allow requests to the local network from hooks and services** option in GitLab admin settings. +{{% /tab %}} +{{% tab "Bitbucket Cloud" %}} + +_Available as of v2.2.0_ + +1. From the **Global** view, navigate to the project that you want to configure pipelines. + +1. Select **Tools > Pipelines** in the navigation bar. + +1. Choose the **Use public Bitbucket Cloud** option. + +1. Follow the directions displayed to **Setup a Bitbucket Cloud application**. Rancher redirects you to Bitbucket to setup an OAuth consumer in Bitbucket. + +1. From Bitbucket, copy the consumer **Key** and **Secret**. Paste them into Rancher. + +1. Click **Authenticate**. + +{{% /tab %}} +{{% tab "Bitbucket Server" %}} + +_Available as of v2.2.0_ + +1. From the **Global** view, navigate to the project that you want to configure pipelines. + +1. Select **Tools > Pipelines** in the navigation bar. + +1. Choose the **Use private Bitbucket Server setup** option. + +1. Follow the directions displayed to **Setup a Bitbucket Server application**. + +1. Enter the host address of your Bitbucket server installation. + +1. Click **Authenticate**. + +>**Note:** +> Bitbucket server needs to do SSL verification when sending webhooks to Rancher. Please ensure that Rancher server's certificate is trusted by the Bitbucket server. There are two options: +> +> 1. Setup Rancher server with a certificate from a trusted CA. +> 1. If you're using self-signed certificates, import Rancher server's certificate to the Bitbucket server. For instructions, see the Bitbucket server documentation for [configuring self-signed certificates](https://confluence.atlassian.com/bitbucketserver/if-you-use-self-signed-certificates-938028692.html). +> +{{% /tab %}} +{{% /tabs %}} + +**Result:** After the version control provider is authenticated, you will be automatically re-directed to start configuring which repositories you want start using with a pipeline. + +### 2. Configure Repositories After the version control provider is authorized, you are automatically re-directed to start configuring which repositories that you want start using pipelines with. Even if someone else has set up the version control provider, you will see their repositories and can build a pipeline. @@ -53,187 +193,58 @@ After the version control provider is authorized, you are automatically re-direc **Results:** You have a list of repositories that you can start configuring pipelines for. -## Pipeline Configuration +### 3. Configure the Pipeline -Now that repositories are added to your project, you can start configuring the pipeline by adding automated stages and steps. For your convenience, there are multiple built-in [step types](#step-types) for dedicated tasks. +Now that repositories are added to your project, you can start configuring the pipeline by adding automated stages and steps. For your convenience, there are multiple built-in step types for dedicated tasks. 1. From the **Global** view, navigate to the project that you want to configure pipelines. 1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** -1. Find the repository that you want to set up a pipeline for. Pipelines can be configured either through the UI or using a yaml file in the repository, i.e. `.rancher-pipeline.yml` or `.rancher-pipeline.yaml`. Throughout the next couple of steps, we'll provide the options of how to do pipeline configuration through the UI or the YAML file. +1. Find the repository that you want to set up a pipeline for. + +1. Configure the pipeline through the UI or using a yaml file in the repository, i.e. `.rancher-pipeline.yml` or `.rancher-pipeline.yaml`. Pipeline configuration is split into stages and steps. Stages must fully complete before moving onto the next stage, but steps in a stage run concurrently. For each stage, you can add different step types. Note: As you build out each step, there are different advanced options based on the step type. Advanced options include trigger rules, environment variables, and secrets. For more information on configuring the pipeline through the UI or the YAML file, refer to the [pipeline configuration reference.]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/config) * If you are going to use the UI, select the vertical **Ellipsis (...) > Edit Config** to configure the pipeline using the UI. After the pipeline is configured, you must view the YAML file and push it to the repository. - * If you are going to use the YAML file, select the vertical **Ellipsis (...) **View/Edit YAML** to configure the pipeline. If you choose to use a YAML file, you need to push it to the repository after any changes in order for it to be updated in the repository. - - >**Note:** When editing the pipeline configuration, it takes a few moments for Rancher to check for an existing pipeline configuration. + * If you are going to use the YAML file, select the vertical **Ellipsis (...) > View/Edit YAML** to configure the pipeline. If you choose to use a YAML file, you need to push it to the repository after any changes in order for it to be updated in the repository. When editing the pipeline configuration, it takes a few moments for Rancher to check for an existing pipeline configuration. 1. Select which `branch` to use from the list of branches. -1. Pipeline configuration is split into stages and [steps](#step-types). Remember that stages must fully complete before moving onto the next stage, but steps in a stage run concurrently. +1. _Available as of v2.2.0_ Optional: Set up notifications. - For each stage, you can add different step types. Learn more about how to configure each step type: +1. Set up the trigger rules for the pipeline. - - [Run Script](#run-script) - - [Build and Publish Images](#build-and-publish-images) - - [Publish Catalog Template](#publish-catalog-template) - - [Deploy YAML](#deploy-yaml) - - [Deploy Catalog App](#deploy-catalog-app) - - >**Note:** As you build out each step, there are different [advanced options](#advanced-options) based on the step type. - - {{% accordion id="stages-and-steps" label="Adding Stages and Steps" %}} -{{% tabs %}} -{{% tab "By UI" %}} -
-If you haven't added any stages, click **Configure pipeline for this branch** to configure the pipeline through the UI. - -1. Add stages to your pipeline execution by clicking **Add Stage**. - - 1. Enter a **Name** for each stage of your pipeline. - 1. For each stage, you can configure [trigger rules](#trigger-rules) by clicking on **Show Advanced Options**. Note: this can always be updated at a later time. - -1. After you've created a stage, start [adding steps](#step-types) by clicking **Add a Step**. You can add multiple steps to each stage. -
-
-{{% /tab %}} -{{% tab "By YAML" %}} -
-For each stage, you can add multiple steps. Read more about each [step type](#step-types) and the [advanced options](#advanced-options) to get all the details on how to configure the YAML. This is only a small example of how to have multiple stages with a singular step in each stage. - -```yaml -# example -stages: - - name: Build something - # Conditions for stages - when: - branch: master - event: [ push, pull_request ] - # Multiple steps run concurrently - steps: - - runScriptConfig: - image: busybox - shellScript: date -R - - name: Publish my image - steps: - - publishImageConfig: - dockerfilePath: ./Dockerfile - buildContext: . - tag: rancher/rancher:v2.0.0 - # Optionally push to remote registry - pushRemote: true - registry: reg.example.com -``` -
-{{% /tab %}} -{{% /tabs %}} - {{% /accordion %}} - -1. _Available as of v2.2.0_ - - **Notifications:** Decide if you want to set up notifications for your pipeline. You can enable notifications to any [notifiers]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers/) based on the build status of a pipeline. Before enabling notifications, Rancher recommends [setting up notifiers]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers/#adding-notifiers) so it will be easy to add recipients immediately. - - {{% accordion id="notification" label="Configuring Notifications" %}} - - -{{% tabs %}} -{{% tab "By UI" %}} -
-_Available as of v2.2.0_ - -1. Within the **Notification** section, turn on notifications by clicking **Enable**. - -1. Select the conditions for the notification. You can select to get a notification for the following statuses: `Failed`, `Success`, `Changed`. For example, if you want to receive notifications when an execution fails, select **Failed**. - -1. If you don't have any existing [notifiers]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers), Rancher will provide a warning that no notifiers are set up and provide a link to be able to go to the notifiers page. Follow the [instructions]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers/#adding-notifiers) to add a notifier. If you already have notifiers, you can add them to the notification by clicking the **Add Recipient** button. - - > **Note:** Notifiers are configured at a cluster level and require a different level of permissions. - -1. For each recipient, select which notifier type from the dropdown. Based on the type of notifier, you can use the default recipient or override the recipient with a different one. For example, if you have a notifier for _Slack_, you can update which channel to send the notification to. You can add additional notifiers by clicking **Add Recipient**. -
-
-{{% /tab %}} -{{% tab "By YAML" %}} -
-_Available as of v2.2.0_ - -In the `notification` section, you will provide the following information: - -* **Recipients:** This will be the list of notifiers/recipients that will receive the notification. - * **Notifier:** The ID of the notifier. This can be found by finding the notifier and selecting **View in API** to get the ID. - * **Recipient:** Depending on the type of the notifier, the "default recipient" can be used or you can override this with a different recipient. For example, when configuring a slack notifier, you select a channel as your default recipient, but if you wanted to send notifications to a different channel, you can select a different recipient. -* **Condition:** Select which conditions of when you want the notification to be sent. -* **Message (Optional):** If you want to change the default notification message, you can edit this in the yaml. Note: This option is not available in the UI. - -```yaml -# Example -stages: - - name: Build something - steps: - - runScriptConfig: - image: busybox - shellScript: ls -notification: - recipients: - - # Recipient - recipient: "#mychannel" - # ID of Notifier - notifier: "c-wdcsr:n-c9pg7" - - recipient: "test@example.com" - notifier: "c-wdcsr:n-lkrhd" - # Select which statuses you want the notification to be sent - condition: ["Failed", "Success", "Changed"] - # Ability to override the default message (Optional) - message: "my-message" -``` -
-{{% /tab %}} -{{% /tabs %}} - - {{% /accordion %}} - -1. Set up the **[Trigger Rules](#trigger-rules)** for the pipeline. - -1. Enter a **Timeout** for the pipeline. By default, each pipeline execution has a timeout of 60 minutes. If the pipeline execution cannot complete within its timeout period, the pipeline is aborted. - - {{% accordion id="timeout" label="Setting up Timeout" %}} - -{{% tabs %}} -{{% tab "By UI" %}} -
-Enter a new value in the **Timeout** field. -
-
-{{% /tab %}} -{{% tab "By YAML" %}} -
-In the `timeout` section, enter the timeout value in minutes. -```yaml -# example -stages: - - name: Build something - steps: - - runScriptConfig: - image: busybox - shellScript: ls -# timeout in minutes -timeout: 30 -``` -
-{{% /tab %}} -{{% /tabs %}} - - {{% /accordion %}} +1. Enter a **Timeout** for the pipeline. 1. When all the stages and steps are configured, click **Done**. **Results:** Your pipeline is now configured and ready to be run. -## Running your Pipelines + +# Pipeline Configuration Reference + +Refer to [this page]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/config) for details on how to configure a pipeline to: + +- Run a script +- Build and publish images +- Publish catalog templates +- Deploy YAML +- Deploy a catalog app + +The configuration reference also covers how to configure: + +- Notifications +- Timeouts +- The rules that trigger a pipeline +- Environment variables +- Secrets + + +# Running your Pipelines Run your pipeline for the first time. From the project view in Rancher, go to **Resources > Pipelines.** (In versions prior to v2.3.0, go to the **Pipelines** tab.) Find your pipeline and select the vertical **Ellipsis (...) > Run**. -During this initial run, your pipeline is tested, and the following [pipeline components]({{}}/rancher/v2.x/en/project-admin/pipelines/#how-pipelines-work) are deployed to your project as workloads in a new namespace dedicated to the pipeline: +During this initial run, your pipeline is tested, and the following pipeline components are deployed to your project as workloads in a new namespace dedicated to the pipeline: - `docker-registry` - `jenkins` @@ -241,7 +252,7 @@ During this initial run, your pipeline is tested, and the following [pipeline co This process takes several minutes. When it completes, you can view each pipeline component from the project **Workloads** tab. -## Pipeline Setting +# Triggering a Pipeline When a repository is enabled, a webhook is automatically set in the version control provider. By default, the pipeline is triggered by a **push** event to a repository, but you can modify the event(s) that trigger running the pipeline. @@ -251,7 +262,7 @@ Available Events: * **Pull Request**: Whenever a pull request is made to the repository, the pipeline is triggered. * **Tag**: When a tag is created in the repository, the pipeline is triggered. -> **Note:** This option doesn't exist for Rancher's [example repositories]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/example/). +> **Note:** This option doesn't exist for Rancher's [example repositories]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/). ### Modifying the Event Triggers for the Repository @@ -263,494 +274,4 @@ Available Events: 1. Select which event triggers (**Push**, **Pull Request** or **Tag**) you want for the repository. -1. Click **Save**. - -## Step Types - -Within each stage, you can add as many steps as you'd like. When there are multiple steps in one stage, they run concurrently. - -- [Run Script](#run-script) -- [Build and Publish Images](#build-and-publish-images) -- [Publish Catalog Template](#publish-catalog-template) -- [Deploy YAML](#deploy-yaml) -- [Deploy Catalog App](#deploy-catalog-app) - - - -### Run Script - -The **Run Script** step executes arbitrary commands in the workspace inside a specified container. You can use it to build, test and do more, given whatever utilities the base image provides. For your convenience, you can use variables to refer to metadata of a pipeline execution. Please refer to the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) for the list of available variables. - -{{% tabs %}} - -{{% tab "By UI" %}} - -1. From the **Step Type** drop-down, choose **Run Script** and fill in the form. - -1. Click **Add**. - -{{% /tab %}} - -{{% tab "By YAML" %}} -
-```yaml -# example -stages: -- name: Build something - steps: - - runScriptConfig: - image: golang - shellScript: go build -``` -
-{{% /tab %}} - -{{% /tabs %}} - -### Build and Publish Images - -The **Build and Publish Image** step builds and publishes a Docker image. This process requires a Dockerfile in your source code's repository to complete successfully. - -_Available as of Rancher v2.1.0_ - -The option to publish an image to an insecure registry is not exposed in the UI, but you can specify an environment variable in the YAML that allows you to publish an image insecurely. - -{{% tabs %}} - -{{% tab "By UI" %}} -1. From the **Step Type** drop-down, choose **Build and Publish**. - -1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**. - - Field | Description | - ---------|----------| - Dockerfile Path | The relative path to the Dockerfile in the source code repo. By default, this path is `./Dockerfile`, which assumes the Dockerfile is in the root directory. You can set it to other paths in different use cases (`./path/to/myDockerfile` for example). | - Image Name | The image name in `name:tag` format. The registry address is not required. For example, to build `example.com/repo/my-image:dev`, enter `repo/my-image:dev`. | - Push image to remote repository | An option to set the registry that publishes the image that's built. To use this option, enable it and choose a registry from the drop-down. If this option is disabled, the image is pushed to the internal registry. | - Build Context

(**Show advanced options**)| By default, the root directory of the source code (`.`). For more details, see the Docker [build command documentation](https://docs.docker.com/engine/reference/commandline/build/). - -{{% /tab %}} - -{{% tab "By YAML" %}} - -You can use specific arguments for Docker daemon and the build. They are not exposed in the UI, but they are available in pipeline YAML format, as indicated in the example below. Available environment variables include: - -Variable Name | Description -------------------------|------------------------------------------------------------ -PLUGIN_DRY_RUN | Disable docker push -PLUGIN_DEBUG | Docker daemon executes in debug mode -PLUGIN_MIRROR | Docker daemon registry mirror -PLUGIN_INSECURE | Docker daemon allows insecure registries -PLUGIN_BUILD_ARGS | Docker build args, a comma separated list - -
-```yaml -# This example shows an environment variable being used -# in the Publish Image step. This variable allows you to -# publish an image to an insecure registry: - -stages: -- name: Publish Image - steps: - - publishImageConfig: - dockerfilePath: ./Dockerfile - buildContext: . - tag: repo/app:v1 - pushRemote: true - registry: example.com - env: - PLUGIN_INSECURE: "true" -``` -
-{{% /tab %}} - -{{% /tabs %}} - -### Publish Catalog Template - -_Available as of v2.2.0_ - -The **Publish Catalog Template** step publishes a version of a catalog app template (i.e. Helm chart) to a [git hosted chart repository]({{}}/rancher/v2.x/en/catalog/custom/). It generates a git commit and pushes it to your chart repository. This process requires a chart folder in your source code's repository and a pre-configured secret in the dedicated pipeline namespace to complete successfully. Any variables in the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) is supported for any file in the chart folder. - -{{% tabs %}} - -{{% tab "By UI" %}} -
- -1. From the **Step Type** drop-down, choose **Publish Catalog Template**. - -1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**. - - Field | Description | - ---------|----------| - Chart Folder | The relative path to the chart folder in the source code repo, where the `Chart.yaml` file is located. | - Catalog Template Name | The name of the template. For example, wordpress. | - Catalog Template Version | The version of the template you want to publish, it should be consistent with the version defined in the `Chart.yaml` file. | - Protocol | You can choose to publish via HTTP(S) or SSH protocol. | - Secret | The secret that stores your Git credentials. You need to create a secret in dedicated pipeline namespace in the project before adding this step. If you use HTTP(S) protocol, store Git username and password in `USERNAME` and `PASSWORD` key of the secret. If you use SSH protocol, store Git deploy key in `DEPLOY_KEY` key of the secret. After the secret is created, select it in this option. | - Git URL | The Git URL of the chart repository that the template will be published to. | - Git Branch | The Git branch of the chart repository that the template will be published to. | - Author Name | The author name used in the commit message. | - Author Email | The author email used in the commit message. | - - -{{% /tab %}} - -{{% tab "By YAML" %}} -
-You can add **Publish Catalog Template** steps directly in the `.rancher-pipeline.yml` file. - -Under the `steps` section, add a step with `publishCatalogConfig`. You will provide the following information: - -* Path: The relative path to the chart folder in the source code repo, where the `Chart.yaml` file is located. -* CatalogTemplate: The name of the template. -* Version: The version of the template you want to publish, it should be consistent with the version defined in the `Chart.yaml` file. -* GitUrl: The git URL of the chart repository that the template will be published to. -* GitBranch: The git branch of the chart repository that the template will be published to. -* GitAuthor: The author name used in the commit message. -* GitEmail: The author email used in the commit message. -* Credentials: You should provide Git credentials by referencing secrets in dedicated pipeline namespace. If you publish via SSH protocol, inject your deploy key to the `DEPLOY_KEY` environment variable. If you publish via HTTP(S) protocol, inject your username and password to `USERNAME` and `PASSWORD` environment variables. - -```yaml -# example -stages: -- name: Publish Wordpress Template - steps: - - publishCatalogConfig: - path: ./charts/wordpress/latest - catalogTemplate: wordpress - version: ${CICD_GIT_TAG} - gitUrl: git@github.com:myrepo/charts.git - gitBranch: master - gitAuthor: example-user - gitEmail: user@example.com - envFrom: - - sourceName: publish-keys - sourceKey: DEPLOY_KEY -``` - -
-{{% /tab %}} - -{{% /tabs %}} - -### Deploy YAML - -This step deploys arbitrary Kubernetes resources to the project. This deployment requires a Kubernetes manifest file to be present in the source code repository. Pipeline variable substitution is supported in the manifest file. You can view an example file at [GitHub](https://github.com/rancher/pipeline-example-go/blob/master/deployment.yaml). Please refer to the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) for the list of available variables. - -{{% tabs %}} - -{{% tab "By UI" %}} - -1. From the **Step Type** drop-down, choose **Deploy YAML** and fill in the form. - -1. Enter the **YAML Path**, which is the path to the manifest file in the source code. - -1. Click **Add**. - -{{% /tab %}} - -{{% tab "By YAML" %}} -
-```yaml -# example -stages: -- name: Deploy - steps: - - applyYamlConfig: - path: ./deployment.yaml -``` -
-{{% /tab %}} - -{{% /tabs %}} - -### Deploy Catalog App - -_Available as of v2.2.0_ - -The **Deploy Catalog App** step deploys a catalog app in the project. It will install a new app if it is not present, or upgrade an existing one. - -{{% tabs %}} - -{{% tab "By UI" %}} - -1. From the **Step Type** drop-down, choose **Deploy Catalog App**. - -1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**. - - Field | Description | - ---------|----------| - Catalog | The catalog from which the app template will be used. | - Template Name | The name of the app template. For example, wordpress. | - Template Version | The version of the app template you want to deploy. | - Namespace | The target namespace where you want to deploy the app. | - App Name | The name of the app you want to deploy. | - Answers | Key-value pairs of answers used to deploy the app. | - - -{{% /tab %}} - -{{% tab "By YAML" %}} -
-You can add **Deploy Catalog App** steps directly in the `.rancher-pipeline.yml` file. - -Under the `steps` section, add a step with `applyAppConfig`. You will provide the following information: - -* CatalogTemplate: The ID of the template. This can be found by clicking `Launch app` and selecting `View details` for the app. It is the last part of the URL. -* Version: The version of the template you want to deploy. -* Answers: Key-value pairs of answers used to deploy the app. -* Name: The name of the app you want to deploy. -* TargetNamespace: The target namespace where you want to deploy the app. - -```yaml -# example -stages: -- name: Deploy App - steps: - - applyAppConfig: - catalogTemplate: cattle-global-data:library-mysql - version: 0.3.8 - answers: - persistence.enabled: "false" - name: testmysql - targetNamespace: test -``` -
-{{% /tab %}} -{{% /tabs %}} - -## Advanced Options - -Within a pipeline, there are multiple advanced options for different parts of the pipeline. - -- [Trigger Rules](#trigger-rules) -- [Environment Variables](#environment-variables) -- [Secrets](#secrets) - -### Trigger Rules - -Trigger rules can be created to have fine-grained control of pipeline executions in your pipeline configuration. Trigger rules come in two types: - -- **Run this when:** - - This type of rule starts the pipeline, stage, or step when a trigger explicitly occurs. - -- **Do Not Run this when:** - - This type of rule skips the pipeline, stage, or step when a trigger explicitly occurs. - -If all conditions evaluate to `true`, then the pipeline/stage/step is executed. Otherwise it is skipped. When a pipeline is skipped, none of the pipeline is executed. When a stage/step is skipped, it is considered successful and follow-up stages/steps continue to run. - -Wildcard character (`*`) expansion is supported in `branch` conditions. - -{{% tabs %}} -{{% tab "Pipeline Trigger" %}} - -1. From the **Global** view, navigate to the project that you want to configure a pipeline trigger rule. - -1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** - -1. From the repository for which you want to manage trigger rules, select the vertical **Ellipsis (...) > Edit Config**. - -1. Click on **Show Advanced Options**. - -1. In the **Trigger Rules** section, configure rules to run or skip the pipeline. - - 1. Click **Add Rule**. In the **Value** field, enter the name of the branch that triggers the pipeline. - - 1. **Optional:** Add more branches that trigger a build. - -1. Click **Done.** - -{{% /tab %}} -{{% tab "Stage Trigger" %}} -1. From the **Global** view, navigate to the project that you want to configure a stage trigger rule. - -1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** - -1. From the repository for which you want to manage trigger rules, select the vertical **Ellipsis (...) > Edit Config**. - -1. Find the **stage** that you want to manage trigger rules, click the **Edit** icon for that stage. - -1. Click **Show advanced options**. - -1. In the **Trigger Rules** section, configure rules to run or skip the stage. - - 1. Click **Add Rule**. - - 1. Choose the **Type** that triggers the stage and enter a value. - - | Type | Value | - | ------ | -------------------------------------------------------------------- | - | Branch | The name of the branch that triggers the stage. | - | Event | The type of event that triggers the stage. Values are: `Push`, `Pull Request`, `Tag` | - -1. Click **Save**. - -{{% /tab %}} -{{% tab "Step Trigger" %}} -1. From the **Global** view, navigate to the project that you want to configure a stage trigger rule. - -1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** - -1. From the repository for which you want to manage trigger rules, select the vertical **Ellipsis (...) > Edit Config**. - -1. Find the **step** that you want to manage trigger rules, click the **Edit** icon for that step. - -1. Click **Show advanced options**. - -1. In the **Trigger Rules** section, configure rules to run or skip the step. - - 1. Click **Add Rule**. - - 1. Choose the **Type** that triggers the step and enter a value. - - | Type | Value | - | ------ | -------------------------------------------------------------------- | - | Branch | The name of the branch that triggers the step. | - | Event | The type of event that triggers the step. Values are: `Push`, `Pull Request`, `Tag` | - -1. Click **Save**. - -{{% /tab %}} -{{% tab "By YAML" %}} -
-```yaml -# example -stages: - - name: Build something - # Conditions for stages - when: - branch: master - event: [ push, pull_request ] - # Multiple steps run concurrently - steps: - - runScriptConfig: - image: busybox - shellScript: date -R - # Conditions for steps - when: - branch: [ master, dev ] - event: push -# branch conditions for the pipeline -branch: - include: [ master, feature/*] - exclude: [ dev ] -``` -
-{{% /tab %}} -{{% /tabs %}} - -### Environment Variables - -When configuring a pipeline, certain [step types](#step-types) allow you to use environment variables to configure the step's script. - -{{% tabs %}} -{{% tab "By UI" %}} -1. From the **Global** view, navigate to the project that you want to configure pipelines. - -1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** - -1. From the pipeline for which you want to edit build triggers, select **Ellipsis (...) > Edit Config**. - -1. Within one of the stages, find the **step** that you want to add an environment variable for, click the **Edit** icon. - -1. Click **Show advanced options**. - -1. Click **Add Variable**, and then enter a key and value in the fields that appear. Add more variables if needed. - -1. Add your environment variable(s) into either the script or file. - -1. Click **Save**. - -{{% /tab %}} - -{{% tab "By YAML" %}} -
-```yaml -# example -stages: - - name: Build something - steps: - - runScriptConfig: - image: busybox - shellScript: echo ${FIRST_KEY} && echo ${SECOND_KEY} - env: - FIRST_KEY: VALUE - SECOND_KEY: VALUE2 -``` -
-{{% /tab %}} - -{{% /tabs %}} - -### Secrets - -If you need to use security-sensitive information in your pipeline scripts (like a password), you can pass them in using Kubernetes [secrets]({{}}/rancher/v2.x/en/k8s-in-rancher/secrets/). - -#### Prerequisite -Create a secret in the same project as your pipeline, or explicitly in the namespace where pipeline build pods run. -
- ->**Note:** Secret injection is disabled on [pull request events](#pipeline-setting). - -{{% tabs %}} -{{% tab "By UI" %}} -1. From the **Global** view, navigate to the project that you want to configure pipelines. - -1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** - -1. From the pipeline for which you want to edit build triggers, select **Ellipsis (...) > Edit Config**. - -1. Within one of the stages, find the **step** that you want to use a secret for, click the **Edit** icon. - -1. Click **Show advanced options**. - -1. Click **Add From Secret**. Select the secret file that you want to use. Then choose a key. Optionally, you can enter an alias for the key. - -1. Click **Save**. - -{{% /tab %}} -{{% tab "By YAML" %}} -
-```yaml -# example -stages: - - name: Build something - steps: - - runScriptConfig: - image: busybox - shellScript: echo ${ALIAS_ENV} - # environment variables from project secrets - envFrom: - - sourceName: my-secret - sourceKey: secret-key - targetKey: ALIAS_ENV -``` -
-{{% /tab %}} -{{% /tabs %}} - -## Pipeline Variable Substitution Reference - -For your convenience, the following variables are available for your pipeline configuration scripts. During pipeline executions, these variables are replaced by metadata. You can reference them in the form of `${VAR_NAME}`. - -Variable Name | Description -------------------------|------------------------------------------------------------ -`CICD_GIT_REPO_NAME` | Repository name (Github organization omitted). -`CICD_GIT_URL` | URL of the Git repository. -`CICD_GIT_COMMIT` | Git commit ID being executed. -`CICD_GIT_BRANCH` | Git branch of this event. -`CICD_GIT_REF` | Git reference specification of this event. -`CICD_GIT_TAG` | Git tag name, set on tag event. -`CICD_EVENT` | Event that triggered the build (`push`, `pull_request` or `tag`). -`CICD_PIPELINE_ID` | Rancher ID for the pipeline. -`CICD_EXECUTION_SEQUENCE` | Build number of the pipeline. -`CICD_EXECUTION_ID` | Combination of `{CICD_PIPELINE_ID}-{CICD_EXECUTION_SEQUENCE}`. -`CICD_REGISTRY` | Address for the Docker registry for the previous publish image step, available in the Kubernetes manifest file of a `Deploy YAML` step. -`CICD_IMAGE` | Name of the image built from the previous publish image step, available in the Kubernetes manifest file of a `Deploy YAML` step. It does not contain the image tag.

[Example](https://github.com/rancher/pipeline-example-go/blob/master/deployment.yaml) +1. Click **Save**. \ No newline at end of file diff --git a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/concepts/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/concepts/_index.md new file mode 100644 index 00000000000..db8e3a24a58 --- /dev/null +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/concepts/_index.md @@ -0,0 +1,36 @@ +--- +title: Concepts +weight: 1 +--- + +The purpose of this page is to explain common concepts and terminology related to pipelines. + +- **Pipeline:** + + A _pipeline_ is a software delivery process that is broken into different stages and steps. Setting up a pipeline can help developers deliver new software as quickly and efficiently as possible. Within Rancher, you can configure pipelines for each of your Rancher projects. A pipeline is based on a specific repository. It defines the process to build, test, and deploy your code. Rancher uses the [pipeline as code](https://jenkins.io/doc/book/pipeline-as-code/) model. Pipeline configuration is represented as a pipeline file in the source code repository, using the file name `.rancher-pipeline.yml` or `.rancher-pipeline.yaml`. + +- **Stages:** + + A pipeline stage consists of multiple steps. Stages are executed in the order defined in the pipeline file. The steps in a stage are executed concurrently. A stage starts when all steps in the former stage finish without failure. + +- **Steps:** + + A pipeline step is executed inside a specified stage. A step fails if it exits with a code other than `0`. If a step exits with this failure code, the entire pipeline fails and terminates. + +- **Workspace:** + + The workspace is the working directory shared by all pipeline steps. In the beginning of a pipeline, source code is checked out to the workspace. The command for every step bootstraps in the workspace. During a pipeline execution, the artifacts from a previous step will be available in future steps. The working directory is an ephemeral volume and will be cleaned out with the executor pod when a pipeline execution is finished. + +Typically, pipeline stages include: + +- **Build:** + + Each time code is checked into your repository, the pipeline automatically clones the repo and builds a new iteration of your software. Throughout this process, the software is typically reviewed by automated tests. + +- **Publish:** + + After the build is completed, either a Docker image is built and published to a Docker registry or a catalog template is published. + +- **Deploy:** + + After the artifacts are published, you would release your application so users could start using the updated product. \ No newline at end of file diff --git a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/config/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/config/_index.md new file mode 100644 index 00000000000..9bffc8eb1ee --- /dev/null +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/config/_index.md @@ -0,0 +1,658 @@ +--- +title: Pipeline Configuration Reference +weight: 1 +--- + +In this section, you'll learn how to configure pipelines. + +- [Step Types](#step-types) +- [Step Type: Run Script](#step-type-run-script) +- [Step Type: Build and Publish Images](#step-type-build-and-publish-images) +- [Step Type: Publish Catalog Template](#step-type-publish-catalog-template) +- [Step Type: Deploy YAML](#step-type-deploy-yaml) +- [Step Type: Deploy Catalog App](#step-type-deploy-catalog-app) +- [Notifications](#notifications) +- [Timeouts](#timeouts) +- [Triggers and Trigger Rules](#triggers-and-trigger-rules) +- [Environment Variables](#environment-variables) +- [Secrets](#secrets) +- [Pipeline Variable Substitution Reference](#pipeline-variable-substitution-reference) +- [Global Pipeline Execution Settings](#global-pipeline-execution-settings) + - [Executor Quota](#executor-quota) + - [Resource Quota for Executors](#resource-quota-for-executors) + - [Custom CA](#custom-ca) +- [Persistent Data for Pipeline Components](#persistent-data-for-pipeline-components) +- [Example rancher-pipeline.yml](#example-rancher-pipeline-yml) + +# Step Types + +Within each stage, you can add as many steps as you'd like. When there are multiple steps in one stage, they run concurrently. + +Step types include: + +- [Run Script](#step-type-run-script) +- [Build and Publish Images](#step-type-build-and-publish-images) +- [Publish Catalog Template](#step-type-publish-catalog-template) +- [Deploy YAML](#step-type-deploy-yaml) +- [Deploy Catalog App](#step-type-deploy-catalog-app) + + + +### Configuring Steps By UI + +If you haven't added any stages, click **Configure pipeline for this branch** to configure the pipeline through the UI. + +1. Add stages to your pipeline execution by clicking **Add Stage**. + + 1. Enter a **Name** for each stage of your pipeline. + 1. For each stage, you can configure [trigger rules](#triggers-and-trigger-rules) by clicking on **Show Advanced Options**. Note: this can always be updated at a later time. + +1. After you've created a stage, start [adding steps](#step-types) by clicking **Add a Step**. You can add multiple steps to each stage. + +### Configuring Steps by YAML + +For each stage, you can add multiple steps. Read more about each [step type](#step-types) and the advanced options to get all the details on how to configure the YAML. This is only a small example of how to have multiple stages with a singular step in each stage. + +```yaml +# example +stages: + - name: Build something + # Conditions for stages + when: + branch: master + event: [ push, pull_request ] + # Multiple steps run concurrently + steps: + - runScriptConfig: + image: busybox + shellScript: date -R + - name: Publish my image + steps: + - publishImageConfig: + dockerfilePath: ./Dockerfile + buildContext: . + tag: rancher/rancher:v2.0.0 + # Optionally push to remote registry + pushRemote: true + registry: reg.example.com +``` +# Step Type: Run Script + +The **Run Script** step executes arbitrary commands in the workspace inside a specified container. You can use it to build, test and do more, given whatever utilities the base image provides. For your convenience, you can use variables to refer to metadata of a pipeline execution. Please refer to the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) for the list of available variables. + +### Configuring Script by UI + +1. From the **Step Type** drop-down, choose **Run Script** and fill in the form. + +1. Click **Add**. + +### Configuring Script by YAML +```yaml +# example +stages: +- name: Build something + steps: + - runScriptConfig: + image: golang + shellScript: go build +``` +# Step Type: Build and Publish Images + +_Available as of Rancher v2.1.0_ + +The **Build and Publish Image** step builds and publishes a Docker image. This process requires a Dockerfile in your source code's repository to complete successfully. + +The option to publish an image to an insecure registry is not exposed in the UI, but you can specify an environment variable in the YAML that allows you to publish an image insecurely. + +### Configuring Building and Publishing Images by UI +1. From the **Step Type** drop-down, choose **Build and Publish**. + +1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**. + + Field | Description | + ---------|----------| + Dockerfile Path | The relative path to the Dockerfile in the source code repo. By default, this path is `./Dockerfile`, which assumes the Dockerfile is in the root directory. You can set it to other paths in different use cases (`./path/to/myDockerfile` for example). | + Image Name | The image name in `name:tag` format. The registry address is not required. For example, to build `example.com/repo/my-image:dev`, enter `repo/my-image:dev`. | + Push image to remote repository | An option to set the registry that publishes the image that's built. To use this option, enable it and choose a registry from the drop-down. If this option is disabled, the image is pushed to the internal registry. | + Build Context

(**Show advanced options**)| By default, the root directory of the source code (`.`). For more details, see the Docker [build command documentation](https://docs.docker.com/engine/reference/commandline/build/). + +### Configuring Building and Publishing Images by YAML + +You can use specific arguments for Docker daemon and the build. They are not exposed in the UI, but they are available in pipeline YAML format, as indicated in the example below. Available environment variables include: + +Variable Name | Description +------------------------|------------------------------------------------------------ +PLUGIN_DRY_RUN | Disable docker push +PLUGIN_DEBUG | Docker daemon executes in debug mode +PLUGIN_MIRROR | Docker daemon registry mirror +PLUGIN_INSECURE | Docker daemon allows insecure registries +PLUGIN_BUILD_ARGS | Docker build args, a comma separated list + +
+ +```yaml +# This example shows an environment variable being used +# in the Publish Image step. This variable allows you to +# publish an image to an insecure registry: + +stages: +- name: Publish Image + steps: + - publishImageConfig: + dockerfilePath: ./Dockerfile + buildContext: . + tag: repo/app:v1 + pushRemote: true + registry: example.com + env: + PLUGIN_INSECURE: "true" +``` + +# Step Type: Publish Catalog Template + +_Available as of v2.2.0_ + +The **Publish Catalog Template** step publishes a version of a catalog app template (i.e. Helm chart) to a [git hosted chart repository]({{}}/rancher/v2.x/en/catalog/custom/). It generates a git commit and pushes it to your chart repository. This process requires a chart folder in your source code's repository and a pre-configured secret in the dedicated pipeline namespace to complete successfully. Any variables in the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) is supported for any file in the chart folder. + +### Configuring Publishing a Catalog Template by UI + +1. From the **Step Type** drop-down, choose **Publish Catalog Template**. + +1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**. + + Field | Description | + ---------|----------| + Chart Folder | The relative path to the chart folder in the source code repo, where the `Chart.yaml` file is located. | + Catalog Template Name | The name of the template. For example, wordpress. | + Catalog Template Version | The version of the template you want to publish, it should be consistent with the version defined in the `Chart.yaml` file. | + Protocol | You can choose to publish via HTTP(S) or SSH protocol. | + Secret | The secret that stores your Git credentials. You need to create a secret in dedicated pipeline namespace in the project before adding this step. If you use HTTP(S) protocol, store Git username and password in `USERNAME` and `PASSWORD` key of the secret. If you use SSH protocol, store Git deploy key in `DEPLOY_KEY` key of the secret. After the secret is created, select it in this option. | + Git URL | The Git URL of the chart repository that the template will be published to. | + Git Branch | The Git branch of the chart repository that the template will be published to. | + Author Name | The author name used in the commit message. | + Author Email | The author email used in the commit message. | + + +### Configuring Publishing a Catalog Template by YAML + +You can add **Publish Catalog Template** steps directly in the `.rancher-pipeline.yml` file. + +Under the `steps` section, add a step with `publishCatalogConfig`. You will provide the following information: + +* Path: The relative path to the chart folder in the source code repo, where the `Chart.yaml` file is located. +* CatalogTemplate: The name of the template. +* Version: The version of the template you want to publish, it should be consistent with the version defined in the `Chart.yaml` file. +* GitUrl: The git URL of the chart repository that the template will be published to. +* GitBranch: The git branch of the chart repository that the template will be published to. +* GitAuthor: The author name used in the commit message. +* GitEmail: The author email used in the commit message. +* Credentials: You should provide Git credentials by referencing secrets in dedicated pipeline namespace. If you publish via SSH protocol, inject your deploy key to the `DEPLOY_KEY` environment variable. If you publish via HTTP(S) protocol, inject your username and password to `USERNAME` and `PASSWORD` environment variables. + +```yaml +# example +stages: +- name: Publish Wordpress Template + steps: + - publishCatalogConfig: + path: ./charts/wordpress/latest + catalogTemplate: wordpress + version: ${CICD_GIT_TAG} + gitUrl: git@github.com:myrepo/charts.git + gitBranch: master + gitAuthor: example-user + gitEmail: user@example.com + envFrom: + - sourceName: publish-keys + sourceKey: DEPLOY_KEY +``` + +# Step Type: Deploy YAML + +This step deploys arbitrary Kubernetes resources to the project. This deployment requires a Kubernetes manifest file to be present in the source code repository. Pipeline variable substitution is supported in the manifest file. You can view an example file at [GitHub](https://github.com/rancher/pipeline-example-go/blob/master/deployment.yaml). Please refer to the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) for the list of available variables. + +### Configure Deploying YAML by UI + +1. From the **Step Type** drop-down, choose **Deploy YAML** and fill in the form. + +1. Enter the **YAML Path**, which is the path to the manifest file in the source code. + +1. Click **Add**. + +### Configure Deploying YAML by YAML + +```yaml +# example +stages: +- name: Deploy + steps: + - applyYamlConfig: + path: ./deployment.yaml +``` + +# Step Type :Deploy Catalog App + +_Available as of v2.2.0_ + +The **Deploy Catalog App** step deploys a catalog app in the project. It will install a new app if it is not present, or upgrade an existing one. + +### Configure Deploying Catalog App by UI + +1. From the **Step Type** drop-down, choose **Deploy Catalog App**. + +1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**. + + Field | Description | + ---------|----------| + Catalog | The catalog from which the app template will be used. | + Template Name | The name of the app template. For example, wordpress. | + Template Version | The version of the app template you want to deploy. | + Namespace | The target namespace where you want to deploy the app. | + App Name | The name of the app you want to deploy. | + Answers | Key-value pairs of answers used to deploy the app. | + + +### Configure Deploying Catalog App by YAML + +You can add **Deploy Catalog App** steps directly in the `.rancher-pipeline.yml` file. + +Under the `steps` section, add a step with `applyAppConfig`. You will provide the following information: + +* CatalogTemplate: The ID of the template. This can be found by clicking `Launch app` and selecting `View details` for the app. It is the last part of the URL. +* Version: The version of the template you want to deploy. +* Answers: Key-value pairs of answers used to deploy the app. +* Name: The name of the app you want to deploy. +* TargetNamespace: The target namespace where you want to deploy the app. + +```yaml +# example +stages: +- name: Deploy App + steps: + - applyAppConfig: + catalogTemplate: cattle-global-data:library-mysql + version: 0.3.8 + answers: + persistence.enabled: "false" + name: testmysql + targetNamespace: test +``` + +# Timeouts + +By default, each pipeline execution has a timeout of 60 minutes. If the pipeline execution cannot complete within its timeout period, the pipeline is aborted. + +### Configuring Timeouts by UI + +Enter a new value in the **Timeout** field. + +### Configuring Timeouts by YAML + +In the `timeout` section, enter the timeout value in minutes. + +```yaml +# example +stages: + - name: Build something + steps: + - runScriptConfig: + image: busybox + shellScript: ls +# timeout in minutes +timeout: 30 +``` + +# Notifications + +You can enable notifications to any [notifiers]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers/) based on the build status of a pipeline. Before enabling notifications, Rancher recommends [setting up notifiers]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers/#adding-notifiers) so it will be easy to add recipients immediately. + +### Configuring Notifications by UI + +_Available as of v2.2.0_ + +1. Within the **Notification** section, turn on notifications by clicking **Enable**. + +1. Select the conditions for the notification. You can select to get a notification for the following statuses: `Failed`, `Success`, `Changed`. For example, if you want to receive notifications when an execution fails, select **Failed**. + +1. If you don't have any existing [notifiers]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers), Rancher will provide a warning that no notifiers are set up and provide a link to be able to go to the notifiers page. Follow the [instructions]({{}}/rancher/v2.x/en/cluster-admin/tools/notifiers/#adding-notifiers) to add a notifier. If you already have notifiers, you can add them to the notification by clicking the **Add Recipient** button. + + > **Note:** Notifiers are configured at a cluster level and require a different level of permissions. + +1. For each recipient, select which notifier type from the dropdown. Based on the type of notifier, you can use the default recipient or override the recipient with a different one. For example, if you have a notifier for _Slack_, you can update which channel to send the notification to. You can add additional notifiers by clicking **Add Recipient**. + +### Configuring Notifications by YAML +_Available as of v2.2.0_ + +In the `notification` section, you will provide the following information: + +* **Recipients:** This will be the list of notifiers/recipients that will receive the notification. + * **Notifier:** The ID of the notifier. This can be found by finding the notifier and selecting **View in API** to get the ID. + * **Recipient:** Depending on the type of the notifier, the "default recipient" can be used or you can override this with a different recipient. For example, when configuring a slack notifier, you select a channel as your default recipient, but if you wanted to send notifications to a different channel, you can select a different recipient. +* **Condition:** Select which conditions of when you want the notification to be sent. +* **Message (Optional):** If you want to change the default notification message, you can edit this in the yaml. Note: This option is not available in the UI. + +```yaml +# Example +stages: + - name: Build something + steps: + - runScriptConfig: + image: busybox + shellScript: ls +notification: + recipients: + - # Recipient + recipient: "#mychannel" + # ID of Notifier + notifier: "c-wdcsr:n-c9pg7" + - recipient: "test@example.com" + notifier: "c-wdcsr:n-lkrhd" + # Select which statuses you want the notification to be sent + condition: ["Failed", "Success", "Changed"] + # Ability to override the default message (Optional) + message: "my-message" +``` + +# Triggers and Trigger Rules + +After you configure a pipeline, you can trigger it using different methods: + +- **Manually:** + + After you configure a pipeline, you can trigger a build using the latest CI definition from Rancher UI. When a pipeline execution is triggered, Rancher dynamically provisions a Kubernetes pod to run your CI tasks and then remove it upon completion. + +- **Automatically:** + + When you enable a repository for a pipeline, webhooks are automatically added to the version control system. When project users interact with the repo by pushing code, opening pull requests, or creating a tag, the version control system sends a webhook to Rancher Server, triggering a pipeline execution. + + To use this automation, webhook management permission is required for the repository. Therefore, when users authenticate and fetch their repositories, only those on which they have webhook management permission will be shown. + +Trigger rules can be created to have fine-grained control of pipeline executions in your pipeline configuration. Trigger rules come in two types: + +- **Run this when:** This type of rule starts the pipeline, stage, or step when a trigger explicitly occurs. + +- **Do Not Run this when:** This type of rule skips the pipeline, stage, or step when a trigger explicitly occurs. + +If all conditions evaluate to `true`, then the pipeline/stage/step is executed. Otherwise it is skipped. When a pipeline is skipped, none of the pipeline is executed. When a stage/step is skipped, it is considered successful and follow-up stages/steps continue to run. + +Wildcard character (`*`) expansion is supported in `branch` conditions. + +This section covers the following topics: + +- [Configuring pipeline triggers](#configuring-pipeline-triggers) +- [Configuring stage triggers](#configuring-stage-triggers) +- [Configuring step triggers](#configuring-step-triggers) +- [Configuring triggers by YAML](#configuring-triggers-by-yaml) + +### Configuring Pipeline Triggers + +1. From the **Global** view, navigate to the project that you want to configure a pipeline trigger rule. + +1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** + +1. From the repository for which you want to manage trigger rules, select the vertical **Ellipsis (...) > Edit Config**. + +1. Click on **Show Advanced Options**. + +1. In the **Trigger Rules** section, configure rules to run or skip the pipeline. + + 1. Click **Add Rule**. In the **Value** field, enter the name of the branch that triggers the pipeline. + + 1. **Optional:** Add more branches that trigger a build. + +1. Click **Done.** + +### Configuring Stage Triggers + +1. From the **Global** view, navigate to the project that you want to configure a stage trigger rule. + +1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** + +1. From the repository for which you want to manage trigger rules, select the vertical **Ellipsis (...) > Edit Config**. + +1. Find the **stage** that you want to manage trigger rules, click the **Edit** icon for that stage. + +1. Click **Show advanced options**. + +1. In the **Trigger Rules** section, configure rules to run or skip the stage. + + 1. Click **Add Rule**. + + 1. Choose the **Type** that triggers the stage and enter a value. + + | Type | Value | + | ------ | -------------------------------------------------------------------- | + | Branch | The name of the branch that triggers the stage. | + | Event | The type of event that triggers the stage. Values are: `Push`, `Pull Request`, `Tag` | + +1. Click **Save**. + +### Configuring Step Triggers + +1. From the **Global** view, navigate to the project that you want to configure a stage trigger rule. + +1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** + +1. From the repository for which you want to manage trigger rules, select the vertical **Ellipsis (...) > Edit Config**. + +1. Find the **step** that you want to manage trigger rules, click the **Edit** icon for that step. + +1. Click **Show advanced options**. + +1. In the **Trigger Rules** section, configure rules to run or skip the step. + + 1. Click **Add Rule**. + + 1. Choose the **Type** that triggers the step and enter a value. + + | Type | Value | + | ------ | -------------------------------------------------------------------- | + | Branch | The name of the branch that triggers the step. | + | Event | The type of event that triggers the step. Values are: `Push`, `Pull Request`, `Tag` | + +1. Click **Save**. + + +### Configuring Triggers by YAML + +```yaml +# example +stages: + - name: Build something + # Conditions for stages + when: + branch: master + event: [ push, pull_request ] + # Multiple steps run concurrently + steps: + - runScriptConfig: + image: busybox + shellScript: date -R + # Conditions for steps + when: + branch: [ master, dev ] + event: push +# branch conditions for the pipeline +branch: + include: [ master, feature/*] + exclude: [ dev ] +``` + +# Environment Variables + +When configuring a pipeline, certain [step types](#step-types) allow you to use environment variables to configure the step's script. + +### Configuring Environment Variables by UI + +1. From the **Global** view, navigate to the project that you want to configure pipelines. + +1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** + +1. From the pipeline for which you want to edit build triggers, select **Ellipsis (...) > Edit Config**. + +1. Within one of the stages, find the **step** that you want to add an environment variable for, click the **Edit** icon. + +1. Click **Show advanced options**. + +1. Click **Add Variable**, and then enter a key and value in the fields that appear. Add more variables if needed. + +1. Add your environment variable(s) into either the script or file. + +1. Click **Save**. + +### Configuring Environment Variables by YAML + +```yaml +# example +stages: + - name: Build something + steps: + - runScriptConfig: + image: busybox + shellScript: echo ${FIRST_KEY} && echo ${SECOND_KEY} + env: + FIRST_KEY: VALUE + SECOND_KEY: VALUE2 +``` + +# Secrets + +If you need to use security-sensitive information in your pipeline scripts (like a password), you can pass them in using Kubernetes [secrets]({{}}/rancher/v2.x/en/k8s-in-rancher/secrets/). + +### Prerequisite +Create a secret in the same project as your pipeline, or explicitly in the namespace where pipeline build pods run. +
+ +>**Note:** Secret injection is disabled on [pull request events](#triggers-and-trigger-rules). + +### Configuring Secrets by UI + +1. From the **Global** view, navigate to the project that you want to configure pipelines. + +1. Click **Resources > Pipelines.** In versions prior to v2.3.0, click **Workloads > Pipelines.** + +1. From the pipeline for which you want to edit build triggers, select **Ellipsis (...) > Edit Config**. + +1. Within one of the stages, find the **step** that you want to use a secret for, click the **Edit** icon. + +1. Click **Show advanced options**. + +1. Click **Add From Secret**. Select the secret file that you want to use. Then choose a key. Optionally, you can enter an alias for the key. + +1. Click **Save**. + +### Configuring Secrets by YAML + +```yaml +# example +stages: + - name: Build something + steps: + - runScriptConfig: + image: busybox + shellScript: echo ${ALIAS_ENV} + # environment variables from project secrets + envFrom: + - sourceName: my-secret + sourceKey: secret-key + targetKey: ALIAS_ENV +``` + +# Pipeline Variable Substitution Reference + +For your convenience, the following variables are available for your pipeline configuration scripts. During pipeline executions, these variables are replaced by metadata. You can reference them in the form of `${VAR_NAME}`. + +Variable Name | Description +------------------------|------------------------------------------------------------ +`CICD_GIT_REPO_NAME` | Repository name (Github organization omitted). +`CICD_GIT_URL` | URL of the Git repository. +`CICD_GIT_COMMIT` | Git commit ID being executed. +`CICD_GIT_BRANCH` | Git branch of this event. +`CICD_GIT_REF` | Git reference specification of this event. +`CICD_GIT_TAG` | Git tag name, set on tag event. +`CICD_EVENT` | Event that triggered the build (`push`, `pull_request` or `tag`). +`CICD_PIPELINE_ID` | Rancher ID for the pipeline. +`CICD_EXECUTION_SEQUENCE` | Build number of the pipeline. +`CICD_EXECUTION_ID` | Combination of `{CICD_PIPELINE_ID}-{CICD_EXECUTION_SEQUENCE}`. +`CICD_REGISTRY` | Address for the Docker registry for the previous publish image step, available in the Kubernetes manifest file of a `Deploy YAML` step. +`CICD_IMAGE` | Name of the image built from the previous publish image step, available in the Kubernetes manifest file of a `Deploy YAML` step. It does not contain the image tag.

[Example](https://github.com/rancher/pipeline-example-go/blob/master/deployment.yaml) + +# Global Pipeline Execution Settings + +After configuring a version control provider, there are several options that can be configured globally on how pipelines are executed in Rancher. These settings can be edited by selecting **Tools > Pipelines** in the navigation bar. In versions prior to v2.2.0, you can select **Resources > Pipelines**. + +- [Executor Quota](#executor-quota) +- [Resource Quota for Executors](#resource-quota-for-executors) +- [Custom CA](#custom-ca) + +### Executor Quota + +Select the maximum number of pipeline executors. The _executor quota_ decides how many builds can run simultaneously in the project. If the number of triggered builds exceeds the quota, subsequent builds will queue until a vacancy opens. By default, the quota is `2`. A value of `0` or less removes the quota limit. + +### Resource Quota for Executors + +_Available as of v2.2.0_ + +Configure compute resources for Jenkins agent containers. When a pipeline execution is triggered, a build pod is dynamically provisioned to run your CI tasks. Under the hood, A build pod consists of one Jenkins agent container and one container for each pipeline step. You can [manage compute resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for every containers in the pod. + +Edit the **Memory Reservation**, **Memory Limit**, **CPU Reservation** or **CPU Limit**, then click **Update Limit and Reservation**. + +To configure compute resources for pipeline-step containers: + +You can configure compute resources for pipeline-step containers in the `.rancher-pipeline.yml` file. + +In a [step type]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#step-types), you will provide the following information: + +* **CPU Reservation (`CpuRequest`)**: CPU request for the container of a pipeline step. +* **CPU Limit (`CpuLimit`)**: CPU limit for the container of a pipeline step. +* **Memory Reservation (`MemoryRequest`)**: Memory request for the container of a pipeline step. +* **Memory Limit (`MemoryLimit`)**: Memory limit for the container of a pipeline step. + +```yaml +# example +stages: + - name: Build something + steps: + - runScriptConfig: + image: busybox + shellScript: ls + cpuRequest: 100m + cpuLimit: 1 + memoryRequest:100Mi + memoryLimit: 1Gi + - publishImageConfig: + dockerfilePath: ./Dockerfile + buildContext: . + tag: repo/app:v1 + cpuRequest: 100m + cpuLimit: 1 + memoryRequest:100Mi + memoryLimit: 1Gi +``` + +>**Note:** Rancher sets default compute resources for pipeline steps except for `Build and Publish Images` and `Run Script` steps. You can override the default value by specifying compute resources in the same way. + +### Custom CA + +_Available as of v2.2.0_ + +If you want to use a version control provider with a certificate from a custom/internal CA root, the CA root certificates need to be added as part of the version control provider configuration in order for the pipeline build pods to succeed. + +1. Click **Edit cacerts**. + +1. Paste in the CA root certificates and click **Save cacerts**. + +**Result:** Pipelines can be used and new pods will be able to work with the self-signed-certificate. + +# Persistent Data for Pipeline Components + +The internal Docker registry and the Minio workloads use ephemeral volumes by default. This default storage works out-of-the-box and makes testing easy, but you lose the build images and build logs if the node running the Docker Registry or Minio fails. In most cases this is fine. If you want build images and logs to survive node failures, you can configure the Docker Registry and Minio to use persistent volumes. + +For details on setting up persistent storage for pipelines, refer to [this page.]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/storage) + +# Example rancher-pipeline.yml + +An example pipeline configuration file is on [this page.]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/example) \ No newline at end of file diff --git a/content/rancher/v2.x/en/project-admin/pipelines/docs-for-v2.0.x/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/docs-for-v2.0.x/_index.md similarity index 98% rename from content/rancher/v2.x/en/project-admin/pipelines/docs-for-v2.0.x/_index.md rename to content/rancher/v2.x/en/k8s-in-rancher/pipelines/docs-for-v2.0.x/_index.md index e408ef05817..412822f0f96 100644 --- a/content/rancher/v2.x/en/project-admin/pipelines/docs-for-v2.0.x/_index.md +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/docs-for-v2.0.x/_index.md @@ -3,6 +3,7 @@ title: v2.0.x Pipeline Documentation weight: 9000 aliases: - /rancher/v2.x/en/project-admin/tools/pipelines/docs-for-v2.0.x + - /rancher/v2.x/en/project-admin/pipelines/docs-for-v2.0.x --- >**Note:** This section describes the pipeline feature as implemented in Rancher v2.0.x. If you are using Rancher v2.1 or later, where pipelines have been significantly improved, please refer to the new documentation for [v2.1 or later]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/). diff --git a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md index 75619cd6702..c57dd613f2d 100644 --- a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md @@ -5,7 +5,7 @@ aliases: - /rancher/v2.x/en/tools/pipelines/quick-start-guide/ --- -Rancher ships with several example repositories that you can use to familiarize yourself with pipelines. We recommend configuring and testing the example repository that most resembles your environment before using pipelines with your own repositories in a production environment. Use this example repository as a sandbox for repo configuration, build demonstration, etc. Rancher includes example repositories for: +Rancher ships with several example repositories that you can use to familiarize yourself with pipelines. We recommend configuring and testing the example repository that most resembles your environment before using pipelines with your own repositories in a production environment. Use this example repository as a sandbox for repo configuration, build demonstration, etc. Rancher includes example repositories for: - Go - Maven @@ -13,7 +13,13 @@ Rancher ships with several example repositories that you can use to familiarize > **Note:** The example repositories are only available if you have not [configured a version control provider]({{}}/rancher/v2.x/en/project-admin/pipelines). -## Configure Repositories +To start using these example repositories, + +1. [Enable the example repositories](#1-enable-the-example-repositories) +2. [View the example pipeline](#2-view-the-example-pipeline) +3. [Run the example pipeline](#3-run-the-example-pipeline) + +### 1. Enable the Example Repositories By default, the example pipeline repositories are disabled. Enable one (or more) to test out the pipeline feature and see how it works. @@ -39,7 +45,7 @@ By default, the example pipeline repositories are disabled. Enable one (or more) - `jenkins` - `minio` -## View the Example Pipeline +### 2. View the Example Pipeline After enabling an example repository, review the pipeline to see how it is set up. @@ -51,7 +57,7 @@ After enabling an example repository, review the pipeline to see how it is set u * **Rancher UI**: Click on **Edit Config** to view the stages and steps of the pipeline. * **YAML**: Click on View/Edit YAML to view the `./rancher-pipeline.yml` file. -## Run the Example Pipeline +### 3. Run the Example Pipeline After enabling an example repository, run the pipeline to see how it works. @@ -65,6 +71,6 @@ After enabling an example repository, run the pipeline to see how it works. **Result:** The pipeline runs. You can see the results in the logs. -## What's Next? +### What's Next? -For detailed information about setting up your own pipeline for your repository, [configure a version control provider]({{}}/rancher/v2.x/en/project-admin/pipelines), [enable a repository](#configure-repositories) and finally [configure your pipeline]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#pipeline-configuration). +For detailed information about setting up your own pipeline for your repository, [configure a version control provider]({{}}/rancher/v2.x/en/project-admin/pipelines), [enable a repository](#configure-repositories) and finally [configure your pipeline]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#pipeline-configuration). \ No newline at end of file diff --git a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md index 82e10d93ddf..512c87af456 100644 --- a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example/_index.md @@ -7,7 +7,9 @@ aliases: Pipelines can be configured either through the UI or using a yaml file in the repository, i.e. `.rancher-pipeline.yml` or `.rancher-pipeline.yaml`. -In the [pipeline configuration docs]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#pipeline-configuration), we provide examples of each available feature within pipelines. Here is a full example `rancher-pipeline.yml` for those who want to jump right in. +In the [pipeline configuration reference]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/config), we provide examples of how to configure each feature using the Rancher UI or using YAML configuration. + +Below is a full example `rancher-pipeline.yml` for those who want to jump right in. ```yaml # example diff --git a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/storage/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/storage/_index.md new file mode 100644 index 00000000000..e3b911bc9a2 --- /dev/null +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/storage/_index.md @@ -0,0 +1,103 @@ +--- +title: Configuring Persistent Data for Pipeline Components +weight: 600 +--- + +The internal [Docker registry](#how-pipelines-work) and the [Minio](#how-pipelines-work) workloads use ephemeral volumes by default. This default storage works out-of-the-box and makes testing easy, but you lose the build images and build logs if the node running the Docker Registry or Minio fails. In most cases this is fine. If you want build images and logs to survive node failures, you can configure the Docker Registry and Minio to use persistent volumes. + +This section assumes that you understand how persistent storage works in Kubernetes. For more information, refer to the section on [how storage works.]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/how-storage-works/) + +>**Prerequisites (for both parts A and B):** +> +>[Persistent volumes]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/) must be available for the cluster. + +### A. Configuring Persistent Data for Docker Registry + +1. From the project that you're configuring a pipeline for, and click **Resources > Workloads.** In versions prior to v2.3.0, select the **Workloads** tab. + +1. Find the `docker-registry` workload and select **Ellipsis (...) > Edit**. + +1. Scroll to the **Volumes** section and expand it. Make one of the following selections from the **Add Volume** menu, which is near the bottom of the section: + + - **Add Volume > Add a new persistent volume (claim)** + - **Add Volume > Use an existing persistent volume (claim)** + +1. Complete the form that displays to choose a persistent volume for the internal Docker registry. +{{% tabs %}} +{{% tab "Add a new persistent volume" %}} +
+1. Enter a **Name** for the volume claim. + +1. Select a volume claim **Source**: + + - If you select **Use a Storage Class to provision a new persistent volume**, select a storage class and enter a **Capacity**. + + - If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down. +1. From the **Customize** section, choose the read/write access for the volume. + +1. Click **Define**. + +{{% /tab %}} + +{{% tab "Use an existing persistent volume" %}} +
+1. Enter a **Name** for the volume claim. + +1. Choose a **Persistent Volume Claim** from the drop-down. + +1. From the **Customize** section, choose the read/write access for the volume. + +1. Click **Define**. + +{{% /tab %}} + +{{% /tabs %}} + +1. From the **Mount Point** field, enter `/var/lib/registry`, which is the data storage path inside the Docker registry container. + +1. Click **Upgrade**. + +### B. Configuring Persistent Data for Minio + +1. From the project view, click **Resources > Workloads.** (In versions prior to v2.3.0, click the **Workloads** tab.) Find the `minio` workload and select **Ellipsis (...) > Edit**. + +1. Scroll to the **Volumes** section and expand it. Make one of the following selections from the **Add Volume** menu, which is near the bottom of the section: + + - **Add Volume > Add a new persistent volume (claim)** + - **Add Volume > Use an existing persistent volume (claim)** + +1. Complete the form that displays to choose a persistent volume for the internal Docker registry. +{{% tabs %}} + +{{% tab "Add a new persistent volume" %}} +
+1. Enter a **Name** for the volume claim. + +1. Select a volume claim **Source**: + + - If you select **Use a Storage Class to provision a new persistent volume**, select a storage class and enter a **Capacity**. + + - If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down. +1. From the **Customize** section, choose the read/write access for the volume. + +1. Click **Define**. + +{{% /tab %}} +{{% tab "Use an existing persistent volume" %}} +
+1. Enter a **Name** for the volume claim. + +1. Choose a **Persistent Volume Claim** from the drop-down. + +1. From the **Customize** section, choose the read/write access for the volume. + +1. Click **Define**. + +{{% /tab %}} +{{% /tabs %}} + +1. From the **Mount Point** field, enter `/data`, which is the data storage path inside the Minio container. + +1. Click **Upgrade**. + +**Result:** Persistent storage is configured for your pipeline components. diff --git a/content/rancher/v2.x/en/project-admin/pipelines/_index.md b/content/rancher/v2.x/en/project-admin/pipelines/_index.md index 521b01f16de..7eea9d66735 100644 --- a/content/rancher/v2.x/en/project-admin/pipelines/_index.md +++ b/content/rancher/v2.x/en/project-admin/pipelines/_index.md @@ -9,8 +9,6 @@ aliases: --- Using Rancher, you can integrate with a GitHub repository to setup a continuous integration (CI) pipeline. -To set up a pipeline, you'll first need to authorize Rancher using your GitHub settings. Directions are provided in the Rancher UI. After authorizing Rancher in GitHub, provide Rancher with a client ID and secret to authenticate. - After configuring Rancher and GitHub, you can deploy containers running Jenkins to automate a pipeline execution: - Build your application from code to image. @@ -19,346 +17,4 @@ After configuring Rancher and GitHub, you can deploy containers running Jenkins - Run unit tests. - Run regression tests. - - - - -A _pipeline_ is a software delivery process that is broken into different stages and steps. Setting up a pipeline can help developers deliver new software as quickly and efficiently as possible. Within Rancher, you can configure pipelines for each of your Rancher projects. - -Typically, pipeline stages include: - -- **Build:** - - Each time code is checked into your repository, the pipeline automatically clones the repo and builds a new iteration of your software. Throughout this process, the software is typically reviewed by automated tests. - -- **Publish:** - - After the build is completed, either a Docker image is built and published to a Docker registry or a catalog template is published. - -- **Deploy:** - - After the artifacts are published, you would release your application so users could start using the updated product. - -Only [administrators]({{}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/), [cluster owners or members]({{}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), or [project owners]({{}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) can [configure version control providers](#version-control-providers) and [manage global pipeline execution settings](#managing-global-pipeline-execution-settings). Project members can only configure [repositories]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#configuring-repositories) and [pipelines]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#pipeline-configuration). - - -> **Notes:** -> -> - Pipelines were improved in Rancher v2.1. Therefore, if you configured pipelines while using v2.0.x, you'll have to reconfigure them after upgrading to v2.1. -> - Still using v2.0.x? See the pipeline documentation for [previous versions]({{}}/rancher/v2.x/en/tools/pipelines/docs-for-v2.0.x). - -## Overview - -Rancher's pipeline provides a simple CI/CD experience. Use it to automatically checkout code, run builds or scripts, publish Docker images or catalog applications, and deploy the updated software to users. - -After enabling the ability to use pipelines in a project, you can configure multiple pipelines in each project. Each pipeline is unique and can be configured independently. - -A pipeline is configured off of a group of files that are checked into source code repositories. Users can configure their pipelines either through the Rancher UI or by adding a `.rancher-pipeline.yml` into the repository. - ->**Note:** Rancher's pipeline provides a simple CI/CD experience, but it does not offer the full power and flexibility of and is not a replacement of enterprise-grade Jenkins or other CI tools your team uses. - - -## How Pipelines Work - -When you configure a pipeline in one of your projects, a namespace specifically for the pipeline is automatically created. The following components are deployed to it: - - - **Jenkins:** - - The pipeline's build engine. Because project users do not directly interact with Jenkins, it's managed and locked. - - >**Note:** There is no option to use existing Jenkins deployments as the pipeline engine. - - - **Docker Registry:** - - Out-of-the-box, the default target for your build-publish step is an internal Docker Registry. However, you can make configurations to push to a remote registry instead. The internal Docker Registry is only accessible from cluster nodes and cannot be directly accessed by users. Images are not persisted beyond the lifetime of the pipeline and should only be used in pipeline runs. If you need to access your images outside of pipeline runs, please push to an external registry. - - - **Minio:** - - Minio storage is used to store the logs for pipeline executions. - - >**Note:** The managed Jenkins instance works statelessly, so don't worry about its data persistency. The Docker Registry and Minio instances use ephemeral volumes by default, which is fine for most use cases. If you want to make sure pipeline logs can survive node failures, you can configure persistent volumes for them, as described in [data persistency for pipeline components](#configuring-persistent-data-for-pipeline-components). - -## Pipeline Triggers - -After you configure a pipeline, you can trigger it using different methods: - - -- **Manually:** - - After you configure a pipeline, you can trigger a build using the latest CI definition from Rancher UI. When a pipeline execution is triggered, Rancher dynamically provisions a Kubernetes pod to run your CI tasks and then remove it upon completion. - -- **Automatically:** - - When you enable a repository for a pipeline, webhooks are automatically added to the version control system. When project users interact with the repo—push code, open pull requests, or create a tag—the version control system sends a webhook to Rancher Server, triggering a pipeline execution. - - To use this automation, webhook management permission is required for the repository. Therefore, when users authenticate and fetch their repositories, only those on which they have webhook management permission will be shown. - -## Version Control Providers - -Before you can start [configuring a pipeline]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/) for your repository, you must configure and authorize a version control provider. - -| Provider | Available as of | -| --- | --- | -| GitHub | v2.0.0 | -| GitLab | v2.1.0 | -| Bitbucket | v2.2.0 | - -Select your provider's tab below and follow the directions. - -{{% tabs %}} -{{% tab "GitHub" %}} -1. From the **Global** view, navigate to the project that you want to configure pipelines. - -1. Select **Tools > Pipelines** in the navigation bar. In versions prior to v2.2.0, you can select **Resources > Pipelines**. - -1. Follow the directions displayed to **Setup a Github application**. Rancher redirects you to Github to setup an OAuth App in Github. - -1. From GitHub, copy the **Client ID** and **Client Secret**. Paste them into Rancher. - -1. If you're using GitHub for enterprise, select **Use a private github enterprise installation**. Enter the host address of your GitHub installation. - -1. Click **Authenticate**. - -{{% /tab %}} -{{% tab "GitLab" %}} - -_Available as of v2.1.0_ - -1. From the **Global** view, navigate to the project that you want to configure pipelines. - -1. Select **Tools > Pipelines** in the navigation bar. In versions prior to v2.2.0, you can select **Resources > Pipelines**. - -1. Follow the directions displayed to **Setup a GitLab application**. Rancher redirects you to GitLab. - -1. From GitLab, copy the **Application ID** and **Secret**. Paste them into Rancher. - -1. If you're using GitLab for enterprise setup, select **Use a private gitlab enterprise installation**. Enter the host address of your GitLab installation. - -1. Click **Authenticate**. - ->**Note:** -> 1. Pipeline uses Gitlab [v4 API](https://docs.gitlab.com/ee/api/v3_to_v4.html) and the supported Gitlab version is 9.0+. -> 2. If you use GitLab 10.7+ and your Rancher setup is in a local network, enable the **Allow requests to the local network from hooks and services** option in GitLab admin settings. -{{% /tab %}} -{{% tab "Bitbucket Cloud" %}} - -_Available as of v2.2.0_ - -1. From the **Global** view, navigate to the project that you want to configure pipelines. - -1. Select **Tools > Pipelines** in the navigation bar. - -1. Choose the **Use public Bitbucket Cloud** option. - -1. Follow the directions displayed to **Setup a Bitbucket Cloud application**. Rancher redirects you to Bitbucket to setup an OAuth consumer in Bitbucket. - -1. From Bitbucket, copy the consumer **Key** and **Secret**. Paste them into Rancher. - -1. Click **Authenticate**. - -{{% /tab %}} -{{% tab "Bitbucket Server" %}} - -_Available as of v2.2.0_ - -1. From the **Global** view, navigate to the project that you want to configure pipelines. - -1. Select **Tools > Pipelines** in the navigation bar. - -1. Choose the **Use private Bitbucket Server setup** option. - -1. Follow the directions displayed to **Setup a Bitbucket Server application**. - -1. Enter the host address of your Bitbucket server installation. - -1. Click **Authenticate**. - ->**Note:** -> Bitbucket server needs to do SSL verification when sending webhooks to Rancher. Please ensure that Rancher server's certificate is trusted by the Bitbucket server. There are two options: -> -> 1. Setup Rancher server with a certificate from a trusted CA. -> 1. If you're using self-signed certificates, import Rancher server's certificate to the Bitbucket server. For instructions, see the Bitbucket server documentation for [configuring self-signed certificates](https://confluence.atlassian.com/bitbucketserver/if-you-use-self-signed-certificates-938028692.html). -> -{{% /tab %}} -{{% /tabs %}} - -**Result:** After the version control provider is authenticated, you will be automatically re-directed to start [configuring which repositories]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#configuring-repositories) that you want start using with a pipeline. Once a repository is enabled, you can start to [configure the pipeline]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#pipeline-configuration). - -## Managing Global Pipeline Execution Settings - -After configuring a version control provider, there are several options that can be configured globally on how [pipelines]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/) are executed in Rancher. - -1. From the **Global** view, navigate to the project that you want to configure pipelines. - -1. Select **Tools > Pipelines** in the navigation bar. In versions prior to v2.2.0, you can select **Resources > Pipelines**. - -1. Edit the different settings: - - {{% accordion id="executor-quota" label="Executor Quota" %}} - -Select the maximum number of pipeline executors. The _executor quota_ decides how many builds can run simultaneously in the project. If the number of triggered builds exceeds the quota, subsequent builds will queue until a vacancy opens. By default, the quota is `2`. A value of `0` or less removes the quota limit. - {{% /accordion %}} - - {{% accordion id="resource-quota" label="Resource Quota for Executors" %}} - -_Available as of v2.2.0_ - -Configure compute resources for Jenkins agent containers. When a pipeline execution is triggered, a build pod is dynamically provisioned to run your CI tasks. Under the hood, A build pod consists of one Jenkins agent container and one container for each pipeline step. You can [manage compute resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for every containers in the pod. - -Edit the **Memory Reservation**, **Memory Limit**, **CPU Reservation** or **CPU Limit**, then click **Update Limit and Reservation**. - -To configure compute resources for pipeline-step containers: -{{% tabs %}} -{{% tab "By YAML" %}} - -You can configure compute resources for pipeline-step containers in the `.rancher-pipeline.yml` file. - -In a [step type]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#step-types), you will provide the following information: - -* **CPU Reservation (`CpuRequest`)**: CPU request for the container of a pipeline step. -* **CPU Limit (`CpuLimit`)**: CPU limit for the container of a pipeline step. -* **Memory Reservation (`MemoryRequest`)**: Memory request for the container of a pipeline step. -* **Memory Limit (`MemoryLimit`)**: Memory limit for the container of a pipeline step. - -```yaml -# example -stages: - - name: Build something - steps: - - runScriptConfig: - image: busybox - shellScript: ls - cpuRequest: 100m - cpuLimit: 1 - memoryRequest:100Mi - memoryLimit: 1Gi - - publishImageConfig: - dockerfilePath: ./Dockerfile - buildContext: . - tag: repo/app:v1 - cpuRequest: 100m - cpuLimit: 1 - memoryRequest:100Mi - memoryLimit: 1Gi -``` - ->**Note:** Rancher sets default compute resources for pipeline steps except for `Build and Publish Images` and `Run Script` steps. You can override the default value by specifying compute resources in the same way. -{{% /tab %}} -{{% /tabs %}} - - {{% /accordion %}} - {{% accordion id="cacerts" label="Custom CA" %}} - -_Available as of v2.2.0_ - -If you want to use a version control provider with a certificate from a custom/internal CA root, the CA root certificates need to be added as part of the version control provider configuration in order for the pipeline build pods to succeed. - -1. Click **Edit cacerts**. - -1. Paste in the CA root certificates and click **Save cacerts**. - -**Result:** Pipelines can be used and new pods will be able to work with the self-signed-certificate. - - {{% /accordion %}} - -## Configuring Persistent Data for Pipeline Components - -The internal [Docker registry](#how-pipelines-work) and the [Minio](#how-pipelines-work) workloads use ephemeral volumes by default. This default storage works out-of-the-box and makes testing easy, but you lose the build images and build logs if the node running the Docker Registry or Minio fails. In most cases this is fine. If you want build images and logs to survive node failures, you can configure the Docker Registry and Minio to use persistent volumes. - ->**Prerequisites (for both parts A and B):** -> ->[Persistent volumes]({{}}/rancher/v2.x/en/k8s-in-rancher/volumes-and-storage/#persistent-volumes) must be available for the cluster. - -### A. Configuring Persistent Data for Docker Registry - -1. From the project that you're configuring a pipeline for, and click **Resources > Workloads.** In versions prior to v2.3.0, select the **Workloads** tab. - -1. Find the `docker-registry` workload and select **Ellipsis (...) > Edit**. - -1. Scroll to the **Volumes** section and expand it. Make one of the following selections from the **Add Volume** menu, which is near the bottom of the section: - - - **Add Volume > Add a new persistent volume (claim)** - - **Add Volume > Use an existing persistent volume (claim)** - -1. Complete the form that displays to choose a persistent volume for the internal Docker registry. -{{% tabs %}} - -{{% tab "Add a new persistent volume" %}} -
-1. Enter a **Name** for the volume claim. - -1. Select a volume claim **Source**: - - - If you select **Use a Storage Class to provision a new persistent volume**, select a [Storage Class]({{}}/rancher/v2.x/en/k8s-in-rancher/volumes-and-storage/#storage-classes) and enter a **Capacity**. - - - If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down. -1. From the **Customize** section, choose the read/write access for the volume. - -1. Click **Define**. - -{{% /tab %}} - -{{% tab "Use an existing persistent volume" %}} -
-1. Enter a **Name** for the volume claim. - -1. Choose a **Persistent Volume Claim** from the drop-down. - -1. From the **Customize** section, choose the read/write access for the volume. - -1. Click **Define**. - -{{% /tab %}} - -{{% /tabs %}} - -1. From the **Mount Point** field, enter `/var/lib/registry`, which is the data storage path inside the Docker registry container. - -1. Click **Upgrade**. - -### B. Configuring Persistent Data for Minio - -1. From the project view, click **Resources > Workloads.** (In versions prior to v2.3.0, click the **Workloads** tab.) Find the `minio` workload and select **Ellipsis (...) > Edit**. - -1. Scroll to the **Volumes** section and expand it. Make one of the following selections from the **Add Volume** menu, which is near the bottom of the section: - - - **Add Volume > Add a new persistent volume (claim)** - - **Add Volume > Use an existing persistent volume (claim)** - -1. Complete the form that displays to choose a persistent volume for the internal Docker registry. -{{% tabs %}} - -{{% tab "Add a new persistent volume" %}} -
-1. Enter a **Name** for the volume claim. - -1. Select a volume claim **Source**: - - - If you select **Use a Storage Class to provision a new persistent volume**, select a [Storage Class]({{}}/rancher/v2.x/en/k8s-in-rancher/volumes-and-storage/#storage-classes) and enter a **Capacity**. - - - If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down. -1. From the **Customize** section, choose the read/write access for the volume. - -1. Click **Define**. - -{{% /tab %}} - -{{% tab "Use an existing persistent volume" %}} -
-1. Enter a **Name** for the volume claim. - -1. Choose a **Persistent Volume Claim** from the drop-down. - -1. From the **Customize** section, choose the read/write access for the volume. - -1. Click **Define**. - -{{% /tab %}} - -{{% /tabs %}} - -1. From the **Mount Point** field, enter `/data`, which is the data storage path inside the Minio container. - -1. Click **Upgrade**. - -**Result:** Persistent storage is configured for your pipeline components. +For details, refer to the [pipelines]({{}}/rancher/v2.x/en/k8s-in-rancher/pipelines) section. \ No newline at end of file