Compare commits

..
Author SHA1 Message Date
github-actions[bot] 49dfcc300b deploy: fb49f4b953 2024-05-30 18:40:58 +00:00
github-actions[bot] a26f1c8134 deploy: 3c0a23c564 2024-05-30 18:00:32 +00:00
github-actions[bot] 00fb8f4d46 deploy: a90c5f94a1 2024-05-30 16:40:08 +00:00
github-actions[bot] 11eb997458 deploy: 6351e1a4cb 2024-05-30 16:35:20 +00:00
github-actions[bot] 90a697e952 deploy: 8a26132d03 2024-05-29 21:39:49 +00:00
github-actions[bot] 7ac18f2764 deploy: 510c47827c 2024-05-28 00:23:50 +00:00
github-actions[bot] b5aba687a0 deploy: b69f371be3 2024-05-24 16:08:22 +00:00
github-actions[bot] ccb9012030 deploy: 2611f98cbb 2024-05-23 19:23:07 +00:00
github-actions[bot] c90220ad18 deploy: 7245df6e2b 2024-05-23 19:17:39 +00:00
github-actions[bot] d74f6a4313 deploy: 7b4e17c4bb 2024-05-23 18:40:14 +00:00
github-actions[bot] 9eb1c27bdf deploy: ed074fe196 2024-05-23 14:53:36 +00:00
github-actions[bot] e5352ade2a deploy: 18e5625b0c 2024-05-21 20:07:19 +00:00
github-actions[bot] 88d0559b78 deploy: 4051ea3814 2024-05-21 17:07:37 +00:00
github-actions[bot] 17192e5e14 deploy: d4796a1ae8 2024-05-21 16:09:43 +00:00
github-actions[bot] a823bf6bdf deploy: c9e7c6bced 2024-05-21 15:30:12 +00:00
github-actions[bot] 6bc5479c84 deploy: 1f2cc96089 2024-05-20 18:07:54 +00:00
github-actions[bot] dd301a6626 deploy: fcd6037152 2024-05-20 18:07:08 +00:00
github-actions[bot] 2ee8b0e161 deploy: 2d437d065d 2024-05-17 21:49:31 +00:00
github-actions[bot] 1370256956 deploy: 5b9f233c98 2024-05-17 04:57:32 +00:00
github-actions[bot] e82090a9d6 deploy: a9c989f0eb 2024-05-17 04:56:39 +00:00
github-actions[bot] 78398f9385 deploy: 777b6f45a0 2024-05-17 04:56:03 +00:00
github-actions[bot] 76e5068f6f deploy: c6356bcaa0 2024-05-17 04:55:14 +00:00
github-actions[bot] d59115ce3e deploy: 16968b839b 2024-05-16 15:41:21 +00:00
github-actions[bot] 09a9f449c4 deploy: 5bf5c87b3f 2024-05-15 16:46:15 +00:00
github-actions[bot] 72bed2379f deploy: 7827b6e79f 2024-05-14 17:19:51 +00:00
github-actions[bot] 45c4577837 deploy: 97a321a759 2024-05-09 17:41:47 +00:00
github-actions[bot] 90a2bc5b84 deploy: 6ae22a0431 2024-05-09 16:11:01 +00:00
github-actions[bot] 3e309024ac deploy: d02b44a237 2024-05-09 00:24:29 +00:00
github-actions[bot] 3b5acf9f29 deploy: 932544b627 2024-05-09 00:19:27 +00:00
github-actions[bot] 30fcd187e8 deploy: cbb2a6db85 2024-05-08 23:57:42 +00:00
github-actions[bot] 1bd4b48ba0 deploy: 8e3985e633 2024-05-08 23:54:54 +00:00
github-actions[bot] d23fdb5d40 deploy: 6b13f7ba92 2024-05-08 23:52:20 +00:00
github-actions[bot] 560f1fac28 deploy: 3e7357a89d 2024-05-08 23:51:01 +00:00
github-actions[bot] 09b3b46ad1 deploy: c129dbbfcf 2024-05-08 20:22:27 +00:00
github-actions[bot] 220a075d1d deploy: 7d3d40ae83 2024-05-07 15:24:12 +00:00
github-actions[bot] 3f8e1d9efd deploy: 3c0b963f9f 2024-05-06 21:18:17 +00:00
github-actions[bot] b9cf07be4c deploy: 31c279bd42 2024-05-06 19:36:27 +00:00
github-actions[bot] e9cbc6c227 deploy: ae896ecbba 2024-05-06 18:29:06 +00:00
github-actions[bot] 8fbe7602f0 deploy: 84d4214bb2 2024-05-06 17:12:41 +00:00
github-actions[bot] b6d1855dfb deploy: 9ea762eb4e 2024-05-01 22:03:13 +00:00
github-actions[bot] c8cd1f3715 deploy: 32d05e9489 2024-05-01 17:33:57 +00:00
github-actions[bot] 146af67d15 deploy: c735cf2402 2024-04-30 21:59:04 +00:00
github-actions[bot] 0b603602ad deploy: 54dc6b187b 2024-04-30 21:44:46 +00:00
github-actions[bot] 71e561d951 deploy: 2980926dd8 2024-04-25 16:31:50 +00:00
github-actions[bot] 47d855f47b deploy: f83cb0297f 2024-04-16 18:50:40 +00:00
github-actions[bot] ce77f35fb1 deploy: cef7e95970 2024-04-15 19:25:17 +00:00
github-actions[bot] 25bc57ad6e deploy: b1aff707a9 2024-04-15 18:39:47 +00:00
github-actions[bot] 60ea49dffb deploy: 489b54c333 2024-04-15 15:53:42 +00:00
github-actions[bot] d1524ce692 deploy: 957bd368a7 2024-04-15 13:28:49 +00:00
github-actions[bot] b98aa8d16f deploy: 1ac5230a1c 2024-04-11 15:30:17 +00:00
github-actions[bot] c21b7c9872 deploy: 2b0a778a7d 2024-04-10 21:45:14 +00:00
github-actions[bot] a421976111 deploy: 4f043afbb7 2024-04-10 21:41:57 +00:00
github-actions[bot] 742d28fd35 deploy: b14d053624 2024-04-10 21:16:24 +00:00
github-actions[bot] 6b08856554 deploy: 209f133f04 2024-04-10 21:15:05 +00:00
github-actions[bot] 42f0ebe8c2 deploy: b3a2a2ac48 2024-04-10 21:13:11 +00:00
github-actions[bot] 49d6ef55fd deploy: 9a59743908 2024-04-10 16:37:39 +00:00
github-actions[bot] ed6fe561aa deploy: f76c9e5790 2024-04-10 15:00:04 +00:00
github-actions[bot] e34534fa26 deploy: 00ca7f6630 2024-04-10 14:58:54 +00:00
github-actions[bot] 8942f82ef9 deploy: eda6da5c7f 2024-04-09 21:36:11 +00:00
github-actions[bot] 40c5e16c26 deploy: 6ccd0b7b6c 2024-04-09 21:33:51 +00:00
github-actions[bot] 8701d7a775 deploy: 0f8d17de31 2024-04-09 18:44:07 +00:00
github-actions[bot] 7d51649420 deploy: fcff8576f7 2024-04-09 16:42:43 +00:00
github-actions[bot] 54c301378e deploy: 82fdb5d185 2024-04-09 15:26:57 +00:00
github-actions[bot] 0bb902af4d deploy: d60a015493 2024-04-08 20:55:20 +00:00
github-actions[bot] 9a5de6a1ed deploy: 6f4861fc77 2024-04-08 18:48:43 +00:00
github-actions[bot] bb4a88d044 deploy: 51b462f97d 2024-04-08 18:11:50 +00:00
github-actions[bot] faea5178ac deploy: aff1de294b 2024-04-08 16:07:48 +00:00
github-actions[bot] badfff39f5 deploy: a8470170cc 2024-04-04 16:28:31 +00:00
github-actions[bot] 6dafb36674 deploy: 7ef80ffcca 2024-04-02 14:49:33 +00:00
github-actions[bot] fd63211d15 deploy: 78acd021f2 2024-04-01 21:50:35 +00:00
github-actions[bot] 497636bd23 deploy: 2cb7e383fa 2024-03-29 20:46:35 +00:00
github-actions[bot] a2c30477e7 deploy: 7b3067229e 2024-03-29 17:20:18 +00:00
github-actions[bot] d788922739 deploy: 79bef2c1ff 2024-03-29 15:43:16 +00:00
github-actions[bot] fcfa7b441e deploy: 785db2b776 2024-03-29 15:12:02 +00:00
github-actions[bot] 18d407af15 deploy: 5330e6e002 2024-03-29 00:22:26 +00:00
github-actions[bot] 28821109d0 deploy: fd56e5239a 2024-03-29 00:15:18 +00:00
github-actions[bot] 77eaea9b5a deploy: 89d3efd25f 2024-03-28 23:24:31 +00:00
github-actions[bot] 6b6d9b5bd7 deploy: 3fb94fe646 2024-03-28 23:19:22 +00:00
github-actions[bot] ce9bc30a60 deploy: cd940d28c5 2024-03-28 20:14:52 +00:00
github-actions[bot] ea75d257f0 deploy: 48f4b33402 2024-03-28 20:14:04 +00:00
github-actions[bot] f0a26f781a deploy: 29c8fe37f9 2024-03-28 20:13:14 +00:00
github-actions[bot] a098d1a96a deploy: c1d73dd091 2024-03-28 16:46:53 +00:00
github-actions[bot] 7888dd1ed2 deploy: 3c043cac1e 2024-03-26 22:06:48 +00:00
github-actions[bot] 01b85f346b deploy: 0a7c7e7230 2024-03-26 14:37:08 +00:00
github-actions[bot] bac6fc0ce4 deploy: c256118aa9 2024-03-25 22:28:53 +00:00
github-actions[bot] 1a3b6f0ef3 deploy: cb9ac7370c 2024-03-25 18:45:03 +00:00
github-actions[bot] f307b6356f deploy: ab8df7f920 2024-03-21 16:18:13 +00:00
github-actions[bot] 05599162a3 deploy: bc33662b1f 2024-03-21 16:04:00 +00:00
github-actions[bot] 1d4091deb2 deploy: 51beaa2aeb 2024-03-20 20:02:29 +00:00
github-actions[bot] 2e1d4579bd deploy: d6a5c9cb78 2024-03-19 18:51:31 +00:00
github-actions[bot] deece65444 deploy: 558e533dd1 2024-03-15 01:37:53 +00:00
github-actions[bot] 8304cbb6cb deploy: 31cdce4a59 2024-03-14 18:34:02 +00:00
github-actions[bot] a663246f23 deploy: 4db4014a20 2024-03-11 21:51:21 +00:00
github-actions[bot] 654c00a015 deploy: ef19e6525d 2024-03-11 14:39:37 +00:00
github-actions[bot] 6394957585 deploy: 1f6a00bef3 2024-03-08 21:19:05 +00:00
github-actions[bot] f8c18eac6f deploy: f37b7655f9 2024-03-08 19:13:01 +00:00
github-actions[bot] 1bd2731aa0 deploy: dfcb3b7fde 2024-03-07 20:49:52 +00:00
github-actions[bot] 6e2f09e807 deploy: 0e76fd2d59 2024-03-07 17:19:07 +00:00
github-actions[bot] 67c3b31a2b deploy: 5664965fa1 2024-03-07 16:43:07 +00:00
github-actions[bot] 18f5d2083e deploy: 224bc8168b 2024-03-07 15:52:00 +00:00
github-actions[bot] d29fbbc32a deploy: 8e216e1700 2024-03-06 21:16:08 +00:00
github-actions[bot] a5bf5d006b deploy: 6b0b653e7c 2024-03-06 21:14:20 +00:00
github-actions[bot] 3ff67ea51e deploy: bbbb18eede 2024-03-05 00:48:14 +00:00
github-actions[bot] 4e3c99c356 deploy: 485b3c0c5d 2024-03-04 18:16:15 +00:00
github-actions[bot] 21bdda51be deploy: 18b43d1c9c 2024-03-04 17:54:24 +00:00
github-actions[bot] c6fbab5d4d deploy: 0563bcabc1 2024-03-01 23:57:34 +00:00
github-actions[bot] adfc609772 deploy: 87f7929549 2024-03-01 23:56:55 +00:00
github-actions[bot] beaf59c28c deploy: b0b6b185b9 2024-03-01 20:28:47 +00:00
github-actions[bot] b408ee7ddc deploy: b78a5cb897 2024-02-29 20:44:48 +00:00
github-actions[bot] 001ef63683 deploy: de7cab1891 2024-02-28 21:23:15 +00:00
github-actions[bot] b18be25682 deploy: 9ad451f11f 2024-02-27 17:57:34 +00:00
github-actions[bot] a8775919be deploy: e332ff4ffa 2024-02-27 00:59:28 +00:00
github-actions[bot] fe7ece88c2 deploy: 0ab94d8800 2024-02-23 23:42:33 +00:00
github-actions[bot] 3a485a60dd deploy: 9a0df1f1ba 2024-02-23 23:06:54 +00:00
github-actions[bot] 22c91e7685 deploy: b187a46a75 2024-02-23 21:01:33 +00:00
github-actions[bot] f2fef459d7 deploy: fa11eeda9f 2024-02-21 21:17:13 +00:00
github-actions[bot] dd53753230 deploy: afa972e2b2 2024-02-21 21:07:24 +00:00
github-actions[bot] 7e7db54808 deploy: d56729e94a 2024-02-21 18:48:16 +00:00
github-actions[bot] ee4bd490df deploy: 17af76d76f 2024-02-21 17:30:06 +00:00
github-actions[bot] 80392f037b deploy: d7dae21ca7 2024-02-20 21:52:35 +00:00
github-actions[bot] af1ccbab32 deploy: d9ba7f2e95 2024-02-20 19:43:49 +00:00
github-actions[bot] 460a3a37cb deploy: 513cc5c340 2024-02-16 16:25:16 +00:00
github-actions[bot] 1a1b47b4ed deploy: 3a6b7e866a 2024-02-13 21:33:04 +00:00
github-actions[bot] 9e116d475d deploy: 44ac9a470a 2024-02-13 19:35:55 +00:00
github-actions[bot] eca22b2392 deploy: cab46bd291 2024-02-13 19:29:23 +00:00
github-actions[bot] ce9cc05628 deploy: 5fd5c7f9db 2024-02-13 18:16:10 +00:00
github-actions[bot] 435bb33374 deploy: 74f23ff609 2024-02-13 18:10:05 +00:00
github-actions[bot] e15c8a3e18 deploy: 3d10005273 2024-02-12 19:45:50 +00:00
github-actions[bot] 3be627bf86 deploy: 6fdd90be51 2024-02-12 19:43:54 +00:00
github-actions[bot] 3ea4fa8f4b deploy: 81637f366a 2024-02-12 17:33:36 +00:00
github-actions[bot] 307d813218 deploy: 2e142301f6 2024-02-12 16:33:05 +00:00
github-actions[bot] 518d931eab deploy: c5a198f142 2024-02-09 19:52:50 +00:00
github-actions[bot] 0a8eda9d24 deploy: 4a9e9d0d0c 2024-02-08 18:29:57 +00:00
github-actions[bot] 63d1f472b9 deploy: 22ed868912 2024-02-08 18:29:20 +00:00
github-actions[bot] 38c28959a5 deploy: ddabfef05e 2024-02-08 18:27:15 +00:00
github-actions[bot] 1681745981 deploy: adf6d83398 2024-02-08 18:15:49 +00:00
github-actions[bot] 9501ee46ff deploy: 9012432eb8 2024-02-08 01:14:06 +00:00
github-actions[bot] 3146450d0f deploy: c5bd3a9126 2024-02-07 22:38:28 +00:00
github-actions[bot] 896b564c10 deploy: 00d9654e47 2024-02-06 19:34:24 +00:00
github-actions[bot] 8a659f0c14 deploy: ed88739bef 2024-02-06 16:45:30 +00:00
github-actions[bot] bf6df4593c deploy: 00c791500f 2024-02-05 20:26:11 +00:00
github-actions[bot] 7f8fe300bb deploy: c6a11eaeab 2024-02-02 15:35:32 +00:00
github-actions[bot] c53216920b deploy: 395d484bc7 2024-02-01 21:19:42 +00:00
github-actions[bot] 7ed831accf deploy: a3d005272b 2024-01-29 21:23:32 +00:00
github-actions[bot] 8940cc71cc deploy: 0b12ffe2ee 2024-01-29 18:36:14 +00:00
github-actions[bot] fa3a9406d6 deploy: cdc3587166 2024-01-26 19:06:35 +00:00
github-actions[bot] 0e88260a77 deploy: 1621ea5b5d 2024-01-26 19:00:25 +00:00
github-actions[bot] 0a143585ed deploy: 76b61df25b 2024-01-26 18:33:04 +00:00
github-actions[bot] 9c9fc28d80 deploy: 34dda87c71 2024-01-26 17:52:20 +00:00
github-actions[bot] 36228da753 deploy: 5f6b134ddd 2024-01-25 21:13:15 +00:00
github-actions[bot] 3ed4e23888 deploy: 03f245ac3b 2024-01-24 00:14:25 +00:00
github-actions[bot] 04a68403e3 deploy: fe366979ff 2024-01-23 00:24:08 +00:00
github-actions[bot] 1b0da46030 deploy: eceb869b73 2024-01-22 23:01:32 +00:00
github-actions[bot] 545c98d739 deploy: 6cddcfd3d7 2024-01-22 21:14:06 +00:00
github-actions[bot] a2475f64b9 deploy: 8699d8fac7 2024-01-22 21:10:11 +00:00
github-actions[bot] 15ed904f18 deploy: 825ff74299 2024-01-22 18:08:51 +00:00
github-actions[bot] 508cf47604 deploy: 5f8f405efa 2024-01-19 23:40:57 +00:00
github-actions[bot] 0c618c792f deploy: 412eafecf1 2024-01-19 23:15:25 +00:00
github-actions[bot] 04383cef84 deploy: bf13d30c39 2024-01-19 22:49:47 +00:00
github-actions[bot] 9c1ce70a06 deploy: 293cb54f6a 2024-01-19 22:27:52 +00:00
github-actions[bot] 8005cc4a7a deploy: 83bf367f00 2024-01-18 21:14:53 +00:00
github-actions[bot] c4a87bf11f deploy: 2302e37376 2024-01-17 18:54:20 +00:00
github-actions[bot] 4d05d97f2e deploy: 95604bb338 2024-01-16 19:58:58 +00:00
github-actions[bot] d0b888d607 deploy: 13219aad0c 2024-01-16 16:25:21 +00:00
github-actions[bot] 4faadecc61 deploy: 13daace65b 2024-01-13 00:58:02 +00:00
github-actions[bot] 0a6b5848e5 deploy: 3a4b7e73c0 2024-01-11 18:09:53 +00:00
github-actions[bot] 212dad17b4 deploy: 42b352e25d 2024-01-10 23:51:25 +00:00
github-actions[bot] 4f95209223 deploy: 7bdc1c8d2a 2024-01-09 22:38:12 +00:00
github-actions[bot] c1c451a688 deploy: fd62929c65 2024-01-09 20:27:06 +00:00
github-actions[bot] be94c45a34 deploy: 02d48c12c8 2024-01-09 20:01:49 +00:00
github-actions[bot] 5a3deafa82 deploy: f0ef55014c 2024-01-09 18:26:17 +00:00
github-actions[bot] 5ae144a6d6 deploy: f47734f15b 2024-01-09 18:22:14 +00:00
github-actions[bot] ccc6e529e0 deploy: 09183ca7cd 2024-01-08 22:42:31 +00:00
github-actions[bot] 159f916c43 deploy: 1d9047d714 2024-01-02 23:54:34 +00:00
github-actions[bot] 6b29b31aa0 deploy: cdba2fedb2 2023-12-29 19:06:17 +00:00
github-actions[bot] 43dbb52f89 deploy: 3f4691f985 2023-12-28 21:38:44 +00:00
github-actions[bot] 635ee41497 deploy: 3644d67b06 2023-12-26 16:21:15 +00:00
github-actions[bot] e952148cfe deploy: 2267d1c735 2023-12-20 22:58:02 +00:00
github-actions[bot] 6c0d93f4f6 deploy: 01ef2065c0 2023-12-20 18:14:02 +00:00
github-actions[bot] ac6af869ec deploy: 2d61152c07 2023-12-13 16:39:12 +00:00
github-actions[bot] f37fe435df deploy: fc4731a933 2023-12-12 19:59:44 +00:00
github-actions[bot] fb33036380 deploy: d7b9502172 2023-12-12 19:25:47 +00:00
github-actions[bot] 03caa46aba deploy: b20b29918f 2023-12-11 19:21:04 +00:00
github-actions[bot] 736c638c27 deploy: b5dfa00ca8 2023-12-11 18:53:01 +00:00
github-actions[bot] 909d52414a deploy: e12d294cfd 2023-12-11 18:49:47 +00:00
github-actions[bot] 84167acc42 deploy: 8203da887a 2023-12-08 19:16:05 +00:00
github-actions[bot] 689b48ebba deploy: e9b3fe89a9 2023-12-07 15:36:28 +00:00
github-actions[bot] d1f7588cda deploy: e479cc33e3 2023-12-07 00:02:22 +00:00
github-actions[bot] debc7e73fe deploy: 9bcdc440c5 2023-12-06 23:53:19 +00:00
github-actions[bot] 389ecf417b deploy: 64e58f644c 2023-12-06 22:16:24 +00:00
github-actions[bot] f75d34595c deploy: 437ab199da 2023-12-06 21:43:16 +00:00
github-actions[bot] 56c1559ff1 deploy: 79ac876275 2023-12-06 20:16:27 +00:00
github-actions[bot] a835bdceb4 deploy: 3d8918839e 2023-12-06 20:15:17 +00:00
github-actions[bot] 327369be3b deploy: 3b8defdbb1 2023-12-06 20:14:36 +00:00
github-actions[bot] 5871128f13 deploy: 7ea840eefe 2023-12-06 20:12:48 +00:00
github-actions[bot] 8d896ba609 deploy: f175eae6b4 2023-12-06 15:31:01 +00:00
github-actions[bot] 5bb3d0aa0a deploy: 1f39a6ffe8 2023-11-30 17:29:00 +00:00
github-actions[bot] fc198c1b60 deploy: 662afc641b 2023-11-30 15:34:43 +00:00
github-actions[bot] 87aa7b8c1b deploy: 43df88dca9 2023-11-29 22:27:09 +00:00
github-actions[bot] bddd0053bf deploy: 07c03a2bd0 2023-11-29 21:30:16 +00:00
github-actions[bot] 31d55d06b1 deploy: 0b3fceefdc 2023-11-28 19:16:43 +00:00
github-actions[bot] 0ff4a4448a deploy: f202e03526 2023-11-22 22:09:07 +00:00
github-actions[bot] 09a2ac3bd5 deploy: b447e91535 2023-11-22 16:34:05 +00:00
github-actions[bot] 7cf7da8010 deploy: b9e971e9c6 2023-11-21 23:38:11 +00:00
github-actions[bot] 811a0faa36 deploy: 7115adb87a 2023-11-21 22:06:23 +00:00
github-actions[bot] 47cd5ce701 deploy: de807522d3 2023-11-17 18:31:11 +00:00
github-actions[bot] a12582889b deploy: b7f1b59fa9 2023-11-17 15:47:31 +00:00
github-actions[bot] a7339f2119 deploy: ccd59cb482 2023-11-16 20:19:32 +00:00
github-actions[bot] ef33640516 deploy: 25771e2843 2023-11-16 16:59:36 +00:00
github-actions[bot] cf22f14364 deploy: b0435c7827 2023-11-16 16:34:53 +00:00
github-actions[bot] d1cd7d881f deploy: 0719859803 2023-11-14 21:39:17 +00:00
github-actions[bot] 960dd8a45d deploy: 2dd001f632 2023-11-13 18:41:38 +00:00
github-actions[bot] 436e2774f0 deploy: f3dfd92555 2023-11-06 23:19:12 +00:00
github-actions[bot] ed3e19ad77 deploy: dec6fa41f1 2023-11-04 00:01:18 +00:00
github-actions[bot] 40ba367dfb deploy: 01fd1933b4 2023-11-03 23:39:40 +00:00
github-actions[bot] 70d7c3bdb0 deploy: a1a687576e 2023-11-03 23:34:11 +00:00
github-actions[bot] b501249b2b deploy: 4efd181f29 2023-11-03 19:46:14 +00:00
github-actions[bot] 6a75871da6 deploy: 7b1c47a156 2023-11-02 16:30:43 +00:00
github-actions[bot] 1fa5c35554 deploy: 8033f02abb 2023-11-01 22:14:37 +00:00
github-actions[bot] 28f59e8cb8 deploy: f154dbf90a 2023-10-31 00:36:20 +00:00
github-actions[bot] 162e88775b deploy: 9615b78a9a 2023-10-30 15:31:10 +00:00
github-actions[bot] e9f0b0536b deploy: 5a0e3c4ffa 2023-10-27 22:38:43 +00:00
github-actions[bot] 3596ea40f8 deploy: 3ae30045a0 2023-10-27 18:44:55 +00:00
github-actions[bot] bda8a5a3f1 deploy: 01c0d1503c 2023-10-26 23:42:34 +00:00
github-actions[bot] 9bf65346ab deploy: 74d684f77b 2023-10-25 00:36:38 +00:00
github-actions[bot] 74a9c67368 deploy: 6a2a83efe3 2023-10-24 21:20:33 +00:00
github-actions[bot] c4f7c9bd50 deploy: 1bf4fd260f 2023-10-24 21:06:30 +00:00
github-actions[bot] 9dc9df3b30 deploy: 8100893a0e 2023-10-24 19:13:07 +00:00
github-actions[bot] 8fde37b6e2 deploy: c2ab70ef57 2023-10-24 18:44:17 +00:00
github-actions[bot] 9f001081e8 deploy: db1ec45f40 2023-10-23 22:58:17 +00:00
github-actions[bot] 6355e10f23 deploy: 66bf595711 2023-10-23 21:09:01 +00:00
github-actions[bot] 4238fe5377 deploy: 704912aff7 2023-10-20 21:49:08 +00:00
github-actions[bot] c145bfbfa5 deploy: ea60ca4558 2023-10-20 20:29:30 +00:00
github-actions[bot] 16709f85aa deploy: 4d91876215 2023-10-20 18:47:22 +00:00
github-actions[bot] c394b3d0c0 deploy: 9df2eb08d5 2023-10-19 23:12:51 +00:00
github-actions[bot] adf27c5b4e deploy: 8d3474bdb8 2023-10-19 23:02:15 +00:00
github-actions[bot] 41e53fabb8 deploy: 671b947fad 2023-10-19 22:24:26 +00:00
github-actions[bot] b0b79a6edb deploy: a97d474370 2023-10-19 21:42:24 +00:00
github-actions[bot] 84877e13d9 deploy: f404b30784 2023-10-19 20:12:50 +00:00
github-actions[bot] d076407c42 deploy: 48d3be7e5e 2023-10-19 19:26:13 +00:00
github-actions[bot] 086dafdaad deploy: cb4bad2b46 2023-10-19 19:16:12 +00:00
github-actions[bot] a9a84868c7 deploy: 3c40971c1d 2023-10-19 18:31:04 +00:00
github-actions[bot] 4935c9c7c0 deploy: 26cce61205 2023-10-19 18:29:50 +00:00
github-actions[bot] 55b19cdb83 deploy: 0940a22d46 2023-10-18 20:07:20 +00:00
github-actions[bot] 6a6838d6e3 deploy: 59ccda44d6 2023-10-18 17:57:41 +00:00
github-actions[bot] 79da7a35ca deploy: 8f33e627b7 2023-10-18 17:55:34 +00:00
github-actions[bot] b4959db293 deploy: f79994a08d 2023-10-13 22:19:09 +00:00
github-actions[bot] 16bbe17116 deploy: bf54ae4351 2023-10-12 18:54:50 +00:00
github-actions[bot] ee80035ee0 deploy: e60feac44c 2023-10-12 16:49:17 +00:00
github-actions[bot] bf23a1ebf6 deploy: ad1469c19b 2023-10-12 16:40:00 +00:00
github-actions[bot] 744991ec36 deploy: d2a00b73e4 2023-10-11 22:29:18 +00:00
github-actions[bot] 6b29f7fc0b deploy: 4616a1928b 2023-10-11 14:51:32 +00:00
github-actions[bot] c1353ad1bb deploy: 8c7c4aa207 2023-10-06 22:39:17 +00:00
github-actions[bot] 45e1312e1c deploy: 8a95b68bd4 2023-10-06 03:50:42 +00:00
github-actions[bot] bc6d95b906 deploy: b19e67d0c5 2023-10-03 14:25:48 +00:00
github-actions[bot] 53285a810f deploy: cead220aaf 2023-10-02 15:18:10 +00:00
github-actions[bot] 13a9979778 deploy: a1178a1485 2023-09-29 15:43:49 +00:00
github-actions[bot] a400a5189e deploy: a7b41dd423 2023-09-28 20:35:10 +00:00
github-actions[bot] 0a6793d7b4 deploy: 9af9982d68 2023-09-28 18:47:45 +00:00
github-actions[bot] 3a9b23686e deploy: 57bd584612 2023-09-28 18:46:35 +00:00
github-actions[bot] d09d2f9bee deploy: e8497d1aae 2023-09-28 17:43:11 +00:00
github-actions[bot] b2d555a1e4 deploy: 892f92b40e 2023-09-27 23:37:14 +00:00
github-actions[bot] ec7938ac98 deploy: 03d60e2851 2023-09-27 22:56:03 +00:00
github-actions[bot] 4e50955d68 deploy: bcfcf6f62c 2023-09-27 21:03:42 +00:00
github-actions[bot] 668143f450 deploy: c7c9caa3e6 2023-09-22 21:58:38 +00:00
github-actions[bot] 1ac2625bc9 deploy: 38959f4ba9 2023-09-22 21:04:15 +00:00
github-actions[bot] bb9e0559eb deploy: 0baf42a3e9 2023-09-22 20:47:58 +00:00
github-actions[bot] c07b3bf9a6 deploy: 82226c2444 2023-09-21 13:20:57 +00:00
github-actions[bot] 2b77b9536f deploy: d532ef0ded 2023-09-20 21:56:41 +00:00
github-actions[bot] 2fc9516a1d deploy: 92fb7cea60 2023-09-20 21:14:07 +00:00
github-actions[bot] c9a635bf97 deploy: 1c2b3f3cce 2023-09-20 20:05:38 +00:00
github-actions[bot] 8caa4ce2bd deploy: 00c18c19cd 2023-09-20 19:59:03 +00:00
github-actions[bot] e245627bec deploy: da2c4a0f17 2023-09-20 19:57:28 +00:00
github-actions[bot] 48bf8f7c43 deploy: 50ccc06da9 2023-09-20 17:03:32 +00:00
github-actions[bot] 9b60bce6c3 deploy: 3dcb680c48 2023-09-19 22:50:35 +00:00
github-actions[bot] 9f8387cff5 deploy: 07da386179 2023-09-19 21:20:37 +00:00
github-actions[bot] 770e151b3c deploy: cf0e7a280d 2023-09-19 17:23:49 +00:00
github-actions[bot] c8930bbdb6 deploy: 08f6c59d81 2023-09-18 16:38:34 +00:00
github-actions[bot] 829ec3eef8 deploy: aa1223263d 2023-09-18 16:13:12 +00:00
github-actions[bot] 0d6dcc6b76 deploy: 6dd093f6c5 2023-09-18 16:07:01 +00:00
github-actions[bot] be37b4f595 deploy: e4326feac0 2023-09-18 15:45:28 +00:00
github-actions[bot] eb0a61ee77 deploy: bcfd5b25aa 2023-09-13 23:36:51 +00:00
github-actions[bot] 87177160de deploy: f6a70125b1 2023-09-13 22:26:03 +00:00
github-actions[bot] 3e121711a7 deploy: 7922b74630 2023-09-13 21:39:34 +00:00
github-actions[bot] 7ae41875ad deploy: a7c4d606ee 2023-09-13 21:03:26 +00:00
github-actions[bot] 956024ce18 deploy: 58d3cc3f17 2023-09-13 20:53:37 +00:00
github-actions[bot] b62dfe7201 deploy: 89af32fc1e 2023-09-12 17:32:17 +00:00
github-actions[bot] 71173319ac deploy: 3840a41654 2023-09-12 17:30:03 +00:00
github-actions[bot] edb8a53d2a deploy: acedf5470d 2023-09-12 14:19:59 +00:00
github-actions[bot] 3b3e72a91c deploy: fed7507b14 2023-09-11 17:58:30 +00:00
github-actions[bot] baee920286 deploy: 946c22780a 2023-09-11 15:32:30 +00:00
github-actions[bot] 3acd49272e deploy: 8cc47cf07f 2023-09-11 14:35:10 +00:00
github-actions[bot] b035e03aba deploy: ae0281478a 2023-09-08 21:45:30 +00:00
github-actions[bot] 782bdc452c deploy: 82668462be 2023-09-08 21:06:04 +00:00
github-actions[bot] d560c1dc64 deploy: a2952cd469 2023-09-08 20:00:30 +00:00
github-actions[bot] 7887a83a3a deploy: d5e9e0d111 2023-09-08 14:59:30 +00:00
github-actions[bot] de4bfcaabe deploy: 039fa9c1db 2023-09-08 01:32:57 +00:00
github-actions[bot] 346baf3403 deploy: f308f802e1 2023-09-07 21:30:05 +00:00
github-actions[bot] e9b8acf584 deploy: 02912f266c 2023-09-06 22:12:59 +00:00
github-actions[bot] 90328db61b deploy: 5274e4e000 2023-09-06 17:56:22 +00:00
github-actions[bot] 79e48489a6 deploy: 4560574a74 2023-08-31 22:48:01 +00:00
github-actions[bot] 8b76afa9a5 deploy: 162d410af1 2023-08-25 21:10:30 +00:00
github-actions[bot] 678500e8c2 deploy: 36f84327ea 2023-08-25 20:25:03 +00:00
github-actions[bot] a238e40965 deploy: bea15afa17 2023-08-25 19:00:29 +00:00
github-actions[bot] 861d012251 deploy: 1e4d576296 2023-08-25 18:58:13 +00:00
github-actions[bot] 3aab905eb5 deploy: f7557d8e99 2023-08-25 18:30:14 +00:00
github-actions[bot] 6f080a40ec deploy: ac655e308b 2023-08-23 23:44:29 +00:00
github-actions[bot] 954c62dcc5 deploy: 8c850bb5a5 2023-08-23 21:44:29 +00:00
github-actions[bot] 8a9da6dacb deploy: cc1b56c892 2023-08-23 21:41:14 +00:00
github-actions[bot] a55f06ab1d deploy: d08532ac18 2023-08-23 19:41:36 +00:00
github-actions[bot] da250db021 deploy: 1fd885c669 2023-08-22 22:45:24 +00:00
github-actions[bot] 89b05fbb77 deploy: 411825bbdc 2023-08-22 21:28:18 +00:00
github-actions[bot] 3e943882d1 deploy: d9bf7f41a5 2023-08-22 21:23:48 +00:00
github-actions[bot] 3ae0a55b4e deploy: 95b8aa9530 2023-08-22 20:25:12 +00:00
github-actions[bot] c5dae15a6d deploy: aca79758ee 2023-08-22 16:42:17 +00:00
github-actions[bot] 44ee64ca9c deploy: aed5f8fbbe 2023-08-18 14:17:07 +00:00
github-actions[bot] f447f27315 deploy: 1ef9919aea 2023-08-15 22:41:46 +00:00
github-actions[bot] 5f32194c5c deploy: ef7a25fab8 2023-08-15 20:30:26 +00:00
github-actions[bot] ad4e2a1372 deploy: 61b02185af 2023-08-15 19:39:47 +00:00
github-actions[bot] d2d8a5ca76 deploy: 8cfddecb5a 2023-08-15 16:55:10 +00:00
github-actions[bot] 56649c3b63 deploy: 28acde1f41 2023-08-14 21:01:27 +00:00
github-actions[bot] 4c2bf2f5f9 deploy: 7e47e6640c 2023-08-14 18:29:49 +00:00
github-actions[bot] 5e3d95a536 deploy: 153da4ce5f 2023-08-14 17:29:31 +00:00
github-actions[bot] 2db7e6f91a deploy: 23d6ec9a62 2023-08-14 17:00:33 +00:00
github-actions[bot] b3abafc97a deploy: 7a7d6904fb 2023-08-11 23:30:09 +00:00
github-actions[bot] e8db65fb73 deploy: 714ba53fe3 2023-08-11 22:58:31 +00:00
github-actions[bot] b047186fd7 deploy: 4b2fadcc59 2023-08-11 00:25:38 +00:00
github-actions[bot] e5eeb51d95 deploy: e739394756 2023-08-10 22:39:13 +00:00
github-actions[bot] 7e14386adf deploy: 3ded90a630 2023-08-10 22:21:44 +00:00
github-actions[bot] 2005d32845 deploy: 351ce06436 2023-08-07 16:57:32 +00:00
github-actions[bot] 7fe818ccee deploy: ec88ecd160 2023-08-07 16:11:03 +00:00
github-actions[bot] 0fd597b47d deploy: 5693bf2c1a 2023-08-04 15:31:06 +00:00
github-actions[bot] 2f0172e210 deploy: a2cb32710d 2023-08-03 21:41:18 +00:00
github-actions[bot] f1de21988b deploy: e88dacdfb3 2023-08-03 17:17:21 +00:00
github-actions[bot] fc66d77aa1 deploy: e55e90fa29 2023-08-02 15:59:06 +00:00
github-actions[bot] f18a5c5f0d deploy: a437928bb5 2023-08-02 04:23:48 +00:00
github-actions[bot] f805b39abc deploy: 68af55043a 2023-08-01 21:50:06 +00:00
github-actions[bot] afbe0a5b52 deploy: d0e62d0ef4 2023-08-01 19:39:26 +00:00
github-actions[bot] 7cc87ab9f8 deploy: fcb7ef7fee 2023-07-28 20:54:44 +00:00
github-actions[bot] 4a0855636b deploy: 04d25bb850 2023-07-25 21:37:59 +00:00
github-actions[bot] 4321cef2e4 deploy: 3d08fad69b 2023-07-25 17:24:08 +00:00
github-actions[bot] 2f4882a13a deploy: a093a611f2 2023-07-24 15:40:11 +00:00
github-actions[bot] 8d00ac9af3 deploy: 47ad664d53 2023-07-21 19:05:05 +00:00
github-actions[bot] ccc3686370 deploy: 320b78dbf2 2023-07-20 21:25:06 +00:00
github-actions[bot] 376c8f76cf deploy: aaa622292a 2023-07-18 23:53:09 +00:00
github-actions[bot] ff02684c55 deploy: 4276a1413d 2023-07-18 21:32:37 +00:00
github-actions[bot] 8ee52c00e6 deploy: 7287344fc1 2023-07-18 20:46:09 +00:00
github-actions[bot] b9c19b3417 deploy: 8af72cf44a 2023-07-07 21:57:54 +00:00
github-actions[bot] 08c5f4926b deploy: 93e5534265 2023-07-07 20:55:38 +00:00
github-actions[bot] 8f0f2b08e8 deploy: 4bd1f1d0c7 2023-07-06 18:09:57 +00:00
github-actions[bot] a1d5dbb014 deploy: 7045bcb5e6 2023-07-06 18:08:18 +00:00
github-actions[bot] 3ba3b82c1e deploy: a937e2e9e1 2023-07-06 00:25:31 +00:00
github-actions[bot] 5d2994613e deploy: f7b3a8dbcb 2023-07-04 23:12:57 +00:00
github-actions[bot] 3b2320e228 deploy: 42fd7fce5d 2023-07-04 23:11:07 +00:00
github-actions[bot] dc892a9f09 deploy: 932256a16f 2023-07-04 19:10:45 +00:00
github-actions[bot] 7a59ab8359 deploy: 6058de9d29 2023-07-04 18:31:41 +00:00
github-actions[bot] b40f44bac7 deploy: 6233d3a42e 2023-07-03 20:34:09 +00:00
github-actions[bot] 23b1038690 deploy: e915165bb8 2023-07-03 17:55:24 +00:00
github-actions[bot] b6819041db deploy: a46ff028e1 2023-06-29 21:48:29 +00:00
github-actions[bot] b54ceeab05 deploy: fdebc8fcc1 2023-06-29 19:23:46 +00:00
github-actions[bot] 3fb79c05be deploy: e3b0c03ef8 2023-06-28 21:35:04 +00:00
github-actions[bot] 861c1a97b4 deploy: a25a4e7c36 2023-06-28 18:17:09 +00:00
github-actions[bot] db1cd05170 deploy: fbfaf54bc4 2023-06-28 17:41:18 +00:00
github-actions[bot] 839fe62032 deploy: 8edddaef3b 2023-06-22 21:33:20 +00:00
github-actions[bot] accc830b32 deploy: cab1ea018d 2023-06-21 22:09:50 +00:00
github-actions[bot] 48e3cf6a43 deploy: 815c37835b 2023-06-21 20:35:11 +00:00
github-actions[bot] 0f24c65f5c deploy: 32d2697f4f 2023-06-21 20:33:47 +00:00
github-actions[bot] 14a3114b4f deploy: 71b5476f55 2023-06-20 23:51:51 +00:00
github-actions[bot] 699d512272 deploy: 8fcb90a372 2023-06-20 23:27:51 +00:00
github-actions[bot] cbbd3a3618 deploy: 8cd8ed012b 2023-06-19 18:22:50 +00:00
github-actions[bot] b997ace9df deploy: ab5a86161b 2023-06-13 18:17:47 +00:00
github-actions[bot] 0ea54f299c deploy: 95969b2c2e 2023-06-13 17:54:37 +00:00
github-actions[bot] 09bfdb7aa2 deploy: b4af04de98 2023-06-09 00:21:01 +00:00
github-actions[bot] f972e1f2c3 deploy: 078b4f4c9a 2023-06-08 23:07:30 +00:00
github-actions[bot] 7d300c5867 deploy: d8ace55db5 2023-06-08 21:36:55 +00:00
github-actions[bot] 53807d31d9 deploy: ae741d3b8b 2023-06-06 17:23:59 +00:00
github-actions[bot] 8e67c6a6c2 deploy: 7084157c48 2023-06-06 17:10:05 +00:00
github-actions[bot] cc99697558 deploy: bebd4a4df8 2023-06-06 16:58:58 +00:00
github-actions[bot] af97f6fd58 deploy: 5f688b5d1a 2023-06-05 22:05:58 +00:00
github-actions[bot] bb0d53a8ef deploy: 6a1600c46c 2023-06-05 21:52:01 +00:00
github-actions[bot] 57deadc313 deploy: 207c528405 2023-06-05 21:03:54 +00:00
github-actions[bot] ea0eb6ce69 deploy: 9f65c72d92 2023-06-05 20:15:28 +00:00
github-actions[bot] 181994d787 deploy: 05f194a3f3 2023-06-03 13:35:02 +00:00
github-actions[bot] 070f4ad99a deploy: 9eeaf37b1a 2023-06-03 13:33:52 +00:00
github-actions[bot] 141e0ff20c deploy: 4783588940 2023-06-02 21:59:49 +00:00
github-actions[bot] b1c4c0e7c7 deploy: 9a801b3310 2023-06-02 21:44:01 +00:00
github-actions[bot] 13a0056588 deploy: 3486cb7626 2023-06-02 21:00:09 +00:00
github-actions[bot] 0d81f22fcd deploy: 0f2356c8d3 2023-06-02 20:58:07 +00:00
github-actions[bot] 0db0a35187 deploy: c8bf3eff9e 2023-06-02 15:26:42 +00:00
github-actions[bot] 55602a1053 deploy: 93451fe58d 2023-06-02 15:16:50 +00:00
github-actions[bot] 81753a2951 deploy: e0afaa5783 2023-06-01 23:25:04 +00:00
github-actions[bot] fc73a1b778 deploy: 150b77d6e0 2023-06-01 23:09:28 +00:00
github-actions[bot] ac1d4a8cac deploy: 1e0fdad446 2023-06-01 22:39:24 +00:00
github-actions[bot] 3a45d27b6f deploy: 2352ff4b4f 2023-06-01 22:36:56 +00:00
github-actions[bot] 5ba1be26c8 deploy: 7be7b1c337 2023-06-01 21:55:11 +00:00
github-actions[bot] 7575640130 deploy: 14f94957f5 2023-06-01 21:45:52 +00:00
github-actions[bot] e68dc6fb3a deploy: b183dae1fa 2023-06-01 17:36:20 +00:00
github-actions[bot] 25e3973c5b deploy: 33d0cb34f2 2023-06-01 17:07:28 +00:00
github-actions[bot] dccc8fa055 deploy: e648b427cd 2023-06-01 16:56:02 +00:00
github-actions[bot] 08cd399f86 deploy: d1ff3db699 2023-06-01 16:50:11 +00:00
github-actions[bot] e1b952a096 deploy: 9af052e965 2023-06-01 14:56:36 +00:00
github-actions[bot] 219477d2f5 deploy: fa3b5a7784 2023-05-31 22:56:51 +00:00
github-actions[bot] f02a2281bd deploy: 16755c1211 2023-05-31 20:35:53 +00:00
github-actions[bot] 6c96b8d3c4 deploy: 0d67e2eefc 2023-05-31 17:11:02 +00:00
github-actions[bot] a2c0fcbc95 deploy: 8a3d8b1068 2023-05-26 18:47:28 +00:00
github-actions[bot] 090e5d996c deploy: c7f7b91719 2023-05-26 18:44:31 +00:00
github-actions[bot] a5ffbedf5c deploy: fdd21a192d 2023-05-26 14:54:59 +00:00
github-actions[bot] 44e858f2f7 deploy: 6a13f9767a 2023-05-25 22:06:28 +00:00
github-actions[bot] b9130efcf7 deploy: a24c628687 2023-05-25 21:57:40 +00:00
github-actions[bot] 51c2ad6cbf deploy: ee82ada796 2023-05-25 21:54:57 +00:00
github-actions[bot] e288480ccd deploy: c961ddb196 2023-05-25 21:53:51 +00:00
github-actions[bot] eb41e8d55c deploy: b90ca52a07 2023-05-25 21:52:32 +00:00
github-actions[bot] 1c7a91b51e deploy: 6cf9c2c078 2023-05-25 21:50:16 +00:00
github-actions[bot] 4659d996fc deploy: 0eef634031 2023-05-24 20:43:52 +00:00
github-actions[bot] d8ec54db65 deploy: 6969567f87 2023-05-24 18:21:36 +00:00
github-actions[bot] 7c12c6cd18 deploy: 15a2ec845b 2023-05-19 21:25:02 +00:00
github-actions[bot] 15e75e948e deploy: 9eb16cb360 2023-05-19 16:59:19 +00:00
github-actions[bot] f6034eb5b8 deploy: 04074b9a24 2023-05-19 14:21:11 +00:00
github-actions[bot] 8b77d016fb deploy: 1a69f49003 2023-05-19 14:14:44 +00:00
github-actions[bot] 9f99a90b45 deploy: 349aa76683 2023-05-18 17:15:41 +00:00
github-actions[bot] d52662adc1 deploy: 07ac0dbad1 2023-05-16 16:29:23 +00:00
github-actions[bot] c1f2040a82 deploy: ae1fc31875 2023-05-15 22:50:08 +00:00
github-actions[bot] 168b58f24c deploy: d4527d6d13 2023-05-15 21:58:11 +00:00
github-actions[bot] e0b7588b3d deploy: 08b0b914ca 2023-05-12 23:12:48 +00:00
github-actions[bot] 2e005fd8cb deploy: 73469fd513 2023-05-09 16:49:04 +00:00
github-actions[bot] 37914a9360 deploy: cbeda1c87c 2023-05-09 16:47:38 +00:00
github-actions[bot] 723ceaa523 deploy: 9145a012eb 2023-05-09 16:39:35 +00:00
github-actions[bot] af3454e1ba deploy: 10eb5c4250 2023-05-08 19:43:18 +00:00
github-actions[bot] a0be43e5f9 deploy: d50edbd53a 2023-05-08 19:41:52 +00:00
github-actions[bot] f97bd20029 deploy: e79b907ff0 2023-05-05 21:44:44 +00:00
github-actions[bot] 3b0fb79a6d deploy: 98ffa615bc 2023-05-05 19:46:56 +00:00
github-actions[bot] a7330ac2d0 deploy: 31c0640c42 2023-05-05 17:33:05 +00:00
github-actions[bot] d48b690030 deploy: 47a155d9af 2023-05-05 17:30:06 +00:00
github-actions[bot] 51f7e1b62b deploy: ddb2626739 2023-05-05 15:23:40 +00:00
github-actions[bot] 9e7607db44 deploy: 5c7e125389 2023-05-04 21:28:37 +00:00
github-actions[bot] 697eaf8f9f deploy: d26f8de3a2 2023-05-02 22:20:24 +00:00
github-actions[bot] 19af872aaf deploy: a75cb161ab 2023-05-02 20:28:50 +00:00
github-actions[bot] f10fc75f3c deploy: 196680efb4 2023-05-01 21:00:02 +00:00
github-actions[bot] 809aa9a2dd deploy: 10be541c16 2023-04-28 22:42:05 +00:00
github-actions[bot] bbabad1752 deploy: a5b1a04ea3 2023-04-28 22:28:29 +00:00
github-actions[bot] 7dc1a1bf0f deploy: e2d6bd25a3 2023-04-28 16:26:31 +00:00
github-actions[bot] 5e1032db38 deploy: 4c332fa834 2023-04-27 22:10:25 +00:00
github-actions[bot] c2b1e6d950 deploy: 8e532034bb 2023-04-27 21:48:26 +00:00
github-actions[bot] 6c44ffd4aa deploy: 08aa193918 2023-04-26 21:26:03 +00:00
github-actions[bot] 855a90f364 deploy: 9b9a73dd59 2023-04-26 18:39:55 +00:00
github-actions[bot] 6c7c494616 deploy: 9f02a5816c 2023-04-26 18:35:05 +00:00
github-actions[bot] f3b2bded7a deploy: bf955668af 2023-04-26 18:29:05 +00:00
github-actions[bot] af1fe3a959 deploy: a5e5525984 2023-04-25 23:54:59 +00:00
github-actions[bot] fe302296da deploy: 7862a6d6bf 2023-04-25 19:13:34 +00:00
github-actions[bot] 19c9e3e4bd deploy: 0fd699ae09 2023-04-24 21:57:59 +00:00
github-actions[bot] 4c01da5491 deploy: ea7d4485b3 2023-04-24 21:51:03 +00:00
github-actions[bot] 4207b1c2d4 deploy: 68ec61cdf6 2023-04-21 19:23:16 +00:00
github-actions[bot] 559cabdea4 deploy: b909661aa0 2023-04-20 18:29:32 +00:00
github-actions[bot] 5d7d21f661 deploy: 80747c8d62 2023-04-19 20:11:59 +00:00
github-actions[bot] aff30c7528 deploy: a56a63ad9e 2023-04-19 19:28:13 +00:00
github-actions[bot] 7cbe657ff2 deploy: 241b2b4557 2023-04-19 16:45:26 +00:00
github-actions[bot] 547559cac9 deploy: e48aabb3dd 2023-04-18 22:37:46 +00:00
github-actions[bot] c94177d2e4 deploy: 470d730424 2023-04-17 15:53:06 +00:00
github-actions[bot] 58a1b51ca1 deploy: f3612708d0 2023-04-13 21:52:41 +00:00
github-actions[bot] 268b5a7f0f deploy: 2862f3d506 2023-04-13 16:38:23 +00:00
github-actions[bot] 91d1b5cf8b deploy: c17c68516a 2023-04-13 14:57:55 +00:00
github-actions[bot] 5383c45568 deploy: 13d71a5142 2023-04-12 04:51:21 +00:00
github-actions[bot] de59a1f028 deploy: 1b5278c3a9 2023-04-11 15:01:29 +00:00
github-actions[bot] cd5a8101fa deploy: a5e3b54e32 2023-04-03 19:20:08 +00:00
github-actions[bot] 4f3f025cc5 deploy: 6461455e1f 2023-04-03 17:40:59 +00:00
github-actions[bot] ecf0161de5 deploy: fba622e38c 2023-04-03 15:32:56 +00:00
github-actions[bot] 09cada1277 deploy: d6579e988d 2023-03-29 21:07:06 +00:00
github-actions[bot] ac8d8c6343 deploy: 57a79a6931 2023-03-29 20:59:42 +00:00
github-actions[bot] 3eb78ceb37 deploy: 9c4fa7b21c 2023-03-27 15:15:07 +00:00
github-actions[bot] 87c76525a5 deploy: 637f6a1c54 2023-03-27 14:47:25 +00:00
github-actions[bot] 39711d1293 deploy: 3b371ab635 2023-03-24 15:30:38 +00:00
github-actions[bot] 2158240110 deploy: 9a6dadaf40 2023-03-24 15:14:13 +00:00
github-actions[bot] 3a2bb3b316 deploy: 5323ae3ab1 2023-03-20 16:59:56 +00:00
github-actions[bot] 9a491986c2 deploy: d48546c707 2023-03-17 22:42:20 +00:00
github-actions[bot] 59d2eace0b deploy: 86b4b5fd97 2023-03-17 22:36:04 +00:00
github-actions[bot] 84027a9840 deploy: da145c71a4 2023-03-10 20:17:28 +00:00
github-actions[bot] 0f2e601559 deploy: 3b0ceffe94 2023-03-10 19:29:42 +00:00
github-actions[bot] 417ada2723 deploy: 58d34d8bdd 2023-03-08 19:38:55 +00:00
github-actions[bot] 30e7e4b446 deploy: 3c50c59e49 2023-03-07 22:36:58 +00:00
github-actions[bot] 718be93d0d deploy: 2dcdb6afb4 2023-03-07 18:34:40 +00:00
github-actions[bot] 1b192b0bff deploy: af3f3171ea 2023-03-06 22:22:05 +00:00
github-actions[bot] 7009a55511 deploy: 0a3d074a7a 2023-03-01 18:53:20 +00:00
github-actions[bot] 0b7cb4287b deploy: 92df620622 2023-02-28 19:21:09 +00:00
github-actions[bot] 2e342f7a1d deploy: bbd475ef40 2023-02-28 17:32:26 +00:00
github-actions[bot] bdeb90cf8c deploy: ddc6129b95 2023-02-28 00:27:02 +00:00
github-actions[bot] 76106f5c6a deploy: df7d053f16 2023-02-27 22:28:21 +00:00
github-actions[bot] ffa59026c9 deploy: 47b0f2657b 2023-02-27 16:29:49 +00:00
github-actions[bot] 8c6aaed212 deploy: 81b576006f 2023-02-24 00:33:15 +00:00
github-actions[bot] 3ea1b85e61 deploy: 9fefbc7dff 2023-02-24 00:05:50 +00:00
github-actions[bot] 2c57f83a55 deploy: 75f5001ea4 2023-02-22 17:55:17 +00:00
github-actions[bot] 28e582c129 deploy: 00c2d6fadd 2023-02-21 23:14:10 +00:00
github-actions[bot] 7f0115e7f9 deploy: 9e3b1df3e7 2023-02-21 22:20:23 +00:00
github-actions[bot] a6f2e6376d deploy: 407c0b6a23 2023-02-17 23:59:55 +00:00
github-actions[bot] 8df364ff2e deploy: a30412f49d 2023-02-17 22:26:02 +00:00
github-actions[bot] 0d4704a88a deploy: e0259a4170 2023-02-17 19:57:18 +00:00
github-actions[bot] 4e34728b51 deploy: 0a2fc5e2d5 2023-02-15 22:21:06 +00:00
github-actions[bot] bd8f03ef0a deploy: b71712fcf2 2023-02-15 21:34:08 +00:00
github-actions[bot] 9eadf5ac90 deploy: 99bd431f60 2023-02-10 23:39:04 +00:00
github-actions[bot] d378ae6de7 deploy: 3951e884dd 2023-02-09 22:05:37 +00:00
github-actions[bot] 868999682d deploy: ecc036caba 2023-02-09 21:15:36 +00:00
github-actions[bot] d0af0dc533 deploy: 9813766f60 2023-02-09 21:04:32 +00:00
github-actions[bot] 03de32feb8 deploy: 788cf97657 2023-02-09 02:35:44 +00:00
github-actions[bot] 00f03332ad deploy: 2901a394d1 2023-02-09 02:34:59 +00:00
github-actions[bot] a019535f26 deploy: d6f9bf67e9 2023-02-09 01:35:50 +00:00
github-actions[bot] 6fa5611494 deploy: da240c5db0 2023-02-09 01:30:24 +00:00
github-actions[bot] b321e482a3 deploy: 57e398ca58 2023-02-08 23:02:07 +00:00
github-actions[bot] 9931f5d189 deploy: 8eb7d11f88 2023-02-08 02:34:46 +00:00
github-actions[bot] bcf2b79670 deploy: c7a60831bf 2023-02-07 19:15:14 +00:00
github-actions[bot] e259676292 deploy: 144dc06b47 2023-02-07 18:14:15 +00:00
github-actions[bot] a5549cd1bb deploy: 907ef1a4e3 2023-02-07 01:18:04 +00:00
github-actions[bot] 271571c449 deploy: e8869c3a9c 2023-02-07 01:06:34 +00:00
github-actions[bot] bdaa0f00cb deploy: 16cbb8302c 2023-02-07 00:39:20 +00:00
github-actions[bot] 4159383f3c deploy: 33e43652a5 2023-02-06 21:40:57 +00:00
github-actions[bot] 2c4b4292da deploy: 8a019a77f8 2023-02-06 21:40:25 +00:00
github-actions[bot] 9e443220e6 deploy: 6b57eada01 2023-02-03 17:17:14 +00:00
github-actions[bot] fa95f271af deploy: 77643faaf5 2023-02-02 22:12:08 +00:00
github-actions[bot] d625af22af deploy: 30992dd2a5 2023-01-31 20:16:09 +00:00
github-actions[bot] ba69cfee5b deploy: f9e6d8dc21 2023-01-31 18:22:33 +00:00
github-actions[bot] 8872468dbb deploy: ebcf7321d4 2023-01-27 22:23:24 +00:00
github-actions[bot] 70c7c660b1 deploy: cee3d6bc96 2023-01-27 21:50:48 +00:00
github-actions[bot] bb0db513b2 deploy: d501973808 2023-01-27 21:38:54 +00:00
github-actions[bot] ea029c92ff deploy: 699c8eb449 2023-01-26 15:37:39 +00:00
github-actions[bot] 54951ef090 deploy: 1e493e5641 2023-01-25 18:26:00 +00:00
github-actions[bot] 187ad0ddb1 deploy: 0727c7bb9b 2023-01-24 00:00:32 +00:00
github-actions[bot] a10bc05c7d deploy: 5107bc39ed 2023-01-23 23:36:44 +00:00
github-actions[bot] a73a22babd deploy: ecd63824ca 2023-01-23 23:36:02 +00:00
github-actions[bot] 9c5cd612e1 deploy: 950d95077c 2023-01-19 20:10:42 +00:00
github-actions[bot] 164a24f6d0 deploy: dca5002eae 2023-01-19 20:08:59 +00:00
github-actions[bot] 0b02db0d1a deploy: 40042c6f0d 2023-01-19 20:04:04 +00:00
github-actions[bot] be86bb62c5 deploy: 2204c3446a 2023-01-18 18:42:19 +00:00
github-actions[bot] 6b22fdf93b deploy: 15ea1bccc0 2023-01-16 17:29:04 +00:00
github-actions[bot] a4b42a3a7e deploy: 5f57b9c41e 2023-01-13 18:55:02 +00:00
github-actions[bot] 14a8e1e639 deploy: 6825dcd247 2023-01-12 18:20:01 +00:00
github-actions[bot] 38f1927fe0 deploy: e3f43e64df 2023-01-07 00:07:59 +00:00
github-actions[bot] 1c216d5ba4 deploy: 743dd5e1f3 2023-01-04 17:52:25 +00:00
github-actions[bot] 7cce57861f deploy: 6514a58d4b 2023-01-04 00:38:57 +00:00
github-actions[bot] ec2bdf2fef deploy: 2810df9a9c 2023-01-04 00:18:14 +00:00
github-actions[bot] 747ee887ab deploy: 6d0fb3daf2 2022-12-22 22:00:17 +00:00
github-actions[bot] eef769b122 deploy: cfd19d21b2 2022-12-22 00:37:39 +00:00
github-actions[bot] 36933b9206 deploy: 6a4b382632 2022-12-19 18:41:11 +00:00
github-actions[bot] e62dad6f3b deploy: 2ba362105b 2022-12-15 00:10:14 +00:00
github-actions[bot] ad38b24769 deploy: 38bc4f8fa0 2022-12-13 21:13:48 +00:00
github-actions[bot] 9222cec822 deploy: 9b48b7c7d4 2022-12-13 18:38:20 +00:00
github-actions[bot] a6efcef175 deploy: ec97d80f1e 2022-12-12 20:53:52 +00:00
github-actions[bot] ed2627b985 deploy: a52be2e254 2022-12-10 00:52:23 +00:00
github-actions[bot] bcd934a274 deploy: 3a97dae8af 2022-12-09 02:20:14 +00:00
github-actions[bot] 590e5cd737 deploy: 37e2e6e63b 2022-12-08 22:55:52 +00:00
github-actions[bot] c1aab17cf4 deploy: ac60735704 2022-12-08 21:36:07 +00:00
github-actions[bot] 0d18b75d0a deploy: 17132e732e 2022-12-07 23:52:21 +00:00
github-actions[bot] 9813b91a72 deploy: 3d15613ef2 2022-12-07 02:17:56 +00:00
github-actions[bot] 2749ddb0e9 deploy: 2d087b8390 2022-12-07 00:01:49 +00:00
github-actions[bot] 4aaff787b6 deploy: 319fa80361 2022-12-06 21:54:34 +00:00
github-actions[bot] 9577e96d36 deploy: aa29a86080 2022-12-03 02:40:18 +00:00
github-actions[bot] dd688897b6 deploy: d277a3701c 2022-12-02 23:50:19 +00:00
github-actions[bot] 4abe0698fc deploy: 8ac152bcaf 2022-12-01 01:04:34 +00:00
github-actions[bot] 146696faca deploy: b029562ede 2022-12-01 00:20:19 +00:00
github-actions[bot] 6718e5b6ae deploy: 00efeb90cc 2022-11-30 22:17:34 +00:00
github-actions[bot] 5f4a043bf9 deploy: 534fe8c386 2022-11-30 00:30:24 +00:00
github-actions[bot] 1d84dd172b deploy: f491c5f9ad 2022-11-28 22:40:13 +00:00
github-actions[bot] c5c27a8c25 deploy: 3c68b4cfc9 2022-11-28 21:54:58 +00:00
github-actions[bot] cddb0c2ffa deploy: 7e8d557cc8 2022-11-25 00:08:25 +00:00
github-actions[bot] 99173c98a2 deploy: 80ff1da9f2 2022-11-24 21:55:34 +00:00
github-actions[bot] e487bd0d35 deploy: 14ee69daef 2022-11-23 22:03:56 +00:00
github-actions[bot] 9767456391 deploy: c1d08f92ab 2022-11-23 19:04:57 +00:00
github-actions[bot] fd099431c1 deploy: cf894b9c2d 2022-11-18 19:00:10 +00:00
github-actions[bot] 1ccd17f123 deploy: c17115f170 2022-11-18 01:26:41 +00:00
github-actions[bot] 01b68fdfd5 deploy: abd777b1fa 2022-11-17 22:46:15 +00:00
github-actions[bot] 2577c4b677 deploy: 15c4258120 2022-11-17 17:42:07 +00:00
github-actions[bot] 890b0daabc deploy: 9eec85db53 2022-11-16 19:57:14 +00:00
github-actions[bot] c600d2c44e deploy: c8fd4b6102 2022-11-14 21:25:49 +00:00
github-actions[bot] bde39d324b deploy: 27b3cd2003 2022-11-10 00:29:02 +00:00
github-actions[bot] 9f696240b1 deploy: bd8c0f3ee5 2022-11-07 23:05:16 +00:00
github-actions[bot] 2a94478157 deploy: c3a2fddd71 2022-11-04 04:10:45 +00:00
github-actions[bot] 4ae817d18b deploy: aa0c456ba1 2022-10-31 20:50:04 +00:00
github-actions[bot] da29b0dd34 deploy: abad812246 2022-10-29 02:34:56 +00:00
github-actions[bot] 92da899ddc deploy: 658c4dec4d 2022-10-26 17:24:02 +00:00
github-actions[bot] b5461fa14a deploy: 951b9f0403 2022-10-25 21:20:05 +00:00
github-actions[bot] 71679c6878 deploy: 47c1e6e048 2022-10-25 16:45:41 +00:00
github-actions[bot] ef6178d267 deploy: 3c786e223a 2022-10-24 22:23:41 +00:00
github-actions[bot] b271269a15 deploy: 27b925f973 2022-10-18 21:52:10 +00:00
github-actions[bot] 2e762e1cb1 deploy: 1cbb88936c 2022-10-17 18:35:04 +00:00
github-actions[bot] 162084f662 deploy: df1460603a 2022-10-17 16:13:51 +00:00
github-actions[bot] 6971f06b12 deploy: 211bba52aa 2022-10-13 23:19:07 +00:00
github-actions[bot] 10d3c7cf0d deploy: a136956216 2022-10-13 21:26:30 +00:00
github-actions[bot] 4027a1f853 deploy: 21093e682c 2022-10-11 23:39:58 +00:00
github-actions[bot] 492258125a deploy: f5141d5c4c 2022-10-11 23:07:02 +00:00
github-actions[bot] 0579d1f341 deploy: 260637b847 2022-10-11 22:58:49 +00:00
github-actions[bot] 74c81f3faf deploy: d0010f2058 2022-10-03 22:46:36 +00:00
github-actions[bot] 537bcb24d2 deploy: 97152def71 2022-10-03 17:08:13 +00:00
github-actions[bot] be8a01cd8c deploy: 3dab366eb4 2022-10-03 17:02:31 +00:00
github-actions[bot] c4d5c919c5 deploy: 4bc4dcbbc3 2022-10-03 16:58:50 +00:00
github-actions[bot] 0cceea3308 deploy: af4a295486 2022-09-28 21:52:37 +00:00
github-actions[bot] 27ba76a2bb deploy: 6167c9fbb9 2022-09-28 16:15:54 +00:00
github-actions[bot] 2ca8f48e61 deploy: 1c0d662469 2022-09-27 21:03:52 +00:00
github-actions[bot] ade5338dec deploy: 408c4c3998 2022-09-26 17:08:48 +00:00
github-actions[bot] 908cd76c9b deploy: fc8ec03c13 2022-09-22 17:31:36 +00:00
github-actions[bot] 2316f7b25c deploy: 10e5370240 2022-09-21 21:17:23 +00:00
github-actions[bot] cdda1234d6 deploy: d5e271caff 2022-09-21 16:47:53 +00:00
github-actions[bot] 1ea5c372c5 deploy: 4a155328f9 2022-09-20 22:04:51 +00:00
github-actions[bot] 8970fb542a deploy: bbbf5ea98f 2022-09-20 17:04:38 +00:00
github-actions[bot] 1e240e2ff3 deploy: c9458f36e6 2022-09-20 17:00:22 +00:00
github-actions[bot] 9077d4f374 deploy: 91e4501594 2022-09-16 16:54:03 +00:00
github-actions[bot] 8c44da5408 deploy: f1c2e1f17a 2022-09-16 16:46:46 +00:00
github-actions[bot] 5bc1a5a27f deploy: e736c03f65 2022-09-15 23:39:59 +00:00
github-actions[bot] 351bb4e544 deploy: 74e2c3d259 2022-09-15 23:20:45 +00:00
github-actions[bot] e6b4008937 deploy: 5a3bfa4271 2022-09-14 22:32:25 +00:00
github-actions[bot] 682a8ba01a deploy: ab06355467 2022-09-14 17:07:41 +00:00
github-actions[bot] 9a93065306 deploy: db4d8c31ae 2022-09-13 23:30:12 +00:00
github-actions[bot] 868712c57a deploy: 2f6b161a44 2022-09-13 20:27:25 +00:00
github-actions[bot] 2e836d4b48 deploy: 070338c2e4 2022-09-13 19:02:06 +00:00
github-actions[bot] dee8f2b801 deploy: 2a1aecc321 2022-09-13 16:50:35 +00:00
github-actions[bot] 8e5513dbc0 deploy: 9f188317d3 2022-09-13 03:36:43 +00:00
github-actions[bot] 183c5da85d deploy: e8c112709a 2022-09-12 23:19:08 +00:00
github-actions[bot] a72c9fa516 deploy: 0643860cc1 2022-09-12 22:09:52 +00:00
github-actions[bot] 1a81a57060 deploy: 0051aca3e9 2022-09-12 20:13:05 +00:00
github-actions[bot] 339838b1ff deploy: 01ba9e77e4 2022-09-12 16:57:16 +00:00
github-actions[bot] fa416c23ab deploy: 36389237ce 2022-09-12 16:21:58 +00:00
github-actions[bot] 0b1b76ab6e deploy: 872809ba37 2022-09-10 07:39:00 +00:00
github-actions[bot] 12d408485f deploy: 11ebb7d341 2022-09-10 04:01:04 +00:00
github-actions[bot] e348927cfa deploy: 927b54a88f 2022-09-10 03:55:53 +00:00
github-actions[bot] 4c26482059 deploy: 4280d6efbf 2022-09-10 02:59:51 +00:00
github-actions[bot] be34e9a44b deploy: fb7f0f8da9 2022-09-10 00:06:20 +00:00
github-actions[bot] 9ef8ba92bf deploy: 878aa86c1a 2022-09-09 22:17:39 +00:00
github-actions[bot] 9260af7548 deploy: 38da4eb019 2022-09-08 22:44:57 +00:00
Chris Olstrom f15c497b43 Create CNAME 2022-09-07 16:46:18 -07:00
Chris Olstrom 28447c9f4f Delete CNAME 2022-09-07 16:46:07 -07:00
github-actions[bot] 4c8ebdb197 deploy: 879f4c94de 2022-09-07 19:14:10 +00:00
github-actions[bot] 7e0e3b8fa2 deploy: 55b02e5057 2022-09-06 17:52:02 +00:00
Billy Tat aba9c1d954 Deploy website - based on 88bdf37af4 2022-09-02 16:37:03 -07:00
Billy Tat 8541cb34d4 Deploy website - based on 7ff88f27f3aec5026e5181cdbe10446c63ed726b 2022-09-02 16:24:35 -07:00
Billy Tat 1d128bb26f Deploy website - based on ffaee23f59280cfc56f819699132a2ebacf046b3 2022-09-02 16:14:22 -07:00
Billy Tat 43b91137c1 Deploy website - based on a44ed9e1a4c21cb786a8de3c8c06bf163321ce46 2022-09-02 15:56:45 -07:00
Billy Tat f8fab32b00 Deploy website - based on a44ed9e1a4c21cb786a8de3c8c06bf163321ce46 2022-09-02 15:41:35 -07:00
Billy Tat b715d71827 Deploy website - based on 99896f6cac 2022-09-02 15:02:00 -07:00
35948 changed files with 4537107 additions and 1115574 deletions
-7
View File
@@ -1,7 +0,0 @@
--ignore-dir=public
--ignore-dir=static
--ignore-dir=src/img
--ignore-dir=src/img
--type-add=svg:ext:svg
--no-xml
--no-svg
-34
View File
@@ -1,34 +0,0 @@
# EditorConfig helps developers define and maintain consistent
# coding styles between different editors and IDEs
# editorconfig.org
root = true
[*]
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
indent_style = space
indent_size = 2
[*.js]
indent_style = space
indent_size = 2
[*.hbs]
insert_final_newline = false
indent_style = space
indent_size = 2
[*.css]
indent_style = space
indent_size = 2
[*.html]
indent_style = space
indent_size = 2
[*.{diff,md}]
trim_trailing_whitespace = false
-48
View File
@@ -1,48 +0,0 @@
module.exports = {
root: true,
parserOptions: {
ecmaVersion: 6,
sourceType: 'module',
},
extends: 'eslint:recommended',
globals: {
"document": true,
"window": true,
"jQuery": true,
"$": true,
"require": true,
"__dirname": true,
"process": true,
"ENV": true,
"module": true
},
env: {
browser: true,
"es6": true
},
rules: {
"no-cond-assign": [
2,
"except-parens"
],
"curly": 2,
"no-console": 0,
"no-debugger": 0,
"eqeqeq": 2,
"no-eval": 2,
"guard-for-in": 0,
"wrap-iife": 0,
"linebreak-style": 0,
"new-cap": 0,
"no-caller": 2,
"no-empty": 0,
"no-extra-boolean-cast": 0,
"no-new": 0,
"no-plusplus": 0,
"no-undef": 2,
"dot-notation": 0,
"strict": 0,
"no-eq-null": 2,
"no-unused-vars": 2
}
};
@@ -1,18 +0,0 @@
---
name: Release Maintenance Task Checklist
about: Track tasks that need to be done every release.
title: '<VERSION> - Rancher Manager Release Maintenance Task Checklist'
---
This issue is to track tasks that need to be done every release regardless of whether the release has new feature content or not.
- [ ] Create a new branch for the release. Release-specific updates should use this branch as its base
- [ ] Update the [versions table](https://ranchermanager.docs.rancher.com/versions)
- [ ] Update the [Rancher:webhook version mapping table](https://ranchermanager.docs.rancher.com/reference-guides/rancher-webhook)
- [ ] Update the [CNI popularity table](https://ranchermanager.docs.rancher.com/faq/container-network-interface-providers#cni-community-popularity)
- [ ] Update the [CSP adapter compatibility matrix](https://ranchermanager.docs.rancher.com/integrations-in-rancher/cloud-marketplace/aws-cloud-marketplace/install-adapter#rancher-vs-adapter-compatibility-matrix)
- [ ] Update the [deprecated features table](https://ranchermanager.docs.rancher.com/faq/deprecated-features)
- [ ] Update the swagger-<VERSION>.json file
- [ ] Create a PR merging the release branch back into the main branch
- [ ] Create a new [release](https://github.com/rancher/rancher-docs/releases)
- [ ] Update Algolia search index
@@ -1,23 +0,0 @@
---
name: Request a New Feature
about: For requesting new feature(s) to be added to the docs.
---
## Related Issues
<!--
List any issues or tickets on other platforms that are associated with the request. For example, include a link to the issue tracking that feature in the Rancher repo, or list the Jira ticket number for the request.
-->
## Summary
<!--
Describe the new feature. If QA has not yet tested the new feature/process, please also file a ticket with QA for their review.
-->
## Details
<!--
- Include all pertinent information, e.g., screenshots, resource requirements, workarounds, links, etc.
- List page link(s) in the current docs where the new feature applies, if applicable.
-->
@@ -1,15 +0,0 @@
---
name: Request an Update
about: For fixing docs errors/typos, adding needed/missing information, updating verbiage, deleting outdated info, etc.
---
## Related Issues
<!--
List any issues or tickets on other platforms that are associated with the request. For example, include a link to the issue tracking that feature in the Rancher repo, or list the Jira ticket number for the request.
-->
## Summary
<!--
Describe the requested update giving as much detail as possible. Please also list page link(s) in the current docs where the update applies.
-->
-8
View File
@@ -1,8 +0,0 @@
version: 2
updates:
- package-ecosystem: gitsubmodule
schedule:
interval: "daily"
directory: /
-27
View File
@@ -1,27 +0,0 @@
<!--
Check the Rancher docs issues to see if there is an existing issue for this pull request. If there is, enter the issue number below.
-->
Fixes #[issue_number]
## Reminders
- See the [README](../README.md) for more details on how to work with the Rancher docs.
- Verify if changes pertain to other versions of Rancher. If they do, finalize the edits on one version of the page, then apply the edits to the other versions.
- If the pull request is dependent on an upcoming release, remember to add a "MERGE ON RELEASE" label and set the proper milestone.
## Description
<!--
- What is the goal of this pull request?
- What did you change?
- Are there any other pull requests, tickets, or issues associated with this pull request?
-->
## Comments
<!--
Any additional notes a reviewer should know before we review.
-->
-1
View File
@@ -1 +0,0 @@
f288d286db3b7c65e02cb6635d0c048f0f7a88fa
-29
View File
@@ -1,29 +0,0 @@
name: Create issue to track porting between Community and Product docs
on:
pull_request_target:
types:
- closed
paths-ignore:
- '**/README.md'
permissions:
issues: write
pull-requests: read
jobs:
create_issue:
if: github.event.pull_request.merged == true && contains( github.event.pull_request.labels.*.name, 'port/community-product')
runs-on: ubuntu-latest
steps:
- name: Create issue
env:
GH_TOKEN: ${{ github.token }}
REPO_TYPE: ${{ contains( github.repository, 'product-docs') && 'Product' || 'Community' }}
PR_TITLE: ${{ github.event.pull_request.title }}
run: |
gh issue create \
--repo ${{ github.repository }} \
--title "Port $REPO_TYPE docs PR #${{ github.event.pull_request.number }}: $PR_TITLE" \
--body "Reference: https://github.com/${{ github.repository }}/pull/${{ github.event.pull_request.number }}" \
--label port/community-product
-52
View File
@@ -1,52 +0,0 @@
name: Deploy to GitHub Pages
on:
push:
branches:
- main
paths-ignore:
- '**/README.md'
- '**/.github/ISSUE_TEMPLATE/**'
jobs:
build:
name: Build Docusaurus
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 18
cache: yarn
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Build website
env:
NODE_OPTIONS: "--max_old_space_size=8192"
run: yarn build --no-minify
- name: Upload Build Artifact
uses: actions/upload-pages-artifact@v3
with:
path: build
deploy:
name: Deploy to GitHub Pages
needs: build
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
-31
View File
@@ -1,31 +0,0 @@
name: Test deployment
on:
pull_request:
paths-ignore:
- '**/README.md'
- '**/.github/ISSUE_TEMPLATE/**'
jobs:
test-deploy:
name: Test deployment
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 18
cache: yarn
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Check Markdown links
run: yarn run remark --quiet --frail --use remark-validate-links ./docs
- name: Check External links
run: yarn run remark --quiet --use remark-lint-no-dead-urls ./docs
- name: Test build website
env:
NODE_OPTIONS: "--max_old_space_size=7168"
run: yarn build --no-minify
-25
View File
@@ -1,25 +0,0 @@
# Dependencies
/node_modules
# Production
/build
# Generated files
.docusaurus
.cache-loader
# Misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.idea/
/dev-resources/algolia-search/.env
/dev-resources/algolia-search/algolia-index.sh
-3
View File
@@ -1,3 +0,0 @@
[submodule ".github/styles/suse-vale-styleguide"]
path = .github/styles/suse-vale-styleguide
url = https://github.com/openSUSE/suse-vale-styleguide
View File
-122
View File
@@ -1,122 +0,0 @@
---
kind: pipeline
name: pull-request
platform:
os: linux
arch: amd64
steps:
- name: test
pull: default
image: plugins/docker
settings:
context: .
dockerfile: Dockerfile.prod
dry_run: true
repo: rancher/docs
when:
event:
- pull_request
---
kind: pipeline
name: dev
platform:
os: linux
arch: amd64
steps:
- name: publish-dev
pull: default
image: plugins/docker
settings:
context: .
dockerfile: Dockerfile.dev
group: publish
repo: rancher/docs
tag: dev
password:
from_secret: docker_password
username:
from_secret: docker_username
when:
branch:
- master
event:
- push
---
kind: pipeline
name: staging
platform:
os: linux
arch: amd64
steps:
- name: publish-staging
pull: default
image: plugins/docker
settings:
context: .
dockerfile: Dockerfile.staging
group: publish
repo: rancher/docs
tag: staging
password:
from_secret: docker_password
username:
from_secret: docker_username
when:
branch:
- staging
event:
- push
---
kind: pipeline
name: production
platform:
os: linux
arch: amd64
steps:
- name: publish-latest
pull: default
image: plugins/docker
settings:
context: .
dockerfile: Dockerfile.prod
group: publish
repo: rancher/docs
tag: latest
password:
from_secret: docker_password
username:
from_secret: docker_username
when:
branch:
- master
event:
- push
- name: publish-algolia
pull: default
image: plugins/docker
settings:
context: .
dockerfile: Dockerfile.algolia
repo: rancher/docs
tag: algolia
password:
from_secret: docker_password
username:
from_secret: docker_username
when:
branch:
- master
event:
- push
-24
View File
@@ -1,24 +0,0 @@
options:
formatter: stylish
merge-default-rules: true
files:
include: 'src/sass/**/*.scss'
rules:
bem-depth: 1
brace-style: 0
class-name-format:
- 1
- convention: 'hyphenatedbem'
- allow-single-line: false
empty-args: 0
indentation:
- 1
- size: 4
mixin-name-format:
- 1
- convention: 'hyphenatedbem'
no-color-keywords: 0
no-color-literals: 0
property-sort-order: 0
no-ids: 0
one-declaration-per-line: 1
-7
View File
@@ -1,7 +0,0 @@
StylesPath = .github/styles/suse-vale-styleguide
[formtats]
mdx = md
[*.md]
BasedOnStyles = common
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/v2.7/reference-guides/rancher-security/hardening-guides/k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.24-k8s-v1.24">
<link rel="canonical" href="/v2.7/reference-guides/rancher-security/hardening-guides/k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.24-k8s-v1.24" />
</head>
<script>
window.location.href = '/v2.7/reference-guides/rancher-security/hardening-guides/k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.24-k8s-v1.24' + window.location.search + window.location.hash;
</script>
</html>
+30
View File
File diff suppressed because one or more lines are too long
View File
-1
View File
@@ -1 +0,0 @@
* @btat @LucasSaintarbor @pmkovar @sunilarjun
-178
View File
@@ -1,178 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
-108
View File
@@ -1,108 +0,0 @@
# Contribute to Rancher Docs
Welcome to the [Rancher docs](https://ranchermanager.docs.rancher.com/) repository. See the [Rancher software](https://github.com/rancher/rancher) repository if you have questions or requests for the Rancher platform.
## Make a Suggestion
You can suggest changes to the Rancher docs in two ways:
1. [Open an issue](https://github.com/rancher/rancher-docs/issues/new/choose).
1. Edit the docs in the way you see fit and open a pull request.
## Edit the Docs
To get started, [fork](https://github.com/rancher/rancher-docs/fork) and clone the rancher-docs repository.
Our repository doesn't allow you to make changes directly to the `main` branch. Create a working branch and make pull requests from your fork to [rancher/rancher-docs](https://github.com/rancher/rancher-docs).
For most updates, you'll need to edit a file in the `/docs` directory, which represents the ["Latest"](https://ranchermanager.docs.rancher.com/) version of our published documentation. The "Latest" version is a mirror of the most recently released version of Rancher. As of July 2025, the most recently released version of Rancher is 2.12.
Whenever an update is made to `/docs`, you should apply the same change to the corresponding file in `/versioned_docs/version-2.9`. If a change only affects older versions, you don't need to mirror it to the `/docs` directory.
If a file is moved or renamed, you'll also need to edit the `sidebars.js` files for each affected version, as well as the list of redirects in `docusaurus.config.js`. See [Moving or Renaming Docs](./moving-or-renaming-docs.md).
### Navigate the Repo
The file paths in the repo correspond to the URLs for pages on the docs website. The docs for the latest version of Rancher are located in `/docs`. All images are in `/static/img` in the top level of the repo. Older docs are found within `/versioned_docs` and generally follow the same structure as the files in `/docs`.
### Style & Formatting
The docs are written in [Markdown](https://www.markdownguide.org/getting-started/). We use standard American English and many pages are also available in Simplified Chinese.
Moving forward, we are referring to the SUSE [style guide](https://documentation.suse.com/style/current/pdf/style-guide_en.pdf). The **Style check / runner / vale (pull_request)** check used [Vale](https://vale.sh/) to make style and grammar suggestions for new or updated documentation based on the SUSE style guide. To review these suggestions when working on a PR:
1. Select the details of the **Style check / runner / vale (pull_request)** check.
1. In the logs, go to **Run errata-ai/vale-action@v2.1.0** and select **Running vale with reviewdog 🐶 ...** to view the suggestions.
1. New or updated files are checked against the SUSE style guide. Suggestions have the following format: '{"message": "[suse-vale-styleguide.Rule] Rule description", "location": {"path": "file-path", "range": {"start": {"line": , "column": }}}, "severity": " "}'
For example: '{"message": "[suse-vale-styleguide.Usage] Use 'certain' instead of 'some'", "location": {"path": "docs/contribute-to-rancher.md", "range": {"start": {"line": 3, "column": 132}}}, "severity": "WARNING"}'
1. Incorporate the suggestions when possible and appropriate.
Every docs page contain metadata in the first few lines:
```
---
title: Some Title
---
```
The `title` is rendered as the page's headline. The site renderer wraps the `title` value in `H1` tags, which are equivalent to `#` in Markdown syntax. This means that all subsequent headers on the page should be second level (`##`) or more.
## Run the Docs Website
The Rancher Docs website is built with [Docusaurus 2](https://docusaurus.io/), a modern static website generator.
You can run the site on your local machine, to preview how pages on your working branch will look live.
First, install Docusaurus 2:
1. If you haven't already, install [Node](https://nodejs.org/en/download/) and [Yarn](https://yarnpkg.com/getting-started/install).
1. Go into your local rancher-docs folder.
1. The Rancher Docs repository already contains a yarn.lock file, which contains the dependencies you need to build the website. Run `yarn` to install Docusaurus and associated dependencies.
### Start Site
```
yarn start
```
This command starts a local development server for Docusaurus 2, and opens up a browser window. Most changes are reflected live without having to restart the server.
**Note:** The `yarn start` command won't include some important static site features. For example, switching between languages from the site's dropdown menu is not available. If you need these features, use `yarn build`.
### Build Site
```
yarn build
```
This command generates static content into the `build` directory and can be served using any static contents hosting service.
### Launch With Docker
You can also use [Docker](https://www.docker.com/) to launch the website.
The below command can be used to install the dependencies and run the site inside a container:
```
docker run --rm -it -v $PWD:$PWD -w $PWD -p 3000:3000 node:18 /bin/sh -c "yarn install && yarn start -h 0.0.0.0"
```
Subsequent executions will check for updated dependencies, if there are none, it will skip the updates and quickly start the server.
License
=======
Copyright (c) 2014-2026 [SUSE, LLC.](https://www.suse.com/)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/api-reference">
<link rel="canonical" href="/api/api-reference" />
</head>
<script>
window.location.href = '/api/api-reference' + window.location.search + window.location.hash;
</script>
</html>
File diff suppressed because one or more lines are too long
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/api-reference">
<link rel="canonical" href="/api/api-reference" />
</head>
<script>
window.location.href = '/api/api-reference' + window.location.search + window.location.hash;
</script>
</html>
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/api-tokens">
<link rel="canonical" href="/api/api-tokens" />
</head>
<script>
window.location.href = '/api/api-tokens' + window.location.search + window.location.hash;
</script>
</html>
File diff suppressed because one or more lines are too long
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/api-tokens">
<link rel="canonical" href="/api/api-tokens" />
</head>
<script>
window.location.href = '/api/api-tokens' + window.location.search + window.location.hash;
</script>
</html>
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/quickstart">
<link rel="canonical" href="/api/quickstart" />
</head>
<script>
window.location.href = '/api/quickstart' + window.location.search + window.location.hash;
</script>
</html>
File diff suppressed because one or more lines are too long
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/quickstart">
<link rel="canonical" href="/api/quickstart" />
</head>
<script>
window.location.href = '/api/quickstart' + window.location.search + window.location.hash;
</script>
</html>
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/v3-rancher-api-guide">
<link rel="canonical" href="/api/v3-rancher-api-guide" />
</head>
<script>
window.location.href = '/api/v3-rancher-api-guide' + window.location.search + window.location.hash;
</script>
</html>
File diff suppressed because one or more lines are too long
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/v3-rancher-api-guide">
<link rel="canonical" href="/api/v3-rancher-api-guide" />
</head>
<script>
window.location.href = '/api/v3-rancher-api-guide' + window.location.search + window.location.hash;
</script>
</html>
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/workflows/projects">
<link rel="canonical" href="/api/workflows/projects" />
</head>
<script>
window.location.href = '/api/workflows/projects' + window.location.search + window.location.hash;
</script>
</html>
File diff suppressed because one or more lines are too long
+11
View File
@@ -0,0 +1,11 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta http-equiv="refresh" content="0; url=/api/workflows/projects">
<link rel="canonical" href="/api/workflows/projects" />
</head>
<script>
window.location.href = '/api/workflows/projects' + window.location.search + window.location.hash;
</script>
</html>
@@ -1,140 +0,0 @@
---
title: Contributing to Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/contribute-to-rancher"/>
</head>
Learn about the repositories used for Rancher and Rancher docs, how to build Rancher repositories, and what information to include when you file an issue.
For more detailed information on how to contribute to the development of Rancher projects, refer to the [Rancher Developer Wiki](https://github.com/rancher/rancher/wiki). The wiki has resources on many topics, including the following:
- How to set up the Rancher development environment and run tests
- The typical flow of an issue through the development lifecycle
- Coding guidelines and development best practices
- Debugging and troubleshooting
- Developing the Rancher API
On the Rancher Users Slack, the channel for developers is **#developer**.
## Rancher Docs
If you have suggestions for the documentation on this website, [open](https://github.com/rancher/rancher-docs/issues/new/choose) an issue in the main [Rancher docs](https://github.com/rancher/rancher-docs) repository. This repo contains documentation for Rancher v2.0 and later.
See the [Rancher docs README](https://github.com/rancher/rancher-docs#readme) for more details on contributing to and building the Rancher v2.x docs repo.
For documentation describing Rancher v1.6 and earlier, see the [Rancher 1.x docs](https://github.com/rancher/rancher.github.io) repo, which contains source files for https://rancher.com/docs/rancher/v1.6/en/.
## Rancher Repositories
All of repositories are located within our main GitHub organization. There are many repositories used for Rancher, but we'll provide descriptions of some of the main ones used in Rancher.
Repository | URL | Description
-----------|-----|-------------
Rancher | https://github.com/rancher/rancher | This repository is the main source code for Rancher 2.x.
Types | https://github.com/rancher/types | This repository is the repository that has all the API types for Rancher 2.x.
API Framework | https://github.com/rancher/norman | This repository is an API framework for building Rancher style APIs backed by Kubernetes Custom Resources.
User Interface | https://github.com/rancher/dashboard/ | This repository is the source of the Dashboard UI.
(Rancher) Docker Machine | https://github.com/rancher/machine | This repository is the source of the Docker Machine binary used when using Node Drivers. This is a fork of the `docker/machine` repository.
machine-package | https://github.com/rancher/machine-package | This repository is used to build the Rancher Docker Machine binary.
kontainer-engine | https://github.com/rancher/kontainer-engine | This repository is the source of kontainer-engine, the tool to provision hosted Kubernetes clusters.
RKE repository | https://github.com/rancher/rke | This repository is the source of Rancher Kubernetes Engine, the tool to provision Kubernetes clusters on any machine.
CLI | https://github.com/rancher/cli | This repository is the source code for the Rancher CLI used in Rancher 2.x.
(Rancher) Helm repository | https://github.com/rancher/helm | This repository is the source of the packaged Helm binary. This is a fork of the `helm/helm` repository.
Telemetry repository | https://github.com/rancher/telemetry | This repository is the source for the Telemetry binary.
loglevel repository | https://github.com/rancher/loglevel | This repository is the source of the loglevel binary, used to dynamically change log levels.
To see all libraries/projects used in Rancher, see the [`go.mod` file](https://github.com/rancher/rancher/blob/master/go.mod) in the `rancher/rancher` repository.
![Rancher diagram](/img/ranchercomponentsdiagram-2.6.svg)<br/>
<sup>Rancher components used for provisioning/managing Kubernetes clusters.</sup>
### Building Rancher Repositories
Every repository should have a Makefile and can be built using the `make` command. The `make` targets are based on the scripts in the `/scripts` directory in the repository, and each target will use [Dapper](https://github.com/rancher/dapper) to run the target in an isolated environment. The `Dockerfile.dapper` will be used for this process, and includes all the necessary build tooling needed.
The default target is `ci`, and will run `./scripts/validate`, `./scripts/build`, `./scripts/test` and `./scripts/package`. The resulting binaries of the build will be in `./build/bin` and are usually also packaged in a Docker image.
### Rancher Bugs, Issues or Questions
If you find any bugs or are having any trouble, please search the [reported issue](https://github.com/rancher/rancher/issues) as someone may have experienced the same issue or we are actively working on a solution.
If you can't find anything related to your issue, contact us by [filing an issue](https://github.com/rancher/rancher/issues/new). Though we have many repositories related to Rancher, we want the bugs filed in the Rancher repository so we won't miss them! If you want to ask a question or ask fellow users about an use case, we suggest creating a post on the [Rancher Forums](https://forums.rancher.com).
#### Checklist for Filing Issues
Please follow this checklist when filing an issue which will helps us investigate and fix the issue. More info means more data we can use to determine what is causing the issue or what might be related to the issue.
:::note
For large amounts of data, please use [GitHub Gist](https://gist.github.com/) or similar and link the created resource in the issue.
:::
:::note Important:
Please remove any sensitive data as it will be publicly viewable.
:::
- **Resources:** Provide as much as detail as possible on the used resources. As the source of the issue can be many things, including as much of detail as possible helps to determine the root cause. See some examples below:
- **Hosts:** What specifications does the host have, like CPU/memory/disk, what cloud does it happen on, what Amazon Machine Image are you using, what DigitalOcean droplet are you using, what image are you provisioning that we can rebuild or use when we try to reproduce
- **Operating System:** What operating system are you using? Providing specifics helps here like the output of `cat /etc/os-release` for exact OS release and `uname -r` for exact kernel used
- **Docker:** What Docker version are you using, how did you install it? Most of the details of Docker can be found by supplying output of `docker version` and `docker info`
- **Environment:** Are you in a proxy environment, are you using recognized CA/self signed certificates, are you using an external loadbalancer
- **Rancher:** What version of Rancher are you using, this can be found on the bottom left of the UI or be retrieved from the image tag you are running on the host
- **Clusters:** What kind of cluster did you create, how did you create it, what did you specify when you were creating it
- **Steps to reproduce the issue:** Provide as much detail on how you got into the reported situation. This helps the person to reproduce the situation you are in.
- Provide manual steps or automation scripts used to get from a newly created setup to the situation you reported.
- **Logs:** Provide data/logs from the used resources.
- Rancher
- Docker install
```
docker logs \
--timestamps \
$(docker ps | grep -E "rancher/rancher:|rancher/rancher " | awk '{ print $1 }')
```
- Kubernetes install using `kubectl`
:::note
Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG=$PWD/kube_config_cluster.yml` if Rancher is installed on a Kubernetes cluster) or are using the embedded kubectl via the UI.
:::
```
kubectl -n cattle-system \
logs \
-l app=rancher \
--timestamps=true
```
- Docker install using `docker` on each of the nodes in the RKE cluster
```
docker logs \
--timestamps \
$(docker ps | grep -E "rancher/rancher@|rancher_rancher" | awk '{ print $1 }')
```
- Kubernetes Install with RKE Add-On
:::note
Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG=$PWD/kube_config_cluster.yml` if the Rancher server is installed on a Kubernetes cluster) or are using the embedded kubectl via the UI.
:::
```
kubectl -n cattle-system \
logs \
--timestamps=true \
-f $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name="cattle-server") | .metadata.name')
```
- System logging (these might not all exist, depending on operating system)
- `/var/log/messages`
- `/var/log/syslog`
- `/var/log/kern.log`
- Docker daemon logging (these might not all exist, depending on operating system)
- `/var/log/docker.log`
- **Metrics:** If you are experiencing performance issues, please provide as much of data (files or screenshots) of metrics which can help determining what is going on. If you have an issue related to a machine, it helps to supply output of `top`, `free -m`, `df` which shows processes/memory/disk usage.
@@ -1,5 +0,0 @@
---
title: Explanations
---
**Explanatory docs** are concerned primarily with providing theoretical knowledge for the "why" behind a task or a topic. Explanations are "understanding-oriented" in nature and will clarify a topic in order to broaden the user's knowledge. In this section, users can find additional context and background, alternatives or even opinions on topics, and often historical reasons, constraints, and insights into why a process works the way that it does.
@@ -1,26 +0,0 @@
---
title: CIS Scans
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/cis-scans"/>
</head>
_Available as of v2.4.0_
- [Prerequisites](#prerequisites)
- [How-to Guides](#how-to-guides)
## Prerequisites
To run security scans on a cluster and access the generated reports, you must be an [Administrator](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md) or [Cluster Owner.](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md)
Rancher can only run security scans on clusters that were created with RKE, which includes custom clusters and clusters that Rancher created in an infrastructure provider such as Amazon EC2 or GCE. Imported clusters and clusters in hosted Kubernetes providers can't be scanned by Rancher.
The security scan cannot run in a cluster that has Windows nodes.
You will only be able to see the CIS scan reports for clusters that you have access to.
## How-to Guides
Please refer [here](../../../how-to-guides/advanced-user-guides/cis-scan-guides/cis-scan-guides.md) for how-to guides on CIS scans.
@@ -1,106 +0,0 @@
---
title: Skipped and Not Applicable Tests
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/cis-scans/skipped-and-not-applicable-tests"/>
</head>
This section lists the tests that are skipped in the permissive test profile for RKE.
All the tests that are skipped and not applicable on this page will be counted as Not Applicable in the generated report. The skipped test count will only mention the user-defined skipped tests. This allows user-skipped tests to be distinguished from the tests that are skipped by default in the RKE permissive test profile.
## CIS Benchmark v1.5
### CIS Benchmark v1.5 Skipped Tests
| Number | Description | Reason for Skipping |
| ---------- | ------------- | --------- |
| 1.1.12 | Ensure that the etcd data directory ownership is set to etcd:etcd (Scored) | A system service account is required for etcd data directory ownership. Refer to Rancher's hardening guide for more details on how to configure this ownership. |
| 1.2.6 | Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored) | When generating serving certificates, functionality could break in conjunction with hostname overrides which are required for certain cloud providers. |
| 1.2.16 | Ensure that the admission control plugin PodSecurityPolicy is set (Scored) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 1.2.33 | Ensure that the --encryption-provider-config argument is set as appropriate (Not Scored) | Enabling encryption changes how data can be recovered as data is encrypted. |
| 1.2.34 | Ensure that encryption providers are appropriately configured (Not Scored) | Enabling encryption changes how data can be recovered as data is encrypted. |
| 4.2.6 | Ensure that the --protect-kernel-defaults argument is set to true (Scored) | System level configurations are required before provisioning the cluster in order for this argument to be set to true. |
| 4.2.10 | Ensure that the--tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored) | When generating serving certificates, functionality could break in conjunction with hostname overrides which are required for certain cloud providers. |
| 5.1.5 | Ensure that default service accounts are not actively used. (Scored) | Kubernetes provides default service accounts to be used. |
| 5.2.2 | Minimize the admission of containers wishing to share the host process ID namespace (Scored) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 5.2.3 | Minimize the admission of containers wishing to share the host IPC namespace (Scored) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 5.2.4 | Minimize the admission of containers wishing to share the host network namespace (Scored) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 5.2.5 | Minimize the admission of containers with allowPrivilegeEscalation (Scored) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 5.3.2 | Ensure that all Namespaces have Network Policies defined (Scored) | Enabling Network Policies can prevent certain applications from communicating with each other. |
| 5.6.4 | The default namespace should not be used (Scored) | Kubernetes provides a default namespace. |
### CIS Benchmark v1.5 Not Applicable Tests
| Number | Description | Reason for being not applicable |
| ---------- | ------------- | --------- |
| 1.1.1 | Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for kube-apiserver. All configuration is passed in as arguments at container run time. |
| 1.1.2 | Ensure that the API server pod specification file ownership is set to root:root (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for kube-apiserver. All configuration is passed in as arguments at container run time. |
| 1.1.3 | Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for controller-manager. All configuration is passed in as arguments at container run time. |
| 1.1.4 | Ensure that the controller manager pod specification file ownership is set to root:root (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for controller-manager. All configuration is passed in as arguments at container run time. |
| 1.1.5 | Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for scheduler. All configuration is passed in as arguments at container run time. |
| 1.1.6 | Ensure that the scheduler pod specification file ownership is set to root:root (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for scheduler. All configuration is passed in as arguments at container run time. |
| 1.1.7 | Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for etcd. All configuration is passed in as arguments at container run time. |
| 1.1.8 | Ensure that the etcd pod specification file ownership is set to root:root (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for etcd. All configuration is passed in as arguments at container run time. |
| 1.1.13 | Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored) | Clusters provisioned by RKE does not store the kubernetes default kubeconfig credentials file on the nodes. |
| 1.1.14 | Ensure that the admin.conf file ownership is set to root:root (Scored) | Clusters provisioned by RKE does not store the kubernetes default kubeconfig credentials file on the nodes. |
| 1.1.15 | Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for scheduler. All configuration is passed in as arguments at container run time. |
| 1.1.16 | Ensure that the scheduler.conf file ownership is set to root:root (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for scheduler. All configuration is passed in as arguments at container run time. |
| 1.1.17 | Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for controller-manager. All configuration is passed in as arguments at container run time. |
| 1.1.18 | Ensure that the controller-manager.conf file ownership is set to root:root (Scored) | Clusters provisioned by RKE doesn't require or maintain a configuration file for controller-manager. All configuration is passed in as arguments at container run time. |
| 1.3.6 | Ensure that the RotateKubeletServerCertificate argument is set to true (Scored) | Clusters provisioned by RKE handles certificate rotation directly through RKE. |
| 4.1.1 | Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored) | Clusters provisioned by RKE doesn’t require or maintain a configuration file for the kubelet service. All configuration is passed in as arguments at container run time. |
| 4.1.2 | Ensure that the kubelet service file ownership is set to root:root (Scored) | Clusters provisioned by RKE doesn’t require or maintain a configuration file for the kubelet service. All configuration is passed in as arguments at container run time. |
| 4.1.9 | Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored) | Clusters provisioned by RKE doesn’t require or maintain a configuration file for the kubelet. All configuration is passed in as arguments at container run time. |
| 4.1.10 | Ensure that the kubelet configuration file ownership is set to root:root (Scored) | Clusters provisioned by RKE doesn’t require or maintain a configuration file for the kubelet. All configuration is passed in as arguments at container run time. |
| 4.2.12 | Ensure that the RotateKubeletServerCertificate argument is set to true (Scored) | Clusters provisioned by RKE handles certificate rotation directly through RKE. |
## CIS Benchmark v1.4
The skipped and not applicable tests for CIS Benchmark v1.4 are as follows:
### CIS Benchmark v1.4 Skipped Tests
Number | Description | Reason for Skipping
---|---|---
1.1.11 | "Ensure that the admission control plugin AlwaysPullImages is set (Scored)" | Enabling AlwaysPullImages can use significant bandwidth.
1.1.21 | "Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)" | When generating serving certificates, functionality could break in conjunction with hostname overrides which are required for certain cloud providers.
1.1.24 | "Ensure that the admission control plugin PodSecurityPolicy is set (Scored)" | Enabling Pod Security Policy can cause applications to unexpectedly fail.
1.1.34 | "Ensure that the --encryption-provider-config argument is set as appropriate (Scored)" | Enabling encryption changes how data can be recovered as data is encrypted.
1.1.35 | "Ensure that the encryption provider is set to aescbc (Scored)" | Enabling encryption changes how data can be recovered as data is encrypted.
1.1.36 | "Ensure that the admission control plugin EventRateLimit is set (Scored)" | EventRateLimit needs to be tuned depending on the cluster.
1.2.2 | "Ensure that the --address argument is set to 127.0.0.1 (Scored)" | Adding this argument prevents Rancher's monitoring tool to collect metrics on the scheduler.
1.3.7 | "Ensure that the --address argument is set to 127.0.0.1 (Scored)" | Adding this argument prevents Rancher's monitoring tool to collect metrics on the controller manager.
1.4.12 | "Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)" | A system service account is required for etcd data directory ownership. Refer to Rancher's hardening guide for more details on how to configure this ownership.
1.7.2 | "Do not admit containers wishing to share the host process ID namespace (Scored)" | Enabling Pod Security Policy can cause applications to unexpectedly fail.
1.7.3 | "Do not admit containers wishing to share the host IPC namespace (Scored)" | Enabling Pod Security Policy can cause applications to unexpectedly fail.
1.7.4 | "Do not admit containers wishing to share the host network namespace (Scored)" | Enabling Pod Security Policy can cause applications to unexpectedly fail.
1.7.5 | " Do not admit containers with allowPrivilegeEscalation (Scored)" | Enabling Pod Security Policy can cause applications to unexpectedly fail.
2.1.6 | "Ensure that the --protect-kernel-defaults argument is set to true (Scored)" | System level configurations are required before provisioning the cluster in order for this argument to be set to true.
2.1.10 | "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)" | When generating serving certificates, functionality could break in conjunction with hostname overrides which are required for certain cloud providers.
### CIS Benchmark v1.4 Not Applicable Tests
Number | Description | Reason for being not applicable
---|---|---
1.1.9 | "Ensure that the --repair-malformed-updates argument is set to false (Scored)" | The argument --repair-malformed-updates has been removed as of Kubernetes version 1.14
1.3.6 | "Ensure that the RotateKubeletServerCertificate argument is set to true" | Cluster provisioned by RKE handles certificate rotation directly through RKE.
1.4.1 | "Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)" | Cluster provisioned by RKE doesn't require or maintain a configuration file for kube-apiserver.
1.4.2 | "Ensure that the API server pod specification file ownership is set to root:root (Scored)" | Cluster provisioned by RKE doesn't require or maintain a configuration file for kube-apiserver.
1.4.3 | "Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)" | Cluster provisioned by RKE doesn't require or maintain a configuration file for controller-manager.
1.4.4 | "Ensure that the controller manager pod specification file ownership is set to root:root (Scored)" | Cluster provisioned by RKE doesn't require or maintain a configuration file for controller-manager.
1.4.5 | "Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)" | Cluster provisioned by RKE doesn't require or maintain a configuration file for scheduler.
1.4.6 | "Ensure that the scheduler pod specification file ownership is set to root:root (Scored)" | Cluster provisioned by RKE doesn't require or maintain a configuration file for scheduler.
1.4.7 | "Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)" | Cluster provisioned by RKE doesn't require or maintain a configuration file for etcd.
1.4.8 | "Ensure that the etcd pod specification file ownership is set to root:root (Scored)" | Cluster provisioned by RKE doesn't require or maintain a configuration file for etcd.
1.4.13 | "Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)" | Cluster provisioned by RKE does not store the kubernetes default kubeconfig credentials file on the nodes.
1.4.14 | "Ensure that the admin.conf file ownership is set to root:root (Scored)" | Cluster provisioned by RKE does not store the kubernetes default kubeconfig credentials file on the nodes.
2.1.8 | "Ensure that the --hostname-override argument is not set (Scored)" | Clusters provisioned by RKE clusters and most cloud providers require hostnames.
2.1.12 | "Ensure that the --rotate-certificates argument is not set to false (Scored)" | Cluster provisioned by RKE handles certificate rotation directly through RKE.
2.1.13 | "Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)" | Cluster provisioned by RKE handles certificate rotation directly through RKE.
2.2.3 | "Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)" | Cluster provisioned by RKE doesn’t require or maintain a configuration file for the kubelet service.
2.2.4 | "Ensure that the kubelet service file ownership is set to root:root (Scored)" | Cluster provisioned by RKE doesn’t require or maintain a configuration file for the kubelet service.
2.2.9 | "Ensure that the kubelet configuration file ownership is set to root:root (Scored)" | RKE doesn’t require or maintain a configuration file for the kubelet.
2.2.10 | "Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)" | RKE doesn’t require or maintain a configuration file for the kubelet.
@@ -1,328 +0,0 @@
---
title: Cluster Alerts
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/pages-for-subheaders/monitoring-and-alerting"/>
</head>
To keep your clusters and applications healthy and driving your organizational productivity forward, you need to stay informed of events occurring in your clusters and projects, both planned and unplanned. When an event occurs, your alert is triggered, and you are sent a notification. You can then, if necessary, follow up with corrective actions.
## About Alerts
Notifiers and alerts are built on top of the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/alertmanager/). Leveraging these tools, Rancher can notify [cluster owners](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#cluster-roles) and [project owners](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#project-roles) of events they need to address.
Before you can receive alerts, you must configure one or more notifier in Rancher.
When you create a cluster, some alert rules are predefined. You can receive these alerts if you configure a [notifier](../notifiers.md) for them.
For details about what triggers the predefined alerts, refer to the [documentation on default alerts.](default-alerts.md)
### Alert Event Examples
Some examples of alert events are:
- A Kubernetes master component entering an unhealthy state.
- A node or workload error occurring.
- A scheduled deployment taking place as planned.
- A node's hardware resources becoming overstressed.
### Alerts Triggered by Prometheus Queries
When you edit an alert rule, you will have the opportunity to configure the alert to be triggered based on a Prometheus expression. For examples of expressions, refer to [this page.](../cluster-monitoring/expression.md)
Monitoring must be [enabled](../cluster-monitoring/cluster-monitoring.md) before you can trigger alerts with custom Prometheus queries or expressions.
### Urgency Levels
You can set an urgency level for each alert. This urgency appears in the notification you receive, helping you to prioritize your response actions. For example, if you have an alert configured to inform you of a routine deployment, no action is required. These alerts can be assigned a low priority level. However, if a deployment fails, it can critically impact your organization, and you need to react quickly. Assign these alerts a high priority level.
### Scope of Alerts
The scope for alerts can be set at either the cluster level or [project level](../../../reference-guides/rancher-project-tools/project-alerts.md).
At the cluster level, Rancher monitors components in your Kubernetes cluster, and sends you alerts related to:
- The state of your nodes.
- The system services that manage your Kubernetes cluster.
- The resource events from specific system services.
- The Prometheus expression cross the thresholds
### Managing Cluster Alerts
After you set up cluster alerts, you can manage each alert object. To manage alerts, browse to the cluster containing the alerts, and then select **Tools > Alerts** that you want to manage. You can:
- Deactivate/Reactive alerts
- Edit alert settings
- Delete unnecessary alerts
- Mute firing alerts
- Unmute muted alerts
## Adding Cluster Alerts
As a [cluster owner](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#cluster-roles), you can configure Rancher to send you alerts for cluster events.
>**Prerequisite:** Before you can receive cluster alerts, you must [add a notifier](../notifiers.md).
1. From the **Global** view, navigate to the cluster that you want to configure cluster alerts for. Select **Tools > Alerts**. Then click **Add Alert Group**.
1. Enter a **Name** for the alert that describes its purpose, you could group alert rules for the different purpose.
1. Based on the type of alert you want to create, refer to the [cluster alert configuration section.](#cluster-alert-configuration)
1. Continue adding more **Alert Rule** to the group.
1. Finally, choose the [notifiers](../notifiers.md) to send the alerts to.
- You can set up multiple notifiers.
- You can change notifier recipients on the fly.
1. Click **Create.**
**Result:** Your alert is configured. A notification is sent when the alert is triggered.
## Cluster Alert Configuration
- [System Service Alerts](#system-service-alerts)
- [Resource Event Alerts](#resource-event-alerts)
- [Node Alerts](#node-alerts)
- [Node Selector Alerts](#node-selector-alerts)
- [CIS Scan Alerts](#cis-scan-alerts)
- [Metric Expression Alerts](#metric-expression-alerts)
## System Service Alerts
This alert type monitor for events that affect one of the Kubernetes master components, regardless of the node it occurs on.
Each of the below sections corresponds to a part of the alert rule configuration section in the Rancher UI.
### When a
Select the **System Services** option, and then select an option from the dropdown:
- [controller-manager](https://kubernetes.io/docs/concepts/overview/components/#kube-controller-manager)
- [etcd](https://kubernetes.io/docs/concepts/overview/components/#etcd)
- [scheduler](https://kubernetes.io/docs/concepts/overview/components/#kube-scheduler)
### Is
The alert will be triggered when the selected Kubernetes master component is unhealthy.
### Send a
Select the urgency level of the alert. The options are:
- **Critical**: Most urgent
- **Warning**: Normal urgency
- **Info**: Least urgent
Select the urgency level based on the importance of the service and how many nodes fill the role within your cluster. For example, if you're making an alert for the `etcd` service, select **Critical**. If you're making an alert for redundant schedulers, **Warning** is more appropriate.
### Advanced Options
By default, the below options will apply to all alert rules within the group. You can disable these advanced options when configuring a specific rule.
- **Group Wait Time**: How long to wait to buffer alerts of the same group before sending initially, default to 30 seconds.
- **Group Interval Time**: How long to wait before sending an alert that has been added to a group which contains already fired alerts, default to 30 seconds.
- **Repeat Wait Time**: How long to wait before re-sending a given alert that has already been sent, default to 1 hour.
## Resource Event Alerts
This alert type monitors for specific events that are thrown from a resource type.
Each of the below sections corresponds to a part of the alert rule configuration section in the Rancher UI.
### When a
Choose the type of resource event that triggers an alert. The options are:
- **Normal**: triggers an alert when any standard resource event occurs.
- **Warning**: triggers an alert when unexpected resource events occur.
Select a resource type from the **Choose a Resource** drop-down that you want to trigger an alert.
- [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/)
- [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/)
- [Node](https://kubernetes.io/docs/concepts/architecture/nodes/)
- [Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod/)
- [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/)
### Send a
Select the urgency level of the alert.
- **Critical**: Most urgent
- **Warning**: Normal urgency
- **Info**: Least urgent
Select the urgency level of the alert by considering factors such as how often the event occurs or its importance. For example:
- If you set a normal alert for pods, you're likely to receive alerts often, and individual pods usually self-heal, so select an urgency of **Info**.
- If you set a warning alert for StatefulSets, it's very likely to impact operations, so select an urgency of **Critical**.
### Advanced Options
By default, the below options will apply to all alert rules within the group. You can disable these advanced options when configuring a specific rule.
- **Group Wait Time**: How long to wait to buffer alerts of the same group before sending initially, default to 30 seconds.
- **Group Interval Time**: How long to wait before sending an alert that has been added to a group which contains already fired alerts, default to 30 seconds.
- **Repeat Wait Time**: How long to wait before re-sending a given alert that has already been sent, default to 1 hour.
## Node Alerts
This alert type monitors for events that occur on a specific node.
Each of the below sections corresponds to a part of the alert rule configuration section in the Rancher UI.
### When a
Select the **Node** option, and then make a selection from the **Choose a Node** drop-down.
### Is
Choose an event to trigger the alert.
- **Not Ready**: Sends you an alert when the node is unresponsive.
- **CPU usage over**: Sends you an alert when the node raises above an entered percentage of its processing allocation.
- **Mem usage over**: Sends you an alert when the node raises above an entered percentage of its memory allocation.
### Send a
Select the urgency level of the alert.
- **Critical**: Most urgent
- **Warning**: Normal urgency
- **Info**: Least urgent
Select the urgency level of the alert based on its impact on operations. For example, an alert triggered when a node's CPU raises above 60% deems an urgency of **Info**, but a node that is **Not Ready** deems an urgency of **Critical**.
### Advanced Options
By default, the below options will apply to all alert rules within the group. You can disable these advanced options when configuring a specific rule.
- **Group Wait Time**: How long to wait to buffer alerts of the same group before sending initially, default to 30 seconds.
- **Group Interval Time**: How long to wait before sending an alert that has been added to a group which contains already fired alerts, default to 30 seconds.
- **Repeat Wait Time**: How long to wait before re-sending a given alert that has already been sent, default to 1 hour.
## Node Selector Alerts
This alert type monitors for events that occur on any node on marked with a label. For more information, see the Kubernetes documentation for [Labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).
Each of the below sections corresponds to a part of the alert rule configuration section in the Rancher UI.
### When a
Select the **Node Selector** option, and then click **Add Selector** to enter a key value pair for a label. This label should be applied to one or more of your nodes. Add as many selectors as you'd like.
### Is
Choose an event to trigger the alert.
- **Not Ready**: Sends you an alert when selected nodes are unresponsive.
- **CPU usage over**: Sends you an alert when selected nodes raise above an entered percentage of processing allocation.
- **Mem usage over**: Sends you an alert when selected nodes raise above an entered percentage of memory allocation.
### Send a
Select the urgency level of the alert.
- **Critical**: Most urgent
- **Warning**: Normal urgency
- **Info**: Least urgent
Select the urgency level of the alert based on its impact on operations. For example, an alert triggered when a node's CPU raises above 60% deems an urgency of **Info**, but a node that is **Not Ready** deems an urgency of **Critical**.
### Advanced Options
By default, the below options will apply to all alert rules within the group. You can disable these advanced options when configuring a specific rule.
- **Group Wait Time**: How long to wait to buffer alerts of the same group before sending initially, default to 30 seconds.
- **Group Interval Time**: How long to wait before sending an alert that has been added to a group which contains already fired alerts, default to 30 seconds.
- **Repeat Wait Time**: How long to wait before re-sending a given alert that has already been sent, default to 1 hour.
## CIS Scan Alerts
_Available as of v2.4.0_
This alert type is triggered based on the results of a CIS scan.
Each of the below sections corresponds to a part of the alert rule configuration section in the Rancher UI.
### When a
Select **CIS Scan.**
### Is
Choose an event to trigger the alert:
- Completed Scan
- Has Failure
### Send a
Select the urgency level of the alert.
- **Critical**: Most urgent
- **Warning**: Normal urgency
- **Info**: Least urgent
Select the urgency level of the alert based on its impact on operations. For example, an alert triggered when a node's CPU raises above 60% deems an urgency of **Info**, but a node that is **Not Ready** deems an urgency of **Critical**.
### Advanced Options
By default, the below options will apply to all alert rules within the group. You can disable these advanced options when configuring a specific rule.
- **Group Wait Time**: How long to wait to buffer alerts of the same group before sending initially, default to 30 seconds.
- **Group Interval Time**: How long to wait before sending an alert that has been added to a group which contains already fired alerts, default to 30 seconds.
- **Repeat Wait Time**: How long to wait before re-sending a given alert that has already been sent, default to 1 hour.
## Metric Expression Alerts
This alert type monitors for the overload from Prometheus expression querying, it would be available after you enable monitoring.
Each of the below sections corresponds to a part of the alert rule configuration section in the Rancher UI.
### When a
Input or select an **Expression**, the dropdown shows the original metrics from Prometheus, including:
- [**Node**](https://github.com/prometheus/node_exporter)
- [**Container**](https://github.com/google/cadvisor)
- [**ETCD**](https://etcd.io/docs/v3.3/op-guide/monitoring/)
- [**Kubernetes Components**](https://github.com/kubernetes/metrics)
- [**Kubernetes Resources**](https://github.com/kubernetes/kube-state-metrics)
- [**Fluentd**](https://docs.fluentd.org/v1.0/articles/monitoring-prometheus) (supported by [Logging](../cluster-logging/cluster-logging.md))
- [**Cluster Level Grafana**](https://grafana.com/docs/grafana/latest/setup-grafana/set-up-grafana-monitoring/)
- **Cluster Level Prometheus**
### Is
Choose a comparison:
- **Equal**: Trigger alert when expression value equal to the threshold.
- **Not Equal**: Trigger alert when expression value not equal to the threshold.
- **Greater Than**: Trigger alert when expression value greater than to threshold.
- **Less Than**: Trigger alert when expression value equal or less than the threshold.
- **Greater or Equal**: Trigger alert when expression value greater to equal to the threshold.
- **Less or Equal**: Trigger alert when expression value less or equal to the threshold.
If applicable, choose a comparison value or a threshold for the alert to be triggered.
### For
Select a duration for a trigger alert when the expression value crosses the threshold longer than the configured duration.
### Send a
Select the urgency level of the alert.
- **Critical**: Most urgent
- **Warning**: Normal urgency
- **Info**: Least urgent
Select the urgency level of the alert based on its impact on operations. For example, an alert triggered when a node's load expression ```sum(node_load5) / count(node_cpu_seconds_total{mode="system"})``` raises above 0.6 deems an urgency of **Info**, but 1 deems an urgency of **Critical**.
### Advanced Options
By default, the below options will apply to all alert rules within the group. You can disable these advanced options when configuring a specific rule.
- **Group Wait Time**: How long to wait to buffer alerts of the same group before sending initially, default to 30 seconds.
- **Group Interval Time**: How long to wait before sending an alert that has been added to a group which contains already fired alerts, default to 30 seconds.
- **Repeat Wait Time**: How long to wait before re-sending a given alert that has already been sent, default to 1 hour.
@@ -1,56 +0,0 @@
---
title: Default Alerts for Cluster Monitoring
---
When you create a cluster, some alert rules are predefined. These alerts notify you about signs that the cluster could be unhealthy. You can receive these alerts if you configure a [notifier](../notifiers.md) for them.
Several of the alerts use Prometheus expressions as the metric that triggers the alert. For more information on how expressions work, you can refer to the Rancher [documentation about Prometheus expressions](../cluster-monitoring/expression.md) or the Prometheus [documentation about querying metrics](https://prometheus.io/docs/prometheus/latest/querying/basics/).
## Alerts for etcd
Etcd is the key-value store that contains the state of the Kubernetes cluster. Rancher provides default alerts if the built-in monitoring detects a potential problem with etcd. You don't have to enable monitoring to receive these alerts.
A leader is the node that handles all client requests that need cluster consensus. For more information, you can refer to this [explanation of how etcd works.](https://www.rancher.cn/blog/2019/2019-01-29-what-is-etcd/#how-does-etcd-work)
The leader of the cluster can change in response to certain events. It is normal for the leader to change, but too many changes can indicate a problem with the network or a high CPU load. With longer latencies, the default etcd configuration may cause frequent heartbeat timeouts, which trigger a new leader election.
| Alert | Explanation |
|-------|-------------|
| A high number of leader changes within the etcd cluster are happening | A warning alert is triggered when the leader changes more than three times in one hour. |
| Database usage close to the quota 500M | A warning alert is triggered when the size of etcd exceeds 500M.|
| Etcd is unavailable | A critical alert is triggered when etcd becomes unavailable. |
| Etcd member has no leader | A critical alert is triggered when the etcd cluster does not have a leader for at least three minutes. |
## Alerts for Kubernetes Components
Rancher provides alerts when core Kubernetes system components become unhealthy.
Controllers update Kubernetes resources based on changes in etcd. The [controller manager](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) monitors the cluster desired state through the Kubernetes API server and makes the necessary changes to the current state to reach the desired state.
The [scheduler](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/) service is a core component of Kubernetes. It is responsible for scheduling cluster workloads to nodes, based on various configurations, metrics, resource requirements and workload-specific requirements.
| Alert | Explanation |
|-------|-------------|
| Controller Manager is unavailable | A critical warning is triggered when the cluster’s controller-manager becomes unavailable. |
| Scheduler is unavailable | A critical warning is triggered when the cluster’s scheduler becomes unavailable. |
## Alerts for Events
Kubernetes events are objects that provide insight into what is happening inside a cluster, such as what decisions were made by the scheduler or why some pods were evicted from the node. In the Rancher UI, from the project view, you can see events for each workload.
| Alert | Explanation |
|-------|-------------|
| Get warning deployment event | A warning alert is triggered when a warning event happens on a deployment. |
## Alerts for Nodes
Alerts can be triggered based on node metrics. Each computing resource in a Kubernetes cluster is called a node. Nodes can be either bare-metal servers or virtual machines.
| Alert | Explanation |
|-------|-------------|
| High CPU load | A warning alert is triggered if the node uses more than 100 percent of the node’s available CPU seconds for at least three minutes. |
| High node memory utilization | A warning alert is triggered if the node uses more than 80 percent of its available memory for at least three minutes. |
| Node disk is running full within 24 hours | A critical alert is triggered if the disk space on the node is expected to run out in the next 24 hours based on the disk growth over the last 6 hours. |
## Project-level Alerts
When you enable monitoring for the project, some project-level alerts are provided. For details, refer to the [section on project-level alerts.](../../../reference-guides/rancher-project-tools/project-alerts.md)
@@ -1,115 +0,0 @@
---
title: Cluster Logging
description: Rancher integrates with popular logging services. Learn the requirements and benefits of integrating with logging services, and enable logging on your cluster.
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/pages-for-subheaders/logging"/>
</head>
Logging is helpful because it allows you to:
- Capture and analyze the state of your cluster
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debug and troubleshoot problems
Rancher supports integration with the following services:
- Elasticsearch
- Splunk
- Kafka
- Syslog
- Fluentd
## How Logging Integrations Work
Rancher can integrate with popular external services used for event streams, telemetry, or search. These services can log errors and warnings in your Kubernetes infrastructure to a stream.
These services collect container log events, which are saved to the `/var/log/containers` directory on each of your nodes. The service collects both standard and error events. You can then log into your services to review the events collected, leveraging each service's unique features.
When configuring Rancher to integrate with these services, you'll have to point Rancher toward the service's endpoint and provide authentication information.
Additionally, you'll have the opportunity to enter key-value pairs to filter the log events collected. The service will only collect events for containers marked with your configured key-value pairs.
>**Note:** You can only configure one logging service per cluster or per project.
## Requirements
The Docker daemon on each node in the cluster should be [configured](https://docs.docker.com/config/containers/logging/configure/) with the (default) log-driver: `json-file`. You can check the log-driver by running the following command:
```
$ docker info | grep 'Logging Driver'
Logging Driver: json-file
```
## Logging Scope
You can configure logging at either cluster level or project level.
- Cluster logging writes logs for every pod in the cluster, i.e. in all the projects. For [RKE clusters](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md), it also writes logs for all the Kubernetes system components.
- [Project logging](../../../reference-guides/rancher-project-tools/project-logging.md) writes logs for every pod in that particular project.
Logs that are sent to your logging service are from the following locations:
- Pod logs stored at `/var/log/containers`.
- Kubernetes system components logs stored at `/var/lib/rancher/rke/log/`.
## Enabling Cluster Logging
As an [administrator](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md) or [cluster owner](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#cluster-roles), you can configure Rancher to send Kubernetes logs to a logging service.
1. From the **Global** view, navigate to the cluster that you want to configure cluster logging.
1. Select **Tools > Logging** in the navigation bar.
1. Select a logging service and enter the configuration. Refer to the specific service for detailed configuration. Rancher supports integration with the following services:
- [Elasticsearch](elasticsearch.md)
- [Splunk](splunk.md)
- [Kafka](kafka.md)
- [Syslog](syslog.md)
- [Fluentd](fluentd.md)
1. (Optional) Instead of using the UI to configure the logging services, you can enter custom advanced configurations by clicking on **Edit as File**, which is located above the logging targets. This link is only visible after you select a logging service.
- With the file editor, enter raw fluentd configuration for any logging service. Refer to the documentation for each logging service on how to setup the output configuration.
- [Elasticsearch Documentation](https://github.com/uken/fluent-plugin-elasticsearch)
- [Splunk Documentation](https://github.com/fluent/fluent-plugin-splunk)
- [Kafka Documentation](https://github.com/fluent/fluent-plugin-kafka)
- [Syslog Documentation](https://github.com/dlackty/fluent-plugin-remote_syslog)
- [Fluentd Documentation](https://docs.fluentd.org/v1.0/articles/out_forward)
- If the logging service is using TLS, you also need to complete the **SSL Configuration** form.
1. Provide the **Client Private Key** and **Client Certificate**. You can either copy and paste them or upload them by using the **Read from a file** button.
- You can use either a self-signed certificate or one provided by a certificate authority.
- You can generate a self-signed certificate using an openssl command. For example:
```
openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=myservice.example.com"
```
2. If you are using a self-signed certificate, provide the **CA Certificate PEM**.
1. (Optional) Complete the **Additional Logging Configuration** form.
1. **Optional:** Use the **Add Field** button to add custom log fields to your logging configuration. These fields are key value pairs (such as `foo=bar`) that you can use to filter the logs from another system.
1. Enter a **Flush Interval**. This value determines how often [Fluentd](https://www.fluentd.org/) flushes data to the logging server. Intervals are measured in seconds.
1. **Include System Log**. The logs from pods in system project and RKE components will be sent to the target. Uncheck it to exclude the system logs.
1. Click **Test**. Rancher sends a test log to the service.
> **Note:** This button is replaced with _Dry Run_ if you are using the custom configuration editor. In this case, Rancher calls the fluentd dry run command to validate the configuration.
1. Click **Save**.
**Result:** Rancher is now configured to send logs to the selected service. Log into the logging service so that you can start viewing the logs.
## Related Links
[Logging Architecture](https://kubernetes.io/docs/concepts/cluster-administration/logging/)
@@ -1,40 +0,0 @@
---
title: Elasticsearch
---
If your organization uses [Elasticsearch](https://www.elastic.co/), either on premise or in the cloud, you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Elasticsearch deployment to view logs.
>**Prerequisites:** Configure an [Elasticsearch deployment](https://www.elastic.co/guide/en/cloud/saas-release/ec-create-deployment.html).
## Elasticsearch Deployment Configuration
1. In the **Endpoint** field, enter the IP address and port of your Elasticsearch instance. You can find this information from the dashboard of your Elasticsearch deployment.
* Elasticsearch usually uses port `9200` for HTTP and `9243` for HTTPS.
1. If you are using [X-Pack Security](https://www.elastic.co/guide/en/x-pack/current/xpack-introduction.html), enter your Elasticsearch **Username** and **Password** for authentication.
1. Enter an [Index Pattern](https://www.elastic.co/guide/en/kibana/current/index-patterns.html).
## SSL Configuration
If your instance of Elasticsearch uses SSL, your **Endpoint** will need to begin with `https://`. With the correct endpoint, the **SSL Configuration** form is enabled and ready to be completed.
1. Provide the **Client Private Key** and **Client Certificate**. You can either copy and paste them or upload them by using the **Read from a file** button.
- You can use either a self-signed certificate or one provided by a certificate authority.
- You can generate a self-signed certificate using an openssl command. For example:
```
openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=myservice.example.com"
```
1. Enter your **Client Key Password**.
1. Enter your **SSL Version**. The default version is `TLSv1_2`.
1. Select whether or not you want to verify your SSL.
* If you are using a self-signed certificate, select **Enabled - Input trusted server certificate**, provide the **CA Certificate PEM**. You can copy and paste the certificate or upload it using the **Read from a file** button.
* If you are using a certificate from a certificate authority, select **Enabled - Input trusted server certificate**. You do not need to provide a **CA Certificate PEM**.
@@ -1,33 +0,0 @@
---
title: Fluentd
---
If your organization uses [Fluentd](https://www.fluentd.org/), you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Fluentd server to view logs.
>**Prerequisites:** Configure Fluentd input forward to receive the event stream.
>
>See [Fluentd Documentation](https://docs.fluentd.org/v1.0/articles/in_forward) for details.
## Fluentd Configuration
You can add multiple Fluentd Servers. First, click **Add Fluentd Server**. For each Fluentd server, complete the configuration information:
1. In the **Endpoint** field, enter the address and port of your Fluentd instance, e.g. `http://Fluentd-server:24224`.
1. Enter the **Shared Key** if your Fluentd Server is using a shared key for authentication.
1. Enter the **Username** and **Password** if your Fluentd Server is using username and password for authentication.
1. **Optional:** Enter the **Hostname** of the Fluentd server.
1. Enter the load balancing **Weight** of the Fluentd server. If the weight of one server is 20 and the other server is 30, events will be sent in a 2:3 ratio. If you do not enter a weight, the default weight is 60.
1. If this server is a standby server, check **Use as Standby Only**. Standby servers are used when all other servers are not available.
After adding all the Fluentd servers, you have the option to select **Enable Gzip Compression**. By default, this is enabled because the transferred payload size will be reduced.
## SSL Configuration
If your Fluentd servers are using TLS, you need to select **Use TLS**. If you are using a self-signed certificate, provide the **CA Certificate PEM**. You can copy and paste the certificate or upload it using the **Read from a file** button.
>**Note:** Fluentd does not support self-signed certificates when client authentication is enabled.
@@ -1,40 +0,0 @@
---
title: Kafka
---
If your organization uses [Kafka](https://kafka.apache.org/), you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Kafka server to view logs.
>**Prerequisite:** You must have a Kafka server configured.
## Kafka Server Configuration
1. Select the type of **Endpoint** your Kafka server is using:
* **Zookeeper**: Enter the IP address and port. By default, Zookeeper uses port `2181`. Please note that a Zookeeper endpoint cannot enable TLS.
* **Broker**: Click on **Add Endpoint**. For each Kafka broker, enter the IP address and port. By default, Kafka brokers use port `9092`.
1. In the **Topic** field, enter the name of a Kafka [topic](https://kafka.apache.org/documentation/#basic_ops_add_topic) that your Kubernetes cluster submits logs to.
## **Broker** Endpoint Type
### SSL Configuration
If your Kafka cluster is using SSL for the **Broker**, you need to complete the **SSL Configuration** form.
1. Provide the **Client Private Key** and **Client Certificate**. You can either copy and paste them or upload them by using the **Read from a file** button.
1. Provide the **CA Certificate PEM**. You can either copy and paste the certificate or upload it using the **Read from a file** button.
>**Note:** Kafka does not support self-signed certificates when client authentication is enabled.
### SASL configuration
If your Kafka cluster is using [SASL authentication](https://kafka.apache.org/documentation/#security_sasl) for the Broker, you need to complete the **SASL Configuration** form.
1. Enter the SASL **Username** and **Password**.
1. Select the **SASL Type** that your Kafka cluster is using.
* If your Kafka is using **Plain**, please ensure your Kafka cluster is using SSL.
* If your Kafka is using **Scram**, you need to select which **Scram Mechanism** Kafka is using.
@@ -1,72 +0,0 @@
---
title: Splunk
---
If your organization uses [Splunk](https://www.splunk.com/), you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Splunk server to view logs.
>**Prerequisites:**
>
>- Configure HTTP event collection for your Splunk Server (Splunk Enterprise or Splunk Cloud).
>- Either create a new token or copy an existing token.
>
>For more information, see [Splunk Documentation](http://docs.splunk.com/Documentation/Splunk/7.1.2/Data/UsetheHTTPEventCollector#About_Event_Collector_tokens).
## Splunk Configuration
1. In the **Endpoint** field, enter the IP address and port for you Splunk instance (i.e. `http://splunk-server:8088`)
* Splunk usually uses port `8088`. If you're using Splunk Cloud, you'll need to work with [Splunk support](https://www.splunk.com/en_us/support-and-services.html) to get an endpoint URL.
1. Enter the **Token** you obtained while completing the prerequisites (i.e., when you created a token in Splunk).
1. In the **Source** field, enter the name of the token as entered in Splunk.
1. **Optional:** Provide one or more [index](http://docs.splunk.com/Documentation/Splunk/7.1.2/Indexer/Aboutindexesandindexers) that's allowed for your token.
## SSL Configuration
If your instance of Splunk uses SSL, your **Endpoint** will need to begin with `https://`. With the correct endpoint, the **SSL Configuration** form is enabled and ready to be completed.
1. Provide the **Client Private Key** and **Client Certificate**. You can either copy and paste them or upload them by using the **Read from a file** button.
- You can use either a self-signed certificate or one provided by a certificate authority.
- You can generate a self-signed certificate using an openssl command. For example:
```
openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=myservice.example.com"
```
1. Enter your **Client Key Password**.
1. Select whether or not you want to verify your SSL.
* If you are using a self-signed certificate, select **Enabled - Input trusted server certificate**, provide the **CA Certificate PEM**. You can copy and paste the certificate or upload it using the **Read from a file** button.
* If you are using a certificate from a certificate authority, select **Enabled - Input trusted server certificate**. You do not need to provide a **CA Certificate PEM**.
## Viewing Logs
1. Log into your Splunk server.
1. Click on **Search & Reporting**. The number of **Indexed Events** listed should be increasing.
1. Click on Data Summary and select the Sources tab.
![View Logs](/img/splunk/splunk4.jpg)
1. To view the actual logs, click on the source that you declared earlier.
![View Logs](/img/splunk/splunk5.jpg)
## Troubleshooting
You can use curl to see if **HEC** is listening for HTTP event data.
```
$ curl http://splunk-server:8088/services/collector/event \
-H 'Authorization: Splunk 8da70994-b1b0-4a79-b154-bfaae8f93432' \
-d '{"event": "hello world"}'
```
If Splunk is configured correctly, you should receive **json** data returning `success code 0`. You should be able
to send logging data to HEC.
If you received an error, check your configuration in Splunk and Rancher.
@@ -1,40 +0,0 @@
---
title: Syslog
---
If your organization uses [Syslog](https://tools.ietf.org/html/rfc5424), you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Syslog server to view logs.
>**Prerequisite:** You must have a Syslog server configured.
If you are using rsyslog, please make sure your rsyslog authentication mode is `x509/name`.
## Syslog Server Configuration
1. In the **Endpoint** field, enter the IP address and port for your Syslog server. Additionally, in the dropdown, select the protocol that your Syslog server uses.
1. In the **Program** field, enter the name of the application sending logs to your Syslog server, e.g. `Rancher`.
1. If you are using a cloud logging service, e.g. [Sumologic](https://www.sumologic.com/), enter a **Token** that authenticates with your Syslog server. You will need to create this token in the cloud logging service.
1. Select a **Log Severity** for events that are logged to the Syslog server. For more information on each severity level, see the [Syslog protocol documentation](https://tools.ietf.org/html/rfc5424#page-11).
- By specifying a **Log Severity** does not mean that will act as a filtering mechanism for logs. To do that you should use a parser on the Syslog server.
## Encryption Configuration
If your Syslog server is using **TCP** protocol and uses TLS, you need to select **Use TLS** and complete the **Encryption Configuration** form.
1. Provide the **Client Private Key** and **Client Certificate**. You can either copy and paste them or upload them by using the **Read from a file** button.
- You can use either a self-signed certificate or one provided by a certificate authority.
- You can generate a self-signed certificate using an openssl command. For example:
```
openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=myservice.example.com"
```
1. Select whether or not you want to verify your SSL.
* If you are using a self-signed certificate, select **Enabled - Input trusted server certificate**, provide the **CA Certificate PEM**. You can copy and paste the certificate or upload it using the **Read from a file** button.
* If you are using a certificate from a certificate authority, select **Enabled - Input trusted server certificate**. You do not need to provide a **CA Certificate PEM**.
@@ -1,112 +0,0 @@
---
title: Cluster Metrics
---
_Available as of v2.2.0_
Cluster metrics display the hardware utilization for all nodes in your cluster, regardless of its role. They give you a global monitoring insight into the cluster.
Some of the biggest metrics to look out for:
- **CPU Utilization**
High load either indicates that your cluster is running efficiently or that you're running out of CPU resources.
- **Disk Utilization**
Be on the lookout for increased read and write rates on nodes nearing their disk capacity. This advice is especially true for etcd nodes, as running out of storage on an etcd node leads to cluster failure.
- **Memory Utilization**
Deltas in memory utilization usually indicate a memory leak.
- **Load Average**
Generally, you want your load average to match your number of logical CPUs for the cluster. For example, if your cluster has 8 logical CPUs, the ideal load average would be 8 as well. If you load average is well under the number of logical CPUs for the cluster, you may want to reduce cluster resources. On the other hand, if your average is over 8, your cluster may need more resources.
## Finding Node Metrics
1. From the **Global** view, navigate to the cluster that you want to view metrics.
1. Select **Nodes** in the navigation bar.
1. Select a specific node and click on its name.
1. Click on **Node Metrics**.
[_Get expressions for Cluster Metrics_](./expression.md#cluster-metrics)
### Etcd Metrics
>**Note:** Only supported for [Rancher launched Kubernetes clusters](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
Etcd metrics display the operations of the etcd database on each of your cluster nodes. After establishing a baseline of normal etcd operational metrics, observe them for abnormal deltas between metric refreshes, which indicate potential issues with etcd. Always address etcd issues immediately!
You should also pay attention to the text at the top of the etcd metrics, which displays leader election statistics. This text indicates if etcd currently has a leader, which is the etcd instance that coordinates the other etcd instances in your cluster. A large increase in leader changes implies etcd is unstable. If you notice a change in leader election statistics, you should investigate them for issues.
Some of the biggest metrics to look out for:
- **Etcd has a leader**
etcd is usually deployed on multiple nodes and elects a leader to coordinate its operations. If etcd does not have a leader, its operations are not being coordinated.
- **Number of leader changes**
If this statistic suddenly grows, it usually indicates network communication issues that constantly force the cluster to elect a new leader.
[_Get expressions for Etcd Metrics_](./expression.md#etcd-metrics)
### Kubernetes Components Metrics
Kubernetes components metrics display data about the cluster's individual Kubernetes components. Primarily, it displays information about connections and latency for each component: the API server, controller manager, scheduler, and ingress controller.
>**Note:** The metrics for the controller manager, scheduler and ingress controller are only supported for [Rancher launched Kubernetes clusters](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
When analyzing Kubernetes component metrics, don't be concerned about any single standalone metric in the charts and graphs that display. Rather, you should establish a baseline for metrics considered normal following a period of observation, e.g. the range of values that your components usually operate within and are considered normal. After you establish this baseline, be on the lookout for large deltas in the charts and graphs, as these big changes usually indicate a problem that you need to investigate.
Some of the more important component metrics to monitor are:
- **API Server Request Latency**
Increasing API response times indicate there's a generalized problem that requires investigation.
- **API Server Request Rate**
Rising API request rates usually coincide with increased API response times. Increased request rates also indicate a generalized problem requiring investigation.
- **Scheduler Preemption Attempts**
If you see a spike in scheduler preemptions, it's an indication that you're running out of hardware resources, as Kubernetes is recognizing it doesn't have enough resources to run all your pods and is prioritizing the more important ones.
- **Scheduling Failed Pods**
Failed pods can have a variety of causes, such as unbound persistent volume claims, exhausted hardware resources, non-responsive nodes, etc.
- **Ingress Controller Request Process Time**
How fast ingress is routing connections to your cluster services.
[_Get expressions for Kubernetes Component Metrics_](./expression.md#kubernetes-components-metrics)
## Rancher Logging Metrics
Although the Dashboard for a cluster primarily displays data sourced from Prometheus, it also displays information for cluster logging, provided that you have [configured Rancher to use a logging service](../cluster-logging/cluster-logging.md).
[_Get expressions for Rancher Logging Metrics_](./expression.md#rancher-logging-metrics)
## Finding Workload Metrics
Workload metrics display the hardware utilization for a Kubernetes workload. You can also view metrics for [deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/), [stateful sets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) and so on.
1. From the **Global** view, navigate to the project that you want to view workload metrics.
1. From the main navigation bar, choose **Resources > Workloads.** In versions before v2.3.0, choose **Workloads** on the main navigation bar.
1. Select a specific workload and click on its name.
1. In the **Pods** section, select a specific pod and click on its name.
- **View the Pod Metrics:** Click on **Pod Metrics**.
- **View the Container Metrics:** In the **Containers** section, select a specific container and click on its name. Click on **Container Metrics**.
[_Get expressions for Workload Metrics_](./expression.md#workload-metrics)
@@ -1,110 +0,0 @@
---
title: Integrating Rancher and Prometheus for Cluster Monitoring
description: Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Learn about the scope of monitoring and how to enable cluster monitoring
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/pages-for-subheaders/monitoring-and-alerting"/>
</head>
_Available as of v2.2.0_
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution.
## About Prometheus
Prometheus provides a _time series_ of your data, which is, according to [Prometheus documentation](https://prometheus.io/docs/concepts/data_model/):
You can configure these services to collect logs at either the cluster level or the project level. This page describes how to enable monitoring for a cluster. For details on enabling monitoring for a project, refer to the [project administration section](project-monitoring.md).
>A stream of timestamped values belonging to the same metric and the same set of labeled dimensions, along with comprehensive statistics and metrics of the monitored cluster.
In other words, Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Using timestamps, Prometheus lets you query and view these metrics in easy-to-read graphs and visuals, either through the Rancher UI or [Grafana](https://grafana.com/), which is an analytics viewing platform deployed along with Prometheus.
By viewing data that Prometheus scrapes from your cluster control plane, nodes, and deployments, you can stay on top of everything happening in your cluster. You can then use these analytics to better run your organization: stop system emergencies before they start, develop maintenance strategies, restore crashed servers, etc.
Multi-tenancy support in terms of cluster-only and project-only Prometheus instances are also supported.
## Monitoring Scope
Using Prometheus, you can monitor Rancher at both the cluster level and [project level](project-monitoring.md). For each cluster and project that is enabled for monitoring, Rancher deploys a Prometheus server.
- Cluster monitoring allows you to view the health of your Kubernetes cluster. Prometheus collects metrics from the cluster components below, which you can view in graphs and charts.
- Kubernetes control plane
- etcd database
- All nodes (including workers)
- [Project monitoring](project-monitoring.md) allows you to view the state of pods running in a given project. Prometheus collects metrics from the project's deployed HTTP and TCP/UDP workloads.
## Enabling Cluster Monitoring
As an [administrator](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md) or [cluster owner](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#cluster-roles), you can configure Rancher to deploy Prometheus to monitor your Kubernetes cluster.
> **Prerequisites:** The following TCP ports need to be opened for metrics scraping:
>
> | Port | Node type | Component |
> | --- | --- | --- |
> | 9796 | Worker | Node exporter |
> | 10254 | Worker | Nginx Ingress Controller |
> | 10250 | Worker/Controlplane | Kubelet |
> | 10251 | Controlplane | Kube scheduler |
> | 10252 | Controlplane | Kube controller manager |
> | 2379 | Etcd | Etcd server |
> Monitoring V1 requires a Kubernetes verison less than or equal to v1.20.x. To install monitoring on Kubernetes v1.21+, you will need to [migrate to Monitoring V2.](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.5/how-to-guides/advanced-user-guides/monitoring-alerting-guides/migrate-to-rancher-v2.5%2B-monitoring.md)
1. From the **Global** view, navigate to the cluster that you want to configure cluster monitoring.
1. Select **Tools > Monitoring** in the navigation bar.
1. Select **Enable** to show the [Prometheus configuration options](prometheus.md). Review the [resource consumption recommendations](#resource-consumption) to ensure you have enough resources for Prometheus and on your worker nodes to enable monitoring. Enter in your desired configuration options.
1. Click **Save**.
**Result:** The Prometheus server will be deployed as well as two monitoring applications. The two monitoring applications, `cluster-monitoring` and `monitoring-operator`, are added as an [application](../../../how-to-guides/new-user-guides/helm-charts-in-rancher/helm-charts-in-rancher.md) to the cluster's `system` project. After the applications are `active`, you can start viewing [cluster metrics](cluster-metrics.md) through the Rancher dashboard or directly from Grafana.
> The default username and password for the Grafana instance will be `admin/admin`. However, Grafana dashboards are served via the Rancher authentication proxy, so only users who are currently authenticated into the Rancher server have access to the Grafana dashboard.
## Resource Consumption
When enabling cluster monitoring, you need to ensure your worker nodes and Prometheus pod have enough resources. The tables below provides a guide of how much resource consumption will be used. In larger deployments, it is strongly advised that the monitoring infrastructure be placed on dedicated nodes in the cluster.
### Resource Consumption of Prometheus Pods
This table is the resource consumption of the Prometheus pod, which is based on the number of all the nodes in the cluster. The count of nodes includes the worker, control plane and etcd nodes. Total disk space allocation should be approximated by the `rate * retention` period set at the cluster level. When enabling cluster level monitoring, you should adjust the CPU and Memory limits and reservation.
Number of Cluster Nodes | CPU (milli CPU) | Memory | Disk
------------------------|-----|--------|------
5 | 500 | 650 MB | ~1 GB/Day
50| 2000 | 2 GB | ~5 GB/Day
256| 4000 | 6 GB | ~18 GB/Day
Additional pod resource requirements for cluster level monitoring.
| Workload | Container | CPU - Request | Mem - Request | CPU - Limit | Mem - Limit | Configurable |
|---------------------|---------------------------------|---------------|---------------|-------------|-------------|--------------|
| Prometheus | prometheus | 750m | 750Mi | 1000m | 1000Mi | Y |
| | prometheus-proxy | 50m | 50Mi | 100m | 100Mi | Y |
| | prometheus-auth | 100m | 100Mi | 500m | 200Mi | Y |
| | prometheus-config-reloader | - | - | 50m | 50Mi | N |
| | rules-configmap-reloader | - | - | 100m | 25Mi | N |
| Grafana | grafana-init-plugin-json-copy | 50m | 50Mi | 50m | 50Mi | Y |
| | grafana-init-plugin-json-modify | 50m | 50Mi | 50m | 50Mi | Y |
| | grafana | 100m | 100Mi | 200m | 200Mi | Y |
| | grafana-proxy | 50m | 50Mi | 100m | 100Mi | Y |
| Kube-State Exporter | kube-state | 100m | 130Mi | 100m | 200Mi | Y |
| Node Exporter | exporter-node | 200m | 200Mi | 200m | 200Mi | Y |
| Operator | prometheus-operator | 100m | 50Mi | 200m | 100Mi | Y |
### Resource Consumption of Other Pods
Besides the Prometheus pod, there are components that are deployed that require additional resources on the worker nodes.
Pod | CPU (milli CPU) | Memory (MB)
----|-----------------|------------
Node Exporter (Per Node) | 100 | 30
Kube State Cluster Monitor | 100 | 130
Grafana | 100 | 150
Prometheus Cluster Monitoring Nginx | 50 | 50
@@ -1,486 +0,0 @@
---
title: Prometheus Custom Metrics Adapter
---
After you've enabled [cluster level monitoring](cluster-monitoring.md), You can view the metrics data from Rancher. You can also deploy the Prometheus custom metrics adapter then you can use the HPA with metrics stored in cluster monitoring.
## Deploy Prometheus Custom Metrics Adapter
We are going to use the [Prometheus custom metrics adapter](https://github.com/DirectXMan12/k8s-prometheus-adapter/releases/tag/v0.5.0), version v0.5.0. This is a great example for the [custom metrics server](https://github.com/kubernetes-incubator/custom-metrics-apiserver). And you must be the *cluster owner* to execute following steps.
- Get the service account of the cluster monitoring is using. It should be configured in the workload ID: `statefulset:cattle-prometheus:prometheus-cluster-monitoring`. And if you didn't customize anything, the service account name should be `cluster-monitoring`.
- Grant permission to that service account. You will need two kinds of permission.
One role is `extension-apiserver-authentication-reader` in `kube-system`, so you will need to create a `Rolebinding` to in `kube-system`. This permission is to get api aggregation configuration from config map in `kube-system`.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: custom-metrics-auth-reader
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
name: cluster-monitoring
namespace: cattle-prometheus
```
The other one is cluster role `system:auth-delegator`, so you will need to create a `ClusterRoleBinding`. This permission is to have subject access review permission.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: custom-metrics:system:auth-delegator
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:auth-delegator
subjects:
- kind: ServiceAccount
name: cluster-monitoring
namespace: cattle-prometheus
```
- Create configuration for custom metrics adapter. Following is an example configuration. There will be a configuration details in next session.
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: adapter-config
namespace: cattle-prometheus
data:
config.yaml: |
rules:
- seriesQuery: '{__name__=~"^container_.*",container_name!="POD",namespace!="",pod_name!=""}'
seriesFilters: []
resources:
overrides:
namespace:
resource: namespace
pod_name:
resource: pod
name:
matches: ^container_(.*)_seconds_total$
as: ""
metricsQuery: sum(rate(<<.Series>>{<<.LabelMatchers>>,container_name!="POD"}[1m])) by (<<.GroupBy>>)
- seriesQuery: '{__name__=~"^container_.*",container_name!="POD",namespace!="",pod_name!=""}'
seriesFilters:
- isNot: ^container_.*_seconds_total$
resources:
overrides:
namespace:
resource: namespace
pod_name:
resource: pod
name:
matches: ^container_(.*)_total$
as: ""
metricsQuery: sum(rate(<<.Series>>{<<.LabelMatchers>>,container_name!="POD"}[1m])) by (<<.GroupBy>>)
- seriesQuery: '{__name__=~"^container_.*",container_name!="POD",namespace!="",pod_name!=""}'
seriesFilters:
- isNot: ^container_.*_total$
resources:
overrides:
namespace:
resource: namespace
pod_name:
resource: pod
name:
matches: ^container_(.*)$
as: ""
metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>,container_name!="POD"}) by (<<.GroupBy>>)
- seriesQuery: '{namespace!="",__name__!~"^container_.*"}'
seriesFilters:
- isNot: .*_total$
resources:
template: <<.Resource>>
name:
matches: ""
as: ""
metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)
- seriesQuery: '{namespace!="",__name__!~"^container_.*"}'
seriesFilters:
- isNot: .*_seconds_total
resources:
template: <<.Resource>>
name:
matches: ^(.*)_total$
as: ""
metricsQuery: sum(rate(<<.Series>>{<<.LabelMatchers>>}[1m])) by (<<.GroupBy>>)
- seriesQuery: '{namespace!="",__name__!~"^container_.*"}'
seriesFilters: []
resources:
template: <<.Resource>>
name:
matches: ^(.*)_seconds_total$
as: ""
metricsQuery: sum(rate(<<.Series>>{<<.LabelMatchers>>}[1m])) by (<<.GroupBy>>)
resourceRules:
cpu:
containerQuery: sum(rate(container_cpu_usage_seconds_total{<<.LabelMatchers>>}[1m])) by (<<.GroupBy>>)
nodeQuery: sum(rate(container_cpu_usage_seconds_total{<<.LabelMatchers>>, id='/'}[1m])) by (<<.GroupBy>>)
resources:
overrides:
instance:
resource: node
namespace:
resource: namespace
pod_name:
resource: pod
containerLabel: container_name
memory:
containerQuery: sum(container_memory_working_set_bytes{<<.LabelMatchers>>}) by (<<.GroupBy>>)
nodeQuery: sum(container_memory_working_set_bytes{<<.LabelMatchers>>,id='/'}) by (<<.GroupBy>>)
resources:
overrides:
instance:
resource: node
namespace:
resource: namespace
pod_name:
resource: pod
containerLabel: container_name
window: 1m
```
- Create HTTPS TLS certs for your api server. You can use following command to create a self-signed cert.
```bash
openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out serving.crt -keyout serving.key -subj "/C=CN/CN=custom-metrics-apiserver.cattle-prometheus.svc.cluster.local"
# And you will find serving.crt and serving.key in your path. And then you are going to create a secret in cattle-prometheus namespace.
kubectl create secret generic -n cattle-prometheus cm-adapter-serving-certs --from-file=serving.key=./serving.key --from-file=serving.crt=./serving.crt
```
- Then you can create the prometheus custom metrics adapter. And you will need a service for this deployment too. Creating it via Import YAML or Rancher would do. Please create those resources in `cattle-prometheus` namespaces.
Here is the prometheus custom metrics adapter deployment.
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: custom-metrics-apiserver
name: custom-metrics-apiserver
namespace: cattle-prometheus
spec:
replicas: 1
selector:
matchLabels:
app: custom-metrics-apiserver
template:
metadata:
labels:
app: custom-metrics-apiserver
name: custom-metrics-apiserver
spec:
serviceAccountName: cluster-monitoring
containers:
- name: custom-metrics-apiserver
image: directxman12/k8s-prometheus-adapter-amd64:v0.5.0
args:
- --secure-port=6443
- --tls-cert-file=/var/run/serving-cert/serving.crt
- --tls-private-key-file=/var/run/serving-cert/serving.key
- --logtostderr=true
- --prometheus-url=http://prometheus-operated/
- --metrics-relist-interval=1m
- --v=10
- --config=/etc/adapter/config.yaml
ports:
- containerPort: 6443
volumeMounts:
- mountPath: /var/run/serving-cert
name: volume-serving-cert
readOnly: true
- mountPath: /etc/adapter/
name: config
readOnly: true
- mountPath: /tmp
name: tmp-vol
volumes:
- name: volume-serving-cert
secret:
secretName: cm-adapter-serving-certs
- name: config
configMap:
name: adapter-config
- name: tmp-vol
emptyDir: {}
```
Here is the service of the deployment.
```yaml
apiVersion: v1
kind: Service
metadata:
name: custom-metrics-apiserver
namespace: cattle-prometheus
spec:
ports:
- port: 443
targetPort: 6443
selector:
app: custom-metrics-apiserver
```
- Create API service for your custom metric server.
```yaml
apiVersion: apiregistration.k8s.io/v1beta1
kind: APIService
metadata:
name: v1beta1.custom.metrics.k8s.io
spec:
service:
name: custom-metrics-apiserver
namespace: cattle-prometheus
group: custom.metrics.k8s.io
version: v1beta1
insecureSkipTLSVerify: true
groupPriorityMinimum: 100
versionPriority: 100
```
- Then you can verify your custom metrics server by `kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1`. If you see the return datas from the api, it means that the metrics server has been successfully set up.
- You create HPA with custom metrics now. Here is an example of HPA. You will need to create a nginx deployment in your namespace first.
```yaml
kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta1
metadata:
name: nginx
spec:
scaleTargetRef:
# point the HPA at the nginx deployment you just created
apiVersion: apps/v1
kind: Deployment
name: nginx
# autoscale between 1 and 10 replicas
minReplicas: 1
maxReplicas: 10
metrics:
# use a "Pods" metric, which takes the average of the
# given metric across all pods controlled by the autoscaling target
- type: Pods
pods:
metricName: memory_usage_bytes
targetAverageValue: 5000000
```
And then, you should see your nginx is scaling up. HPA with custom metrics works.
## Configuration of prometheus custom metrics adapter
> Refer to https://github.com/DirectXMan12/k8s-prometheus-adapter/blob/master/docs/config.md
The adapter determines which metrics to expose, and how to expose them,
through a set of "discovery" rules. Each rule is executed independently
(so make sure that your rules are mutually exclusive), and specifies each
of the steps the adapter needs to take to expose a metric in the API.
Each rule can be broken down into roughly four parts:
- *Discovery*, which specifies how the adapter should find all Prometheus
metrics for this rule.
- *Association*, which specifies how the adapter should determine which
Kubernetes resources a particular metric is associated with.
- *Naming*, which specifies how the adapter should expose the metric in
the custom metrics API.
- *Querying*, which specifies how a request for a particular metric on one
or more Kubernetes objects should be turned into a query to Prometheus.
A basic config with one rule might look like:
```yaml
rules:
# this rule matches cumulative cAdvisor metrics measured in seconds
- seriesQuery: '{__name__=~"^container_.*",container_name!="POD",namespace!="",pod_name!=""}'
resources:
# skip specifying generic resource<->label mappings, and just
# attach only pod and namespace resources by mapping label names to group-resources
overrides:
namespace: {resource: "namespace"},
pod_name: {resource: "pod"},
# specify that the `container_` and `_seconds_total` suffixes should be removed.
# this also introduces an implicit filter on metric family names
name:
# we use the value of the capture group implicitly as the API name
# we could also explicitly write `as: "$1"`
matches: "^container_(.*)_seconds_total$"
# specify how to construct a query to fetch samples for a given series
# This is a Go template where the `.Series` and `.LabelMatchers` string values
# are available, and the delimiters are `<<` and `>>` to avoid conflicts with
# the prometheus query language
metricsQuery: "sum(rate(<<.Series>>{<<.LabelMatchers>>,container_name!="POD"}[2m])) by (<<.GroupBy>>)"
```
### Discovery
Discovery governs the process of finding the metrics that you want to
expose in the custom metrics API. There are two fields that factor into
discovery: `seriesQuery` and `seriesFilters`.
`seriesQuery` specifies Prometheus series query (as passed to the
`/api/v1/series` endpoint in Prometheus) to use to find some set of
Prometheus series. The adapter will strip the label values from this
series, and then use the resulting metric-name-label-names combinations
later on.
In many cases, `seriesQuery` will be sufficient to narrow down the list of
Prometheus series. However, sometimes (especially if two rules might
otherwise overlap), it's useful to do additional filtering on metric
names. In this case, `seriesFilters` can be used. After the list of
series is returned from `seriesQuery`, each series has its metric name
filtered through any specified filters.
Filters may be either:
- `is: <regex>`, which matches any series whose name matches the specified
regex.
- `isNot: <regex>`, which matches any series whose name does not match the
specified regex.
For example:
```yaml
# match all cAdvisor metrics that aren't measured in seconds
seriesQuery: '{__name__=~"^container_.*_total",container_name!="POD",namespace!="",pod_name!=""}'
seriesFilters:
isNot: "^container_.*_seconds_total"
```
### Association
Association governs the process of figuring out which Kubernetes resources
a particular metric could be attached to. The `resources` field controls
this process.
There are two ways to associate resources with a particular metric. In
both cases, the value of the label becomes the name of the particular
object.
One way is to specify that any label name that matches some particular
pattern refers to some group-resource based on the label name. This can
be done using the `template` field. The pattern is specified as a Go
template, with the `Group` and `Resource` fields representing group and
resource. You don't necessarily have to use the `Group` field (in which
case the group is guessed by the system). For instance:
```yaml
# any label `kube_<group>_<resource>` becomes <group>.<resource> in Kubernetes
resources:
template: "kube_<<.Group>>_<<.Resource>>"
```
The other way is to specify that some particular label represents some
particular Kubernetes resource. This can be done using the `overrides`
field. Each override maps a Prometheus label to a Kubernetes
group-resource. For instance:
```yaml
# the microservice label corresponds to the apps.deployment resource
resource:
overrides:
microservice: {group: "apps", resource: "deployment"}
```
These two can be combined, so you can specify both a template and some
individual overrides.
The resources mentioned can be any resource available in your kubernetes
cluster, as long as you've got a corresponding label.
### Naming
Naming governs the process of converting a Prometheus metric name into
a metric in the custom metrics API, and vice versa. It's controlled by
the `name` field.
Naming is controlled by specifying a pattern to extract an API name from
a Prometheus name, and potentially a transformation on that extracted
value.
The pattern is specified in the `matches` field, and is just a regular
expression. If not specified, it defaults to `.*`.
The transformation is specified by the `as` field. You can use any
capture groups defined in the `matches` field. If the `matches` field
doesn't contain capture groups, the `as` field defaults to `$0`. If it
contains a single capture group, the `as` field defautls to `$1`.
Otherwise, it's an error not to specify the as field.
For example:
```yaml
# match turn any name <name>_total to <name>_per_second
# e.g. http_requests_total becomes http_requests_per_second
name:
matches: "^(.*)_total$"
as: "${1}_per_second"
```
### Querying
Querying governs the process of actually fetching values for a particular
metric. It's controlled by the `metricsQuery` field.
The `metricsQuery` field is a Go template that gets turned into
a Prometheus query, using input from a particular call to the custom
metrics API. A given call to the custom metrics API is distilled down to
a metric name, a group-resource, and one or more objects of that
group-resource. These get turned into the following fields in the
template:
- `Series`: the metric name
- `LabelMatchers`: a comma-separated list of label matchers matching the
given objects. Currently, this is the label for the particular
group-resource, plus the label for namespace, if the group-resource is
namespaced.
- `GroupBy`: a comma-separated list of labels to group by. Currently,
this contains the group-resource label used in `LabelMatchers`.
For instance, suppose we had a series `http_requests_total` (exposed as
`http_requests_per_second` in the API) with labels `service`, `pod`,
`ingress`, `namespace`, and `verb`. The first four correspond to
Kubernetes resources. Then, if someone requested the metric
`pods/http_request_per_second` for the pods `pod1` and `pod2` in the
`somens` namespace, we'd have:
- `Series: "http_requests_total"`
- `LabelMatchers: "pod=~\"pod1|pod2",namespace="somens"`
- `GroupBy`: `pod`
Additionally, there are two advanced fields that are "raw" forms of other
fields:
- `LabelValuesByName`: a map mapping the labels and values from the
`LabelMatchers` field. The values are pre-joined by `|`
(for used with the `=~` matcher in Prometheus).
- `GroupBySlice`: the slice form of `GroupBy`.
In general, you'll probably want to use the `Series`, `LabelMatchers`, and
`GroupBy` fields. The other two are for advanced usage.
The query is expected to return one value for each object requested. The
adapter will use the labels on the returned series to associate a given
series back to its corresponding object.
For example:
```yaml
# convert cumulative cAdvisor metrics into rates calculated over 2 minutes
metricsQuery: "sum(rate(<<.Series>>{<<.LabelMatchers>>,container_name!="POD"}[2m])) by (<<.GroupBy>>)"
```
@@ -1,365 +0,0 @@
---
title: Prometheus Expressions
---
The PromQL expressions in this doc can be used to configure [alerts.](../cluster-alerts/cluster-alerts.md)
> Before expressions can be used in alerts, monitoring must be enabled. For more information, refer to the documentation on enabling monitoring [at the cluster level](cluster-monitoring.md) or [at the project level.](./project-monitoring.md)
For more information about querying Prometheus, refer to the official [Prometheus documentation.](https://prometheus.io/docs/prometheus/latest/querying/basics/)
## Cluster Metrics
### Cluster CPU Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `1 - (avg(irate(node_cpu_seconds_total{mode="idle"}[5m])) by (instance))` |
| Summary | `1 - (avg(irate(node_cpu_seconds_total{mode="idle"}[5m])))` |
### Cluster Load Average
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>load1</td><td>`sum(node_load1) by (instance) / count(node_cpu_seconds_total{mode="system"}) by (instance)`</td></tr><tr><td>load5</td><td>`sum(node_load5) by (instance) / count(node_cpu_seconds_total{mode="system"}) by (instance)`</td></tr><tr><td>load15</td><td>`sum(node_load15) by (instance) / count(node_cpu_seconds_total{mode="system"}) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>load1</td><td>`sum(node_load1) by (instance) / count(node_cpu_seconds_total{mode="system"})`</td></tr><tr><td>load5</td><td>`sum(node_load5) by (instance) / count(node_cpu_seconds_total{mode="system"})`</td></tr><tr><td>load15</td><td>`sum(node_load15) by (instance) / count(node_cpu_seconds_total{mode="system"})`</td></tr></table> |
### Cluster Memory Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `1 - sum(node_memory_MemAvailable_bytes) by (instance) / sum(node_memory_MemTotal_bytes) by (instance)` |
| Summary | `1 - sum(node_memory_MemAvailable_bytes) / sum(node_memory_MemTotal_bytes)` |
### Cluster Disk Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `(sum(node_filesystem_size_bytes{device!="rootfs"}) by (instance) - sum(node_filesystem_free_bytes{device!="rootfs"}) by (instance)) / sum(node_filesystem_size_bytes{device!="rootfs"}) by (instance)` |
| Summary | `(sum(node_filesystem_size_bytes{device!="rootfs"}) - sum(node_filesystem_free_bytes{device!="rootfs"})) / sum(node_filesystem_size_bytes{device!="rootfs"})` |
### Cluster Disk I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>read</td><td>`sum(rate(node_disk_read_bytes_total[5m])) by (instance)`</td></tr><tr><td>written</td><td>`sum(rate(node_disk_written_bytes_total[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>read</td><td>`sum(rate(node_disk_read_bytes_total[5m]))`</td></tr><tr><td>written</td><td>`sum(rate(node_disk_written_bytes_total[5m]))`</td></tr></table> |
### Cluster Network Packets
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive-dropped</td><td><code>sum(rate(node_network_receive_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>receive-errs</td><td><code>sum(rate(node_network_receive_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>receive-packets</td><td><code>sum(rate(node_network_receive_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit-dropped</td><td><code>sum(rate(node_network_transmit_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit-errs</td><td><code>sum(rate(node_network_transmit_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit-packets</td><td><code>sum(rate(node_network_transmit_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr></table> |
| Summary | <table><tr><td>receive-dropped</td><td><code>sum(rate(node_network_receive_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>receive-errs</td><td><code>sum(rate(node_network_receive_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>receive-packets</td><td><code>sum(rate(node_network_receive_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit-dropped</td><td><code>sum(rate(node_network_transmit_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit-errs</td><td><code>sum(rate(node_network_transmit_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit-packets</td><td><code>sum(rate(node_network_transmit_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr></table> |
### Cluster Network I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr></table> |
| Summary | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr></table> |
## Node Metrics
### Node CPU Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `avg(irate(node_cpu_seconds_total{mode!="idle", instance=~"$instance"}[5m])) by (mode)` |
| Summary | `1 - (avg(irate(node_cpu_seconds_total{mode="idle", instance=~"$instance"}[5m])))` |
### Node Load Average
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>load1</td><td>`sum(node_load1{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr><tr><td>load5</td><td>`sum(node_load5{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr><tr><td>load15</td><td>`sum(node_load15{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr></table> |
| Summary | <table><tr><td>load1</td><td>`sum(node_load1{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr><tr><td>load5</td><td>`sum(node_load5{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr><tr><td>load15</td><td>`sum(node_load15{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr></table> |
### Node Memory Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `1 - sum(node_memory_MemAvailable_bytes{instance=~"$instance"}) / sum(node_memory_MemTotal_bytes{instance=~"$instance"})` |
| Summary | `1 - sum(node_memory_MemAvailable_bytes{instance=~"$instance"}) / sum(node_memory_MemTotal_bytes{instance=~"$instance"}) ` |
### Node Disk Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `(sum(node_filesystem_size_bytes{device!="rootfs",instance=~"$instance"}) by (device) - sum(node_filesystem_free_bytes{device!="rootfs",instance=~"$instance"}) by (device)) / sum(node_filesystem_size_bytes{device!="rootfs",instance=~"$instance"}) by (device)` |
| Summary | `(sum(node_filesystem_size_bytes{device!="rootfs",instance=~"$instance"}) - sum(node_filesystem_free_bytes{device!="rootfs",instance=~"$instance"})) / sum(node_filesystem_size_bytes{device!="rootfs",instance=~"$instance"})` |
### Node Disk I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>read</td><td>`sum(rate(node_disk_read_bytes_total{instance=~"$instance"}[5m]))`</td></tr><tr><td>written</td><td>`sum(rate(node_disk_written_bytes_total{instance=~"$instance"}[5m]))`</td></tr></table> |
| Summary | <table><tr><td>read</td><td>`sum(rate(node_disk_read_bytes_total{instance=~"$instance"}[5m]))`</td></tr><tr><td>written</td><td>`sum(rate(node_disk_written_bytes_total{instance=~"$instance"}[5m]))`</td></tr></table> |
### Node Network Packets
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive-dropped</td><td><code>sum(rate(node_network_receive_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>receive-errs</td><td><code>sum(rate(node_network_receive_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>receive-packets</td><td><code>sum(rate(node_network_receive_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit-dropped</td><td><code>sum(rate(node_network_transmit_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit-errs</td><td><code>sum(rate(node_network_transmit_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit-packets</td><td><code>sum(rate(node_network_transmit_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr></table> |
| Summary | <table><tr><td>receive-dropped</td><td><code>sum(rate(node_network_receive_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>receive-errs</td><td><code>sum(rate(node_network_receive_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>receive-packets</td><td><code>sum(rate(node_network_receive_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit-dropped</td><td><code>sum(rate(node_network_transmit_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit-errs</td><td><code>sum(rate(node_network_transmit_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit-packets</td><td><code>sum(rate(node_network_transmit_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr></table> |
### Node Network I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr></table> |
| Summary | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr></table> |
## Etcd Metrics
### Etcd Has a Leader
`max(etcd_server_has_leader)`
### Number of Times the Leader Changes
`max(etcd_server_leader_changes_seen_total)`
### Number of Failed Proposals
`sum(etcd_server_proposals_failed_total)`
### GRPC Client Traffic
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>in</td><td>`sum(rate(etcd_network_client_grpc_received_bytes_total[5m])) by (instance)`</td></tr><tr><td>out</td><td>`sum(rate(etcd_network_client_grpc_sent_bytes_total[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>in</td><td>`sum(rate(etcd_network_client_grpc_received_bytes_total[5m]))`</td></tr><tr><td>out</td><td>`sum(rate(etcd_network_client_grpc_sent_bytes_total[5m]))`</td></tr></table> |
### Peer Traffic
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>in</td><td>`sum(rate(etcd_network_peer_received_bytes_total[5m])) by (instance)`</td></tr><tr><td>out</td><td>`sum(rate(etcd_network_peer_sent_bytes_total[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>in</td><td>`sum(rate(etcd_network_peer_received_bytes_total[5m]))`</td></tr><tr><td>out</td><td>`sum(rate(etcd_network_peer_sent_bytes_total[5m]))`</td></tr></table> |
### DB Size
| Catalog | Expression |
| --- | --- |
| Detail | `sum(etcd_debugging_mvcc_db_total_size_in_bytes) by (instance)` |
| Summary | `sum(etcd_debugging_mvcc_db_total_size_in_bytes)` |
### Active Streams
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>lease-watch</td><td>`sum(grpc_server_started_total{grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}) by (instance) - sum(grpc_server_handled_total{grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}) by (instance)`</td></tr><tr><td>watch</td><td>`sum(grpc_server_started_total{grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}) by (instance) - sum(grpc_server_handled_total{grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>lease-watch</td><td>`sum(grpc_server_started_total{grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}) - sum(grpc_server_handled_total{grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"})`</td></tr><tr><td>watch</td><td>`sum(grpc_server_started_total{grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}) - sum(grpc_server_handled_total{grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"})`</td></tr></table> |
### Raft Proposals
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>applied</td><td>`sum(increase(etcd_server_proposals_applied_total[5m])) by (instance)`</td></tr><tr><td>committed</td><td>`sum(increase(etcd_server_proposals_committed_total[5m])) by (instance)`</td></tr><tr><td>pending</td><td>`sum(increase(etcd_server_proposals_pending[5m])) by (instance)`</td></tr><tr><td>failed</td><td>`sum(increase(etcd_server_proposals_failed_total[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>applied</td><td>`sum(increase(etcd_server_proposals_applied_total[5m]))`</td></tr><tr><td>committed</td><td>`sum(increase(etcd_server_proposals_committed_total[5m]))`</td></tr><tr><td>pending</td><td>`sum(increase(etcd_server_proposals_pending[5m]))`</td></tr><tr><td>failed</td><td>`sum(increase(etcd_server_proposals_failed_total[5m]))`</td></tr></table> |
### RPC Rate
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>total</td><td>`sum(rate(grpc_server_started_total{grpc_type="unary"}[5m])) by (instance)`</td></tr><tr><td>fail</td><td>`sum(rate(grpc_server_handled_total{grpc_type="unary",grpc_code!="OK"}[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>total</td><td>`sum(rate(grpc_server_started_total{grpc_type="unary"}[5m]))`</td></tr><tr><td>fail</td><td>`sum(rate(grpc_server_handled_total{grpc_type="unary",grpc_code!="OK"}[5m]))`</td></tr></table> |
### Disk Operations
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>commit-called-by-backend</td><td>`sum(rate(etcd_disk_backend_commit_duration_seconds_sum[1m])) by (instance)`</td></tr><tr><td>fsync-called-by-wal</td><td>`sum(rate(etcd_disk_wal_fsync_duration_seconds_sum[1m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>commit-called-by-backend</td><td>`sum(rate(etcd_disk_backend_commit_duration_seconds_sum[1m]))`</td></tr><tr><td>fsync-called-by-wal</td><td>`sum(rate(etcd_disk_wal_fsync_duration_seconds_sum[1m]))`</td></tr></table> |
### Disk Sync Duration
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>wal</td><td>`histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[5m])) by (instance, le))`</td></tr><tr><td>db</td><td>`histogram_quantile(0.99, sum(rate(etcd_disk_backend_commit_duration_seconds_bucket[5m])) by (instance, le))`</td></tr></table> |
| Summary | <table><tr><td>wal</td><td>`sum(histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[5m])) by (instance, le)))`</td></tr><tr><td>db</td><td>`sum(histogram_quantile(0.99, sum(rate(etcd_disk_backend_commit_duration_seconds_bucket[5m])) by (instance, le)))`</td></tr></table> |
## Kubernetes Components Metrics
### API Server Request Latency
| Catalog | Expression |
| --- | --- |
| Detail | `avg(apiserver_request_latencies_sum / apiserver_request_latencies_count) by (instance, verb) /1e+06` |
| Summary | `avg(apiserver_request_latencies_sum / apiserver_request_latencies_count) by (instance) /1e+06` |
### API Server Request Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(apiserver_request_count[5m])) by (instance, code)` |
| Summary | `sum(rate(apiserver_request_count[5m])) by (instance)` |
### Scheduling Failed Pods
| Catalog | Expression |
| --- | --- |
| Detail | `sum(kube_pod_status_scheduled{condition="false"})` |
| Summary | `sum(kube_pod_status_scheduled{condition="false"})` |
### Controller Manager Queue Depth
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>volumes</td><td>`sum(volumes_depth) by instance`</td></tr><tr><td>deployment</td><td>`sum(deployment_depth) by instance`</td></tr><tr><td>replicaset</td><td>`sum(replicaset_depth) by instance`</td></tr><tr><td>service</td><td>`sum(service_depth) by instance`</td></tr><tr><td>serviceaccount</td><td>`sum(serviceaccount_depth) by instance`</td></tr><tr><td>endpoint</td><td>`sum(endpoint_depth) by instance`</td></tr><tr><td>daemonset</td><td>`sum(daemonset_depth) by instance`</td></tr><tr><td>statefulset</td><td>`sum(statefulset_depth) by instance`</td></tr><tr><td>replicationmanager</td><td>`sum(replicationmanager_depth) by instance`</td></tr></table> |
| Summary | <table><tr><td>volumes</td><td>`sum(volumes_depth)`</td></tr><tr><td>deployment</td><td>`sum(deployment_depth)`</td></tr><tr><td>replicaset</td><td>`sum(replicaset_depth)`</td></tr><tr><td>service</td><td>`sum(service_depth)`</td></tr><tr><td>serviceaccount</td><td>`sum(serviceaccount_depth)`</td></tr><tr><td>endpoint</td><td>`sum(endpoint_depth)`</td></tr><tr><td>daemonset</td><td>`sum(daemonset_depth)`</td></tr><tr><td>statefulset</td><td>`sum(statefulset_depth)`</td></tr><tr><td>replicationmanager</td><td>`sum(replicationmanager_depth)`</td></tr></table> |
### Scheduler E2E Scheduling Latency
| Catalog | Expression |
| --- | --- |
| Detail | `histogram_quantile(0.99, sum(scheduler_e2e_scheduling_latency_microseconds_bucket) by (le, instance)) / 1e+06` |
| Summary | `sum(histogram_quantile(0.99, sum(scheduler_e2e_scheduling_latency_microseconds_bucket) by (le, instance)) / 1e+06)` |
### Scheduler Preemption Attempts
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(scheduler_total_preemption_attempts[5m])) by (instance)` |
| Summary | `sum(rate(scheduler_total_preemption_attempts[5m]))` |
### Ingress Controller Connections
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>reading</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="reading"}) by (instance)`</td></tr><tr><td>waiting</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="waiting"}) by (instance)`</td></tr><tr><td>writing</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="writing"}) by (instance)`</td></tr><tr><td>accepted</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="accepted"}[5m]))) by (instance)`</td></tr><tr><td>active</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="active"}[5m]))) by (instance)`</td></tr><tr><td>handled</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="handled"}[5m]))) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>reading</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="reading"})`</td></tr><tr><td>waiting</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="waiting"})`</td></tr><tr><td>writing</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="writing"})`</td></tr><tr><td>accepted</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="accepted"}[5m])))`</td></tr><tr><td>active</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="active"}[5m])))`</td></tr><tr><td>handled</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="handled"}[5m])))`</td></tr></table> |
### Ingress Controller Request Process Time
| Catalog | Expression |
| --- | --- |
| Detail | `topk(10, histogram_quantile(0.95,sum by (le, host, path)(rate(nginx_ingress_controller_request_duration_seconds_bucket{host!="_"}[5m]))))` |
| Summary | `topk(10, histogram_quantile(0.95,sum by (le, host)(rate(nginx_ingress_controller_request_duration_seconds_bucket{host!="_"}[5m]))))` |
## Rancher Logging Metrics
### Fluentd Buffer Queue Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(fluentd_output_status_buffer_queue_length[5m])) by (instance)` |
| Summary | `sum(rate(fluentd_output_status_buffer_queue_length[5m]))` |
### Fluentd Input Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(fluentd_input_status_num_records_total[5m])) by (instance)` |
| Summary | `sum(rate(fluentd_input_status_num_records_total[5m]))` |
### Fluentd Output Errors Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(fluentd_output_status_num_errors[5m])) by (type)` |
| Summary | `sum(rate(fluentd_output_status_num_errors[5m]))` |
### Fluentd Output Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(fluentd_output_status_num_records_total[5m])) by (instance)` |
| Summary | `sum(rate(fluentd_output_status_num_records_total[5m]))` |
## Workload Metrics
### Workload CPU Utilization
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>cfs throttled seconds</td><td>`sum(rate(container_cpu_cfs_throttled_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>user seconds</td><td>`sum(rate(container_cpu_user_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>system seconds</td><td>`sum(rate(container_cpu_system_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>usage seconds</td><td>`sum(rate(container_cpu_usage_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
| Summary | <table><tr><td>cfs throttled seconds</td><td>`sum(rate(container_cpu_cfs_throttled_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>user seconds</td><td>`sum(rate(container_cpu_user_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>system seconds</td><td>`sum(rate(container_cpu_system_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>usage seconds</td><td>`sum(rate(container_cpu_usage_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |
### Workload Memory Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `sum(container_memory_working_set_bytes{namespace="$namespace",pod_name=~"$podName", container_name!=""}) by (pod_name)` |
| Summary | `sum(container_memory_working_set_bytes{namespace="$namespace",pod_name=~"$podName", container_name!=""})` |
### Workload Network Packets
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive-packets</td><td>`sum(rate(container_network_receive_packets_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>receive-dropped</td><td>`sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>receive-errors</td><td>`sum(rate(container_network_receive_errors_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>transmit-packets</td><td>`sum(rate(container_network_transmit_packets_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>transmit-dropped</td><td>`sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>transmit-errors</td><td>`sum(rate(container_network_transmit_errors_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
| Summary | <table><tr><td>receive-packets</td><td>`sum(rate(container_network_receive_packets_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-dropped</td><td>`sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-errors</td><td>`sum(rate(container_network_receive_errors_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-packets</td><td>`sum(rate(container_network_transmit_packets_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-dropped</td><td>`sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-errors</td><td>`sum(rate(container_network_transmit_errors_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |
### Workload Network I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive</td><td>`sum(rate(container_network_receive_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>transmit</td><td>`sum(rate(container_network_transmit_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
| Summary | <table><tr><td>receive</td><td>`sum(rate(container_network_receive_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit</td><td>`sum(rate(container_network_transmit_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |
### Workload Disk I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
| Summary | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |
## Pod Metrics
### Pod CPU Utilization
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>cfs throttled seconds</td><td>`sum(rate(container_cpu_cfs_throttled_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>usage seconds</td><td>`sum(rate(container_cpu_usage_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>system seconds</td><td>`sum(rate(container_cpu_system_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>user seconds</td><td>`sum(rate(container_cpu_user_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m])) by (container_name)`</td></tr></table> |
| Summary | <table><tr><td>cfs throttled seconds</td><td>`sum(rate(container_cpu_cfs_throttled_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m]))`</td></tr><tr><td>usage seconds</td><td>`sum(rate(container_cpu_usage_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m]))`</td></tr><tr><td>system seconds</td><td>`sum(rate(container_cpu_system_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m]))`</td></tr><tr><td>user seconds</td><td>`sum(rate(container_cpu_user_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m]))`</td></tr></table> |
### Pod Memory Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `sum(container_memory_working_set_bytes{container_name!="POD",namespace="$namespace",pod_name="$podName",container_name!=""}) by (container_name)` |
| Summary | `sum(container_memory_working_set_bytes{container_name!="POD",namespace="$namespace",pod_name="$podName",container_name!=""})` |
### Pod Network Packets
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive-packets</td><td>`sum(rate(container_network_receive_packets_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-dropped</td><td>`sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-errors</td><td>`sum(rate(container_network_receive_errors_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-packets</td><td>`sum(rate(container_network_transmit_packets_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-dropped</td><td>`sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-errors</td><td>`sum(rate(container_network_transmit_errors_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
| Summary | <table><tr><td>receive-packets</td><td>`sum(rate(container_network_receive_packets_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-dropped</td><td>`sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-errors</td><td>`sum(rate(container_network_receive_errors_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-packets</td><td>`sum(rate(container_network_transmit_packets_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-dropped</td><td>`sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-errors</td><td>`sum(rate(container_network_transmit_errors_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
### Pod Network I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive</td><td>`sum(rate(container_network_receive_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit</td><td>`sum(rate(container_network_transmit_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
| Summary | <table><tr><td>receive</td><td>`sum(rate(container_network_receive_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit</td><td>`sum(rate(container_network_transmit_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
### Pod Disk I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m])) by (container_name)`</td></tr></table> |
| Summary | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
## Container Metrics
### Container CPU Utilization
| Catalog | Expression |
| --- | --- |
| cfs throttled seconds | `sum(rate(container_cpu_cfs_throttled_seconds_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
| usage seconds | `sum(rate(container_cpu_usage_seconds_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
| system seconds | `sum(rate(container_cpu_system_seconds_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
| user seconds | `sum(rate(container_cpu_user_seconds_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
### Container Memory Utilization
`sum(container_memory_working_set_bytes{namespace="$namespace",pod_name="$podName",container_name="$containerName"})`
### Container Disk I/O
| Catalog | Expression |
| --- | --- |
| read | `sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
| write | `sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
@@ -1,72 +0,0 @@
---
title: Project Monitoring
---
_Available as of v2.2.4_
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution.
### Monitoring Scope
Using Prometheus, you can monitor Rancher at both the [cluster level](cluster-monitoring.md) and project level. For each cluster and project that is enabled for monitoring, Rancher deploys a Prometheus server.
- [Cluster monitoring](cluster-monitoring.md/) allows you to view the health of your Kubernetes cluster. Prometheus collects metrics from the cluster components below, which you can view in graphs and charts.
- Kubernetes control plane
- etcd database
- All nodes (including workers)
- Project monitoring allows you to view the state of pods running in a given project. Prometheus collects metrics from the project's deployed HTTP and TCP/UDP workloads.
### Permissions to Configure Project Monitoring
Only [administrators](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md), [cluster owners or members](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#cluster-roles), or [project owners](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#project-roles) can configure project level monitoring. Project members can only view monitoring metrics.
### Enabling Project Monitoring
> **Prerequisite:** Cluster monitoring must be [enabled.](cluster-monitoring.md)
1. Go to the project where monitoring should be enabled. Note: When cluster monitoring is enabled, monitoring is also enabled by default in the **System** project.
1. Select **Tools > Monitoring** in the navigation bar.
1. Select **Enable** to show the [Prometheus configuration options](./prometheus.md). Enter in your desired configuration options.
1. Click **Save**.
### Project-Level Monitoring Resource Requirements
Container| CPU - Request | Mem - Request | CPU - Limit | Mem - Limit | Configurable
---------|---------------|---------------|-------------|-------------|-------------
Prometheus|750m| 750Mi | 1000m | 1000Mi | Yes
Grafana | 100m | 100Mi | 200m | 200Mi | No
**Result:** A single application,`project-monitoring`, is added as an [application](../../../how-to-guides/new-user-guides/helm-charts-in-rancher/helm-charts-in-rancher.md) to the project. After the application is `active`, you can start viewing project metrics through the [Rancher dashboard](cluster-monitoring.md/) or directly from Grafana.
> The default username and password for the Grafana instance will be `admin/admin`. However, Grafana dashboards are served via the Rancher authentication proxy, so only users who are currently authenticated into the Rancher server have access to the Grafana dashboard.
### Project Metrics
[Workload metrics](./expression.md#workload-metrics) are available for the project if monitoring is enabled at the [cluster level](cluster-monitoring.md/) and at the [project level.](#enabling-project-monitoring)
You can monitor custom metrics from any [exporters.](https://prometheus.io/docs/instrumenting/exporters/) You can also expose some custom endpoints on deployments without needing to configure Prometheus for your project.
> **Example:**
> A [Redis](https://redis.io/) application is deployed in the namespace `redis-app` in the project `Datacenter`. It is monitored via [Redis exporter](https://github.com/oliver006/redis_exporter). After enabling project monitoring, you can edit the application to configure the <b>Advanced Options -> Custom Metrics</b> section. Enter the `Container Port` and `Path` and select the `Protocol`.
To access a project-level Grafana instance,
1. From the **Global** view, navigate to a cluster that has monitoring enabled.
1. Go to a project that has monitoring enabled.
1. From the project view, click **Apps.** In versions before v2.2.0, choose **Catalog Apps** on the main navigation bar.
1. Go to the `project-monitoring` application.
1. In the `project-monitoring` application, there are two `/index.html` links: one that leads to a Grafana instance and one that leads to a Prometheus instance. When you click the Grafana link, it will redirect you to a new webpage for Grafana, which shows metrics for the cluster.
1. You will be signed in to the Grafana instance automatically. The default username is `admin` and the default password is `admin`. For security, we recommend that you log out of Grafana, log back in with the `admin` password, and change your password.
**Results:** You will be logged into Grafana from the Grafana instance. After logging in, you can view the preset Grafana dashboards, which are imported via the [Grafana provisioning mechanism](https://grafana.com/docs/grafana/latest/administration/provisioning/#dashboards), so you cannot modify them directly. For now, if you want to configure your own dashboards, clone the original and modify the new copy.
@@ -1,105 +0,0 @@
---
title: Prometheus Configuration
---
_Available as of v2.2.0_
While configuring monitoring at either the [cluster level](cluster-monitoring.md) or [project level](./project-monitoring.md), there are multiple options that can be configured.
- [Basic Configuration](#basic-configuration)
- [Advanced Options](#advanced-options)
- [Node Exporter](#node-exporter)
- [Persistent Storage](#persistent-storage)
- [Remote Storage](#remote-storage)
## Basic Configuration
Option | Description
-------|-------------
Data Retention | How long your Prometheus instance retains monitoring data scraped from Rancher objects before it's purged.
[Enable Node Exporter](#node-exporter) | Whether or not to deploy the node exporter.
Node Exporter Host Port | The host port on which data is exposed, i.e. data that Prometheus collects from your node hardware. Required if you have enabled the node exporter.
[Enable Persistent Storage](#persistent-storage) for Prometheus | Whether or not to configure storage for Prometheus so that metrics can be retained even if the Prometheus pod fails.
[Enable Persistent Storage](#persistent-storage) for Grafana | Whether or not to configure storage for Grafana so that the Grafana dashboards and configuration can be retained even if the Grafana pod fails.
Prometheus [CPU Limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu) | CPU resource limit for the Prometheus pod.
Prometheus [CPU Reservation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu) | CPU reservation for the Prometheus pod.
Prometheus [Memory Limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory) | Memory resource limit for the Prometheus pod.
Prometheus [Memory Reservation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory) | Memory resource requests for the Prometheus pod.
Selector | Ability to select the nodes in which Prometheus and Grafana pods are deployed to. To use this option, the nodes must have labels.
## Advanced Options
Since monitoring is an [application](https://github.com/rancher/system-charts/tree/dev/charts/rancher-monitoring) from the [Rancher catalog](../../../how-to-guides/new-user-guides/helm-charts-in-rancher/helm-charts-in-rancher.md), it can be configured like any other catalog application, by passing in values to Helm.
> **Warning:** Any modification to the application without understanding the entire application can lead to catastrophic errors.
### Prometheus RemoteRead and RemoteWrite
_Available as of v2.4.0_
Prometheus RemoteRead and RemoteWrite can be configured as custom answers in the **Advanced Options** section.
For more information on remote endpoints and storage, refer to the [Prometheus documentation.](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage)
The Prometheus operator documentation contains the full [RemoteReadSpec](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#remotereadspec) and [RemoteWriteSpec.](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#remotewritespec)
An example configuration would be:
| Variable | Value |
|--------------|------------|
| `prometheus.remoteWrite[0].url` | `http://mytarget.com` |
### LivenessProbe and ReadinessProbe
_Available as of v2.4.0_
Prometheus LivenessProbe and ReadinessProbe can be configured as custom answers in the **Advanced Options** section.
The Kubernetes probe spec is [here.](https://v1-17.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.17/#probe-v1-core)
Some example key-value pairs are:
| Variable | Value |
|--------------|------------|
| `prometheus.livenessProbe.timeoutSeconds` | 60 |
| `prometheus.readinessProbe.timeoutSeconds` | 60 |
## Node Exporter
The [node exporter](https://github.com/prometheus/node_exporter/blob/master/README.md) is a popular open source exporter, which exposes the metrics for hardware and \*NIX kernels OS. It is designed to monitor the host system. However, there are still issues with namespaces when running it in a container, mostly around filesystem mount spaces. In order to monitor actual network metrics for the container network, the node exporter must be deployed with the `hostNetwork` mode.
When configuring Prometheus and enabling the node exporter, enter a host port in the **Node Exporter Host Port** that will not produce port conflicts with existing applications. The host port chosen must be open to allow internal traffic between Prometheus and the Node Exporter.
>**Warning:** In order for Prometheus to collect the metrics of the node exporter, after enabling cluster monitoring, you must open the <b>Node Exporter Host Port</b> in the host firewall rules to allow intranet access. By default, `9796` is used as that host port.
## Persistent Storage
>**Prerequisite:** Configure one or more StorageClasses to use as [persistent storage](../../../how-to-guides/advanced-user-guides/manage-clusters/create-kubernetes-persistent-storage/create-kubernetes-persistent-storage.md) for your Prometheus or Grafana pod.
By default, when you enable Prometheus for either a cluster or project, all monitoring data that Prometheus collects is stored on its own pod. With local storage, if the Prometheus or Grafana pods fail, all the data is lost. Rancher recommends configuring an external persistent storage to the cluster. With the external persistent storage, if the Prometheus or Grafana pods fail, the new pods can recover using data from the persistent storage.
When enabling persistent storage for Prometheus or Grafana, specify the size of the persistent volume and select the StorageClass.
## Remote Storage
>**Prerequisite:** Need a remote storage endpoint to be available. The possible list of integrations is available [here](https://prometheus.io/docs/operating/integrations/)
Using advanced options, remote storage integration for the Prometheus installation can be configured as follows:
```
prometheus.remoteWrite[0].url = http://remote1/push
prometheus.remoteWrite[0].remoteTimeout = 33s
prometheus.remoteWrite[1].url = http://remote2/push
prometheus.remoteRead[0].url = http://remote1/read
prometheus.remoteRead[0].proxyUrl = http://proxy.url
prometheus.remoteRead[0].bearerToken = token-value
prometheus.remoteRead[1].url = http://remote2/read
prometheus.remoteRead[1].remoteTimeout = 33s
prometheus.remoteRead[1].readRecent = true
```
Additional fields can be set up based on the [ReadSpec](https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#remotereadspec) and [RemoteWriteSpec](https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#remotewritespec)
@@ -1,59 +0,0 @@
---
title: Viewing Metrics
---
_Available as of v2.2.0_
After you've enabled monitoring at either the [cluster level](cluster-monitoring.md) or [project level](./project-monitoring.md), you will want to be start viewing the data being collected. There are multiple ways to view this data.
## Rancher Dashboard
>**Note:** This is only available if you've enabled monitoring at the [cluster level](cluster-monitoring.md). Project specific analytics must be viewed using the project's Grafana instance.
Rancher's dashboards are available at multiple locations:
- **Cluster Dashboard**: From the **Global** view, navigate to the cluster.
- **Node Metrics**: From the **Global** view, navigate to the cluster. Select **Nodes**. Find the individual node and click on its name. Click **Node Metrics.**
- **Workload Metrics**: From the **Global** view, navigate to the project. From the main navigation bar, choose **Resources > Workloads.** (In versions before v2.3.0, choose **Workloads** on the main navigation bar.) Find the individual workload and click on its name. Click **Workload Metrics.**
- **Pod Metrics**: From the **Global** view, navigate to the project. Select **Workloads > Workloads**. Find the individual workload and click on its name. Find the individual pod and click on its name. Click **Pod Metrics.**
- **Container Metrics**: From the **Global** view, navigate to the project. From the main navigation bar, choose **Resources > Workloads.** (In versions before v2.3.0, choose **Workloads** on the main navigation bar.) Find the individual workload and click on its name. Find the individual pod and click on its name. Find the individual container and click on its name. Click **Container Metrics.**
Prometheus metrics are displayed and are denoted with the Grafana icon. If you click on the icon, the metrics will open a new tab in Grafana.
Within each Prometheus metrics widget, there are several ways to customize your view.
- Toggle between two views:
- **Detail**: Displays graphs and charts that let you view each event in a Prometheus time series
- **Summary** Displays events in a Prometheus time series that are outside the norm.
- Change the range of the time series that you're viewing to see a more refined or expansive data sample.
- Customize the data sample to display data between specific dates and times.
When analyzing these metrics, don't be concerned about any single standalone metric in the charts and graphs. Rather, you should establish a baseline for your metrics over the course of time, e.g. the range of values that your components usually operate within and are considered normal. After you establish the baseline, be on the lookout for any large deltas in the charts and graphs, as these big changes usually indicate a problem that you need to investigate.
## Grafana
If you've enabled monitoring at either the [cluster level](cluster-monitoring.md) or [project level](./project-monitoring.md), Rancher automatically creates a link to Grafana instance. Use this link to view monitoring data.
Grafana allows you to query, visualize, alert, and ultimately, understand your cluster and workload data. For more information on Grafana and its capabilities, visit the [Grafana website](https://grafana.com/grafana).
### Authentication
Rancher determines which users can access the new Grafana instance, as well as the objects they can view within it, by validating them against the user's [cluster or project roles](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md). In other words, a user's access in Grafana mirrors their access in Rancher.
When you go to the Grafana instance, you will be logged in with the username `admin` and the password `admin`. If you log out and log in again, you will be prompted to change your password. You will only have access to the URL of the Grafana instance if you have access to view the corresponding metrics in Rancher. So for example, if your Rancher permissions are scoped to the project level, you won't be able to see the Grafana instance for cluster-level metrics.
### Accessing the Cluster-level Grafana Instance
1. From the **Global** view, navigate to a cluster that has monitoring enabled.
1. Go to the **System** project view. This project is where the cluster-level Grafana instance runs.
1. Click **Apps.** In versions before v2.2.0, choose **Catalog Apps** on the main navigation bar.
1. Go to the `cluster-monitoring` application.
1. In the `cluster-monitoring` application, there are two `/index.html` links: one that leads to a Grafana instance and one that leads to a Prometheus instance. When you click the Grafana link, it will redirect you to a new webpage for Grafana, which shows metrics for the cluster.
1. You will be signed in to the Grafana instance automatically. The default username is `admin` and the default password is `admin`. For security, we recommend that you log out of Grafana, log back in with the `admin` password, and change your password.
**Results:** You are logged into Grafana from the Grafana instance. After logging in, you can view the preset Grafana dashboards, which are imported via the [Grafana provisioning mechanism](https://grafana.com/docs/grafana/latest/administration/provisioning/#dashboards), so you cannot modify them directly. For now, if you want to configure your own dashboards, clone the original and modify the new copy.
@@ -1,11 +0,0 @@
---
title: Integrations in Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/pages-for-subheaders/cloud-marketplace"/>
</head>
Over time, Rancher has accrued several products and projects that have been integrated into the Rancher UI.
Examples of some of these integrations are [Istio](istio/istio.md) and [CIS Scans](cis-scans/cis-scans.md).
@@ -1,153 +0,0 @@
---
title: CPU and Memory Allocations
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/cpu-and-memory-allocations"/>
</head>
_Available as of v2.3.0_
This section describes the minimum recommended computing resources for the Istio components in a cluster.
The CPU and memory allocations for each component are [configurable.](#configuring-resource-allocations)
Before enabling Istio, we recommend that you confirm that your Rancher worker nodes have enough CPU and memory to run all of the components of Istio.
> **Tip:** In larger deployments, it is strongly advised that the infrastructure be placed on dedicated nodes in the cluster by adding a node selector for each Istio component.
The table below shows a summary of the minimum recommended resource requests and limits for the CPU and memory of each central Istio component.
In Kubernetes, the resource request indicates that the workload will not deployed on a node unless the node has at least the specified amount of memory and CPU available. If the workload surpasses the limit for CPU or memory, it can be terminated or evicted from the node. For more information on managing resource limits for containers, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
Workload | Container | CPU - Request | Mem - Request | CPU - Limit | Mem - Limit | Configurable
---------|-----------|---------------|---------------|-------------|-------------|-------------
istio-pilot |discovery| 500m | 2048Mi | 1000m | 4096Mi | Y
istio-telemetry |mixer| 1000m | 1024Mi | 4800m | 4096Mi | Y
istio-policy | mixer | 1000m | 1024Mi | 4800m | 4096Mi | Y
istio-tracing | jaeger | 100m | 100Mi | 500m | 1024Mi | Y
prometheus | prometheus | 750m | 750Mi | 1000m | 1024Mi | Y
grafana | grafana | 100m | 100Mi | 200m | 512Mi | Y
Others | - | 500m | 500Mi | - | - | N
**Total** | **-** | **3950m** | **5546Mi** | **>12300m** | **>14848Mi** | **-**
## Configuring Resource Allocations
You can individually configure the resource allocation for each type of Istio component. This section includes the default resource allocations for each component.
To make it easier to schedule the workloads to a node, a cluster administrator can reduce the CPU and memory resource requests for the component. However, the default CPU and memory allocations are the minimum that we recommend.
You can find more information about Istio configuration in the [official Istio documentation](https://istio.io/docs/concepts/what-is-istio).
To configure the resources allocated to an Istio component,
1. In Rancher, go to the cluster where you have Istio installed.
1. Click **Tools > Istio.** This opens the Istio configuration page.
1. Change the CPU or memory allocations, the nodes where each component will be scheduled to, or the node tolerations.
1. Click **Save.**
**Result:** The resource allocations for the Istio components are updated.
## Pilot
[Pilot](https://istio.io/docs/ops/deployment/architecture/#pilot) provides the following:
- Authentication configuration
- Service discovery for the Envoy sidecars
- Traffic management capabilities for intelligent routing (A/B tests and canary rollouts)
- Configuration for resiliency (timeouts, retries, circuit breakers, etc)
For more information on Pilot, refer to the [documentation](https://istio.io/docs/concepts/traffic-management/#pilot-and-envoy).
Option | Description| Required | Default
-------|------------|-------|-------
Pilot CPU Limit | CPU resource limit for the istio-pilot pod.| Yes | 1000
Pilot CPU Reservation | CPU reservation for the istio-pilot pod. | Yes | 500
Pilot Memory Limit | Memory resource limit for the istio-pilot pod. | Yes | 4096
Pilot Memory Reservation | Memory resource requests for the istio-pilot pod. | Yes | 2048
Trace sampling Percentage | [Trace sampling percentage](https://istio.io/docs/tasks/telemetry/distributed-tracing/overview/#trace-sampling) | Yes | 1
Pilot Selector | Ability to select the nodes in which istio-pilot pod is deployed to. To use this option, the nodes must have labels. | No | n/a
## Mixer
[Mixer](https://istio.io/docs/ops/deployment/architecture/#mixer) enforces access control and usage policies across the service mesh. It also integrates with plugins for monitoring tools such as Prometheus. The Envoy sidecar proxy passes telemetry data and monitoring data to Mixer, and Mixer passes the monitoring data to Prometheus.
For more information on Mixer, policies and telemetry, refer to the [documentation](https://istio.io/docs/concepts/policies-and-telemetry/).
Option | Description| Required | Default
-------|------------|-------|-------
Mixer Telemetry CPU Limit | CPU resource limit for the istio-telemetry pod.| Yes | 4800
Mixer Telemetry CPU Reservation | CPU reservation for the istio-telemetry pod.| Yes | 1000
Mixer Telemetry Memory Limit | Memory resource limit for the istio-telemetry pod.| Yes | 4096
Mixer Telemetry Memory Reservation | Memory resource requests for the istio-telemetry pod.| Yes | 1024
Enable Mixer Policy | Whether or not to deploy the istio-policy. | Yes | False
Mixer Policy CPU Limit | CPU resource limit for the istio-policy pod. | Yes, when policy enabled | 4800
Mixer Policy CPU Reservation | CPU reservation for the istio-policy pod. | Yes, when policy enabled | 1000
Mixer Policy Memory Limit | Memory resource limit for the istio-policy pod. | Yes, when policy enabled | 4096
Mixer Policy Memory Reservation | Memory resource requests for the istio-policy pod. | Yes, when policy enabled | 1024
Mixer Selector | Ability to select the nodes in which istio-policy and istio-telemetry pods are deployed to. To use this option, the nodes must have labels. | No | n/a
## Tracing
[Distributed tracing](https://istio.io/docs/tasks/telemetry/distributed-tracing/overview/) enables users to track a request through a service mesh. This makes it easier to troubleshoot problems with latency, parallelism and serialization.
Option | Description| Required | Default
-------|------------|-------|-------
Enable Tracing | Whether or not to deploy the istio-tracing. | Yes | True
Tracing CPU Limit | CPU resource limit for the istio-tracing pod. | Yes | 500
Tracing CPU Reservation | CPU reservation for the istio-tracing pod. | Yes | 100
Tracing Memory Limit | Memory resource limit for the istio-tracing pod. | Yes | 1024
Tracing Memory Reservation | Memory resource requests for the istio-tracing pod. | Yes | 100
Tracing Selector | Ability to select the nodes in which tracing pod is deployed to. To use this option, the nodes must have labels. | No | n/a
## Ingress Gateway
The Istio gateway allows Istio features such as monitoring and route rules to be applied to traffic entering the cluster. This gateway is a prerequisite for outside traffic to make requests to Istio.
For more information, refer to the [documentation](https://istio.io/docs/tasks/traffic-management/ingress/).
Option | Description| Required | Default
-------|------------|-------|-------
Enable Ingress Gateway | Whether or not to deploy the istio-ingressgateway. | Yes | False
Service Type of Istio Ingress Gateway | How to expose the gateway. You can choose NodePort or Loadbalancer | Yes | NodePort
Http2 Port | The NodePort for http2 requests | Yes | 31380
Https Port | The NodePort for https requests | Yes | 31390
Load Balancer IP | Ingress Gateway Load Balancer IP | No | n/a
Load Balancer Source Ranges | Ingress Gateway Load Balancer Source Ranges | No | n/a
Ingress Gateway CPU Limit | CPU resource limit for the istio-ingressgateway pod. | Yes | 2000
Ingress Gateway CPU Reservation | CPU reservation for the istio-ingressgateway pod. | Yes | 100
Ingress Gateway Memory Limit | Memory resource limit for the istio-ingressgateway pod. | Yes | 1024
Ingress Gateway Memory Reservation | Memory resource requests for the istio-ingressgateway pod. | Yes | 128
Ingress Gateway Selector | Ability to select the nodes in which istio-ingressgateway pod is deployed to. To use this option, the nodes must have labels. | No | n/a
## Prometheus
You can query for Istio metrics using Prometheus. Prometheus is an open-source systems monitoring and alerting toolkit.
Option | Description| Required | Default
-------|------------|-------|-------
Prometheus CPU Limit | CPU resource limit for the Prometheus pod.| Yes | 1000
Prometheus CPU Reservation | CPU reservation for the Prometheus pod.| Yes | 750
Prometheus Memory Limit | Memory resource limit for the Prometheus pod.| Yes | 1024
Prometheus Memory Reservation | Memory resource requests for the Prometheus pod.| Yes | 750
Retention for Prometheus | How long your Prometheus instance retains data | Yes | 6
Prometheus Selector | Ability to select the nodes in which Prometheus pod is deployed to. To use this option, the nodes must have labels.| No | n/a
## Grafana
You can visualize metrics with Grafana. Grafana lets you visualize Istio traffic data scraped by Prometheus.
Option | Description| Required | Default
-------|------------|-------|-------
Enable Grafana | Whether or not to deploy the Grafana.| Yes | True
Grafana CPU Limit | CPU resource limit for the Grafana pod.| Yes, when Grafana enabled | 200
Grafana CPU Reservation | CPU reservation for the Grafana pod.| Yes, when Grafana enabled | 100
Grafana Memory Limit | Memory resource limit for the Grafana pod.| Yes, when Grafana enabled | 512
Grafana Memory Reservation | Memory resource requests for the Grafana pod.| Yes, when Grafana enabled | 100
Grafana Selector | Ability to select the nodes in which Grafana pod is deployed to. To use this option, the nodes must have labels. | No | n/a
Enable Persistent Storage for Grafana | Enable Persistent Storage for Grafana | Yes, when Grafana enabled | False
Source | Use a Storage Class to provision a new persistent volume or Use an existing persistent volume claim | Yes, when Grafana enabled and enabled PV | Use SC
Storage Class | Storage Class for provisioning PV for Grafana | Yes, when Grafana enabled, enabled PV and use storage class | Use the default class
Persistent Volume Size | The size for the PV you would like to provision for Grafana | Yes, when Grafana enabled, enabled PV and use storage class | 5Gi
Existing Claim | Use existing PVC for Grafana | Yes, when Grafana enabled, enabled PV and use existing PVC | n/a
@@ -1,30 +0,0 @@
---
title: Disabling Istio
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/disable-istio"/>
</head>
This section describes how to disable Istio in a cluster, namespace, or workload.
## Disable Istio in a Cluster
To disable Istio,
1. From the **Global** view, navigate to the cluster that you want to disable Istio for.
1. Click **Tools > Istio.**
1. Click **Disable,** then click the red button again to confirm the disable action.
**Result:** The `cluster-istio` application in the cluster's `system` project gets removed. The Istio sidecar cannot be deployed on any workloads in the cluster.
## Disable Istio in a Namespace
1. In the Rancher UI, go to the project that has the namespace where you want to disable Istio.
1. On the **Workloads** tab, you will see a list of namespaces and the workloads deployed in them. Go to the namespace where you want to disable and click the **&#8942; > Disable Istio Auto Injection.**
**Result:** When workloads are deployed in this namespace, they will not have the Istio sidecar.
## Remove the Istio Sidecar from a Workload
Disable Istio in the namespace, then redeploy the workloads with in it. They will be deployed without the Istio sidecar.
@@ -1,90 +0,0 @@
---
title: Istio
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio"/>
</head>
_Available as of v2.3.0_
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, control, troubleshoot, and secure the traffic within a complex network of microservices.
As a network of microservices changes and grows, the interactions between them can become more difficult to manage and understand. In such a situation, it is useful to have a service mesh as a separate infrastructure layer. Istio's service mesh lets you manipulate traffic between microservices without changing the microservices directly.
Our integration of Istio is designed so that a Rancher operator, such as an administrator or cluster owner, can deliver Istio to developers. Then developers can use Istio to enforce security policies, troubleshoot problems, or manage traffic for green/blue deployments, canary deployments, or A/B testing.
This service mesh provides features that include but are not limited to the following:
- Traffic management features
- Enhanced monitoring and tracing
- Service discovery and routing
- Secure connections and service-to-service authentication with mutual TLS
- Load balancing
- Automatic retries, backoff, and circuit breaking
After Istio is enabled in a cluster, you can leverage Istio's control plane functionality with `kubectl`.
Rancher's Istio integration comes with comprehensive visualization aids:
- **Trace the root cause of errors with Jaeger.** [Jaeger](https://www.jaegertracing.io/) is an open-source tool that provides a UI for a distributed tracing system, which is useful for root cause analysis and for determining what causes poor performance. Distributed tracing allows you to view an entire chain of calls, which might originate with a user request and traverse dozens of microservices.
- **Get the full picture of your microservice architecture with Kiali.** [Kiali](https://www.kiali.io/) provides a diagram that shows the services within a service mesh and how they are connected, including the traffic rates and latencies between them. You can check the health of the service mesh, or drill down to see the incoming and outgoing requests to a single component.
- **Gain insights from time series analytics with Grafana dashboards.** [Grafana](https://grafana.com/) is an analytics platform that allows you to query, visualize, alert on and understand the data gathered by Prometheus.
- **Write custom queries for time series data with the Prometheus UI.** [Prometheus](https://prometheus.io/) is a systems monitoring and alerting toolkit. Prometheus scrapes data from your cluster, which is then used by Grafana. A Prometheus UI is also integrated into Rancher, and lets you write custom queries for time series data and see the results in the UI.
Istio needs to be set up by a Rancher administrator or cluster administrator before it can be used in a project.
## Prerequisites
Before enabling Istio, we recommend that you confirm that your Rancher worker nodes have enough [CPU and memory](cpu-and-memory-allocations.md) to run all of the components of Istio.
## Setup Guide
Refer to the [setup guide](../../../how-to-guides/advanced-user-guides/istio-setup-guide/istio-setup-guide.md) for instructions on how to set up Istio and use it in a project.
## Disabling Istio
To remove Istio components from a cluster, namespace, or workload, refer to the section on [disabling Istio.](disable-istio.md)
## Accessing Visualizations
> By default, only cluster owners have access to Jaeger and Kiali. For instructions on how to allow project members to access them, see [this section.](rbac-for-istio.md)
After Istio is set up in a cluster, Grafana, Prometheus, Jaeger, and Kiali are available in the Rancher UI.
Your access to the visualizations depend on your role. Grafana and Prometheus are only available for cluster owners. The Kiali and Jaeger UIs are available only to cluster owners by default, but cluster owners can allow project members to access them by editing the Istio settings. When you go to your project and click **Resources > Istio,** you can go to each UI for Kiali, Jaeger, Grafana, and Prometheus by clicking their icons in the top right corner of the page.
To see the visualizations, go to the cluster where Istio is set up and click **Tools > Istio.** You should see links to each UI at the top of the page.
You can also get to the visualization tools from the project view.
## Viewing the Kiali Traffic Graph
1. From the project view in Rancher, click **Resources > Istio.**
1. If you are a cluster owner, you can go to the **Traffic Graph** tab. This tab has the Kiali network visualization integrated into the UI.
## Viewing Traffic Metrics
Istio’s monitoring features provide visibility into the performance of all your services.
1. From the project view in Rancher, click **Resources > Istio.**
1. Go to the **Traffic Metrics** tab. After traffic is generated in your cluster, you should be able to see metrics for **Success Rate, Request Volume, 4xx Response Count, Project 5xx Response Count** and **Request Duration.** Cluster owners can see all of the metrics, while project members can see a subset of the metrics.
## Architecture
Istio installs a service mesh that uses [Envoy](https://www.envoyproxy.io) sidecar proxies to intercept traffic to each workload. These sidecars intercept and manage service-to-service communication, allowing fine-grained observation and control over traffic within the cluster.
Only workloads that have the Istio sidecar injected can be tracked and controlled by Istio.
Enabling Istio in Rancher enables monitoring in the cluster, and enables Istio in all new namespaces that are created in a cluster. You need to manually enable Istio in preexisting namespaces.
When a namespace has Istio enabled, new workloads deployed in the namespace will automatically have the Istio sidecar. You need to manually enable Istio in preexisting workloads.
For more information on the Istio sidecar, refer to the [Istio docs](https://istio.io/docs/setup/kubernetes/additional-setup/sidecar-injection/).
### Two Ingresses
By default, each Rancher-provisioned cluster has one NGINX ingress controller allowing traffic into the cluster. To allow Istio to receive external traffic, you need to enable the Istio ingress gateway for the cluster. The result is that your cluster will have two ingresses.
![In an Istio-enabled cluster, you can have two ingresses: the default Nginx ingress, and the default Istio controller.](/img/istio-ingress.svg)
@@ -1,61 +0,0 @@
---
title: Role-based Access Control
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/rbac-for-istio"/>
</head>
This section describes the permissions required to access Istio features and how to configure access to the Kiali and Jaeger visualizations.
## Cluster-level Access
By default, only cluster administrators can:
- Enable Istio for the cluster
- Configure resource allocations for Istio
- View each UI for Prometheus, Grafana, Kiali, and Jaeger
## Project-level Access
After Istio is enabled in a cluster, project owners and members have permission to:
- Enable and disable Istio sidecar auto-injection for namespaces
- Add the Istio sidecar to workloads
- View the traffic metrics and traffic graph for the cluster
- View the Kiali and Jaeger visualizations if cluster administrators give access to project members
- Configure Istio's resources (such as the gateway, destination rules, or virtual services) with `kubectl` (This does not apply to read-only project members)
## Access to Visualizations
By default, the Kiali and Jaeger visualizations are restricted to the cluster owner because the information in them could be sensitive.
**Jaeger** provides a UI for a distributed tracing system, which is useful for root cause analysis and for determining what causes poor performance.
**Kiali** provides a diagram that shows the services within a service mesh and how they are connected.
Rancher supports giving groups permission to access Kiali and Jaeger, but not individuals.
To configure who has permission to access the Kiali and Jaeger UI,
1. Go to the cluster view and click **Tools > Istio.**
1. Then go to the **Member Access** section. If you want to restrict access to certain groups, choose **Allow cluster owner and specified members to access Kiali and Jaeger UI.** Search for the groups that you want to have access to Kiali and Jaeger. If you want all members to have access to the tools, click **Allow all members to access Kiali and Jaeger UI.**
1. Click **Save.**
**Result:** The access levels for Kiali and Jaeger have been updated.
## Summary of Default Permissions for Istio Users
| Permission | Cluster Administrators | Project Owners | Project Members | Read-only Project Members |
|------------------------------------------|----------------|----------------|-----------------|---------------------------|
| Enable and disable Istio for the cluster | ✓ | | | |
| Configure Istio resource limits | ✓ | | | |
| Control who has access to Kiali and the Jaeger UI | ✓ | | | |
| Enable and disable Istio for a namespace | ✓ | ✓ | ✓ | |
| Enable and disable Istio on workloads | ✓ | ✓ | ✓ | |
| Configure Istio with `kubectl` | ✓ | ✓ | ✓ | |
| View Prometheus UI and Grafana UI | ✓ | | | |
| View Kiali UI and Jaeger UI ([Configurable](#access-to-visualizations)) | ✓ | | | |
| View Istio project dashboard, including traffic metrics* | ✓ | ✓ | ✓ | ✓ |
* By default, only the cluster owner will see the traffic graph. Project members will see only a subset of traffic metrics. Project members cannot see the traffic graph because it comes from Kiali, and access to Kiali is restricted to cluster owners by default.
@@ -1,17 +0,0 @@
---
title: Release Notes
---
## Istio 1.5.8
### Important note on 1.5.x versions
When upgrading from any 1.4 version of Istio to any 1.5 version, the Rancher installer will delete several resources in order to complete the upgrade, at which point they will be immediately re-installed. This includes the `istio-reader-service-account`. If your Istio installation is using this service account be aware that any secrets tied to the service account will be deleted. Most notably this will **break specific [multi-cluster deployments](https://archive.istio.io/v1.4/docs/setup/install/multicluster/)**. Downgrades back to 1.4 are not possible.
See the official upgrade notes for additional information on the 1.5 release and upgrading from 1.4: https://istio.io/latest/news/releases/1.5.x/announcing-1.5/upgrade-notes/
> **Note:** Rancher continues to use the Helm installation method, which produces a different architecture from an istioctl installation.
### Known Issues
* The Kiali traffic graph is currently not working [#24924](https://github.com/istio/istio/issues/24924)
@@ -1,193 +0,0 @@
---
title: Notifiers
---
Notifiers are services that inform you of alert events. You can configure notifiers to send alert notifications to staff best suited to take corrective action.
Rancher integrates with a variety of popular IT services, including:
- **Slack**: Send alert notifications to your Slack channels.
- **Email**: Choose email recipients for alert notifications.
- **PagerDuty**: Route notifications to staff by phone, SMS, or personal email.
- **WebHooks**: Update a webpage with alert notifications.
- **WeChat**: (Available as of v2.2.0) Send alert notifications to your Enterprise WeChat contacts.
- **DingTalk**: (Available as of v2.4.6) Send alert notifications to DingTalk using a webhook.
- **Microsoft Teams**: (Available as of v2.4.6) Send alert notifications to Teams using a webhook.
## Roles-based Access Control for Notifiers
Notifiers are configured at the cluster level. This model ensures that only cluster owners need to configure notifiers, leaving project owners to simply configure alerts in the scope of their projects. You don't need to dispense privileges like SMTP server access or cloud account access.
## Adding Notifiers
Set up a notifier so that you can begin configuring and sending alerts.
1. From the **Global View**, open the cluster that you want to add a notifier.
1. From the main menu, select **Tools > Notifiers**. Then click **Add Notifier**.
1. Select the service you want to use as your notifier, and then fill out the form. For help filling out the form, refer to the configuration section below.
1. Click **Test.** You should receive a notification confirming that the notifier is configured correctly.
1. Click **Add** to complete adding the notifier.
**Result:** Your notifier is added to Rancher.
## Configuration
- [Slack](#slack)
- [Email](#email)
- [PagerDuty](#pagerduty)
- [Webhook](#webhook)
- [WeChat](#wechat)
- [DingTalk](#dingtalk)
- [Microsoft Teams](#microsoft-teams)
### Slack
| Field | Explanation |
|----------|----------------------|
| Name | Enter a **Name** for the notifier. |
| URL | From Slack, create a webhook. For instructions, see the [Slack Documentation](https://get.slack.help/hc/en-us/articles/115005265063-Incoming-WebHooks-for-Slack). Then enter the Slack webhook URL. |
| Default Channel | Enter the name of the channel that you want to send alert notifications in the following format: `#<channelname>`. Both public and private channels are supported. |
| Proxy URL | Proxy for the Slack webhook. |
| Send Resolved Alerts | _Available as of v2.3.0_ Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage) |
**Validation:** Click **Test**. If the test is successful, the Slack channel you're configuring for the notifier outputs **Slack setting validated.**
### Email
| Field | Explanation |
|----------|----------------------|
| Name | Enter a **Name** for the notifier. |
| Default Recipient Address | Enter the email address that you want to receive the notification. |
| Send Resolved Alerts | _Available as of v2.3.0_ Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage) |
SMTP Server Configuration:
| Field | Explanation |
|----------|----------------------|
| Sender | Enter an email address available on your mail server that you want to send the notification. |
| Host | Enter the IP address or hostname for your SMTP server. Example: `smtp.email.com` |
| Port | In the **Port** field, enter the port used for email. Typically, TLS uses `587` and SSL uses `465`. |
| Use TLS | If you're using TLS, make sure **Use TLS** is selected. |
| Username | Username to authenticate with the SMTP server. |
| Password | Password to authenticate with the SMTP server. |
**Validation:** Click **Test**. If the test is successful, Rancher prints **settings validated** and you receive a test notification email.
### PagerDuty
| Field | Explanation |
|----------|----------------------|
| Name | Enter a **Name** for the notifier. |
| Default Integration Key | From PagerDuty, create a Prometheus integration. For instructions, see the [PagerDuty Documentation](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/). Then enter the integration key.
| Service Key | The same as the integration key. For instructions on creating a Prometheus integration, see the [PagerDuty Documentation](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/). Then enter the integration key. |
| Send Resolved Alerts | _Available as of v2.3.0_ Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage) |
**Validation:** Click **Test**. If the test is successful, your PagerDuty endpoint outputs **PagerDuty setting validated.**
### Webhook
| Field | Explanation |
|----------|----------------------|
| Name | Enter a **Name** for the notifier. |
| URL | Using the app of your choice, create a webhook URL. |
| Proxy URL | Proxy for the webhook. |
| Send Resolved Alerts | _Available as of v2.3.0_ Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage) |
**Validation:** Click **Test**. If the test is successful, the URL you're configuring as a notifier outputs **Webhook setting validated.**
### WeChat
_Available as of v2.2.0_
| Field | Explanation |
|----------|----------------------|
| Name | Enter a **Name** for the notifier. |
| Corporation ID | Enter the "EnterpriseID" of your corporation. You can get it fro the [Profile page](https://work.weixin.qq.com/wework_admin/frame#profile). |
| Application Agent ID | From Enterprise WeChat, create an application in the [Application page](https://work.weixin.qq.com/wework_admin/frame#apps), and then enter the "AgentId" of this application. You will also need to enter the application secret. |
| Application Secret | The secret that corresponds to the Application Agent ID. |
| Recipient Type | Party, tag, or user. |
| Default Recipient | The default recipient ID should correspond to the recipient type. It should be the party ID, tag ID or user account that you want to receive the notification. You could get contact information from [Contacts page](https://work.weixin.qq.com/wework_admin/frame#contacts). |
| Proxy URL | If you are using a proxy, enter the proxy URL. |
| Send Resolved Alerts | _Available as of v2.3.0_ Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage) |
**Validation:** Click **Test.** If the test is successful, you should receive an alert message.
### DingTalk
_Available as of v2.4.6_
| Field | Explanation |
|----------|----------------------|
| Name | Enter a **Name** for the notifier. |
| Webhook URL | Enter the DingTalk webhook URL. For help setting up the webhook, refer to the [DingTalk documentation.](https://www.alibabacloud.com/help/doc-detail/52872.htm) |
| Secret | Optional: Enter a secret for the DingTalk webhook. |
| Proxy URL | Optional: Enter a proxy for the DingTalk webhook. |
| Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage) |
**Validation:** Click **Test.** If the test is successful, the DingTalk notifier output is **DingTalk setting validated.**
### Microsoft Teams
_Available as of v2.4.6_
| Field | Explanation |
|----------|----------------------|
| Name | Enter a **Name** for the notifier. |
| Webhook URL | Enter the Microsoft Teams webhook URL. For help setting up the webhook, refer to the [Teams Documentation.](https://docs.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook) |
| Proxy URL | Optional: Enter a proxy for the Teams webhook. |
| Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage) |
**Validation:** Click **Test.** If the test is successful, the Teams notifier output is **MicrosoftTeams setting validated.**
## Managing Notifiers
After you set up notifiers, you can manage them. From the **Global** view, open the cluster that you want to manage your notifiers. Select **Tools > Notifiers**. You can:
- **Edit** their settings that you configured during their initial setup.
- **Clone** them, to quickly setup slightly different notifiers.
- **Delete** them when they're no longer necessary.
## Example Payload for a Webhook Alert Notifier
```json
{
"receiver": "c-2a3bc:kube-components-alert",
"status": "firing",
"alerts": [
{
"status": "firing",
"labels": {
"alert_name": "Scheduler is unavailable",
"alert_type": "systemService",
"cluster_name": "mycluster (ID: c-2a3bc)",
"component_name": "scheduler",
"group_id": "c-2a3bc:kube-components-alert",
"logs": "Get http://127.0.0.1:10251/healthz: dial tcp 127.0.0.1:10251: connect: connection refused",
"rule_id": "c-2a3bc:kube-components-alert_scheduler-system-service",
"severity": "critical"
},
"annotations": {},
"startsAt": "2020-01-30T19:18:13.321684733Z",
"endsAt": "0001-01-01T00:00:00Z",
"generatorURL": ""
}
],
"groupLabels": {
"component_name": "scheduler",
"rule_id": "c-2a3bc:kube-components-alert_scheduler-system-service"
},
"commonLabels": {
"alert_name": "Scheduler is unavailable",
"alert_type": "systemService",
"cluster_name": "mycluster (ID: c-2a3bc)"
}
}
```
## What's Next?
After creating a notifier, set up alerts to receive notifications of Rancher system events.
- [Cluster owners](../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#cluster-roles) can set up alerts at the [cluster level](cluster-alerts/cluster-alerts.md).
- [Project owners](../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md#project-roles) can set up alerts at the [project level](../../reference-guides/rancher-project-tools/project-alerts.md).
@@ -1,99 +0,0 @@
---
title: OPA Gatekeeper
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/opa-gatekeeper"/>
</head>
_Available as of v2.4.0_
To ensure consistency and compliance, every organization needs the ability to define and enforce policies in its environment in an automated way. [OPA (Open Policy Agent)](https://www.openpolicyagent.org/) is a policy engine that facilitates policy-based control for cloud native environments. Rancher provides the ability to enable OPA Gatekeeper in Kubernetes clusters, and also installs a couple of built-in policy definitions, which are also called constraint templates.
OPA provides a high-level declarative language that lets you specify policy as code and ability to extend simple APIs to offload policy decision-making.
[OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper) is a project that provides integration between OPA and Kubernetes. OPA Gatekeeper provides:
- An extensible, parameterized policy library.
- Native Kubernetes CRDs for instantiating the policy library, also called “constraints."
- Native Kubernetes CRDs for extending the policy library, also called "constraint templates."
- Audit functionality.
To read more about OPA, please refer to the [official documentation.](https://www.openpolicyagent.org/docs/latest/)
## How the OPA Gatekeeper Integration Works
Kubernetes provides the ability to extend API server functionality via admission controller webhooks, which are invoked whenever a resource is created, updated or deleted. Gatekeeper is installed as a validating webhook and enforces policies defined by Kubernetes custom resource definitions. In addition to the admission control usage, Gatekeeper provides the capability to audit existing resources in Kubernetes clusters and mark current violations of enabled policies.
OPA Gatekeeper is made available via Rancher's Helm system chart, and it is installed in a namespace named `gatekeeper-system.`
## Enabling OPA Gatekeeper in a Cluster
> **Prerequisites:**
>
> - Only administrators and cluster owners can enable OPA Gatekeeper.
> - The dashboard needs to be enabled using the `dashboard` feature flag. For more information, refer to the [section on enabling experimental features.](../../getting-started/installation-and-upgrade/advanced-options/enable-experimental-features/enable-experimental-features.md)
1. Navigate to the cluster's **Dashboard** view.
1. On the left side menu, expand the cluster menu and click on **OPA Gatekeeper.**
1. To install Gatekeeper with the default configuration, click on **Enable Gatekeeper (v0.1.0) with defaults.**
1. To change any default configuration, click on **Customize Gatekeeper yaml configuration.**
## Constraint Templates
[Constraint templates](https://github.com/open-policy-agent/gatekeeper#constraint-templates) are Kubernetes custom resources that define the schema and Rego logic of the OPA policy to be applied by Gatekeeper. For more information on the Rego policy language, refer to the [official documentation.](https://www.openpolicyagent.org/docs/latest/policy-language/)
When OPA Gatekeeper is enabled, Rancher installs some templates by default.
To list the constraint templates installed in the cluster, go to the left side menu under OPA Gatekeeper and click on **Templates.**
Rancher also provides the ability to create your own constraint templates by importing YAML definitions.
## Creating and Configuring Constraints
[Constraints](https://github.com/open-policy-agent/gatekeeper#constraints) are Kubernetes custom resources that define the scope of objects to which a specific constraint template applies to. The complete policy is defined by constraint templates and constraints together.
> **Prerequisites:** OPA Gatekeeper must be enabled in the cluster.
To list the constraints installed, go to the left side menu under OPA Gatekeeper, and click on **Constraints.**
New constraints can be created from a constraint template.
Rancher provides the ability to create a constraint by using a convenient form that lets you input the various constraint fields.
The **Edit as yaml** option is also available to configure the the constraint's yaml definition.
### Exempting Rancher's System Namespaces from Constraints
When a constraint is created, ensure that it does not apply to any Rancher or Kubernetes system namespaces. If the system namespaces are not excluded, then it is possible to see many resources under them marked as violations of the constraint.
To limit the scope of the constraint only to user namespaces, always specify these namespaces under the **Match** field of the constraint.
Also, the constraint may interfere with other Rancher functionality and deny system workloads from being deployed. To avoid this, exclude all Rancher-specific namespaces from your constraints.
## Enforcing Constraints in your Cluster
When the **Enforcement Action** is **Deny,** the constraint is immediately enabled and will deny any requests that violate the policy defined. By default, the enforcement value is **Deny.**
When the **Enforcement Action** is **Dryrun,** then any resources that violate the policy are only recorded under the constraint's status field.
To enforce constraints, create a constraint using the form. In the **Enforcement Action** field, choose **Deny.**
## Audit and Violations in your Cluster
OPA Gatekeeper runs a periodic audit to check if any existing resource violates any enforced constraint. The audit-interval (default 300s) can be configured while installing Gatekeeper.
On the Gatekeeper page, any violations of the defined constraints are listed.
Also under **Constraints,** the number of violations of the constraint can be found.
The detail view of each constraint lists information about the resource that violated the constraint.
## Disabling Gatekeeper
1. Navigate to the cluster's Dashboard view
1. On the left side menu, expand the cluster menu and click on **OPA Gatekeeper.**
1. Click the **&#8942; > Disable**.
**Result:** Upon disabling OPA Gatekeeper, all constraint templates and constraints will also be deleted.
-73
View File
@@ -1,73 +0,0 @@
---
title: FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/general-faq"/>
</head>
This FAQ is a work in progress designed to answers the questions our users most frequently ask about Rancher v2.x.
See [Technical FAQ](faq/technical-items.md), for frequently asked technical questions.
<br/>
**Does Rancher v2.x support Docker Swarm and Mesos as environment types?**
When creating an environment in Rancher v2.x, Swarm and Mesos will no longer be standard options you can select. However, both Swarm and Mesos will continue to be available as Catalog applications you can deploy. It was a tough decision to make but, in the end, it came down to adoption. For example, out of more than 15,000 clusters, only about 200 or so are running Swarm.
<br/>
**Is it possible to manage Azure Kubernetes Services with Rancher v2.x?**
Yes.
<br/>
**Does Rancher support Windows?**
As of Rancher 2.3.0, we support Windows Server 1809 containers. For details on how to set up a cluster with Windows worker nodes, refer to the section on [configuring custom clusters for Windows.](how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-windows-clusters/use-windows-clusters.md)
<br/>
**Does Rancher support Istio?**
As of Rancher 2.3.0, we support [Istio.](explanations/integrations-in-rancher/istio/istio.md)
Furthermore, Istio is implemented in our micro-PaaS "Rio", which works on Rancher 2.x along with any CNCF compliant Kubernetes cluster. You can read more about it [here](https://rio.io/)
<br/>
**Will Rancher v2.x support Hashicorp's Vault for storing secrets?**
Secrets management is on our roadmap but we haven't assigned it to a specific release yet.
<br/>
**Does Rancher v2.x support RKT containers as well?**
At this time, we only support Docker.
<br/>
**Does Rancher v2.x support Calico, Contiv, Contrail, Flannel, Weave net, etc., for embedded and imported Kubernetes?**
Out-of-the-box, Rancher provides the following CNI network providers for Kubernetes clusters: Canal, Flannel, Calico and Weave (Weave is available as of v2.2.0). Always refer to the [Rancher Support Matrix](https://rancher.com/support-maintenance-terms/) for details about what is officially supported.
<br/>
**Are you planning on supporting Traefik for existing setups?**
We don't currently plan on providing embedded Traefik support, but we're still exploring load-balancing approaches.
<br/>
**Can I import OpenShift Kubernetes clusters into v2.x?**
Our goal is to run any upstream Kubernetes clusters. Therefore, Rancher v2.x should work with OpenShift, but we haven't tested it yet.
<br/>
**Are you going to integrate Longhorn?**
Yes. Longhorn was on a bit of a hiatus while we were working on v2.0. We plan to re-engage on the project.
@@ -1,149 +0,0 @@
---
title: Container Network Interface (CNI) Providers
description: Learn about Container Network Interface (CNI), the CNI providers Rancher provides, the features they offer, and how to choose a provider for you
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/container-network-interface-providers"/>
</head>
## What is CNI?
CNI (Container Network Interface), a [Cloud Native Computing Foundation project](https://cncf.io/), consists of a specification and libraries for writing plugins to configure network interfaces in Linux containers, along with a number of plugins. CNI concerns itself only with network connectivity of containers and removing allocated resources when the container is deleted.
Kubernetes uses CNI as an interface between network providers and Kubernetes pod networking.
![CNI Logo](/img/cni-logo.png)
For more information visit [CNI GitHub project](https://github.com/containernetworking/cni).
### What Network Models are Used in CNI?
CNI network providers implement their network fabric using either an encapsulated network model such as Virtual Extensible Lan ([VXLAN](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan)) or an unencapsulated network model such as Border Gateway Protocol ([BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol)).
#### What is an Encapsulated Network?
This network model provides a logical Layer 2 (L2) network encapsulated over the existing Layer 3 (L3) network topology that spans the Kubernetes cluster nodes. With this model you have an isolated L2 network for containers without needing routing distribution, all at the cost of minimal overhead in terms of processing and increased IP package size, which comes from an IP header generated by overlay encapsulation. Encapsulation information is distributed by UDP ports between Kubernetes workers, interchanging network control plane information about how MAC addresses can be reached. Common encapsulation used in this kind of network model is VXLAN, Internet Protocol Security (IPSec), and IP-in-IP.
In simple terms, this network model generates a kind of network bridge extended between Kubernetes workers, where pods are connected.
This network model is used when an extended L2 bridge is preferred. This network model is sensitive to L3 network latencies of the Kubernetes workers. If datacenters are in distinct geolocations, be sure to have low latencies between them to avoid eventual network segmentation.
CNI network providers using this network model include Flannel, Canal, and Weave.
![Encapsulated Network](/img/encapsulated-network.png)
#### What is an Unencapsulated Network?
This network model provides an L3 network to route packets between containers. This model doesn't generate an isolated l2 network, nor generates overhead. These benefits come at the cost of Kubernetes workers having to manage any route distribution that's needed. Instead of using IP headers for encapsulation, this network model uses a network protocol between Kubernetes workers to distribute routing information to reach pods, such as [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol).
In simple terms, this network model generates a kind of network router extended between Kubernetes workers, which provides information about how to reach pods.
This network model is used when a routed L3 network is preferred. This mode dynamically updates routes at the OS level for Kubernetes workers. It's less sensitive to latency.
CNI network providers using this network model include Calico and Romana.
![Unencapsulated Network](/img/unencapsulated-network.png)
### What CNI Providers are Provided by Rancher?
Out-of-the-box, Rancher provides the following CNI network providers for Kubernetes clusters: Canal, Flannel, Calico and Weave (Weave is available as of v2.2.0). You can choose your CNI network provider when you create new Kubernetes clusters from Rancher.
#### Canal
![Canal Logo](/img/canal-logo.png)
Canal is a CNI network provider that gives you the best of Flannel and Calico. It allows users to easily deploy Calico and Flannel networking together as a unified networking solution, combining Calico’s network policy enforcement with the rich superset of Calico (unencapsulated) and/or Flannel (encapsulated) network connectivity options.
In Rancher, Canal is the default CNI network provider combined with Flannel and VXLAN encapsulation.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). For details, refer to [the port requirements for user clusters.](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md)
![](/img/canal-diagram.png)
For more information, see the [Canal GitHub Page.](https://github.com/projectcalico/canal)
#### Flannel
![Flannel Logo](/img/flannel-logo.png)
Flannel is a simple and easy way to configure L3 network fabric designed for Kubernetes. Flannel runs a single binary agent named flanneld on each host, which is responsible for allocating a subnet lease to each host out of a larger, preconfigured address space. Flannel uses either the Kubernetes API or etcd directly to store the network configuration, the allocated subnets, and any auxiliary data (such as the host's public IP). Packets are forwarded using one of several backend mechanisms, with the default encapsulation being [VXLAN](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan).
Encapsulated traffic is unencrypted by default. Therefore, flannel provides an experimental backend for encryption, [IPSec](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#ipsec), which makes use of [strongSwan](https://www.strongswan.org/) to establish encrypted IPSec tunnels between Kubernetes workers.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). See [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
![Flannel Diagram](/img/flannel-diagram.png)
For more information, see the [Flannel GitHub Page](https://github.com/coreos/flannel).
#### Calico
![Calico Logo](/img/calico-logo.png)
Calico enables networking and network policy in Kubernetes clusters across the cloud. Calico uses a pure, unencapsulated IP network fabric and policy engine to provide networking for your Kubernetes workloads. Workloads are able to communicate over both cloud infrastructure and on-prem using BGP.
Calico also provides a stateless IP-in-IP encapsulation mode that can be used, if necessary. Calico also offers policy isolation, allowing you to secure and govern your Kubernetes workloads using advanced ingress and egress policies.
Kubernetes workers should open TCP port `179` (BGP). See [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
![Calico Diagram](/img/calico-diagram.svg)
For more information, see the following pages:
- [Project Calico Official Site](https://www.projectcalico.org/)
- [Project Calico GitHub Page](https://github.com/projectcalico/calico)
#### Weave
![Weave Logo](/img/weave-logo.png)
_Available as of v2.2.0_
Weave enables networking and network policy in Kubernetes clusters across the cloud. Additionally, it support encrypting traffic between the peers.
Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` and UDP port `6784` (data ports). See the [port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
For more information, see the following pages:
- [Weave Net Official Site](https://github.com/weaveworks/weave/blob/master/site/overview.md)
### CNI Features by Provider
The following table summarizes the different features available for each CNI network provider provided by Rancher.
| Provider | Network Model | Route Distribution | Network Policies | Mesh | External Datastore | Encryption | Ingress/Egress Policies |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| Canal | Encapsulated (VXLAN) | No | Yes | No | K8S API | No | Yes |
| Flannel | Encapsulated (VXLAN) | No | No | No | K8S API | No | No |
| Calico | Encapsulated (VXLAN,IPIP) OR Unencapsulated | Yes | Yes | Yes | Etcd and K8S API | No | Yes |
| Weave | Encapsulated | Yes | Yes | Yes | No | Yes | Yes |
- Network Model: Encapsulated or unencapsulated. For more information, see [What Network Models are Used in CNI?](#what-network-models-are-used-in-cni)
- Route Distribution: An exterior gateway protocol designed to exchange routing and reachability information on the Internet. BGP can assist with pod-to-pod networking between clusters. This feature is a must on unencapsulated CNI network providers, and it is typically done by BGP. If you plan to build clusters split across network segments, route distribution is a feature that's nice-to-have.
- Network Policies: Kubernetes offers functionality to enforce rules about which services can communicate with each other using network policies. This feature is stable as of Kubernetes v1.7 and is ready to use with certain networking plugins.
- Mesh: This feature allows service-to-service networking communication between distinct Kubernetes clusters.
- External Datastore: CNI network providers with this feature need an external datastore for its data.
- Encryption: This feature allows cyphered and secure network control and data planes.
- Ingress/Egress Policies: This feature allows you to manage routing control for both Kubernetes and non-Kubernetes communications.
### CNI Community Popularity
<CNIPopularityTable />
### Which CNI Provider Should I Use?
It depends on your project needs. There are many different providers, which each have various features and options. There isn't one provider that meets everyone's needs.
As of Rancher v2.0.7, Canal is the default CNI network provider. We recommend it for most use cases. It provides encapsulated networking for containers with Flannel, while adding Calico network policies that can provide project/namespace isolation in terms of networking.
### How can I configure a CNI network provider?
Please see [Cluster Options](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md) on how to configure a network provider for your cluster. For more advanced configuration options, please see how to configure your cluster using a [Config File](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#cluster-config-file) and the options for [Network Plug-ins](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/).
@@ -1,33 +0,0 @@
---
title: Installing and Configuring kubectl
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/install-and-configure-kubectl"/>
</head>
`kubectl` is a CLI utility for running commands against Kubernetes clusters. It's required for many maintenance and administrative tasks in Rancher 2.x.
## Installation
See [kubectl Installation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for installation on your operating system.
## Configuration
When you create a Kubernetes cluster with RKE, RKE creates a `kube_config_rancher-cluster.yml` in the local directory that contains credentials to connect to your new cluster with tools like `kubectl` or `helm`.
You can copy this file to `$HOME/.kube/config` or if you are working with multiple Kubernetes clusters, set the `KUBECONFIG` environmental variable to the path of `kube_config_rancher-cluster.yml`.
```
export KUBECONFIG=$(pwd)/kube_config_rancher-cluster.yml
```
Test your connectivity with `kubectl` and see if you can get the list of nodes back.
```
kubectl get nodes
NAME STATUS ROLES AGE VERSION
165.227.114.63 Ready controlplane,etcd,worker 11m v1.10.1
165.227.116.167 Ready controlplane,etcd,worker 11m v1.10.1
165.227.127.226 Ready controlplane,etcd,worker 11m v1.10.1
```
@@ -1,8 +0,0 @@
---
title: Networking
---
Networking FAQ's
- [CNI Providers](container-network-interface-providers.md)
@@ -1,50 +0,0 @@
---
title: Rancher is No Longer Needed
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/rancher-is-no-longer-needed"/>
</head>
This page is intended to answer questions about what happens if you don't want Rancher anymore, if you don't want a cluster to be managed by Rancher anymore, or if the Rancher server is deleted.
## If the Rancher server is deleted, what happens to the workloads in my downstream clusters?
If Rancher is ever deleted or unrecoverable, all workloads in the downstream Kubernetes clusters managed by Rancher will continue to function as normal.
## If the Rancher server is deleted, how do I access my downstream clusters?
The capability to access a downstream cluster without Rancher depends on the type of cluster and the way that the cluster was created. To summarize:
- **Imported clusters:** The cluster will be unaffected and you can access the cluster using the same methods that you did before the cluster was imported into Rancher.
- **Hosted Kubernetes clusters:** If you created the cluster in a cloud-hosted Kubernetes provider such as EKS, GKE, or AKS, you can continue to manage the cluster using your provider's cloud credentials.
- **RKE clusters:** To access an [RKE cluster,](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) the cluster must have the [authorized cluster endpoint](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) enabled, and you must have already downloaded the cluster's kubeconfig file from the Rancher UI. (The authorized cluster endpoint is enabled by default for RKE clusters.) With this endpoint, you can access your cluster with kubectl directly instead of communicating through the Rancher server's [authentication proxy.](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#1-the-authentication-proxy) For instructions on how to configure kubectl to use the authorized cluster endpoint, refer to the section about directly accessing clusters with [kubectl and the kubeconfig file.](../how-to-guides/advanced-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#authenticating-directly-with-a-downstream-cluster) These clusters will use a snapshot of the authentication as it was configured when Rancher was removed.
## What if I don't want Rancher anymore?
If you [installed Rancher on a Kubernetes cluster,](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md) remove Rancher by using the [System Tools](../reference-guides/system-tools.md) with the `remove` subcommand.
If you installed Rancher with Docker, you can uninstall Rancher by removing the single Docker container that it runs in.
Imported clusters will not be affected by Rancher being removed. For other types of clusters, refer to the section on [accessing downstream clusters when Rancher is removed.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
## What if I don't want my imported cluster managed by Rancher?
If an imported cluster is deleted from the Rancher UI, the cluster is detached from Rancher, leaving it intact and accessible by the same methods that were used to access it before it was imported into Rancher.
To detach the cluster,
1. From the **Global** view in Rancher, go to the **Clusters** tab.
2. Go to the imported cluster that should be detached from Rancher and click **&#8942; > Delete.**
3. Click **Delete.**
**Result:** The imported cluster is detached from Rancher and functions normally outside of Rancher.
## What if I don't want my RKE cluster or hosted Kubernetes cluster managed by Rancher?
At this time, there is no functionality to detach these clusters from Rancher. In this context, "detach" is defined as the ability to remove Rancher components from the cluster and manage access to the cluster independently of Rancher.
The capability to manage these clusters without Rancher is being tracked in this [issue.](https://github.com/rancher/rancher/issues/25234)
For information about how to access clusters if the Rancher server is deleted, refer to [this section.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
@@ -1,15 +0,0 @@
---
title: Security FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/security"/>
</head>
## Is there a Hardening Guide?
The Hardening Guide is now located in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
## What are the results of Rancher's Kubernetes cluster when it is CIS benchmarked?
We have run the CIS Kubernetes benchmark against a hardened Rancher Kubernetes cluster. The results of that assessment can be found in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
@@ -1,196 +0,0 @@
---
title: Technical FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/technical-items"/>
</head>
## How can I reset the administrator password?
Docker install:
```
$ docker exec -ti <container_id> reset-password
New password for default administrator (user-xxxxx):
<new_password>
```
Kubernetes install (Helm):
```
$ KUBECONFIG=./kube_config_rancher-cluster.yml
$ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- reset-password
New password for default administrator (user-xxxxx):
<new_password>
```
### RKE add-on install (only supported up to Rancher v2.0.8)
:::tip
> If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/migrating-from-rke-add-on.md) for details on how to start using the Helm chart.
:::
Kubernetes install (RKE add-on):
```
$ KUBECONFIG=./kube_config_rancher-cluster.yml
$ kubectl --kubeconfig $KUBECONFIG exec -n cattle-system $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name=="cattle-server") | .metadata.name') -- reset-password
New password for default administrator (user-xxxxx):
<new_password>
```
## I deleted/deactivated the last admin, how can I fix it?
Docker install:
```
$ docker exec -ti <container_id> ensure-default-admin
New default administrator (user-xxxxx)
New password for default administrator (user-xxxxx):
<new_password>
```
Kubernetes install (Helm):
```
$ KUBECONFIG=./kube_config_rancher-cluster.yml
$ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- ensure-default-admin
New password for default administrator (user-xxxxx):
<new_password>
```
## How can I enable debug logging?
See [Troubleshooting: Logging](../troubleshooting/other-troubleshooting-tips/logging.md)
## My ClusterIP does not respond to ping
ClusterIP is a virtual IP, which will not respond to ping. Best way to test if the ClusterIP is configured correctly, is by using `curl` to access the IP and port to see if it responds.
## Where can I manage Node Templates?
Node Templates can be accessed by opening your account menu (top right) and selecting `Node Templates`.
## Why is my Layer-4 Load Balancer in `Pending` state?
The Layer-4 Load Balancer is created as `type: LoadBalancer`. In Kubernetes, this needs a cloud provider or controller that can satisfy these requests, otherwise these will be in `Pending` state forever. More information can be found on [Cloud Providers](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/set-up-cloud-providers/set-up-cloud-providers.md) or [Create External Load Balancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/)
## Where is the state of Rancher stored?
- Docker Install: in the embedded etcd of the `rancher/rancher` container, located at `/var/lib/rancher`.
- Kubernetes install: in the etcd of the RKE cluster created to run Rancher.
## How are the supported Docker versions determined?
We follow the validated Docker versions for upstream Kubernetes releases. The validated versions can be found under [External Dependencies](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.10.md#external-dependencies) in the Kubernetes release CHANGELOG.md.
## How can I access nodes created by Rancher?
SSH keys to access the nodes created by Rancher can be downloaded via the **Nodes** view. Choose the node which you want to access and click on the vertical &#8942; button at the end of the row, and choose **Download Keys** as shown in the picture below.
![Download Keys](/img/downloadsshkeys.png)
Unzip the downloaded zip file, and use the file `id_rsa` to connect to you host. Be sure to use the correct username (`rancher` or `docker` for RancherOS, `ubuntu` for Ubuntu, `ec2-user` for Amazon Linux)
```
$ ssh -i id_rsa user@ip_of_node
```
## How can I automate task X in Rancher?
The UI consists of static files, and works based on responses of the API. That means every action/task that you can execute in the UI, can be automated via the API. There are 2 ways to do this:
* Visit `https://your_rancher_ip/v3` and browse the API options.
* Capture the API calls when using the UI (Most commonly used for this is [Chrome Developer Tools](https://developers.google.com/web/tools/chrome-devtools/#network) but you can use anything you like)
## The IP address of a node changed, how can I recover?
A node is required to have a static IP configured (or a reserved IP via DHCP). If the IP of a node has changed, you will have to remove it from the cluster and readd it. After it is removed, Rancher will update the cluster to the correct state. If the cluster is no longer in `Provisioning` state, the node is removed from the cluster.
When the IP address of the node changed, Rancher lost connection to the node, so it will be unable to clean the node properly. See [Cleaning cluster nodes](../how-to-guides/advanced-user-guides/manage-clusters/clean-cluster-nodes.md) to clean the node.
When the node is removed from the cluster, and the node is cleaned, you can readd the node to the cluster.
## How can I add more arguments/binds/environment variables to Kubernetes components in a Rancher Launched Kubernetes cluster?
You can add more arguments/binds/environment variables via the [Config File](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#cluster-config-file) option in Cluster Options. For more information, see the [Extra Args, Extra Binds, and Extra Environment Variables](https://rancher.com/docs/rke/latest/en/config-options/services/services-extras/) in the RKE documentation or browse the [Example Cluster.ymls](https://rancher.com/docs/rke/latest/en/example-yamls/).
## How do I check if my certificate chain is valid?
Use the `openssl verify` command to validate your certificate chain:
:::tip
Configure `SSL_CERT_DIR` and `SSL_CERT_FILE` to a dummy location to make sure the OS installed certificates are not used when verifying manually.
:::
```
SSL_CERT_DIR=/dummy SSL_CERT_FILE=/dummy openssl verify -CAfile ca.pem rancher.yourdomain.com.pem
rancher.yourdomain.com.pem: OK
```
If you receive the error `unable to get local issuer certificate`, the chain is incomplete. This usually means that there is an intermediate CA certificate that issued your server certificate. If you already have this certificate, you can use it in the verification of the certificate like shown below:
```
SSL_CERT_DIR=/dummy SSL_CERT_FILE=/dummy openssl verify -CAfile ca.pem -untrusted intermediate.pem rancher.yourdomain.com.pem
rancher.yourdomain.com.pem: OK
```
If you have successfully verified your certificate chain, you should include needed intermediate CA certificates in the server certificate to complete the certificate chain for any connection made to Rancher (for example, by the Rancher agent). The order of the certificates in the server certificate file should be first the server certificate itself (contents of `rancher.yourdomain.com.pem`), followed by intermediate CA certificate(s) (contents of `intermediate.pem`).
```
-----BEGIN CERTIFICATE-----
%YOUR_CERTIFICATE%
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
%YOUR_INTERMEDIATE_CERTIFICATE%
-----END CERTIFICATE-----
```
If you still get errors during verification, you can retrieve the subject and the issuer of the server certificate using the following command:
```
openssl x509 -noout -subject -issuer -in rancher.yourdomain.com.pem
subject= /C=GB/ST=England/O=Alice Ltd/CN=rancher.yourdomain.com
issuer= /C=GB/ST=England/O=Alice Ltd/CN=Alice Intermediate CA
```
## How do I check `Common Name` and `Subject Alternative Names` in my server certificate?
Although technically an entry in `Subject Alternative Names` is required, having the hostname in both `Common Name` and as entry in `Subject Alternative Names` gives you maximum compatibility with older browser/applications.
Check `Common Name`:
```
openssl x509 -noout -subject -in cert.pem
subject= /CN=rancher.my.org
```
Check `Subject Alternative Names`:
```
openssl x509 -noout -in cert.pem -text | grep DNS
DNS:rancher.my.org
```
## Why does it take 5+ minutes for a pod to be rescheduled when a node has failed?
This is due to a combination of the following default Kubernetes settings:
* kubelet
* `node-status-update-frequency`: Specifies how often kubelet posts node status to master (default 10s)
* kube-controller-manager
* `node-monitor-period`: The period for syncing NodeStatus in NodeController (default 5s)
* `node-monitor-grace-period`: Amount of time which we allow running Node to be unresponsive before marking it unhealthy (default 40s)
* `pod-eviction-timeout`: The grace period for deleting pods on failed nodes (default 5m0s)
See [Kubernetes: kubelet](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) and [Kubernetes: kube-controller-manager](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) for more information on these settings.
In Kubernetes v1.13, the `TaintBasedEvictions` feature is enabled by default. See [Kubernetes: Taint based Evictions](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions) for more information.
* kube-apiserver (Kubernetes v1.13 and up)
* `default-not-ready-toleration-seconds`: Indicates the tolerationSeconds of the toleration for notReady:NoExecute that is added by default to every pod that does not already have such a toleration.
* `default-unreachable-toleration-seconds`: Indicates the tolerationSeconds of the toleration for unreachable:NoExecute that is added by default to every pod that does not already have such a toleration.
## Can I use keyboard shortcuts in the UI?
Yes, most parts of the UI can be reached using keyboard shortcuts. For an overview of the available shortcuts, press `?` anywhere in the UI.
@@ -1,35 +0,0 @@
---
title: Telemetry FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/telemetry"/>
</head>
## What is Telemetry?
Telemetry collects aggregate information about the size of Rancher installations, versions of components used, and which features are used. This information is used by Rancher Labs to help make the product better and is not shared with third-parties.
## What information is collected?
No specific identifying information like usernames, passwords, or the names or addresses of user resources will ever be collected.
The primary things collected include:
- Aggregate counts (smallest, average, largest, total) of nodes per-cluster and their size (e.g. CPU cores & RAM).
- Aggregate counts of logical resources like Clusters, Projects, Namespaces, and Pods.
- Counts of what driver was used to deploy clusters and nodes (e.g. GKE vs EC2 vs Imported vs Custom).
- Versions of Kubernetes components, Operating Systems and Docker that are deployed on nodes.
- Whether some optional components are enabled or not (e.g. which auth providers are used).
- The image name & version of Rancher that is running.
- A unique randomly-generated identifier for this installation.
## Can I see the information that is being sent?
If Telemetry is enabled, you can go to `https://<your rancher server>/v1-telemetry` in your installation to see the current data.
If Telemetry is not enabled, the process that collects the data is not running, so there is nothing being collected to look at.
## How do I turn it on or off?
After initial setup, an administrator can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`.
@@ -1,103 +0,0 @@
---
title: Questions about Upgrading to Rancher v2.x
---
This page contains frequently asked questions about the changes between Rancher v1.x and v2.x, and how to upgrade from Rancher v1.x to v2.x.
## Kubernetes
**What does it mean when you say Rancher v2.x is built on Kubernetes?**
Rancher v2.x is a complete container management platform built 100% on Kubernetes leveraging its Custom Resource and Controller framework. All features are written as a CustomResourceDefinition (CRD) which extends the existing Kubernetes API and can leverage native features such as RBAC.
<br/>
**Do you plan to implement upstream Kubernetes, or continue to work on your own fork?**
We're still going to provide our distribution when you select the default option of having us create your Kubernetes cluster, but it will be very close to upstream.
<br/>
**Does this release mean that we need to re-train our support staff in Kubernetes?**
Yes. Rancher will offer the native Kubernetes functionality via `kubectl` but will also offer our own UI dashboard to allow you to deploy Kubernetes workload without having to understand the full complexity of Kubernetes. However, to fully leverage Kubernetes, we do recommend understanding Kubernetes. We do plan on improving our UX with subsequent releases to make Kubernetes easier to use.
<br/>
**Is a Rancher compose going to make a Kubernetes pod? Do we have to learn both now? We usually use the filesystem layer of files, not the UI.**
No. Unfortunately, the differences were enough such that we cannot support Rancher compose anymore in 2.x. We will be providing both a tool and guides to help with this migration.
<br/>
**If we use Kubernetes native YAML files for creating resources, should we expect that to work as expected, or do we need to use Rancher/Docker compose files to deploy infrastructure?**
Absolutely.
## Cattle
**How does Rancher v2.x affect Cattle?**
Cattle will not supported in v2.x as Rancher has been re-architected to be based on Kubernetes. You can, however, expect majority of Cattle features you use will exist and function similarly on Kubernetes. We will develop migration tools in Rancher v2.1 to help you transform your existing Rancher Compose files into Kubernetes YAML files.
<br/>
**Can I migrate existing Cattle workloads into Kubernetes?**
Yes. In the upcoming Rancher v2.1 release we will provide a tool to help translate existing Cattle workloads in Compose format to Kubernetes YAML format. You will then be able to deploy those workloads on the v2.x platform.
## Feature Changes
**Can we still add our own infrastructure services, which had a separate view/filter in 1.6.x?**
Yes. You can manage Kubernetes storage, networking, and its vast ecosystem of add-ons.
<br/>
**Are there changes to default roles available now or going forward? Will the Kubernetes alignment impact plans for roles/RBAC?**
The default roles will be expanded to accommodate the new Rancher 2.x features, and will also take advantage of the Kubernetes RBAC (Role-Based Access Control) capabilities to give you more flexibility.
<br/>
**Will there be any functions like network policies to separate a front-end container from a back-end container through some kind of firewall in v2.x?**
Yes. You can do so by leveraging Kubernetes' network policies.
<br/>
**What about the CLI? Will that work the same way with the same features?**
Yes. Definitely.
## Environments & Clusters
**Can I still create templates for environments and clusters?**
Starting with 2.0, the concept of an environment has now been changed to a Kubernetes cluster as going forward, only the Kubernetes orchestration engine is supported.
Kubernetes RKE Templates is on our roadmap for 2.x. Please refer to our Release Notes and documentation for all the features that we currently support.
<br/>
**Can you still add an existing host to an environment? (i.e. not provisioned directly from Rancher)**
Yes. We still provide you with the same way of executing our Rancher agents directly on hosts.
## Upgrading/Migrating
**How would the migration from v1.x to v2.x work?**
Due to the technical difficulty in transforming a Docker container into a pod running Kubernetes, upgrading will require users to "replay" those workloads from v1.x into new v2.x environments. We plan to ship with a tool in v2.1 to translate existing Rancher Compose files into Kubernetes YAML files. You will then be able to deploy those workloads on the v2.x platform.
<br/>
**Is it possible to upgrade from Rancher v1.x to v2.x without any disruption to Cattle and Kubernetes clusters?**
At this time, we are still exploring this scenario and taking feedback. We anticipate that you will need to launch a new Rancher instance and then relaunch on v2.x. Once you've moved to v2.x, upgrades will be in place, as they are in v1.6.
## Support
**Are you planning some long-term support releases for Rancher v1.6?**
That is definitely the focus of the v1.6 stream. We're continuing to improve that release, fix bugs, and maintain it. New releases of the v1.6 stream are announced in the [Rancher forums.](https://forums.rancher.com/c/announcements) The Rancher wiki contains the [v1.6 release notes.](https://github.com/rancher/rancher/wiki/Rancher-1.6)
@@ -1,9 +0,0 @@
---
title: Getting Started
---
To get up and running with Rancher quickly, we have included a **Getting Started** section.
The goal of this section is to be able to assist users in deploying Rancher and workloads and to install or upgrade Rancher quickly and effectively.
Please see the [introduction](getting-started/introduction/introduction.md), [quick start guides](getting-started/quick-start-guides/quick-start-guides.md), and the [installation and upgrade](getting-started/installation-and-upgrade/installation-and-upgrade.md) sections for more.
@@ -1,9 +0,0 @@
---
title: Advanced
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/pages-for-subheaders/advanced-user-guides"/>
</head>
The documents in this section contain resources for less common use cases.
@@ -1,38 +0,0 @@
---
title: Installing Rancher in an Air Gapped Environment with Helm 2
---
> After Helm 3 was released, the Rancher installation instructions were updated to use Helm 3.
>
> If you are using Helm 2, we recommend [migrating to Helm 3](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) because it is simpler to use and more secure than Helm 2.
>
> This section provides a copy of the older instructions for installing Rancher on a Kubernetes cluster using Helm 2 in an air air gap environment, and it is intended to be used if upgrading to Helm 3 is not feasible.
This section is about installations of Rancher server in an air gapped environment. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy.
Throughout the installations instructions, there will be _tabs_ for either a high availability Kubernetes installation or a single-node Docker installation.
### Air Gapped Kubernetes Installations
This section covers how to install Rancher on a Kubernetes cluster in an air gapped environment.
A Kubernetes installation is comprised of three nodes running the Rancher server components on a Kubernetes cluster. The persistence layer (etcd) is also replicated on these three nodes, providing redundancy and data duplication in case one of the nodes fails.
### Air Gapped Docker Installations
These instructions also cover how to install Rancher on a single node in an air gapped environment.
The Docker installation is for Rancher users that are wanting to test out Rancher. Instead of running on a Kubernetes cluster, you install the Rancher server component on a single node using a `docker run` command. Since there is only one node and a single Docker container, if the node goes down, there is no copy of the etcd data available on other nodes and you will lose all the data of your Rancher server.
> **Important:** If you install Rancher following the Docker installation guide, there is no upgrade path to transition your Docker Installation to a Kubernetes Installation.
Instead of running the Docker installation, you have the option to follow the Kubernetes Install guide, but only use one node to install Rancher. Afterwards, you can scale up the etcd nodes in your Kubernetes cluster to make it a Kubernetes Installation.
## Installation Outline
- [1. Prepare your Node(s)](../../../other-installation-methods/air-gapped-helm-cli-install/infrastructure-private-registry.md)
- [2. Collect and Publish Images to your Private Registry](../../../other-installation-methods/air-gapped-helm-cli-install/publish-images.md)
- [3. Launch a Kubernetes Cluster with RKE](../../../other-installation-methods/air-gapped-helm-cli-install/install-kubernetes.md)
- [4. Install Rancher](../../../other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md)
### [Next: Prepare your Node(s)](../../../other-installation-methods/air-gapped-helm-cli-install/infrastructure-private-registry.md)
@@ -1,340 +0,0 @@
---
title: 4. Install Rancher
---
This section is about how to deploy Rancher for your air gapped environment. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy. There are _tabs_ for either a high availability (recommended) or a Docker installation.
<Tabs>
<TabItem value="Kubernetes Install (Recommended)">
Rancher recommends installing Rancher on a Kubernetes cluster. A highly available Kubernetes Installation is comprised of three nodes running the Rancher server components on a Kubernetes cluster. The persistence layer (etcd) is also replicated on these three nodes, providing redundancy and data duplication in case one of the nodes fails.
### A. Add the Helm Chart Repository
From a system that has access to the internet, fetch the latest Helm chart and copy the resulting manifests to a system that has access to the Rancher server cluster.
1. If you haven't already, initialize Helm locally on a workstation that has internet access.
:::note
Refer to the [Helm version requirements](../../../resources/choose-a-rancher-version.md) to choose a version of Helm to install Rancher.
:::
```plain
helm init -c
```
2. Use the `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher](../../../resources/choose-a-rancher-version.md).
- Latest: Recommended for trying out the newest features
```
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
```
- Stable: Recommended for production environments
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
- Alpha: Experimental preview of upcoming releases.
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
Note: Upgrades are not supported to, from, or between Alphas.
3. Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file.
```plain
helm fetch rancher-<CHART_REPO>/rancher
```
> Want additional options? See the Rancher [Helm chart options](../../../../../reference-guides/installation-references/helm-chart-options.md).
### B. Choose your SSL Configuration
Rancher Server is designed to be secure by default and requires SSL/TLS configuration.
When Rancher is installed on an air gapped Kubernetes cluster, there are two recommended options for the source of the certificate.
> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer](../../../../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination).
| Configuration | Chart option | Description | Requires cert-manager |
| ------------------------------------------ | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- |
| Rancher Generated Self-Signed Certificates | `ingress.tls.source=rancher` | Use certificates issued by Rancher's generated CA (self signed)<br/> This is the **default** and does not need to be added when rendering the Helm template. | yes |
| Certificates from Files | `ingress.tls.source=secret` | Use your own certificate files by creating Kubernetes Secret(s). <br/> This option must be passed when rendering the Rancher Helm template. | no |
### C. Render the Rancher Helm Template
When setting up the Rancher Helm template, there are several options in the Helm chart that are designed specifically for air gap installations.
| Chart Option | Chart Value | Description |
| ----------------------- | -------------------------------- | ---- |
| `certmanager.version` | `<version>` | Configure proper Rancher TLS issuer depending of running cert-manager version. |
| `systemDefaultRegistry` | `<REGISTRY.YOURDOMAIN.COM:PORT>` | Configure Rancher server to always pull from your private registry when provisioning clusters. |
| `useBundledSystemChart` | `true` | Configure Rancher server to use the packaged copy of Helm system charts. The [system charts](https://github.com/rancher/system-charts) repository contains all the catalog items required for features such as monitoring, logging, alerting and global DNS. These [Helm charts](https://github.com/rancher/system-charts) are located in GitHub, but since you are in an air gapped environment, using the charts that are bundled within Rancher is much easier than setting up a Git mirror. _Available as of v2.3.0_ |
Based on the choice your made in [B. Choose your SSL Configuration](#b-choose-your-ssl-configuration), complete one of the procedures below.
<details id="self-signed">
<summary>Option A-Default Self-Signed Certificate</summary>
By default, Rancher generates a CA and uses cert-manager to issue the certificate for access to the Rancher server interface.
> **Note:**
> Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.11.0, please see our [upgrade cert-manager documentation](../../../resources/upgrade-cert-manager.md).
1. From a system connected to the internet, add the cert-manager repo to Helm.
```plain
helm repo add jetstack https://charts.jetstack.io
helm repo update
```
1. Fetch the latest cert-manager chart available from the [Helm chart repository](https://hub.helm.sh/charts/jetstack/cert-manager).
```plain
helm fetch jetstack/cert-manager --version v0.14.2
```
1. Render the cert manager template with the options you would like to use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files.
```plain
helm template ./cert-manager-v0.14.2.tgz --output-dir . \
--name cert-manager --namespace cert-manager \
--set image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-controller
--set webhook.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-webhook
--set cainjector.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-cainjector
```
1. Download the required CRD file for cert-manager
```plain
curl -L -o cert-manager-crd.yaml https://raw.githubusercontent.com/jetstack/cert-manager/release-0.14/deploy/manifests/00-crds.yaml
```
1. Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
Placeholder | Description
------------|-------------
`<VERSION>` | The version number of the output tarball.
`<RANCHER.YOURDOMAIN.COM>` | The DNS name you pointed at your load balancer.
`<REGISTRY.YOURDOMAIN.COM:PORT>` | The DNS name for your private registry.
`<CERTMANAGER_VERSION>` | Cert-manager version running on k8s cluster.
```plain
helm template ./rancher-<VERSION>.tgz --output-dir . \
--name rancher \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set certmanager.version=<CERTMANAGER_VERSION> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Available as of v2.2.0, set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts
```
</details>
<details id="secret">
<summary>Option B: Certificates From Files using Kubernetes Secrets</summary>
Create Kubernetes secrets from your own certificates for Rancher to use. The common name for the cert will need to match the `hostname` option in the command below, or the ingress controller will fail to provision the site for Rancher.
Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------- |
| `<VERSION>` | The version number of the output tarball. |
| `<RANCHER.YOURDOMAIN.COM>` | The DNS name you pointed at your load balancer. |
| `<REGISTRY.YOURDOMAIN.COM:PORT>` | The DNS name for your private registry. |
```plain
helm template ./rancher-<VERSION>.tgz --output-dir . \
--name rancher \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Available as of v2.2.0, set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts
```
If you are using a Private CA signed cert, add `--set privateCA=true` following `--set ingress.tls.source=secret`:
```plain
helm template ./rancher-<VERSION>.tgz --output-dir . \
--name rancher \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set privateCA=true \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Available as of v2.2.0, set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts
```
Then refer to [Adding TLS Secrets](../../../resources/add-tls-secrets.md) to publish the certificate files so Rancher and the ingress controller can use them.
</details>
### D. Install Rancher
Copy the rendered manifest directories to a system that has access to the Rancher server cluster to complete installation.
Use `kubectl` to create namespaces and apply the rendered manifests.
If you choose to use self-signed certificates in [B. Choose your SSL Configuration](#b-choose-your-ssl-configuration), install cert-manager.
<details id="install-cert-manager">
<summary>Self-Signed Certificate Installs - Install Cert-manager</summary>
If you are using self-signed certificates, install cert-manager:
1. Create the namespace for cert-manager.
```plain
kubectl create namespace cert-manager
```
1. Create the cert-manager CustomResourceDefinitions (CRDs).
```plain
kubectl apply -f cert-manager/cert-manager-crd.yaml
```
> **Important:**
> If you are running Kubernetes v1.15 or below, you will need to add the `--validate=false flag to your kubectl apply command above else you will receive a validation error relating to the x-kubernetes-preserve-unknown-fields field in cert-manager’s CustomResourceDefinition resources. This is a benign error and occurs due to the way kubectl performs resource validation.
1. Launch cert-manager.
```plain
kubectl apply -R -f ./cert-manager
```
</details>
Install Rancher:
```plain
kubectl create namespace cattle-system
kubectl -n cattle-system apply -R -f ./rancher
```
**Step Result:** If you are installing Rancher v2.3.0+, the installation is complete.
### E. For Rancher versions before v2.3.0, Configure System Charts
If you are installing Rancher versions before v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in GitHub, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts](../../../resources/local-system-charts.md).
### Additional Resources
These resources could be helpful when installing Rancher:
- [Rancher Helm chart options](../../../../../reference-guides/installation-references/helm-chart-options.md)
- [Adding TLS secrets](../../../resources/add-tls-secrets.md)
- [Troubleshooting Rancher Kubernetes Installations](../../../install-upgrade-on-a-kubernetes-cluster/troubleshooting.md)
</TabItem>
<TabItem value="Docker Install">
The Docker installation is for Rancher users that are wanting to **test** out Rancher. Instead of running on a Kubernetes cluster, you install the Rancher server component on a single node using a `docker run` command. Since there is only one node and a single Docker container, if the node goes down, there is no copy of the etcd data available on other nodes and you will lose all the data of your Rancher server. **Important: If you install Rancher following the Docker installation guide, there is no upgrade path to transition your Docker installation to a Kubernetes Installation.** Instead of running the single node installation, you have the option to follow the Kubernetes Install guide, but only use one node to install Rancher. Afterwards, you can scale up the etcd nodes in your Kubernetes cluster to make it a Kubernetes Installation.
For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.
| Environment Variable Key | Environment Variable Value | Description |
| -------------------------------- | -------------------------------- | ---- |
| `CATTLE_SYSTEM_DEFAULT_REGISTRY` | `<REGISTRY.YOURDOMAIN.COM:PORT>` | Configure Rancher server to always pull from your private registry when provisioning clusters. |
| `CATTLE_SYSTEM_CATALOG` | `bundled` | Configure Rancher server to use the packaged copy of Helm system charts. The [system charts](https://github.com/rancher/system-charts) repository contains all the catalog items required for features such as monitoring, logging, alerting and global DNS. These [Helm charts](https://github.com/rancher/system-charts) are located in GitHub, but since you are in an air gapped environment, using the charts that are bundled within Rancher is much easier than setting up a Git mirror. _Available as of v2.3.0_ |
> **Do you want to...**
>
> - Configure custom CA root certificate to access your services? See [Custom CA root certificate](../../../../../reference-guides/installation-references/helm-chart-options.md#additional-trusted-cas).
> - Record all transactions with the Rancher API? See [API Auditing](../../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#api-audit-log).
- For Rancher before v2.3.0, you will need to mirror the `system-charts` repository to a location in your network that Rancher can reach. Then, after Rancher is installed, you will need to configure Rancher to use that repository. For details, refer to the documentation on [setting up the system charts for Rancher before v2.3.0.](../../../resources/local-system-charts.md)
Choose from the following options:
<details id="option-a">
<summary>Option A-Default Self-Signed Certificate</summary>
If you are installing Rancher in a development or testing environment where identity verification isn't a concern, install Rancher using the self-signed certificate that it generates. This installation option omits the hassle of generating a certificate yourself.
Log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port. |
| `<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../resources/choose-a-rancher-version.md) that you want to install. |
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ #Available as of v2.3.0, use the packaged Rancher system charts
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
</details>
<details id="option-b">
<summary>Option B-Bring Your Own Certificate: Self-Signed</summary>
In development or testing environments where your team will access your Rancher server, create a self-signed certificate for use with your install so that your team can verify they're connecting to your instance of Rancher.
> **Prerequisites:**
> From a computer with an internet connection, create a self-signed certificate using [OpenSSL](https://www.openssl.org/) or another method of your choice.
>
> - The certificate files must be in PEM format.
> - In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Certificate Troubleshooting.](../../../other-installation-methods/rancher-on-a-single-node-with-docker/certificate-troubleshooting.md)
After creating your certificate, log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder. Use the `-v` flag and provide the path to your certificates to mount them in your container.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `<CERT_DIRECTORY>` | The path to the directory containing your certificate files. |
| `<FULL_CHAIN.pem>` | The path to your full certificate chain. |
| `<PRIVATE_KEY.pem>` | The path to the private key for your certificate. |
| `<CA_CERTS>` | The path to the certificate authority's certificate. |
| `<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port. |
| `<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../resources/choose-a-rancher-version.md) that you want to install. |
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
-v /<CERT_DIRECTORY>/<CA_CERTS.pem>:/etc/rancher/ssl/cacerts.pem \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ #Available as of v2.3.0, use the packaged Rancher system charts
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
</details>
<details id="option-c">
<summary>Option C-Bring Your Own Certificate: Signed by Recognized CA</summary>
In development or testing environments where you're exposing an app publicly, use a certificate signed by a recognized CA so that your user base doesn't encounter security warnings.
> **Prerequisite:** The certificate files must be in PEM format.
After obtaining your certificate, log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder. Because your certificate is signed by a recognized CA, mounting an additional CA certificate file is unnecessary.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `<CERT_DIRECTORY>` | The path to the directory containing your certificate files. |
| `<FULL_CHAIN.pem>` | The path to your full certificate chain. |
| `<PRIVATE_KEY.pem>` | The path to the private key for your certificate. |
| `<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port. |
| `<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../resources/choose-a-rancher-version.md) that you want to install. |
> **Note:** Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher.
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
--no-cacerts \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ #Available as of v2.3.0, use the packaged Rancher system charts
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
</details>
If you are installing Rancher v2.3.0+, the installation is complete.
If you are installing Rancher versions before v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in GitHub, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts](../../../resources/local-system-charts.md).
</TabItem>
</Tabs>
@@ -1,83 +0,0 @@
---
title: '3. Install Kubernetes with RKE (Kubernetes Installs Only)'
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-kubernetes"/>
</head>
This section is about how to prepare to launch a Kubernetes cluster which is used to deploy Rancher server for your air gapped environment.
Since a Kubernetes Installation requires a Kubernetes cluster, we will create a Kubernetes cluster using [Rancher Kubernetes Engine](https://rancher.com/docs/rke/latest/en/) (RKE). Before being able to start your Kubernetes cluster, you'll need to [install RKE](https://rancher.com/docs/rke/latest/en/installation/) and create a RKE config file.
- [A. Create an RKE Config File](#a-create-an-rke-config-file)
- [B. Run RKE](#b-run-rke)
- [C. Save Your Files](#c-save-your-files)
### A. Create an RKE Config File
From a system that can access ports 22/tcp and 6443/tcp on your host nodes, use the sample below to create a new file named `rancher-cluster.yml`. This file is a Rancher Kubernetes Engine configuration file (RKE config file), which is a configuration for the cluster you're deploying Rancher to.
Replace values in the code sample below with help of the _RKE Options_ table. Use the IP address or DNS names of the [3 nodes](./prepare-nodes.md) you created.
> **Tip:** For more details on the options available, see the RKE [Config Options](https://rancher.com/docs/rke/latest/en/config-options/).
<figcaption>RKE Options</figcaption>
| Option | Required | Description |
| ------------------ | -------------------- | --------------------------------------------------------------------------------------- |
| `address` | ✓ | The DNS or IP address for the node within the air gap network. |
| `user` | ✓ | A user that can run docker commands. |
| `role` | ✓ | List of Kubernetes roles assigned to the node. |
| `internal_address` | optional<sup>1</sup> | The DNS or IP address used for internal cluster traffic. |
| `ssh_key_path` | | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`). |
> <sup>1</sup> Some services like AWS EC2 require setting the `internal_address` if you want to use self-referencing security groups or firewalls.
```yaml
nodes:
- address: 10.10.3.187 # node air gap network IP
internal_address: 172.31.7.22 # node intra-cluster IP
user: rancher
role: ['controlplane', 'etcd', 'worker']
ssh_key_path: /home/user/.ssh/id_rsa
- address: 10.10.3.254 # node air gap network IP
internal_address: 172.31.13.132 # node intra-cluster IP
user: rancher
role: ['controlplane', 'etcd', 'worker']
ssh_key_path: /home/user/.ssh/id_rsa
- address: 10.10.3.89 # node air gap network IP
internal_address: 172.31.3.216 # node intra-cluster IP
user: rancher
role: ['controlplane', 'etcd', 'worker']
ssh_key_path: /home/user/.ssh/id_rsa
private_registries:
- url: <REGISTRY.YOURDOMAIN.COM:PORT> # private registry url
user: rancher
password: '*********'
is_default: true
```
### B. Run RKE
After configuring `rancher-cluster.yml`, bring up your Kubernetes cluster:
```
rke up --config ./rancher-cluster.yml
```
### C. Save Your Files
> **Important**
> The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster.
Save a copy of the following files in a secure location:
- `rancher-cluster.yml`: The RKE cluster configuration file.
- `kube_config_rancher-cluster.yml`: The [Kubeconfig file](https://rancher.com/docs/rke/latest/en/kubeconfig/) for the cluster, this file contains credentials for full access to the cluster.
- `rancher-cluster.rkestate`: The [Kubernetes Cluster State file](https://rancher.com/docs/rke/latest/en/installation/#kubernetes-cluster-state), this file contains credentials for full access to the cluster.<br/><br/>_The Kubernetes Cluster State file is only created when using RKE v0.2.0 or higher._
> **Note:** The "rancher-cluster" parts of the two latter file names are dependent on how you name the RKE cluster configuration file.
### [Next: Install Rancher](../../../other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md)
@@ -1,274 +0,0 @@
---
title: '2. Collect and Publish Images to your Private Registry'
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/publish-images"/>
</head>
> **Prerequisites:** You must have a [private registry](https://docs.docker.com/registry/deploying/) available to use.
>
> **Note:** Populating the private registry with images is the same process for HA and Docker installations, the differences in this section is based on whether or not you are planning to provision a Windows cluster or not.
By default, all images used to [provision Kubernetes clusters](../../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md) or launch any [tools](../../../../../reference-guides/rancher-cluster-tools.md) in Rancher, e.g. monitoring, pipelines, alerts, are pulled from Docker Hub. In an air gap installation of Rancher, you will need a private registry that is located somewhere accessible by your Rancher server. Then, you will load the registry with all the images.
This section describes how to set up your private registry so that when you install Rancher, Rancher will pull all the required images from this registry.
By default, we provide the steps of how to populate your private registry assuming you are provisioning Linux only clusters, but if you plan on provisioning any [Windows clusters](../../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-windows-clusters/use-windows-clusters.md), there are separate instructions to support the images needed for a Windows cluster.
<Tabs>
<TabItem value="Linux Only Clusters">
For Rancher servers that will only provision Linux clusters, these are the steps to populate your private registry.
A. Find the required assets for your Rancher version <br/>
B. Collect all the required images <br/>
C. Save the images to your workstation <br/>
D. Populate the private registry
### Prerequisites
These steps expect you to use a Linux workstation that has internet access, access to your private registry, and at least 20 GB of disk space.
If you will use ARM64 hosts, the registry must support manifests. As of April 2020, Amazon Elastic Container Registry does not support manifests.
### A. Find the required assets for your Rancher version
1. Browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments. Click **Assets*.*
2. From the release's **Assets** section, download the following files:
| Release File | Description |
| ---------------- | -------------- |
| `rancher-images.txt` | This file contains a list of images needed to install Rancher, provision clusters and user Rancher tools. |
| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. |
| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. |
### B. Collect all the required images (For Kubernetes Installs using Rancher Generated Self-Signed Certificate)
In a Kubernetes Install, if you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://hub.helm.sh/charts/jetstack/cert-manager) image to `rancher-images.txt` as well. You skip this step if you are using you using your own certificates.
1. Fetch the latest `cert-manager` Helm chart and parse the template for image details:
> **Note:** Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.12.0, please see our [upgrade documentation](../../../resources/upgrade-cert-manager.md/).
```plain
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm fetch jetstack/cert-manager --version v0.14.2
helm template ./cert-manager-<version>.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt
```
2. Sort and unique the images list to remove any overlap between the sources:
```plain
sort -u rancher-images.txt -o rancher-images.txt
```
### C. Save the images to your workstation
1. Make `rancher-save-images.sh` an executable:
```
chmod +x rancher-save-images.sh
```
1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images:
```plain
./rancher-save-images.sh --image-list ./rancher-images.txt
```
**Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. Check that the output is in the directory.
### D. Populate the private registry
Move the images in the `rancher-images.tar.gz` to your private registry using the scripts to load the images. The `rancher-images.txt` is expected to be on the workstation in the same directory that you are running the `rancher-load-images.sh` script.
1. Log into your private registry if required:
```plain
docker login <REGISTRY.YOURDOMAIN.COM:PORT>
```
1. Make `rancher-load-images.sh` an executable:
```
chmod +x rancher-load-images.sh
```
1. Use `rancher-load-images.sh` to extract, tag and push `rancher-images.txt` and `rancher-images.tar.gz` to your private registry:
```plain
./rancher-load-images.sh --image-list ./rancher-images.txt --registry <REGISTRY.YOURDOMAIN.COM:PORT>
```
</TabItem>
<TabItem value="Linux and Windows Clusters">
_Available as of v2.3.0_
For Rancher servers that will provision Linux and Windows clusters, there are distinctive steps to populate your private registry for the Windows images and the Linux images. Since a Windows cluster is a mix of Linux and Windows nodes, the Linux images pushed into the private registry are manifests.
### Windows Steps
The Windows images need to be collected and pushed from a Windows server workstation.
A. Find the required assets for your Rancher version <br/>
B. Save the images to your Windows Server workstation <br/>
C. Prepare the Docker daemon <br/>
D. Populate the private registry
<details>
<summary>Collecting and Populating Windows Images into the Private Registry"%}}</summary>
### Prerequisites
These steps expect you to use a Windows Server 1809 workstation that has internet access, access to your private registry, and at least 50 GB of disk space.
The workstation must have Docker 18.02+ in order to support manifests, which are required when provisioning Windows clusters.
Your registry must support manifests. As of April 2020, Amazon Elastic Container Registry does not support manifests.
### A. Find the required assets for your Rancher version
1. Browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments.
2. From the release's "Assets" section, download the following files:
| Release File | Description |
|------------------------|-------------------|
| `rancher-windows-images.txt` | This file contains a list of Windows images needed to provision Windows clusters. |
| `rancher-save-images.ps1` | This script pulls all the images in the `rancher-windows-images.txt` from Docker Hub and saves all of the images as `rancher-windows-images.tar.gz`. |
| `rancher-load-images.ps1` | This script loads the images from the `rancher-windows-images.tar.gz` file and pushes them to your private registry. |
### B. Save the images to your Windows Server workstation
1. Using `powershell`, go to the directory that has the files that were downloaded in the previous step.
1. Run `rancher-save-images.ps1` to create a tarball of all the required images:
```plain
./rancher-save-images.ps1
```
**Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-windows-images.tar.gz`. Check that the output is in the directory.
### C. Prepare the Docker daemon
Append your private registry address to the `allow-nondistributable-artifacts` config field in the Docker daemon (`C:\ProgramData\Docker\config\daemon.json`). Since the base image of Windows images are maintained by the `mcr.microsoft.com` registry, this step is required as the layers in the Microsoft registry are missing from Docker Hub and need to be pulled into the private registry.
```
{
...
"allow-nondistributable-artifacts": [
...
"<REGISTRY.YOURDOMAIN.COM:PORT>"
]
...
}
```
### D. Populate the private registry
Move the images in the `rancher-windows-images.tar.gz` to your private registry using the scripts to load the images. The `rancher-windows-images.txt` is expected to be on the workstation in the same directory that you are running the `rancher-load-images.ps1` script.
1. Using `powershell`, log into your private registry if required:
```plain
docker login <REGISTRY.YOURDOMAIN.COM:PORT>
```
1. Using `powershell`, use `rancher-load-images.ps1` to extract, tag and push the images from `rancher-images.tar.gz` to your private registry:
```plain
./rancher-load-images.ps1 --registry <REGISTRY.YOURDOMAIN.COM:PORT>
```
</details>
### Linux Steps
The Linux images needs to be collected and pushed from a Linux host, but _must be done after_ populating the Windows images into the private registry. These step are different from the Linux only steps as the Linux images that are pushed will actually manifests that support Windows and Linux images.
A. Find the required assets for your Rancher version <br/>
B. Collect all the required images <br/>
C. Save the images to your Linux workstation <br/>
D. Populate the private registry
<details>
<summary>Collecting and Populating Linux Images into the Private Registry</summary>
### Prerequisites
You must populate the private registry with the Windows images before populating the private registry with Linux images. If you have already populated the registry with Linux images, you will need to follow these instructions again as they will publish manifests that support Windows and Linux images.
These steps expect you to use a Linux workstation that has internet access, access to your private registry, and at least 20 GB of disk space.
The workstation must have Docker 18.02+ in order to support manifests, which are required when provisioning Windows clusters.
### A. Find the required assets for your Rancher version
1. Browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments.
2. From the release's **Assets** section, download the following files, which are required to install Rancher in an air gap environment:
| Release File | Description |
|----------------------------|------|
| `rancher-images.txt` | This file contains a list of images needed to install Rancher, provision clusters and user Rancher tools. |
| `rancher-windows-images.txt` | This file contains a list of images needed to provision Windows clusters. |
| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. |
| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. |
### B. Collect all the required images
**For Kubernetes Installs using Rancher Generated Self-Signed Certificate:** In a Kubernetes Install, if you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://hub.helm.sh/charts/jetstack/cert-manager) image to `rancher-images.txt` as well. You skip this step if you are using you using your own certificates.
1. Fetch the latest `cert-manager` Helm chart and parse the template for image details:
> **Note:** Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.12.0, please see our [upgrade documentation](../../../resources/upgrade-cert-manager.md).
```plain
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm fetch jetstack/cert-manager --version v0.14.2
helm template ./cert-manager-<version>.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt
```
2. Sort and unique the images list to remove any overlap between the sources:
```plain
sort -u rancher-images.txt -o rancher-images.txt
```
### C. Save the images to your workstation
1. Make `rancher-save-images.sh` an executable:
```
chmod +x rancher-save-images.sh
```
1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images:
```plain
./rancher-save-images.sh --image-list ./rancher-images.txt
```
**Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. Check that the output is in the directory.
### D. Populate the private registry
Move the images in the `rancher-images.tar.gz` to your private registry using the `rancher-load-images.sh script` to load the images. The `rancher-images.txt` / `rancher-windows-images.txt` image list is expected to be on the workstation in the same directory that you are running the `rancher-load-images.sh` script.
1. Log into your private registry if required:
```plain
docker login <REGISTRY.YOURDOMAIN.COM:PORT>
```
1. Make `rancher-load-images.sh` an executable:
```
chmod +x rancher-load-images.sh
```
1. Use `rancher-load-images.sh` to extract, tag and push the images from `rancher-images.tar.gz` to your private registry:
```plain
./rancher-load-images.sh --image-list ./rancher-images.txt \
--windows-image-list ./rancher-windows-images.txt \
--registry <REGISTRY.YOURDOMAIN.COM:PORT>
```
</details>
</TabItem>
</Tabs>
### [Next: Kubernetes Installs - Launch a Kubernetes Cluster with RKE](../../../other-installation-methods/air-gapped-helm-cli-install/install-kubernetes.md)
### [Next: Docker Installs - Install Rancher](../../../other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md)
@@ -1,107 +0,0 @@
---
title: '1. Prepare your Node(s)'
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/infrastructure-private-registry"/>
</head>
This section is about how to prepare your node(s) to install Rancher for your air gapped environment. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy. There are _tabs_ for either a high availability (recommended) or a Docker installation.
## Prerequisites
<Tabs>
<TabItem value="Kubernetes Install (Recommended)">
### OS, Docker, Hardware, and Networking
Make sure that your node(s) fulfill the general [installation requirements.](../../../installation-requirements/installation-requirements.md)
### Private Registry
Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machines.
If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/).
### CLI Tools
The following CLI tools are required for the Kubernetes Install. Make sure these tools are installed on your workstation and available in your `$PATH`.
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool.
- [rke](https://rancher.com/docs/rke/latest/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters.
- [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. Refer to the [Helm version requirements](../../../resources/choose-a-rancher-version.md) to choose a version of Helm to install Rancher.
</TabItem>
<TabItem value="Docker Install">
### OS, Docker, Hardware, and Networking
Make sure that your node(s) fulfill the general [installation requirements.](../../../installation-requirements/installation-requirements.md)
### Private Registry
Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machines.
If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/).
</TabItem>
</Tabs>
## Set up Infrastructure
<Tabs>
<TabItem value="Kubernetes Install (Recommended)">
Rancher recommends installing Rancher on a Kubernetes cluster. A highly available Kubernetes install is comprised of three nodes running the Rancher server components on a Kubernetes cluster. The persistence layer (etcd) is also replicated on these three nodes, providing redundancy and data duplication in case one of the nodes fails.
### Recommended Architecture
- DNS for Rancher should resolve to a layer 4 load balancer
- The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster.
- The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443.
- The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment.
<figcaption>Rancher installed on a Kubernetes cluster with layer 4 load balancer, depicting SSL termination at ingress controllers</figcaption>
![Rancher HA](/img/ha/rancher2ha.svg)
### A. Provision three air gapped Linux hosts according to our requirements
These hosts will be disconnected from the internet, but require being able to connect with your private registry.
View hardware and software requirements for each of your cluster nodes in [Requirements](../../../installation-requirements/installation-requirements.md).
### B. Set up your Load Balancer
When setting up the Kubernetes cluster that will run the Rancher server components, an Ingress controller pod will be deployed on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server.
You will need to configure a load balancer as a basic Layer 4 TCP forwarder to direct traffic to these ingress controller pods. The exact configuration will vary depending on your environment.
> **Important:**
> Only use this load balancer (i.e, the `local` cluster Ingress) to load balance the Rancher server. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps.
**Load Balancer Configuration Samples:**
- For an example showing how to set up an NGINX load balancer, refer to [this page.](../../../../../how-to-guides/new-user-guides/infrastructure-setup/nginx-load-balancer.md)
- For an example showing how to set up an Amazon NLB load balancer, refer to [this page.](../../../../../how-to-guides/new-user-guides/infrastructure-setup/amazon-elb-load-balancer.md)
</TabItem>
<TabItem value="Docker Install">
The Docker installation is for Rancher users that are wanting to test out Rancher. Instead of running on a Kubernetes cluster, you install the Rancher server component on a single node using a `docker run` command. Since there is only one node and a single Docker container, if the node goes down, there is no copy of the etcd data available on other nodes and you will lose all the data of your Rancher server.
> **Important:** If you install Rancher following the Docker installation guide, there is no upgrade path to transition your Docker installation to a Kubernetes Installation.
Instead of running the Docker installation, you have the option to follow the Kubernetes Install guide, but only use one node to install Rancher. Afterwards, you can scale up the etcd nodes in your Kubernetes cluster to make it a Kubernetes Installation.
### A. Provision a single, air gapped Linux host according to our Requirements
These hosts will be disconnected from the internet, but require being able to connect with your private registry.
View hardware and software requirements for each of your cluster nodes in [Requirements](../../../installation-requirements/installation-requirements.md).
</TabItem>
</Tabs>
### [Next: Collect and Publish Images to your Private Registry](../../../other-installation-methods/air-gapped-helm-cli-install/publish-images.md)
@@ -1,5 +0,0 @@
---
title: cluster.yml Templates for RKE add-on installs
---
These templates are intended to be used for RKE add-on installs, which are only supported up to Rancher v2.0.8. Please use the Rancher Helm chart if you are installing a newer Rancher version.
@@ -1,161 +0,0 @@
---
title: Template for an RKE Cluster with a Certificate Signed by Recognized CA and a Layer 4 Load Balancer
---
RKE uses a cluster.yml file to install and configure your Kubernetes cluster.
This template is intended to be used for RKE add-on installs, which are only supported up to Rancher v2.0.8. Please use the Rancher Helm chart if you are installing a newer Rancher version.
The following template can be used for the cluster.yml if you have a setup with:
- Certificate signed by a recognized CA
- Layer 4 load balancer
- [NGINX Ingress controller](https://kubernetes.github.io/ingress-nginx/)
> For more options, refer to [RKE Documentation: Config Options](https://rancher.com/docs/rke/latest/en/config-options/).
```yaml
nodes:
- address: <IP> # hostname or IP to access nodes
user: <USER> # root user (usually 'root')
role: [controlplane,etcd,worker] # K8s roles for node
ssh_key_path: <PEM_FILE> # path to PEM file
- address: <IP>
user: <USER>
role: [controlplane,etcd,worker]
ssh_key_path: <PEM_FILE>
- address: <IP>
user: <USER>
role: [controlplane,etcd,worker]
ssh_key_path: <PEM_FILE>
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
addons: |-
---
kind: Namespace
apiVersion: v1
metadata:
name: cattle-system
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: cattle-admin
namespace: cattle-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cattle-crb
namespace: cattle-system
subjects:
- kind: ServiceAccount
name: cattle-admin
namespace: cattle-system
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: Secret
metadata:
name: cattle-keys-ingress
namespace: cattle-system
type: Opaque
data:
tls.crt: <BASE64_CRT> # ssl cert for ingress. If self-signed, must be signed by same CA as cattle server
tls.key: <BASE64_KEY> # ssl key for ingress. If self-signed, must be signed by same CA as cattle server
---
apiVersion: v1
kind: Service
metadata:
namespace: cattle-system
name: cattle-service
labels:
app: cattle
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
app: cattle
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: cattle-system
name: cattle-ingress-http
annotations:
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800" # Max time in seconds for ws to remain shell window open
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800" # Max time in seconds for ws to remain shell window open
spec:
rules:
- host: <FQDN> # FQDN to access cattle server
http:
paths:
- backend:
serviceName: cattle-service
servicePort: 80
tls:
- secretName: cattle-keys-ingress
hosts:
- <FQDN> # FQDN to access cattle server
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
namespace: cattle-system
name: cattle
spec:
replicas: 1
template:
metadata:
labels:
app: cattle
spec:
serviceAccountName: cattle-admin
containers:
# Rancher install via RKE addons is only supported up to v2.0.8
- image: rancher/rancher:v2.0.8
args:
- --no-cacerts
imagePullPolicy: Always
name: cattle-server
# env:
# - name: HTTP_PROXY
# value: "http://your_proxy_address:port"
# - name: HTTPS_PROXY
# value: "http://your_proxy_address:port"
# - name: NO_PROXY
# value: "localhost,127.0.0.1,0.0.0.0,10.43.0.0/16,your_network_ranges_that_dont_need_proxy_to_access"
livenessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 60
periodSeconds: 60
readinessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 20
periodSeconds: 10
ports:
- containerPort: 80
protocol: TCP
- containerPort: 443
protocol: TCP
```
@@ -1,177 +0,0 @@
---
title: Template for an RKE Cluster with a Self-signed Certificate and Layer 4 Load Balancer
---
RKE uses a cluster.yml file to install and configure your Kubernetes cluster.
This template is intended to be used for RKE add-on installs, which are only supported up to Rancher v2.0.8. Please use the Rancher Helm chart if you are installing a newer Rancher version.
The following template can be used for the cluster.yml if you have a setup with:
- Self-signed SSL
- Layer 4 load balancer
- [NGINX Ingress controller](https://kubernetes.github.io/ingress-nginx/)
> For more options, refer to [RKE Documentation: Config Options](https://rancher.com/docs/rke/latest/en/config-options/).
```yaml
nodes:
- address: <IP> # hostname or IP to access nodes
user: <USER> # root user (usually 'root')
role: [controlplane,etcd,worker] # K8s roles for node
ssh_key_path: <PEM_FILE> # path to PEM file
- address: <IP>
user: <USER>
role: [controlplane,etcd,worker]
ssh_key_path: <PEM_FILE>
- address: <IP>
user: <USER>
role: [controlplane,etcd,worker]
ssh_key_path: <PEM_FILE>
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
addons: |-
---
kind: Namespace
apiVersion: v1
metadata:
name: cattle-system
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: cattle-admin
namespace: cattle-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cattle-crb
namespace: cattle-system
subjects:
- kind: ServiceAccount
name: cattle-admin
namespace: cattle-system
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: Secret
metadata:
name: cattle-keys-ingress
namespace: cattle-system
type: Opaque
data:
tls.crt: <BASE64_CRT> # ssl cert for ingress. If selfsigned, must be signed by same CA as cattle server
tls.key: <BASE64_KEY> # ssl key for ingress. If selfsigned, must be signed by same CA as cattle server
---
apiVersion: v1
kind: Secret
metadata:
name: cattle-keys-server
namespace: cattle-system
type: Opaque
data:
cacerts.pem: <BASE64_CA> # CA cert used to sign cattle server cert and key
---
apiVersion: v1
kind: Service
metadata:
namespace: cattle-system
name: cattle-service
labels:
app: cattle
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
app: cattle
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: cattle-system
name: cattle-ingress-http
annotations:
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800" # Max time in seconds for ws to remain shell window open
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800" # Max time in seconds for ws to remain shell window open
spec:
rules:
- host: <FQDN> # FQDN to access cattle server
http:
paths:
- backend:
serviceName: cattle-service
servicePort: 80
tls:
- secretName: cattle-keys-ingress
hosts:
- <FQDN> # FQDN to access cattle server
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
namespace: cattle-system
name: cattle
spec:
replicas: 1
template:
metadata:
labels:
app: cattle
spec:
serviceAccountName: cattle-admin
containers:
# Rancher install via RKE addons is only supported up to v2.0.8
- image: rancher/rancher:v2.0.8
imagePullPolicy: Always
name: cattle-server
# env:
# - name: HTTP_PROXY
# value: "http://your_proxy_address:port"
# - name: HTTPS_PROXY
# value: "http://your_proxy_address:port"
# - name: NO_PROXY
# value: "localhost,127.0.0.1,0.0.0.0,10.43.0.0/16,your_network_ranges_that_dont_need_proxy_to_access"
livenessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 60
periodSeconds: 60
readinessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 20
periodSeconds: 10
ports:
- containerPort: 80
protocol: TCP
- containerPort: 443
protocol: TCP
volumeMounts:
- mountPath: /etc/rancher/ssl
name: cattle-keys-volume
readOnly: true
volumes:
- name: cattle-keys-volume
secret:
defaultMode: 420
secretName: cattle-keys-server
```
@@ -1,157 +0,0 @@
---
title: Template for an RKE Cluster with a Self-signed Certificate and SSL Termination on Layer 7 Load Balancer
---
RKE uses a cluster.yml file to install and configure your Kubernetes cluster.
This template is intended to be used for RKE add-on installs, which are only supported up to Rancher v2.0.8. Please use the Rancher Helm chart if you are installing a newer Rancher version.
The following template can be used for the cluster.yml if you have a setup with:
- Layer 7 load balancer with self-signed SSL termination (HTTPS)
- [NGINX Ingress controller](https://kubernetes.github.io/ingress-nginx/)
> For more options, refer to [RKE Documentation: Config Options](https://rancher.com/docs/rke/latest/en/config-options/).
```yaml
nodes:
- address: <IP> # hostname or IP to access nodes
user: <USER> # root user (usually 'root')
role: [controlplane,etcd,worker] # K8s roles for node
ssh_key_path: <PEM_FILE> # path to PEM file
- address: <IP>
user: <USER>
role: [controlplane,etcd,worker]
ssh_key_path: <PEM_FILE>
- address: <IP>
user: <USER>
role: [controlplane,etcd,worker]
ssh_key_path: <PEM_FILE>
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
addons: |-
---
kind: Namespace
apiVersion: v1
metadata:
name: cattle-system
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: cattle-admin
namespace: cattle-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cattle-crb
namespace: cattle-system
subjects:
- kind: ServiceAccount
name: cattle-admin
namespace: cattle-system
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: Secret
metadata:
name: cattle-keys-server
namespace: cattle-system
type: Opaque
data:
cacerts.pem: <BASE64_CA> # CA cert used to sign cattle server cert and key
---
apiVersion: v1
kind: Service
metadata:
namespace: cattle-system
name: cattle-service
labels:
app: cattle
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
selector:
app: cattle
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: cattle-system
name: cattle-ingress-http
annotations:
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800" # Max time in seconds for ws to remain shell window open
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800" # Max time in seconds for ws to remain shell window open
nginx.ingress.kubernetes.io/ssl-redirect: "false" # Disable redirect to ssl
spec:
rules:
- host: <FQDN>
http:
paths:
- backend:
serviceName: cattle-service
servicePort: 80
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
namespace: cattle-system
name: cattle
spec:
replicas: 1
template:
metadata:
labels:
app: cattle
spec:
serviceAccountName: cattle-admin
containers:
# Rancher install via RKE addons is only supported up to v2.0.8
- image: rancher/rancher:v2.0.8
imagePullPolicy: Always
name: cattle-server
# env:
# - name: HTTP_PROXY
# value: "http://your_proxy_address:port"
# - name: HTTPS_PROXY
# value: "http://your_proxy_address:port"
# - name: NO_PROXY
# value: "localhost,127.0.0.1,0.0.0.0,10.43.0.0/16,your_network_ranges_that_dont_need_proxy_to_access"
livenessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 60
periodSeconds: 60
readinessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 20
periodSeconds: 10
ports:
- containerPort: 80
protocol: TCP
volumeMounts:
- mountPath: /etc/rancher/ssl
name: cattle-keys-volume
readOnly: true
volumes:
- name: cattle-keys-volume
secret:
defaultMode: 420
secretName: cattle-keys-server
```
@@ -1,141 +0,0 @@
---
title: Template for an RKE Cluster with a Recognized CA Certificate and SSL Termination on Layer 7 Load Balancer
---
RKE uses a cluster.yml file to install and configure your Kubernetes cluster.
This template is intended to be used for RKE add-on installs, which are only supported up to Rancher v2.0.8. Please use the Rancher Helm chart if you are installing a newer Rancher version.
The following template can be used for the cluster.yml if you have a setup with:
- Layer 7 load balancer with SSL termination (HTTPS)
- [NGINX Ingress controller](https://kubernetes.github.io/ingress-nginx/)
> For more options, refer to [RKE Documentation: Config Options](https://rancher.com/docs/rke/latest/en/config-options/).
```yaml
nodes:
- address: <IP> # hostname or IP to access nodes
user: <USER> # root user (usually 'root')
role: [controlplane,etcd,worker] # K8s roles for node
ssh_key_path: <PEM_FILE> # path to PEM file
- address: <IP>
user: <USER>
role: [controlplane,etcd,worker]
ssh_key_path: <PEM_FILE>
- address: <IP>
user: <USER>
role: [controlplane,etcd,worker]
ssh_key_path: <PEM_FILE>
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
addons: |-
---
kind: Namespace
apiVersion: v1
metadata:
name: cattle-system
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: cattle-admin
namespace: cattle-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cattle-crb
namespace: cattle-system
subjects:
- kind: ServiceAccount
name: cattle-admin
namespace: cattle-system
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: Service
metadata:
namespace: cattle-system
name: cattle-service
labels:
app: cattle
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
selector:
app: cattle
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: cattle-system
name: cattle-ingress-http
annotations:
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800" # Max time in seconds for ws to remain shell window open
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800" # Max time in seconds for ws to remain shell window open
nginx.ingress.kubernetes.io/ssl-redirect: "false" # Disable redirect to ssl
spec:
rules:
- host: <FQDN>
http:
paths:
- backend:
serviceName: cattle-service
servicePort: 80
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
namespace: cattle-system
name: cattle
spec:
replicas: 1
template:
metadata:
labels:
app: cattle
spec:
serviceAccountName: cattle-admin
containers:
# Rancher install via RKE addons is only supported up to v2.0.8
- image: rancher/rancher:v2.0.8
args:
- --no-cacerts
imagePullPolicy: Always
name: cattle-server
# env:
# - name: HTTP_PROXY
# value: "http://your_proxy_address:port"
# - name: HTTPS_PROXY
# value: "http://your_proxy_address:port"
# - name: NO_PROXY
# value: "localhost,127.0.0.1,0.0.0.0,10.43.0.0/16,your_network_ranges_that_dont_need_proxy_to_access"
livenessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 60
periodSeconds: 60
readinessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 20
periodSeconds: 10
ports:
- containerPort: 80
protocol: TCP
```
@@ -1,244 +0,0 @@
---
title: Docker Install with TLS Termination at Layer-7 NGINX Load Balancer
---
For development and testing environments that have a special requirement to terminate TLS/SSL at a load balancer instead of your Rancher Server container, deploy Rancher and configure a load balancer to work with it conjunction.
A layer-7 load balancer can be beneficial if you want to centralize your TLS termination in your infrastructure. Layer-7 load balancing also offers the capability for your load balancer to make decisions based on HTTP attributes such as cookies, etc. that a layer-4 load balancer is not able to concern itself with.
This install procedure walks you through deployment of Rancher using a single container, and then provides a sample configuration for a layer-7 NGINX load balancer.
> **Want to skip the external load balancer?**
> See [Docker Installation](../../other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) instead.
## Requirements for OS, Docker, Hardware, and Networking
Make sure that your node fulfills the general [installation requirements.](../../installation-requirements/installation-requirements.md)
## Installation Outline
## 1. Provision Linux Host
Provision a single Linux host according to our [Requirements](../../installation-requirements/installation-requirements.md) to launch your Rancher Server.
## 2. Choose an SSL Option and Install Rancher
For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.
> **Do you want to...**
>
> - Complete an Air Gap Installation?
> - Record all transactions with the Rancher API?
>
> See [Advanced Options](#advanced-options) below before continuing.
Choose from the following options:
<details id="option-a">
<summary>Option A-Bring Your Own Certificate: Self-Signed</summary>
If you elect to use a self-signed certificate to encrypt communication, you must install the certificate on your load balancer (which you'll do later) and your Rancher container. Run the Docker command to deploy Rancher, pointing it toward your certificate.
> **Prerequisites:**
> Create a self-signed certificate.
>
> - The certificate files must be in PEM format.
**To Install Rancher Using a Self-Signed Cert:**
1. While running the Docker command to deploy Rancher, point Docker toward your CA certificate file.
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /etc/your_certificate_directory/cacerts.pem:/etc/rancher/ssl/cacerts.pem \
rancher/rancher:latest
```
</details>
<details id="option-b">
<summary>Option B-Bring Your Own Certificate: Signed by Recognized CA</summary>
If your cluster is public facing, it's best to use a certificate signed by a recognized CA.
> **Prerequisites:**
>
> - The certificate files must be in PEM format.
**To Install Rancher Using a Cert Signed by a Recognized CA:**
If you use a certificate signed by a recognized CA, installing your certificate in the Rancher container isn't necessary. We do have to make sure there is no default CA certificate generated and stored, you can do this by passing the `--no-cacerts` parameter to the container.
1. Enter the following command.
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
rancher/rancher:latest --no-cacerts
```
</details>
## 3. Configure Load Balancer
When using a load balancer in front of your Rancher container, there's no need for the container to redirect port communication from port 80 or port 443. By passing the header `X-Forwarded-Proto: https` header, this redirect is disabled.
The load balancer or proxy has to be configured to support the following:
- **WebSocket** connections
- **SPDY** / **HTTP/2** protocols
- Passing / setting the following headers:
| Header | Value | Description |
|--------|-------|-------------|
| `Host` | Hostname used to reach Rancher. | To identify the server requested by the client.
| `X-Forwarded-Proto` | `https` | To identify the protocol that a client used to connect to the load balancer or proxy.<br /><br/>**Note:** If this header is present, `rancher/rancher` does not redirect HTTP to HTTPS.
| `X-Forwarded-Port` | Port used to reach Rancher. | To identify the protocol that client used to connect to the load balancer or proxy.
| `X-Forwarded-For` | IP of the client connection. | To identify the originating IP address of a client.
### Example NGINX configuration
This NGINX configuration is tested on NGINX 1.14.
> **Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/).
- Replace `rancher-server` with the IP address or hostname of the node running the Rancher container.
- Replace both occurrences of `FQDN` to the DNS name for Rancher.
- Replace `/certs/fullchain.pem` and `/certs/privkey.pem` to the location of the server certificate and the server certificate key respectively.
```
worker_processes 4;
worker_rlimit_nofile 40000;
events {
worker_connections 8192;
}
http {
upstream rancher {
server rancher-server:80;
}
map $http_upgrade $connection_upgrade {
default Upgrade;
'' close;
}
server {
listen 443 ssl http2;
server_name FQDN;
ssl_certificate /certs/fullchain.pem;
ssl_certificate_key /certs/privkey.pem;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://rancher;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close.
proxy_read_timeout 900s;
proxy_buffering off;
}
}
server {
listen 80;
server_name FQDN;
return 301 https://$server_name$request_uri;
}
}
```
<br/>
## What's Next?
- **Recommended:** Review [Single Node Backup and Restore](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-docker-installed-rancher.md). Although you don't have any data you need to back up right now, we recommend creating backups after regular Rancher use.
- Create a Kubernetes cluster: [Provisioning Kubernetes Clusters](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md).
<br/>
## FAQ and Troubleshooting
For help troubleshooting certificates, see [this section.](../../other-installation-methods/rancher-on-a-single-node-with-docker/certificate-troubleshooting.md)
## Advanced Options
### API Auditing
If you want to record all transactions with the Rancher API, enable the [API Auditing](../../../installation-and-upgrade/advanced-options/advanced-use-cases/enable-api-audit-log.md) feature by adding the flags below into your install command.
-e AUDIT_LEVEL=1 \
-e AUDIT_LOG_PATH=/var/log/auditlog/rancher-api-audit.log \
-e AUDIT_LOG_MAXAGE=20 \
-e AUDIT_LOG_MAXBACKUP=20 \
-e AUDIT_LOG_MAXSIZE=100 \
### Air Gap
If you are visiting this page to complete an [Air Gap Installation](air-gap-helm2/air-gap-helm2.md), you must pre-pend your private registry URL to the server tag when running the installation command in the option that you choose. Add `<REGISTRY.DOMAIN.COM:PORT>` with your private registry URL in front of `rancher/rancher:latest`.
**Example:**
<REGISTRY.DOMAIN.COM:PORT>/rancher/rancher:latest
### Persistent Data
Rancher uses etcd as a datastore. When Rancher is installed with Docker, the embedded etcd is being used. The persistent data is at the following path in the container: `/var/lib/rancher`.
You can bind mount a host volume to this location to preserve data on the host it is running on:
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /opt/rancher:/var/lib/rancher \
rancher/rancher:latest
```
This layer 7 NGINX configuration is tested on NGINX version 1.13 (mainline) and 1.14 (stable).
> **Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - TCP and UDP Load Balancer](https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/).
```
upstream rancher {
server rancher-server:80;
}
map $http_upgrade $connection_upgrade {
default Upgrade;
'' close;
}
server {
listen 443 ssl http2;
server_name rancher.yourdomain.com;
ssl_certificate /etc/your_certificate_directory/fullchain.pem;
ssl_certificate_key /etc/your_certificate_directory/privkey.pem;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://rancher;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close.
proxy_read_timeout 900s;
proxy_buffering off;
}
}
server {
listen 80;
server_name rancher.yourdomain.com;
return 301 https://$server_name$request_uri;
}
```
<br/>
@@ -1,565 +0,0 @@
---
title: Enabling the API Audit Log to Record System Events
---
You can enable the API audit log to record the sequence of system events initiated by individual users. You can know what happened, when it happened, who initiated it, and what cluster it affected. When you enable this feature, all requests to the Rancher API and all responses from it are written to a log.
You can enable API Auditing during Rancher installation or upgrade.
## Enabling API Audit Log
The Audit Log is enabled and configured by passing environment variables to the Rancher server container. See the following to enable on your installation.
- [Docker Install](../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#api-audit-log)
- [Kubernetes Install](../../../../reference-guides/installation-references/helm-chart-options.md#api-audit-log)
## API Audit Log Options
The usage below defines rules about what the audit log should record and what data it should include:
| Parameter | Description |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `AUDIT_LEVEL` | `0` - Disable audit log (default setting).<br/>`1` - Log event metadata.<br/>`2` - Log event metadata and request body.<br/>`3` - Log event metadata, request body, and response body. Each log transaction for a request/response pair uses the same `auditID` value.<br/><br/>See [Audit Level Logging](#audit-log-levels) for a table that displays what each setting logs. |
| `AUDIT_LOG_PATH` | Log path for Rancher Server API. Default path is `/var/log/auditlog/rancher-api-audit.log`. You can mount the log directory to host. <br/><br/>Usage Example: `AUDIT_LOG_PATH=/my/custom/path/`<br/> |
| `AUDIT_LOG_MAXAGE` | Defined the maximum number of days to retain old audit log files. Default is 10 days. |
| `AUDIT_LOG_MAXBACKUP` | Defines the maximum number of audit log files to retain. Default is 10. |
| `AUDIT_LOG_MAXSIZE` | Defines the maximum size in megabytes of the audit log file before it gets rotated. Default size is 100M. |
<br/>
### Audit Log Levels
The following table displays what parts of API transactions are logged for each [`AUDIT_LEVEL`](#api-audit-log-options) setting.
| `AUDIT_LEVEL` Setting | Metadata | Request Body | Response Body |
| --------------------- | -------- | ------------ | ------------- |
| `0` | | | |
| `1` | ✓ | | |
| `2` | ✓ | ✓ | |
| `3` | ✓ | ✓ | ✓ |
## Viewing API Audit Logs
### Docker Install
Share the `AUDIT_LOG_PATH` directory (Default: `/var/log/auditlog`) with the host system. The log can be parsed by standard CLI tools or forwarded on to a log collection tool like Fluentd, Filebeat, Logstash, etc.
### Kubernetes Install
Enabling the API Audit Log with the Helm chart install will create a `rancher-audit-log` sidecar container in the Rancher pod. This container will stream the log to standard output (stdout). You can view the log as you would any container log.
The `rancher-audit-log` container is part of the `rancher` pod in the `cattle-system` namespace.
#### CLI
```bash
kubectl -n cattle-system logs -f rancher-84d886bdbb-s4s69 rancher-audit-log
```
#### Rancher Web GUI
1. From the context menu, select **Cluster: local > System**.
1. From the main navigation bar, choose **Resources > Workloads.** (In versions before v2.3.0, choose **Workloads** on the main navigation bar.) Find the `cattle-system` namespace. Open the `rancher` workload by clicking its link.
1. Pick one of the `rancher` pods and select **&#8942; > View Logs**.
1. From the **Logs** drop-down, select `rancher-audit-log`.
#### Shipping the Audit Log
You can enable Rancher's built in log collection and shipping for the cluster to ship the audit and other services logs to a supported collection endpoint. See [Rancher Tools - Logging](../../../../explanations/integrations-in-rancher/cluster-logging/cluster-logging.md) for details.
## Audit Log Samples
After you enable auditing, each API request or response is logged by Rancher in the form of JSON. Each of the following code samples provide examples of how to identify each API transaction.
### Metadata Level
If you set your `AUDIT_LEVEL` to `1`, Rancher logs the metadata header for every API request, but not the body. The header provides basic information about the API transaction, such as the transaction's ID, who initiated the transaction, the time it occurred, etc.
```json
{
"auditID": "30022177-9e2e-43d1-b0d0-06ef9d3db183",
"requestURI": "/v3/schemas",
"sourceIPs": ["::1"],
"user": {
"name": "user-f4tt2",
"group": ["system:authenticated"]
},
"verb": "GET",
"stage": "RequestReceived",
"stageTimestamp": "2018-07-20 10:22:43 +0800"
}
```
### Metadata and Request Body Level
If you set your `AUDIT_LEVEL` to `2`, Rancher logs the metadata header and body for every API request.
The code sample below depicts an API request, with both its metadata header and body.
```json
{
"auditID": "ef1d249e-bfac-4fd0-a61f-cbdcad53b9bb",
"requestURI": "/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"sourceIPs": ["::1"],
"user": {
"name": "user-f4tt2",
"group": ["system:authenticated"]
},
"verb": "PUT",
"stage": "RequestReceived",
"stageTimestamp": "2018-07-20 10:28:08 +0800",
"requestBody": {
"hostIPC": false,
"hostNetwork": false,
"hostPID": false,
"paused": false,
"annotations": {},
"baseType": "workload",
"containers": [
{
"allowPrivilegeEscalation": false,
"image": "nginx",
"imagePullPolicy": "Always",
"initContainer": false,
"name": "nginx",
"ports": [
{
"containerPort": 80,
"dnsName": "nginx-nodeport",
"kind": "NodePort",
"name": "80tcp01",
"protocol": "TCP",
"sourcePort": 0,
"type": "/v3/project/schemas/containerPort"
}
],
"privileged": false,
"readOnly": false,
"resources": {
"type": "/v3/project/schemas/resourceRequirements",
"requests": {},
"limits": {}
},
"restartCount": 0,
"runAsNonRoot": false,
"stdin": true,
"stdinOnce": false,
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"tty": true,
"type": "/v3/project/schemas/container",
"environmentFrom": [],
"capAdd": [],
"capDrop": [],
"livenessProbe": null,
"volumeMounts": []
}
],
"created": "2018-07-18T07:34:16Z",
"createdTS": 1531899256000,
"creatorId": null,
"deploymentConfig": {
"maxSurge": 1,
"maxUnavailable": 0,
"minReadySeconds": 0,
"progressDeadlineSeconds": 600,
"revisionHistoryLimit": 10,
"strategy": "RollingUpdate"
},
"deploymentStatus": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": "2018-07-18T07:34:38Z",
"lastTransitionTimeTS": 1531899278000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2018-07-18T07:34:16Z",
"lastTransitionTimeTS": 1531899256000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 2,
"readyReplicas": 1,
"replicas": 1,
"type": "/v3/project/schemas/deploymentStatus",
"unavailableReplicas": 0,
"updatedReplicas": 1
},
"dnsPolicy": "ClusterFirst",
"id": "deployment:default:nginx",
"labels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"name": "nginx",
"namespaceId": "default",
"projectId": "c-bcz5t:p-fdr4s",
"publicEndpoints": [
{
"addresses": ["10.64.3.58"],
"allNodes": true,
"ingressId": null,
"nodeId": null,
"podId": null,
"port": 30917,
"protocol": "TCP",
"serviceId": "default:nginx-nodeport",
"type": "publicEndpoint"
}
],
"restartPolicy": "Always",
"scale": 1,
"schedulerName": "default-scheduler",
"selector": {
"matchLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"type": "/v3/project/schemas/labelSelector"
},
"state": "active",
"terminationGracePeriodSeconds": 30,
"transitioning": "no",
"transitioningMessage": "",
"type": "deployment",
"uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd",
"workloadAnnotations": {
"deployment.kubernetes.io/revision": "1",
"field.cattle.io/creatorId": "user-f4tt2"
},
"workloadLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"scheduling": {
"node": {}
},
"description": "my description",
"volumes": []
}
}
```
### Metadata, Request Body, and Response Body Level
If you set your `AUDIT_LEVEL` to `3`, Rancher logs:
- The metadata header and body for every API request.
- The metadata header and body for every API response.
#### Request
The code sample below depicts an API request, with both its metadata header and body.
```json
{
"auditID": "a886fd9f-5d6b-4ae3-9a10-5bff8f3d68af",
"requestURI": "/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"sourceIPs": ["::1"],
"user": {
"name": "user-f4tt2",
"group": ["system:authenticated"]
},
"verb": "PUT",
"stage": "RequestReceived",
"stageTimestamp": "2018-07-20 10:33:06 +0800",
"requestBody": {
"hostIPC": false,
"hostNetwork": false,
"hostPID": false,
"paused": false,
"annotations": {},
"baseType": "workload",
"containers": [
{
"allowPrivilegeEscalation": false,
"image": "nginx",
"imagePullPolicy": "Always",
"initContainer": false,
"name": "nginx",
"ports": [
{
"containerPort": 80,
"dnsName": "nginx-nodeport",
"kind": "NodePort",
"name": "80tcp01",
"protocol": "TCP",
"sourcePort": 0,
"type": "/v3/project/schemas/containerPort"
}
],
"privileged": false,
"readOnly": false,
"resources": {
"type": "/v3/project/schemas/resourceRequirements",
"requests": {},
"limits": {}
},
"restartCount": 0,
"runAsNonRoot": false,
"stdin": true,
"stdinOnce": false,
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"tty": true,
"type": "/v3/project/schemas/container",
"environmentFrom": [],
"capAdd": [],
"capDrop": [],
"livenessProbe": null,
"volumeMounts": []
}
],
"created": "2018-07-18T07:34:16Z",
"createdTS": 1531899256000,
"creatorId": null,
"deploymentConfig": {
"maxSurge": 1,
"maxUnavailable": 0,
"minReadySeconds": 0,
"progressDeadlineSeconds": 600,
"revisionHistoryLimit": 10,
"strategy": "RollingUpdate"
},
"deploymentStatus": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": "2018-07-18T07:34:38Z",
"lastTransitionTimeTS": 1531899278000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2018-07-18T07:34:16Z",
"lastTransitionTimeTS": 1531899256000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 2,
"readyReplicas": 1,
"replicas": 1,
"type": "/v3/project/schemas/deploymentStatus",
"unavailableReplicas": 0,
"updatedReplicas": 1
},
"dnsPolicy": "ClusterFirst",
"id": "deployment:default:nginx",
"labels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"name": "nginx",
"namespaceId": "default",
"projectId": "c-bcz5t:p-fdr4s",
"publicEndpoints": [
{
"addresses": ["10.64.3.58"],
"allNodes": true,
"ingressId": null,
"nodeId": null,
"podId": null,
"port": 30917,
"protocol": "TCP",
"serviceId": "default:nginx-nodeport",
"type": "publicEndpoint"
}
],
"restartPolicy": "Always",
"scale": 1,
"schedulerName": "default-scheduler",
"selector": {
"matchLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"type": "/v3/project/schemas/labelSelector"
},
"state": "active",
"terminationGracePeriodSeconds": 30,
"transitioning": "no",
"transitioningMessage": "",
"type": "deployment",
"uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd",
"workloadAnnotations": {
"deployment.kubernetes.io/revision": "1",
"field.cattle.io/creatorId": "user-f4tt2"
},
"workloadLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"scheduling": {
"node": {}
},
"description": "my decript",
"volumes": []
}
}
```
#### Response
The code sample below depicts an API response, with both its metadata header and body.
```json
{
"auditID": "a886fd9f-5d6b-4ae3-9a10-5bff8f3d68af",
"responseStatus": "200",
"stage": "ResponseComplete",
"stageTimestamp": "2018-07-20 10:33:06 +0800",
"responseBody": {
"actionLinks": {
"pause": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=pause",
"resume": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=resume",
"rollback": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=rollback"
},
"annotations": {},
"baseType": "workload",
"containers": [
{
"allowPrivilegeEscalation": false,
"image": "nginx",
"imagePullPolicy": "Always",
"initContainer": false,
"name": "nginx",
"ports": [
{
"containerPort": 80,
"dnsName": "nginx-nodeport",
"kind": "NodePort",
"name": "80tcp01",
"protocol": "TCP",
"sourcePort": 0,
"type": "/v3/project/schemas/containerPort"
}
],
"privileged": false,
"readOnly": false,
"resources": {
"type": "/v3/project/schemas/resourceRequirements"
},
"restartCount": 0,
"runAsNonRoot": false,
"stdin": true,
"stdinOnce": false,
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"tty": true,
"type": "/v3/project/schemas/container"
}
],
"created": "2018-07-18T07:34:16Z",
"createdTS": 1531899256000,
"creatorId": null,
"deploymentConfig": {
"maxSurge": 1,
"maxUnavailable": 0,
"minReadySeconds": 0,
"progressDeadlineSeconds": 600,
"revisionHistoryLimit": 10,
"strategy": "RollingUpdate"
},
"deploymentStatus": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": "2018-07-18T07:34:38Z",
"lastTransitionTimeTS": 1531899278000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2018-07-18T07:34:16Z",
"lastTransitionTimeTS": 1531899256000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 2,
"readyReplicas": 1,
"replicas": 1,
"type": "/v3/project/schemas/deploymentStatus",
"unavailableReplicas": 0,
"updatedReplicas": 1
},
"dnsPolicy": "ClusterFirst",
"hostIPC": false,
"hostNetwork": false,
"hostPID": false,
"id": "deployment:default:nginx",
"labels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"links": {
"remove": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"revisions": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx/revisions",
"self": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"update": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"yaml": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx/yaml"
},
"name": "nginx",
"namespaceId": "default",
"paused": false,
"projectId": "c-bcz5t:p-fdr4s",
"publicEndpoints": [
{
"addresses": ["10.64.3.58"],
"allNodes": true,
"ingressId": null,
"nodeId": null,
"podId": null,
"port": 30917,
"protocol": "TCP",
"serviceId": "default:nginx-nodeport"
}
],
"restartPolicy": "Always",
"scale": 1,
"schedulerName": "default-scheduler",
"selector": {
"matchLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"type": "/v3/project/schemas/labelSelector"
},
"state": "active",
"terminationGracePeriodSeconds": 30,
"transitioning": "no",
"transitioningMessage": "",
"type": "deployment",
"uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd",
"workloadAnnotations": {
"deployment.kubernetes.io/revision": "1",
"field.cattle.io/creatorId": "user-f4tt2"
},
"workloadLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
}
}
}
```
@@ -1,29 +0,0 @@
---
title: "1. Create Nodes and Load Balancer"
---
Use your provider of choice to provision 3 nodes and a Load Balancer endpoint for your RKE install.
> **Note:** These nodes must be in the same region/datacenter. You may place these servers in separate availability zones.
### Node Requirements
View the supported operating systems and hardware/software/networking requirements for nodes running Rancher at [Node Requirements](../../../../installation-requirements/installation-requirements.md).
View the OS requirements for RKE at [RKE Requirements](https://rancher.com/docs/rke/latest/en/os/)
### Load Balancer
RKE will configure an Ingress controller pod, on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server.
Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configuration will vary depending on your environment.
>**Important:**
>Do not use this load balancer (i.e, the `local` cluster Ingress) to load balance applications other than Rancher following installation. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. We recommend dedicating the `local` cluster to Rancher and no other applications.
#### Examples
* [Nginx](nginx.md)
* [Amazon NLB](nlb.md)
### [Next: Install Kubernetes with RKE](../kubernetes-rke/kubernetes-rke.md)
@@ -1,80 +0,0 @@
---
title: NGINX
---
NGINX will be configured as Layer 4 load balancer (TCP) that forwards connections to one of your Rancher nodes.
>**Note:**
> In this configuration, the load balancer is positioned in front of your nodes. The load balancer can be any host capable of running NGINX.
>
> One caveat: do not use one of your Rancher nodes as the load balancer.
## Install NGINX
Start by installing NGINX on the node you want to use as a load balancer. NGINX has packages available for all known operating systems. The versions tested are `1.14` and `1.15`. For help installing NGINX, refer to their [install documentation](https://www.nginx.com/resources/wiki/start/topics/tutorials/install/).
The `stream` module is required, which is present when using the official NGINX packages. Please refer to your OS documentation on how to install and enable the NGINX `stream` module on your operating system.
## Create NGINX Configuration
After installing NGINX, you need to update the NGINX configuration file, `nginx.conf`, with the IP addresses for your nodes.
1. Copy and paste the code sample below into your favorite text editor. Save it as `nginx.conf`.
2. From `nginx.conf`, replace both occurrences (port 80 and port 443) of `<IP_NODE_1>`, `<IP_NODE_2>`, and `<IP_NODE_3>` with the IPs of your [nodes](create-nodes-lb.md).
>**Note:** See [NGINX Documentation: TCP and UDP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/) for all configuration options.
<figcaption>Example NGINX config</figcaption>
```
worker_processes 4;
worker_rlimit_nofile 40000;
events {
worker_connections 8192;
}
stream {
upstream rancher_servers_http {
least_conn;
server <IP_NODE_1>:80 max_fails=3 fail_timeout=5s;
server <IP_NODE_2>:80 max_fails=3 fail_timeout=5s;
server <IP_NODE_3>:80 max_fails=3 fail_timeout=5s;
}
server {
listen 80;
proxy_pass rancher_servers_http;
}
upstream rancher_servers_https {
least_conn;
server <IP_NODE_1>:443 max_fails=3 fail_timeout=5s;
server <IP_NODE_2>:443 max_fails=3 fail_timeout=5s;
server <IP_NODE_3>:443 max_fails=3 fail_timeout=5s;
}
server {
listen 443;
proxy_pass rancher_servers_https;
}
}
```
3. Save `nginx.conf` to your load balancer at the following path: `/etc/nginx/nginx.conf`.
4. Load the updates to your NGINX configuration by running the following command:
```
# nginx -s reload
```
## Option - Run NGINX as Docker container
Instead of installing NGINX as a package on the operating system, you can rather run it as a Docker container. Save the edited **Example NGINX config** as `/etc/nginx.conf` and run the following command to launch the NGINX container:
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /etc/nginx.conf:/etc/nginx/nginx.conf \
nginx:1.14
```
@@ -1,175 +0,0 @@
---
title: Amazon NLB
---
## Objectives
Configuring an Amazon NLB is a multistage process. We've broken it down into multiple tasks so that it's easy to follow.
1. [Create Target Groups](#create-target-groups)
Begin by creating two target groups for the **TCP** protocol, one regarding TCP port 443 and one regarding TCP port 80 (providing redirect to TCP port 443). You'll add your Linux nodes to these groups.
2. [Register Targets](#register-targets)
Add your Linux nodes to the target groups.
3. [Create Your NLB](#create-your-nlb)
Use Amazon's Wizard to create an Network Load Balancer. As part of this process, you'll add the target groups you created in **1. Create Target Groups**.
> **Note:** Rancher only supports using the Amazon NLB when terminating traffic in `tcp` mode for port 443 rather than `tls` mode. This is due to the fact that the NLB does not inject the correct headers into requests when terminated at the NLB. This means that if you want to use certificates managed by the Amazon Certificate Manager (ACM), you should use an ELB or ALB.
## Create Target Groups
Your first NLB configuration step is to create two target groups. Technically, only port 443 is needed to access Rancher, but its convenient to add a listener for port 80 which will be redirected to port 443 automatically. The NGINX ingress controller on the nodes will make sure that port 80 gets redirected to port 443.
Log into the [Amazon AWS Console](https://console.aws.amazon.com/ec2/) to get started, make sure to select the **Region** where your EC2 instances (Linux nodes) are created.
The Target Groups configuration resides in the **Load Balancing** section of the **EC2** service. Select **Services** and choose **EC2**, find the section **Load Balancing** and open **Target Groups**.
![](/img/ha/nlb/ec2-loadbalancing.png)
Click **Create target group** to create the first target group, regarding TCP port 443.
### Target Group (TCP port 443)
Configure the first target group according to the table below. Screenshots of the configuration are shown just below the table.
Option | Setting
--------------------------------------|------------------------------------
Target Group Name | `rancher-tcp-443`
Protocol | `TCP`
Port | `443`
Target type | `instance`
VPC | Choose your VPC
Protocol<br/>(Health Check) | `HTTP`
Path<br/>(Health Check) | `/healthz`
Port (Advanced health check) | `override`,`80`
Healthy threshold (Advanced health) | `3`
Unhealthy threshold (Advanced) | `3`
Timeout (Advanced) | `6 seconds`
Interval (Advanced) | `10 second`
Success codes | `200-399`
***
**Screenshot Target group TCP port 443 settings**<br/>
![](/img/ha/nlb/create-targetgroup-443.png)
***
**Screenshot Target group TCP port 443 Advanced settings**<br/>
![](/img/ha/nlb/create-targetgroup-443-advanced.png)
***
Click **Create target group** to create the second target group, regarding TCP port 80.
### Target Group (TCP port 80)
Configure the second target group according to the table below. Screenshots of the configuration are shown just below the table.
Option | Setting
--------------------------------------|------------------------------------
Target Group Name | `rancher-tcp-80`
Protocol | `TCP`
Port | `80`
Target type | `instance`
VPC | Choose your VPC
Protocol<br/>(Health Check) | `HTTP`
Path<br/>(Health Check) | `/healthz`
Port (Advanced health check) | `traffic port`
Healthy threshold (Advanced health) | `3`
Unhealthy threshold (Advanced) | `3`
Timeout (Advanced) | `6 seconds`
Interval (Advanced) | `10 second`
Success codes | `200-399`
***
**Screenshot Target group TCP port 80 settings**<br/>
![](/img/ha/nlb/create-targetgroup-80.png)
***
**Screenshot Target group TCP port 80 Advanced settings**<br/>
![](/img/ha/nlb/create-targetgroup-80-advanced.png)
***
## Register Targets
Next, add your Linux nodes to both target groups.
Select the target group named **rancher-tcp-443**, click the tab **Targets** and choose **Edit**.
![](/img/ha/nlb/edit-targetgroup-443.png)
Select the instances (Linux nodes) you want to add, and click **Add to registered**.
***
**Screenshot Add targets to target group TCP port 443**<br/>
![](/img/ha/nlb/add-targets-targetgroup-443.png)
***
**Screenshot Added targets to target group TCP port 443**<br/>
![](/img/ha/nlb/added-targets-targetgroup-443.png)
When the instances are added, click **Save** on the bottom right of the screen.
Repeat those steps, replacing **rancher-tcp-443** with **rancher-tcp-80**. The same instances need to be added as targets to this target group.
## Create Your NLB
Use Amazon's Wizard to create an Network Load Balancer. As part of this process, you'll add the target groups you created in [Create Target Groups](#create-target-groups).
1. From your web browser, navigate to the [Amazon EC2 Console](https://console.aws.amazon.com/ec2/).
2. From the navigation pane, choose **LOAD BALANCING** > **Load Balancers**.
3. Click **Create Load Balancer**.
4. Choose **Network Load Balancer** and click **Create**.
5. Complete the **Step 1: Configure Load Balancer** form.
- **Basic Configuration**
- Name: `rancher`
- Scheme: `internal` or `internet-facing`
The Scheme that you choose for your NLB is dependent on the configuration of your instances/VPC. If your instances do not have public IPs associated with them, or you will only be accessing Rancher internally, you should set your NLB Scheme to `internal` rather than `internet-facing`.
- **Listeners**
Add the **Load Balancer Protocols** and **Load Balancer Ports** below.
- `TCP`: `443`
- **Availability Zones**
- Select Your **VPC** and **Availability Zones**.
6. Complete the **Step 2: Configure Routing** form.
- From the **Target Group** drop-down, choose **Existing target group**.
- From the **Name** drop-down, choose `rancher-tcp-443`.
- Open **Advanced health check settings**, and configure **Interval** to `10 seconds`.
7. Complete **Step 3: Register Targets**. Since you registered your targets earlier, all you have to do is click **Next: Review**.
8. Complete **Step 4: Review**. Look over the load balancer details and click **Create** when you're satisfied.
9. After AWS creates the NLB, click **Close**.
## Add listener to NLB for TCP port 80
1. Select your newly created NLB and select the **Listeners** tab.
2. Click **Add listener**.
3. Use `TCP`:`80` as **Protocol** : **Port**
4. Click **Add action** and choose **Forward to...**
5. From the **Forward to** drop-down, choose `rancher-tcp-80`.
6. Click **Save** in the top right of the screen.
@@ -1,65 +0,0 @@
---
title: "Initialize Helm: Install the Tiller Service"
description: "With Helm, you can create configurable deployments instead of using static files. In order to use Helm, the Tiller service needs to be installed on your cluster."
---
Helm is the package management tool of choice for Kubernetes. Helm "charts" provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at [https://helm.sh/](https://helm.sh/). To be able to use Helm, the server-side component `tiller` needs to be installed on your cluster.
For systems without direct internet access, see [Helm - Air Gap](../../../../other-installation-methods/air-gapped-helm-cli-install/air-gapped-helm-cli-install.md) for install details.
Refer to the [Helm version requirements](../../../../resources/helm-version-requirements.md) to choose a version of Helm to install Rancher.
> **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3)
### Install Tiller on the Cluster
> **Important:** Due to an issue with Helm v2.12.0 and cert-manager, please use Helm v2.12.1 or higher.
Helm installs the `tiller` service on your cluster to manage charts. Since RKE enables RBAC by default we will need to use `kubectl` to create a `serviceaccount` and `clusterrolebinding` so `tiller` has permission to deploy to the cluster.
* Create the `ServiceAccount` in the `kube-system` namespace.
* Create the `ClusterRoleBinding` to give the `tiller` account access to the cluster.
* Finally use `helm` to install the `tiller` service
```plain
kubectl -n kube-system create serviceaccount tiller
kubectl create clusterrolebinding tiller \
--clusterrole=cluster-admin \
--serviceaccount=kube-system:tiller
helm init --service-account tiller
# Users in China: You will need to specify a specific tiller-image in order to initialize tiller.
# The list of tiller image tags are available here: https://dev.aliyun.com/detail.html?spm=5176.1972343.2.18.ErFNgC&repoId=62085.
# When initializing tiller, you'll need to pass in --tiller-image
helm init --service-account tiller \
--tiller-image registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:<tag>
```
> **Note:** This`tiller`install has full cluster access, which should be acceptable if the cluster is dedicated to Rancher server. See the [Helm docs](https://docs.helm.sh/using_helm/#role-based-access-control) for instructions on restricting `tiller` access to suit your security requirements.
### Test your Tiller installation
Run the following command to verify the installation of `tiller` on your cluster:
```
kubectl -n kube-system rollout status deploy/tiller-deploy
Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available...
deployment "tiller-deploy" successfully rolled out
```
And run the following command to validate Helm can talk to the `tiller` service:
```
helm version
Client: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"}
```
### Issues or errors?
See the [Troubleshooting](troubleshooting.md) page.
### [Next: Install Rancher](../helm-rancher/helm-rancher.md)
@@ -1,22 +0,0 @@
---
title: Troubleshooting
---
### Helm commands show forbidden
When Helm is initiated in the cluster without specifying the correct `ServiceAccount`, the command `helm init` will succeed but you won't be able to execute most of the other `helm` commands. The following error will be shown:
```
Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"
```
To resolve this, the server component (`tiller`) needs to be removed and added with the correct `ServiceAccount`. You can use `helm reset --force` to remove the `tiller` from the cluster. Please check if it is removed using `helm version --server`.
```
helm reset --force
Tiller (the Helm server-side component) has been uninstalled from your Kubernetes Cluster.
helm version --server
Error: could not find tiller
```
When you have confirmed that `tiller` has been removed, please follow the steps provided in [Initialize Helm (Install tiller)](helm-init.md) to install `tiller` with the correct `ServiceAccount`.
@@ -1,261 +0,0 @@
---
title: Chart Options
---
### Common Options
| Option | Default Value | Description |
| --- | --- | --- |
| `hostname` | " " | `string` - the Fully Qualified Domain Name for your Rancher Server |
| `ingress.tls.source` | "rancher" | `string` - Where to get the cert for the ingress. - "rancher, letsEncrypt, secret" |
| `letsEncrypt.email` | " " | `string` - Your email address |
| `letsEncrypt.environment` | "production" | `string` - Valid options: "staging, production" |
| `privateCA` | false | `bool` - Set to true if your cert is signed by a private CA |
<br/>
### Advanced Options
| Option | Default Value | Description |
| --- | --- | --- |
| `additionalTrustedCAs` | false | `bool` - See [Additional Trusted CAs](#additional-trusted-cas) |
| `addLocal` | "auto" | `string` - Have Rancher detect and import the local Rancher server cluster |
| `antiAffinity` | "preferred" | `string` - AntiAffinity rule for Rancher pods - "preferred, required" |
| `auditLog.destination` | "sidecar" | `string` - Stream to sidecar container console or hostPath volume - "sidecar, hostPath" |
| `auditLog.hostPath` | "/var/log/rancher/audit" | `string` - log file destination on host (only applies when `auditLog.destination` is set to `hostPath`) |
| `auditLog.level` | 0 | `int` - set the [API Audit Log](../../../advanced-use-cases/enable-api-audit-log.md) level. 0 is off. [0-3] |
| `auditLog.maxAge` | 1 | `int` - maximum number of days to retain old audit log files (only applies when `auditLog.destination` is set to `hostPath`) |
| `auditLog.maxBackups` | 1 | `int` - maximum number of audit log files to retain (only applies when `auditLog.destination` is set to `hostPath`) |
| `auditLog.maxSize` | 100 | `int` - maximum size in megabytes of the audit log file before it gets rotated (only applies when `auditLog.destination` is set to `hostPath`) |
| `busyboxImage` | "busybox" | `string` - Image location for busybox image used to collect audit logs _Note: Available as of v2.2.0_ |
| `debug` | false | `bool` - set debug flag on rancher server |
| `extraEnv` | [] | `list` - set additional environment variables for Rancher _Note: Available as of v2.2.0_ |
| `imagePullSecrets` | [] | `list` - list of names of Secret resource containing private registry credentials |
| `ingress.extraAnnotations` | {} | `map` - additional annotations to customize the ingress |
| `ingress.configurationSnippet` | "" | `string` - additional Nginx configuration. Can be used for proxy configuration. _Note: Available as of v2.0.15, v2.1.10 and v2.2.4_ |
| `proxy` | "" | `string` - HTTP[S] proxy server for Rancher |
| `noProxy` | "127.0.0.0/8,10.0.0.0/8,cattle-system.svc,172.16.0.0/12,192.168.0.0/16" | `string` - comma separated list of hostnames or ip address not to use the proxy |
| `resources` | {} | `map` - rancher pod resource requests & limits |
| `rancherImage` | "rancher/rancher" | `string` - rancher image source |
| `rancherImageTag` | same as chart version | `string` - rancher/rancher image tag |
| `tls` | "ingress" | `string` - See [External TLS Termination](#external-tls-termination) for details. - "ingress, external" |
| `systemDefaultRegistry` | "" | `string` - private registry to be used for all system Docker images, e.g., http://registry.example.com/ _Available as of v2.3.0_ |
| `useBundledSystemChart` | `false` | `bool` - select to use the system-charts packaged with Rancher server. This option is used for air gapped installations. _Available as of v2.3.0_
<br/>
### API Audit Log
Enabling the [API Audit Log](../../enable-api-audit-log.md).
You can collect this log as you would any container log. Enable the [Logging service under Rancher Tools](../../../../../../explanations/integrations-in-rancher/cluster-logging/cluster-logging.md) for the `System` Project on the Rancher server cluster.
```plain
--set auditLog.level=1
```
By default enabling Audit Logging will create a sidecar container in the Rancher pod. This container (`rancher-audit-log`) will stream the log to `stdout`. You can collect this log as you would any container log. When using the sidecar as the audit log destination, the `hostPath`, `maxAge`, `maxBackups`, and `maxSize` options do not apply. It's advised to use your OS or Docker daemon's log rotation features to control disk space use. Enable the [Logging service under Rancher Tools](../../../../../../explanations/integrations-in-rancher/cluster-logging/cluster-logging.md/) for the Rancher server cluster or System Project.
Set the `auditLog.destination` to `hostPath` to forward logs to volume shared with the host system instead of streaming to a sidecar container. When setting the destination to `hostPath` you may want to adjust the other auditLog parameters for log rotation.
### Setting Extra Environment Variables
_Available as of v2.2.0_
You can set extra environment variables for Rancher server using `extraEnv`. This list uses the same `name` and `value` keys as the container manifest definitions. Remember to quote the values.
```plain
--set 'extraEnv[0].name=CATTLE_TLS_MIN_VERSION'
--set 'extraEnv[0].value=1.0'
```
### TLS settings
_Available as of v2.2.0_
To set a different TLS configuration, you can use the `CATTLE_TLS_MIN_VERSION` and `CATTLE_TLS_CIPHERS` environment variables. For example, to configure TLS 1.0 as minimum accepted TLS version:
```plain
--set 'extraEnv[0].name=CATTLE_TLS_MIN_VERSION'
--set 'extraEnv[0].value=1.0'
```
See [TLS settings](../../../../../../reference-guides/installation-references/tls-settings.md) for more information and options.
### Import `local` Cluster
By default Rancher server will detect and import the `local` cluster it's running on. User with access to the `local` cluster will essentially have "root" access to all the clusters managed by Rancher server.
If this is a concern in your environment you can set this option to "false" on your initial install.
> Note: This option is only effective on the initial Rancher install. See [Issue 16522](https://github.com/rancher/rancher/issues/16522) for more information.
```plain
--set addLocal="false"
```
### Customizing your Ingress
To customize or use a different ingress with Rancher server you can set your own Ingress annotations.
Example on setting a custom certificate issuer:
```plain
--set ingress.extraAnnotations.'certmanager\.k8s\.io/cluster-issuer'=ca-key-pair
```
_Available as of v2.0.15, v2.1.10 and v2.2.4_
Example on setting a static proxy header with `ingress.configurationSnippet`. This value is parsed like a template so variables can be used.
```plain
--set ingress.configurationSnippet='more_set_input_headers X-Forwarded-Host {{ .Values.hostname }};'
```
### HTTP Proxy
Rancher requires internet access for some functionality, such as Helm charts. Use `proxy` to set your proxy server.
Add your IP exceptions to the `noProxy` list. Make sure you add the Service cluster IP range (default: 10.43.0.1/16) and any worker cluster `controlplane` nodes. Rancher supports CIDR notation ranges in this list.
```plain
--set proxy="http://<username>:<password>@<proxy_url>:<proxy_port>/"
--set noProxy="127.0.0.0/8\,10.0.0.0/8\,172.16.0.0/12\,192.168.0.0/16"
```
### Additional Trusted CAs
If you have private registries, catalogs or a proxy that intercepts certificates, you may need to add more trusted CAs to Rancher.
```plain
--set additionalTrustedCAs=true
```
Once the Rancher deployment is created, copy your CA certs in pem format into a file named `ca-additional.pem` and use `kubectl` to create the `tls-ca-additional` secret in the `cattle-system` namespace.
```plain
kubectl -n cattle-system create secret generic tls-ca-additional --from-file=ca-additional.pem
```
### Private Registry and Air Gap Installs
For details on installing Rancher with a private registry, see:
- [Air Gap: Docker Install](../../../../../../reference-guides/installation-references/tls-settings.md)
- [Air Gap: Kubernetes Install](../../air-gap-helm2/air-gap-helm2.md)
### External TLS Termination
We recommend configuring your load balancer as a Layer 4 balancer, forwarding plain 80/tcp and 443/tcp to the Rancher Management cluster nodes. The Ingress Controller on the cluster will redirect http traffic on port 80 to https on port 443.
You may terminate the SSL/TLS on a L7 load balancer external to the Rancher cluster (ingress). Use the `--set tls=external` option and point your load balancer at port http 80 on all of the Rancher cluster nodes. This will expose the Rancher interface on http port 80. Be aware that clients that are allowed to connect directly to the Rancher cluster will not be encrypted. If you choose to do this we recommend that you restrict direct access at the network level to just your load balancer.
> **Note:** If you are using a Private CA signed certificate, add `--set privateCA=true` and see [Adding TLS Secrets - Using a Private CA Signed Certificate](./tls-secrets.md) to add the CA cert for Rancher.
Your load balancer must support long lived websocket connections and will need to insert proxy headers so Rancher can route links correctly.
### Configuring Ingress for External TLS when Using NGINX v0.22
In NGINX v0.22, the behavior of NGINX has [changed](https://github.com/kubernetes/ingress-nginx/blob/06efac9f0b6f8f84b553f58ccecf79dc42c75cc6/Changelog.md) regarding forwarding headers and external TLS termination. Therefore, in the scenario that you are using external TLS termination configuration with NGINX v0.22, you must enable the `use-forwarded-headers` option for ingress:
For RKE installations, edit the `cluster.yml` to add the following settings.
```yaml
ingress:
provider: nginx
options:
use-forwarded-headers: 'true'
```
For RKE2 installations, you can create a custom `rke2-ingress-nginx-config.yaml` file at `/var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx-config.yaml` containing this required setting to enable using forwarded headers with external TLS termination. Without this required setting applied, the external LB will continuously respond with redirect loops it receives from the ingress controller. (This can be created before or after rancher is installed, rke2 server agent will notice this addition and automatically apply it.)
```yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: rke2-ingress-nginx
namespace: kube-system
spec:
valuesContent: |-
controller:
config:
use-forwarded-headers: "true"
```
#### Required Headers
* `Host`
* `X-Forwarded-Proto`
* `X-Forwarded-Port`
* `X-Forwarded-For`
#### Recommended Timeouts
* Read Timeout: `1800 seconds`
* Write Timeout: `1800 seconds`
* Connect Timeout: `30 seconds`
#### Health Checks
Rancher will respond `200` to health checks on the `/healthz` endpoint.
#### Example NGINX config
This NGINX configuration is tested on NGINX 1.14.
>**Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/).
* Replace `IP_NODE1`, `IP_NODE2` and `IP_NODE3` with the IP addresses of the nodes in your cluster.
* Replace both occurrences of `FQDN` to the DNS name for Rancher.
* Replace `/certs/fullchain.pem` and `/certs/privkey.pem` to the location of the server certificate and the server certificate key respectively.
```
worker_processes 4;
worker_rlimit_nofile 40000;
events {
worker_connections 8192;
}
http {
upstream rancher {
server IP_NODE_1:80;
server IP_NODE_2:80;
server IP_NODE_3:80;
}
map $http_upgrade $connection_upgrade {
default Upgrade;
'' close;
}
server {
listen 443 ssl http2;
server_name FQDN;
ssl_certificate /certs/fullchain.pem;
ssl_certificate_key /certs/privkey.pem;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://rancher;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close.
proxy_read_timeout 900s;
proxy_buffering off;
}
}
server {
listen 80;
server_name FQDN;
return 301 https://$server_name$request_uri;
}
}
```
@@ -1,231 +0,0 @@
---
title: "4. Install Rancher"
---
Rancher installation is managed using the Helm package manager for Kubernetes. Use `helm` to install the prerequisite and charts to install Rancher.
For systems without direct internet access, see [Air Gap: Kubernetes install](../../air-gap-helm2/install-rancher.md).
Refer to the [Helm version requirements](../../../../resources/helm-version-requirements.md) to choose a version of Helm to install Rancher.
> **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3)
### Add the Helm Chart Repository
Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher](../../../../resources/choose-a-rancher-version.md).
- Latest: Recommended for trying out the newest features
```
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
```
- Stable: Recommended for production environments
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
- Alpha: Experimental preview of upcoming releases.
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
Note: Upgrades are not supported to, from, or between Alphas.
### Choose your SSL Configuration
Rancher Server is designed to be secure by default and requires SSL/TLS configuration.
There are three recommended options for the source of the certificate.
> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer](chart-options.md#external-tls-termination).
| Configuration | Chart option | Description | Requires cert-manager |
|-----|-----|-----|-----|
| [Rancher Generated Certificates](#rancher-generated-certificates) | `ingress.tls.source=rancher` | Use certificates issued by Rancher's generated CA (self signed)<br/>This is the **default** | [yes](#optional-install-cert-manager) |
| [Let’s Encrypt](#let-s-encrypt) | `ingress.tls.source=letsEncrypt` | Use [Let's Encrypt](https://letsencrypt.org/) to issue a certificate | [yes](#optional-install-cert-manager) |
| [Certificates from Files](#certificates-from-files) | `ingress.tls.source=secret` | Use your own certificate files by creating Kubernetes Secret(s) | no |
### Optional: Install cert-manager
**Note:** cert-manager is only required for certificates issued by Rancher's generated CA (`ingress.tls.source=rancher`) and Let's Encrypt issued certificates (`ingress.tls.source=letsEncrypt`). You should skip this step if you are using your own certificate files (option `ingress.tls.source=secret`) or if you use [TLS termination on an External Load Balancer](chart-options.md#external-tls-termination).
> **Important:**
> Due to an issue with Helm v2.12.0 and cert-manager, please use Helm v2.12.1 or higher.
> Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.12.0, please see our [upgrade documentation](../../../../resources/upgrade-cert-manager.md).
Rancher relies on [cert-manager](https://github.com/jetstack/cert-manager) to issue certificates from Rancher's own generated CA or to request Let's Encrypt certificates.
These instructions are adapted from the [official cert-manager documentation](https://docs.cert-manager.io/en/latest/getting-started/install/kubernetes.html#installing-with-helm).
1. Install the CustomResourceDefinition resources separately
```plain
kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.9/deploy/manifests/00-crds.yaml
```
1. Create the namespace for cert-manager
```plain
kubectl create namespace cert-manager
```
1. Label the cert-manager namespace to disable resource validation
```plain
kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true
```
1. Add the Jetstack Helm repository
```plain
helm repo add jetstack https://charts.jetstack.io
```
1. Update your local Helm chart repository cache
```plain
helm repo update
```
1. Install the cert-manager Helm chart
```plain
helm install \
--name cert-manager \
--namespace cert-manager \
--version v0.14.2 \
jetstack/cert-manager
```
Once you’ve installed cert-manager, you can verify it is deployed correctly by checking the cert-manager namespace for running pods:
```
kubectl get pods --namespace cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-7cbdc48784-rpgnt 1/1 Running 0 3m
cert-manager-webhook-5b5dd6999-kst4x 1/1 Running 0 3m
cert-manager-cainjector-3ba5cd2bcd-de332x 1/1 Running 0 3m
```
If the ‘webhook’ pod (2nd line) is in a ContainerCreating state, it may still be waiting for the Secret to be mounted into the pod. Wait a couple of minutes for this to happen but if you experience problems, please check the [troubleshooting](https://docs.cert-manager.io/en/latest/getting-started/troubleshooting.html) guide.
<br/>
#### Rancher Generated Certificates
> **Note:** You need to have [cert-manager](#optional-install-cert-manager) installed before proceeding.
The default is for Rancher to generate a CA and uses `cert-manager` to issue the certificate for access to the Rancher server interface. Because `rancher` is the default option for `ingress.tls.source`, we are not specifying `ingress.tls.source` when running the `helm install` command.
- Set the `hostname` to the DNS name you pointed at your load balancer.
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
```
helm install rancher-<CHART_REPO>/rancher \
--name rancher \
--namespace cattle-system \
--set hostname=rancher.my.org
```
Wait for Rancher to be rolled out:
```
kubectl -n cattle-system rollout status deploy/rancher
Waiting for deployment "rancher" rollout to finish: 0 of 3 updated replicas are available...
deployment "rancher" successfully rolled out
```
#### Let's Encrypt
> **Note:** You need to have [cert-manager](#optional-install-cert-manager) installed before proceeding.
This option uses `cert-manager` to automatically request and renew [Let's Encrypt](https://letsencrypt.org/) certificates. This is a free service that provides you with a valid certificate as Let's Encrypt is a trusted CA. This configuration uses HTTP validation (`HTTP-01`) so the load balancer must have a public DNS record and be accessible from the internet.
In the following command,
- Set `hostname` to the public DNS record that resolves to your load balancer.
- Set `ingress.tls.source` to `letsEncrypt`.
- Set `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices).
- Set `letsEncrypt.ingress.class` to whatever your ingress controller is, e.g., `traefik`, `nginx`, `haproxy`, etc.
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
```
helm install rancher-<CHART_REPO>/rancher \
--name rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set ingress.tls.source=letsEncrypt \
--set letsEncrypt.email=me@example.org \
--set letsEncrypt.ingress.class=nginx
```
Wait for Rancher to be rolled out:
```
kubectl -n cattle-system rollout status deploy/rancher
Waiting for deployment "rancher" rollout to finish: 0 of 3 updated replicas are available...
deployment "rancher" successfully rolled out
```
#### Certificates from Files
Create Kubernetes secrets from your own certificates for Rancher to use.
> **Note:** The `Common Name` or a `Subject Alternative Names` entry in the server certificate must match the `hostname` option, or the ingress controller will fail to configure correctly. Although an entry in the `Subject Alternative Names` is technically required, having a matching `Common Name` maximizes compatibility with older browsers/applications. If you want to check if your certificates are correct, see [How do I check Common Name and Subject Alternative Names in my server certificate?](../../../../../../faq/technical-items.md#how-do-i-check-common-name-and-subject-alternative-names-in-my-server-certificate)
- Set `hostname` and set `ingress.tls.source` to `secret`.
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
```
helm install rancher-<CHART_REPO>/rancher \
--name rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set ingress.tls.source=secret
```
If you are using a Private CA signed certificate , add `--set privateCA=true` to the command:
```
helm install rancher-<CHART_REPO>/rancher \
--name rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set ingress.tls.source=secret
--set privateCA=true
```
Now that Rancher is deployed, see [Adding TLS Secrets](tls-secrets.md) to publish the certificate files so Rancher and the ingress controller can use them.
After adding the secrets, check if Rancher was rolled out successfully:
```
kubectl -n cattle-system rollout status deploy/rancher
Waiting for deployment "rancher" rollout to finish: 0 of 3 updated replicas are available...
deployment "rancher" successfully rolled out
```
If you see the following error: `error: deployment "rancher" exceeded its progress deadline`, you can check the status of the deployment by running the following command:
```
kubectl -n cattle-system get deploy rancher
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
rancher 3 3 3 3 3m
```
It should show the same count for `DESIRED` and `AVAILABLE`.
### Advanced Configurations
The Rancher chart configuration has many options for customizing the install to suit your specific environment. Here are some common advanced scenarios.
* [HTTP Proxy](../../../../../../reference-guides/installation-references/helm-chart-options.md)
* [Private Docker Image Registry](../../../../../../reference-guides/installation-references/helm-chart-options.md#private-registry-and-air-gap-installs)
* [TLS Termination on an External Load Balancer](chart-options.md#external-tls-termination)
See the [Chart Options](chart-options.md) for the full list of options.
### Save your options
Make sure you save the `--set` options you used. You will need to use the same options when you upgrade Rancher to new versions with Helm.
### Finishing Up
That's it you should have a functional Rancher server. Point a browser at the hostname you picked and you should be greeted by the colorful login page.
Doesn't work? Take a look at the [Troubleshooting](troubleshooting.md) Page
@@ -1,31 +0,0 @@
---
title: Adding Kubernetes TLS Secrets
description: Read about how to populate the Kubernetes TLS secret for a Rancher installation
---
Kubernetes will create all the objects and services for Rancher, but it will not become available until we populate the `tls-rancher-ingress` secret in the `cattle-system` namespace with the certificate and key.
Combine the server certificate followed by any intermediate certificate(s) needed into a file named `tls.crt`. Copy your certificate key into a file named `tls.key`.
Use `kubectl` with the `tls` secret type to create the secrets.
```
kubectl -n cattle-system create secret tls tls-rancher-ingress \
--cert=tls.crt \
--key=tls.key
```
> **Note:** If you want to replace the certificate, you can delete the `tls-rancher-ingress` secret using `kubectl -n cattle-system delete secret tls-rancher-ingress` and add a new one using the command shown above. If you are using a private CA signed certificate, replacing the certificate is only possible if the new certificate is signed by the same CA as the certificate currently in use.
### Using a Private CA Signed Certificate
If you are using a private CA, Rancher requires a copy of the private CA's root certificate or certificate chain, which the Rancher Agent uses to validate the connection to the server.
Create a file named `cacerts.pem` that only contains the root CA certificate or certificate chain from your private CA, and use `kubectl` to create the `tls-ca` secret in the `cattle-system` namespace.
>**Important:** Make sure the file is called `cacerts.pem` as Rancher uses that filename to configure the CA certificate.
```
kubectl -n cattle-system create secret generic tls-ca \
--from-file=cacerts.pem
```
@@ -1,132 +0,0 @@
---
title: Troubleshooting
---
### Where is everything
Most of the troubleshooting will be done on objects in these 3 namespaces.
* `cattle-system` - `rancher` deployment and pods.
* `ingress-nginx` - Ingress controller pods and services.
* `kube-system` - `tiller` and `cert-manager` pods.
### "default backend - 404"
A number of things can cause the ingress-controller not to forward traffic to your rancher instance. Most of the time its due to a bad ssl configuration.
Things to check
* [Is Rancher Running](#is-rancher-running)
* [Cert CN is "Kubernetes Ingress Controller Fake Certificate"](#cert-cn-is-kubernetes-ingress-controller-fake-certificate)
### Is Rancher Running
Use `kubectl` to check the `cattle-system` system namespace and see if the Rancher pods are in a Running state.
```
kubectl -n cattle-system get pods
NAME READY STATUS RESTARTS AGE
pod/rancher-784d94f59b-vgqzh 1/1 Running 0 10m
```
If the state is not `Running`, run a `describe` on the pod and check the Events.
```
kubectl -n cattle-system describe pod
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 11m default-scheduler Successfully assigned rancher-784d94f59b-vgqzh to localhost
Normal SuccessfulMountVolume 11m kubelet, localhost MountVolume.SetUp succeeded for volume "rancher-token-dj4mt"
Normal Pulling 11m kubelet, localhost pulling image "rancher/rancher:v2.0.4"
Normal Pulled 11m kubelet, localhost Successfully pulled image "rancher/rancher:v2.0.4"
Normal Created 11m kubelet, localhost Created container
Normal Started 11m kubelet, localhost Started container
```
### Checking the rancher logs
Use `kubectl` to list the pods.
```
kubectl -n cattle-system get pods
NAME READY STATUS RESTARTS AGE
pod/rancher-784d94f59b-vgqzh 1/1 Running 0 10m
```
Use `kubectl` and the pod name to list the logs from the pod.
```
kubectl -n cattle-system logs -f rancher-784d94f59b-vgqzh
```
### Cert CN is "Kubernetes Ingress Controller Fake Certificate"
Use your browser to check the certificate details. If it says the Common Name is "Kubernetes Ingress Controller Fake Certificate", something may have gone wrong with reading or issuing your SSL cert.
> **Note:** if you are using LetsEncrypt to issue certs it can sometimes take a few minuets to issue the cert.
#### cert-manager issued certs (Rancher Generated or LetsEncrypt)
`cert-manager` has 3 parts.
* `cert-manager` pod in the `kube-system` namespace.
* `Issuer` object in the `cattle-system` namespace.
* `Certificate` object in the `cattle-system` namespace.
Work backwards and do a `kubectl describe` on each object and check the events. You can track down what might be missing.
For example there is a problem with the Issuer:
```
kubectl -n cattle-system describe certificate
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning IssuerNotReady 18s (x23 over 19m) cert-manager Issuer rancher not ready
```
```
kubectl -n cattle-system describe issuer
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ErrInitIssuer 19m (x12 over 19m) cert-manager Error initializing issuer: secret "tls-rancher" not found
Warning ErrGetKeyPair 9m (x16 over 19m) cert-manager Error getting keypair for CA issuer: secret "tls-rancher" not found
```
#### Bring Your Own SSL Certs
Your certs get applied directly to the Ingress object in the `cattle-system` namespace.
Check the status of the Ingress object and see if its ready.
```
kubectl -n cattle-system describe ingress
```
If its ready and the SSL is still not working you may have a malformed cert or secret.
Check the nginx-ingress-controller logs. Because the nginx-ingress-controller has multiple containers in its pod you will need to specify the name of the container.
```
kubectl -n ingress-nginx logs -f nginx-ingress-controller-rfjrq nginx-ingress-controller
...
W0705 23:04:58.240571 7 backend_ssl.go:49] error obtaining PEM from secret cattle-system/tls-rancher-ingress: error retrieving secret cattle-system/tls-rancher-ingress: secret cattle-system/tls-rancher-ingress was not found
```
### no matches for kind "Issuer"
The SSL configuration option you have chosen requires cert-manager to be installed before installing Rancher or else the following error is shown:
```
Error: validation failed: unable to recognize "": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1"
```
Install cert-manager and try installing Rancher again.
@@ -1,58 +0,0 @@
---
title: Kubernetes Installation Using Helm 2
---
> After Helm 3 was released, the Rancher installation instructions were updated to use Helm 3.
>
> If you are using Helm 2, we recommend [migrating to Helm 3](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) because it is simpler to use and more secure than Helm 2.
>
> This section provides a copy of the older high-availability Kubernetes Rancher installation instructions that used Helm 2, and it is intended to be used if upgrading to Helm 3 is not feasible.
For production environments, we recommend installing Rancher in a high-availability configuration so that your user base can always access Rancher Server. When installed in a Kubernetes cluster, Rancher will integrate with the cluster's etcd database and take advantage of Kubernetes scheduling for high-availability.
This procedure walks you through setting up a 3-node cluster with Rancher Kubernetes Engine (RKE) and installing the Rancher chart with the Helm package manager.
> **Important:** The Rancher management server can only be run on an RKE-managed Kubernetes cluster. Use of Rancher on hosted Kubernetes or other providers is not supported.
> **Important:** For the best performance, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters](../../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md) for running your workloads.
## Recommended Architecture
- DNS for Rancher should resolve to a Layer 4 load balancer (TCP)
- The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster.
- The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443.
- The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment.
<figcaption>Kubernetes Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers</figcaption>
![High-availability Kubernetes Install](/img/ha/rancher2ha.svg)
<sup>Kubernetes Rancher install with Layer 4 load balancer (TCP), depicting SSL termination at ingress controllers</sup>
## Required Tools
The following CLI tools are required for this install. Please make sure these tools are installed and available in your `$PATH`
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool.
- [rke](https://rancher.com/docs/rke/latest/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters.
- [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. Refer to the [Helm version requirements](../../../resources/helm-version-requirements.md) to choose a version of Helm to install Rancher.
## Installation Outline
- [Create Nodes and Load Balancer](create-nodes-lb/create-nodes-lb.md)
- [Install Kubernetes with RKE](kubernetes-rke/kubernetes-rke.md)
- [Initialize Helm (tiller)](helm-init/helm-init.md)
- [Install Rancher](helm-rancher/helm-rancher.md)
## Additional Install Options
- [Migrating from a Kubernetes Install with an RKE Add-on](../../../install-upgrade-on-a-kubernetes-cluster/upgrades/migrating-from-rke-add-on.md)
## Previous Methods
[RKE add-on install](rke-add-on/rke-add-on.md)
> **Important: RKE add-on install is only supported up to Rancher v2.0.8**
>
> Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install ](../../../resources/helm-version-requirements.md).
>
> If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on](../../../install-upgrade-on-a-kubernetes-cluster/upgrades/migrating-from-rke-add-on.md) for details on how to move to using the Helm chart.
@@ -1,131 +0,0 @@
---
title: "2. Install Kubernetes with RKE"
---
Use RKE to install Kubernetes with a high availability etcd configuration.
>**Note:** For systems without direct internet access see [Air Gap: Kubernetes install](../../air-gap-helm2/air-gap-helm2.md) for install details.
### Create the `rancher-cluster.yml` File
Using the sample below create the `rancher-cluster.yml` file. Replace the IP Addresses in the `nodes` list with the IP address or DNS names of the 3 nodes you created.
> **Note:** If your node has public and internal addresses, it is recommended to set the `internal_address:` so Kubernetes will use it for intra-cluster communication. Some services like AWS EC2 require setting the `internal_address:` if you want to use self-referencing security groups or firewalls.
```yaml
nodes:
- address: 165.227.114.63
internal_address: 172.16.22.12
user: ubuntu
role: [controlplane,worker,etcd]
- address: 165.227.116.167
internal_address: 172.16.32.37
user: ubuntu
role: [controlplane,worker,etcd]
- address: 165.227.127.226
internal_address: 172.16.42.73
user: ubuntu
role: [controlplane,worker,etcd]
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
```
#### Common RKE Nodes Options
| Option | Required | Description |
| --- | --- | --- |
| `address` | yes | The public DNS or IP address |
| `user` | yes | A user that can run docker commands |
| `role` | yes | List of Kubernetes roles assigned to the node |
| `internal_address` | no | The private DNS or IP address for internal cluster traffic |
| `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) |
#### Advanced Configurations
RKE has many configuration options for customizing the install to suit your specific environment.
Please see the [RKE Documentation](https://rancher.com/docs/rke/latest/en/config-options/) for the full list of options and capabilities.
For tuning your etcd cluster for larger Rancher installations see the [etcd settings guide](../../tune-etcd-for-large-installs.md).
### Run RKE
```
rke up --config ./rancher-cluster.yml
```
When finished, it should end with the line: `Finished building Kubernetes cluster successfully`.
### Testing Your Cluster
RKE should have created a file `kube_config_rancher-cluster.yml`. This file has the credentials for `kubectl` and `helm`.
> **Note:** If you have used a different file name from `rancher-cluster.yml`, then the kube config file will be named `kube_config_<FILE_NAME>.yml`.
You can copy this file to `$HOME/.kube/config` or if you are working with multiple Kubernetes clusters, set the `KUBECONFIG` environmental variable to the path of `kube_config_rancher-cluster.yml`.
```
export KUBECONFIG=$(pwd)/kube_config_rancher-cluster.yml
```
Test your connectivity with `kubectl` and see if all your nodes are in `Ready` state.
```
kubectl get nodes
NAME STATUS ROLES AGE VERSION
165.227.114.63 Ready controlplane,etcd,worker 11m v1.13.5
165.227.116.167 Ready controlplane,etcd,worker 11m v1.13.5
165.227.127.226 Ready controlplane,etcd,worker 11m v1.13.5
```
### Check the Health of Your Cluster Pods
Check that all the required pods and containers are healthy are ready to continue.
* Pods are in `Running` or `Completed` state.
* `READY` column shows all the containers are running (i.e. `3/3`) for pods with `STATUS` `Running`
* Pods with `STATUS` `Completed` are run-once Jobs. For these pods `READY` should be `0/1`.
```
kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx nginx-ingress-controller-tnsn4 1/1 Running 0 30s
ingress-nginx nginx-ingress-controller-tw2ht 1/1 Running 0 30s
ingress-nginx nginx-ingress-controller-v874b 1/1 Running 0 30s
kube-system canal-jp4hz 3/3 Running 0 30s
kube-system canal-z2hg8 3/3 Running 0 30s
kube-system canal-z6kpw 3/3 Running 0 30s
kube-system kube-dns-7588d5b5f5-sf4vh 3/3 Running 0 30s
kube-system kube-dns-autoscaler-5db9bbb766-jz2k6 1/1 Running 0 30s
kube-system metrics-server-97bc649d5-4rl2q 1/1 Running 0 30s
kube-system rke-ingress-controller-deploy-job-bhzgm 0/1 Completed 0 30s
kube-system rke-kubedns-addon-deploy-job-gl7t4 0/1 Completed 0 30s
kube-system rke-metrics-addon-deploy-job-7ljkc 0/1 Completed 0 30s
kube-system rke-network-plugin-deploy-job-6pbgj 0/1 Completed 0 30s
```
### Save Your Files
> **Important**
> The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster.
Save a copy of the following files in a secure location:
- `rancher-cluster.yml`: The RKE cluster configuration file.
- `kube_config_rancher-cluster.yml`: The [Kubeconfig file](https://rancher.com/docs/rke/latest/en/kubeconfig/) for the cluster, this file contains credentials for full access to the cluster.
- `rancher-cluster.rkestate`: The [Kubernetes Cluster State file](https://rancher.com/docs/rke/latest/en/installation/#kubernetes-cluster-state), this file contains credentials for full access to the cluster.<br/><br/>_The Kubernetes Cluster State file is only created when using RKE v0.2.0 or higher._
> **Note:** The "rancher-cluster" parts of the two latter file names are dependent on how you name the RKE cluster configuration file.
### Issues or errors?
See the [Troubleshooting](troubleshooting.md) page.
### [Next: Initialize Helm (Install tiller)](../helm-init/helm-init.md)

Some files were not shown because too many files have changed in this diff Show More