Adding troubleshooting info to logging page - inotify system update

Signed-off-by: Sunil Singh <sunil.singh@suse.com>
2026-09-26 04:58:07 +00:00 · 2026-05-22 10:47:37 -07:00
parent c5bc0a85a8
commit 97c7ee878e
12 changed files with 156 additions and 0 deletions
@@ -79,6 +79,19 @@ Rancher Logging 有两个角色，分别是 `logging-admin` 和 `logging-view`

 ## 故障排除

+### Resource Exhaustion of `inotify` Watchers and File Descriptors
+
+When enabling the **Logging** app on Linux systems that heavily monitor the filesystem, you may encounter `Too many open files` or `CrashLoopBackOff` failures related to applications that leverage `inotify` to watch for file changes.
+
+This happens because the Linux kernel caps the number of files a user can open and the number of directory paths a subsystem can watch simultaneously. To resolve this, you must explicitly increase your `inotify` system limits.
+
+Below are example commands an admin user can run to increase system limits for `inotify` user instances and watches:
+
+```shell
+sysctl -w fs.inotify.max_user_instances=8192
+sysctl -w fs.inotify.max_user_watches=524288
+```
+
 ### 日志缓冲区导致 Pod 过载

 根据你的配置，默认缓冲区大小可能太大并导致 Pod 故障。减少负载的一种方法是降低记录器的刷新间隔。这可以防止日志溢出缓冲区。你还可以添加更多刷新线程来处理大量日志试图同时填充缓冲区的情况。
@@ -79,6 +79,19 @@ Rancher Logging 有两个角色，分别是 `logging-admin` 和 `logging-view`

 ## 故障排除

+### Resource Exhaustion of `inotify` Watchers and File Descriptors
+
+When enabling the **Logging** app on Linux systems that heavily monitor the filesystem, you may encounter `Too many open files` or `CrashLoopBackOff` failures related to applications that leverage `inotify` to watch for file changes.
+
+This happens because the Linux kernel caps the number of files a user can open and the number of directory paths a subsystem can watch simultaneously. To resolve this, you must explicitly increase your `inotify` system limits.
+
+Below are example commands an admin user can run to increase system limits for `inotify` user instances and watches:
+
+```shell
+sysctl -w fs.inotify.max_user_instances=8192
+sysctl -w fs.inotify.max_user_watches=524288
+```
+
 ### 日志缓冲区导致 Pod 过载

 根据你的配置，默认缓冲区大小可能太大并导致 Pod 故障。减少负载的一种方法是降低记录器的刷新间隔。这可以防止日志溢出缓冲区。你还可以添加更多刷新线程来处理大量日志试图同时填充缓冲区的情况。
@@ -79,6 +79,19 @@ Rancher Logging 有两个角色，分别是 `logging-admin` 和 `logging-view`

 ## 故障排除

+### Resource Exhaustion of `inotify` Watchers and File Descriptors
+
+When enabling the **Logging** app on Linux systems that heavily monitor the filesystem, you may encounter `Too many open files` or `CrashLoopBackOff` failures related to applications that leverage `inotify` to watch for file changes.
+
+This happens because the Linux kernel caps the number of files a user can open and the number of directory paths a subsystem can watch simultaneously. To resolve this, you must explicitly increase your `inotify` system limits.
+
+Below are example commands an admin user can run to increase system limits for `inotify` user instances and watches:
+
+```shell
+sysctl -w fs.inotify.max_user_instances=8192
+sysctl -w fs.inotify.max_user_watches=524288
+```
+
 ### 日志缓冲区导致 Pod 过载

 根据你的配置，默认缓冲区大小可能太大并导致 Pod 故障。减少负载的一种方法是降低记录器的刷新间隔。这可以防止日志溢出缓冲区。你还可以添加更多刷新线程来处理大量日志试图同时填充缓冲区的情况。
@@ -79,6 +79,19 @@ Rancher Logging 有两个角色，分别是 `logging-admin` 和 `logging-view`

 ## 故障排除

+### Resource Exhaustion of `inotify` Watchers and File Descriptors
+
+When enabling the **Logging** app on Linux systems that heavily monitor the filesystem, you may encounter `Too many open files` or `CrashLoopBackOff` failures related to applications that leverage `inotify` to watch for file changes.
+
+This happens because the Linux kernel caps the number of files a user can open and the number of directory paths a subsystem can watch simultaneously. To resolve this, you must explicitly increase your `inotify` system limits.
+
+Below are example commands an admin user can run to increase system limits for `inotify` user instances and watches:
+
+```shell
+sysctl -w fs.inotify.max_user_instances=8192
+sysctl -w fs.inotify.max_user_watches=524288
+```
+
 ### 日志缓冲区导致 Pod 过载

 根据你的配置，默认缓冲区大小可能太大并导致 Pod 故障。减少负载的一种方法是降低记录器的刷新间隔。这可以防止日志溢出缓冲区。你还可以添加更多刷新线程来处理大量日志试图同时填充缓冲区的情况。
@@ -79,6 +79,19 @@ Rancher Logging 有两个角色，分别是 `logging-admin` 和 `logging-view`

 ## 故障排除

+### Resource Exhaustion of `inotify` Watchers and File Descriptors
+
+When enabling the **Logging** app on Linux systems that heavily monitor the filesystem, you may encounter `Too many open files` or `CrashLoopBackOff` failures related to applications that leverage `inotify` to watch for file changes.
+
+This happens because the Linux kernel caps the number of files a user can open and the number of directory paths a subsystem can watch simultaneously. To resolve this, you must explicitly increase your `inotify` system limits.
+
+Below are example commands an admin user can run to increase system limits for `inotify` user instances and watches:
+
+```shell
+sysctl -w fs.inotify.max_user_instances=8192
+sysctl -w fs.inotify.max_user_watches=524288
+```
+
 ### 日志缓冲区导致 Pod 过载

 根据你的配置，默认缓冲区大小可能太大并导致 Pod 故障。减少负载的一种方法是降低记录器的刷新间隔。这可以防止日志溢出缓冲区。你还可以添加更多刷新线程来处理大量日志试图同时填充缓冲区的情况。
@@ -79,6 +79,19 @@ Rancher Logging 有两个角色，分别是 `logging-admin` 和 `logging-view`

 ## 故障排除

+### Resource Exhaustion of `inotify` Watchers and File Descriptors
+
+When enabling the **Logging** app on Linux systems that heavily monitor the filesystem, you may encounter `Too many open files` or `CrashLoopBackOff` failures related to applications that leverage `inotify` to watch for file changes.
+
+This happens because the Linux kernel caps the number of files a user can open and the number of directory paths a subsystem can watch simultaneously. To resolve this, you must explicitly increase your `inotify` system limits.
+
+Below are example commands an admin user can run to increase system limits for `inotify` user instances and watches:
+
+```shell
+sysctl -w fs.inotify.max_user_instances=8192
+sysctl -w fs.inotify.max_user_watches=524288
+```
+
 ### 日志缓冲区导致 Pod 过载

 根据你的配置，默认缓冲区大小可能太大并导致 Pod 故障。减少负载的一种方法是降低记录器的刷新间隔。这可以防止日志溢出缓冲区。你还可以添加更多刷新线程来处理大量日志试图同时填充缓冲区的情况。