container中ulimit与宿主机关系
question
As I know if we need adjust “open files” nofile
(soft and hard) in linux system, we need run command ulimit
or set in related configuraiton file to get the setting permanently. But I am little bit confused about the setting for containers running in a host
For example, If a Linux OS has ulimit
nofile set to 1024 (soft) and Hard (4096) , and I run docker with --ulimit nofile=10240:40960
, could the container use more nofiles than its host?
In my environment, current setting with dockers running,
- On host (Debian)- 65535 (soft) 65535 (hard)
- Docker Daemon setting Max - 1048576 (soft) 1048576 (hard)
- default docker run - 1024 (soft) 4096 (hard)
- customized docker run - 10240 (soft) 40960 (hard)
I found the application can run with about 100K open files, then crash. How to understand this?
What’s the real limits?
Answer
For example, If a Linux OS has ulimit nofile set to 1024 (soft) and Hard (4096) , and I run docker with —-ulimit nofile=10240:40960, could the container use more nofiles than its host?
- Docker has the
CAP_SYS_RESOURCE
capability set on it’s permissions. This means that Docker is able to set anulimit
different from the host. according toman 2 prlimit
:
A privileged process (under Linux: one with the CAP_SYS_RESOURCE capability in the initial user namespace) may make arbitrary changes to either limit value.
- So, for containers, the limits to be considered are the ones set by the docker daemon. You can check the docker daemon limits with this command:
|
|
- As you can see, the docker 19 has a pretty high limit of
1048576
so your 40960 will work like a charm. - And if you run a docker container with
-ulimit
set to be higher than the node but lower than the daemon itself, you won’t find any problem, and won’t need to give additional permissions like in the example bellow:
|
|
- You can set a new limit for dockerd on the file
/etc/init.d/docker
:
|
|
- As for the container itself having a
ulimit
higher than the docker daemon, it’s a bit more tricky, but doable, refer here. - I saw you have tagged the Kubernetes tag, but didn’t mention it in your question, but in order to make it work on Kubernetes, the container will need
securityContext.priviledged: true
, this way you can run the commandulimit
as root inside the container, here an example:
|
|
参考文档
Need understand “ulimit”’s nofile setting in host and container