How To Fix fork failed: Resource temporarily unavailable
Root Cause
- There is a misbehaving service or process running, consuming more resources than expected.
- The system was not able to create new processes, because of the limits set for nproc in /etc/security/limits.conf.
- The system ran out of memory and new processes were unable to start because they could not allocate memory.
- There is not an available ID to assign to the new process. A unique value less than kernel.pid_max must be available.
Troubleshooting
You can run the below command to find the number of processes opened for every user:
ps --no-headers auxwwwm | cut -f1 -d' ' | sort | uniq -c | sort -n
Compare to /etc/security/limits.conf
or /etc/security/limits.d/*
Check the total number of threads and processes running on the server:
ps -eLf | wc -l
Compare to kernel.pid_max:
sysctl kernel.pid_max
Fixing
nproc
Refer to Amending Maximum Number of Processes in RHEL
Set /etc/security/limits.conf
* soft nproc 65535
* hard nproc 65535
*
should be username
Memory
sar -r 1 10
Check commit%
kernel.pid_max
echo 65535 > /proc/sys/kernel/pid_max
Check for File Descriptors
System Limit
sysctl fs.file-nr
Explain:
<in_use> <unused_but_allocated> <maximum>
User Limit
su - <username>
ulimit -Hn
User Used
lsof -u <username> 2>/dev/null | wc -l
Rewrite for system level
/etc/sysctl.conf
:
fs.file-max = 204708