[TechNote] java.lang.OutOfMemoryError while creating new threads

java.lang.OutOfMemoryError while creating new threads

Question

I was trying to run a script/starting the product services, but the environment was unable to create java threads for it due to OutOfMemoryError exception. Why am I getting this and how can I correct it?

Cause

The "java.lang.OutOfMemoryError: Failed to create a thread" message occurs when the system does not have enough resources to create a new thread. There are three possible causes for this message:

- Inadequate user/application resources.
- The system has run out of native memory to use for the new thread. Threads require a native memory for internal JVM structures, a Java stack, and a native stack.
- There are too many threads already running and the system has run out of internal resources to create new threads.

Sample error stack:
java.lang.OutOfMemoryError: Failed to create a thread: retVal -1073741830, errno 11
at java.lang.Thread.startImpl(Native Method)
at java.lang.Thread.start(Thread.java:891)
at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:738)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:668)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:396)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:353)
at java.lang.Thread.run(Thread.java:738)
...

Note: The native, or system heap, is different from Java heap. The Java heap contains the instances of Java objects and is maintained by Garbage Collection. The maximum size of the Java heap is preallocated during JVM startup as one contiguous area, even if the minimum heap size setting is lower. Meanwhile the native heap is allocated by using the underlying malloc and free mechanisms of the operating system, and is used for the underlying implementation of Java objects.
We can increase or decrease the maximum native heap available by altering the size of the Java heap. This relationship between the heaps occurs because the process address space not used by the Java heap is available for the native heap usage.

Answer

Refer to the following(in order) to correct this error:

1. Linux has a maximum allowed process per user limit, we can check this using the "ulimit -u" command. If this value is low(default is 1024), then either make it unlimited or raise it to a high value, say 131072. This section is also highlighted as the "max user processes" section in "ulimit -a" output. Use the following command to set it to unlimited:
ulimit -u unlimited

2. Increase the amount of native memory available by lowering the size of the Java heap using the -Xmx option. The process address space not used by the Java heap is available for the native heap usage. Java heap is set in $TOP/bin/conf/service_mem_settings.ini file for the 6 services and as custom_java_options in $TOP/bin/conf/env_settings.ini file for the back end scripts.

3. Check space requirements using 'df -k' command.

4. Lower the number of threads being used.

Product Alias/Synonym

WPC
MDMCS
MDMPIM
MDM Server for PIM
WebSphere Product Center
InfoSphere MDM Server for PIM
Infosphere Master Data Management Collaboration Server
InfoSphere Master Data Management Server for Product Information Management