Ошибка джава нет свободной памяти

I got this from crash log.. looks like free memory is super low (1020k) that time, is it caused by too few memory?

I have 10g RAM and my GC settings are:
-Xmx5g -XX:NewRatio=3 -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseAdaptiveSizePolicy -XX:MaxGCPauseMillis=700.

Can someone please give me some hints? Thanks!

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f220198ce10, pid=26470, tid=1154627904
#
# JRE version: 6.0_24-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (19.1-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J  com.twitter.typeahead.server.topic.TopicTypeAheadController$2.onSuccess(Lcom/twitter/typeahead/common/MetastoreResult;)V
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

Current thread (0x00007f21f2ce0800):  JavaThread "New I/O client worker #1-18" [_thread_in_Java, id=26628, stack(0x0000000044c23000,0x0000000044d24000)]


siginfo:si_signo=SIGSEGV: si_errno=0, si_code=2 (SEGV_ACCERR), si_addr=0x00007f2200e4616a

Registers:
RAX=0x00000006bae61950, RBX=0x00000007fafe2be0, RCX=0x0000000000000000, RDX=0x00000006bbe2d580
RSP=0x0000000044d22180, RBP=0x00000006bae61950, RSI=0x00000006bae61950, RDI=0x00007f220198ccf0
R8 =0x00000000d77c5ab0, R9 =0x00000006bbd0a190, R10=0x00000000035df16a, R11=0x00007f21fd867000
R12=0x0000000000000000, R13=0x0000000044d22180, R14=0x0000000000000001, R15=0x00007f21f2ce0800
RIP=0x00007f220198ce10, EFL=0x0000000000010207, CSGSFS=0x0000000000000033, ERR=0x0000000000000006
  TRAPNO=0x000000000000000e

Register to memory mapping:

RAX=0x00000006bae61950

[error occurred during error reporting (printing registers, top of stack, instructions near pc), id 0xb]

Stack: [0x0000000044c23000,0x0000000044d24000],  sp=0x0000000044d22180,  free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

If you keep on allocating & keeping references to object, you will fill up any amount of memory you have.

One option is to do a transparent file close & open when they switch tabs (you only keep a pointer to the file, and when the user switches tab, you close & clean all the objects… it’ll make the file change slower… but…), and maybe keep only 3 or 4 files on memory.

Other thing you should do is, when the user opens a file, load it, and intercept any OutOfMemoryError, then (as it is not possible to open the file) close that file, clean its objects and warn the user that he should close unused files.

Your idea of dynamically extending virtual memory doesn’t solve the issue, for the machine is limited on resources, so you should be carefull & handle memory issues (or at least, be carefull with them).

A couple of hints i’ve seen with memory leaks is:

—> Keep on mind that if you put something into a collection and afterwards forget about it, you still have a strong reference to it, so nullify the collection, clean it or do something with it… if not you will find a memory leak difficult to find.

—> Maybe, using collections with weak references (weakhashmap…) can help with memory issues, but you must be carefull with it, for you might find that the object you look for has been collected.

—> Another idea i’ve found is to develope a persistent collection that stored on database objects least used and transparently loaded. This would probably be the best approach…

java.lang.OutOfMemoryError:
Java heap space

Java applications are only allowed to use a limited amount of memory. This limit is specified during application startup. To make things more complex, Java memory is separated into two different regions. These regions are called Heap space and Permgen (for Permanent Generation):

The size of those regions is set during the Java Virtual Machine (JVM) launch and can be customized by specifying JVM parameters -Xmx and -XX:MaxPermSize. If you do not explicitly set the sizes, platform-specific defaults will be used.

The java.lang.OutOfMemoryError: Java heap space error will be triggered when the application attempts to add more data into the heap space area, but there is not enough room for it.

Note that there might be plenty of physical memory available, but the java.lang.OutOfMemoryError: Java heap space error is thrown whenever the JVM reaches the heap size limit.

What is causing it?

There most common reason for the java.lang.OutOfMemoryError: Java heap space error is simple – you try to fit an XXL application into an S-sized Java heap space. That is – the application just requires more Java heap space than available to it to operate normally. Other causes for this OutOfMemoryError message are more complex and are caused by a programming error:

• Spikes in usage/data volume. The application was designed to handle a certain amount of users or a certain amount of data. When the number of users or the volume of data suddenly spikes and crosses that expected threshold, the operation which functioned normally before the spike ceases to operate and triggers the java.lang.OutOfMemoryError: Java heap space error.
• Memory leaks. A particular type of programming error will lead your application to constantly consume more memory. Every time the leaking functionality of the application is used it leaves some objects behind into the Java heap space. Over time the leaked objects consume all of the available Java heap space and trigger the already familiar java.lang.OutOfMemoryError: Java heap space error.

Give me an example

Trivial example

The first example is truly simple – the following Java code tries to allocate an array of 2M integers. When you compile it and launch with 12MB of Java heap space (java -Xmx12m OOM), it fails with the java.lang.OutOfMemoryError: Java heap space message. With 13MB Java heap space the program runs just fine.

class OOM {
  static final int SIZE=2*1024*1024;
  public static void main(String[] a) {
    int[] i = new int[SIZE];
   }
}

Memory leak example

The second and a more realistic example is of a memory leak. In Java, when developers create and use new objects e.g. new Integer(5), they don’t have to allocate memory themselves – this is being taken care of by the Java Virtual Machine (JVM). During the life of the application the JVM periodically checks which objects in memory are still being used and which are not. Unused objects can be discarded and the memory reclaimed and reused again. This process is called Garbage Collection. The corresponding module in JVM taking care of the collection is called the Garbage Collector (GC).

Java’s automatic memory management relies on GC to periodically look for unused objects and remove them. Simplifying a bit we can say that a memory leak in Java is a situation where some objects are no longer used by the application but Garbage Collection fails to recognize it. As a result these unused objects remain in Java heap space indefinitely. This pileup will eventually trigger the java.lang.OutOfMemoryError: Java heap space error.

It is fairly easy to construct a Java program that satisfies the definition of a memory leak:

class KeylessEntry {
 
   static class Key {
      Integer id;
 
      Key(Integer id) {
         this.id = id;
      }
 
      @Override
      public int hashCode() {
         return id.hashCode();
      }
   }
 
   public static void main(String[] args) {
      Map m = new HashMap();
      while (true)
         for (int i = 0; i < 10000; i++)
            if (!m.containsKey(new Key(i)))
               m.put(new Key(i), "Number:" + i);
   }
}

When you execute the above code above you might expect it to run forever without any problems, assuming that the naive caching solution only expands the underlying Map to 10,000 elements, as beyond that all the keys will already be present in the HashMap. However, in reality the elements will keep being added as the Key class does not contain a proper equals() implementation next to its hashCode().

As a result, over time, with the leaking code constantly used, the “cached” results end up consuming a lot of Java heap space. And when the leaked memory fills all of the available memory in the heap region and Garbage Collection is not able to clean it, the java.lang.OutOfMemoryError:Java heap space is thrown.

The solution would be easy – add the implementation for the equals() method similar to the one below and you will be good to go. But before you manage to find the cause, you will definitely have lose some precious brain cells.

@Override
public boolean equals(Object o) {
   boolean response = false;
   if (o instanceof Key) {
      response = (((Key)o).id).equals(this.id);
   }
   return response;
}

What is the solution?

In some cases, the amount of heap you have allocated to your JVM is just not enough to accommodate the needs of your applications running on that JVM. In that case, you should just allocate more heap – see at the end of this chapter for how to achieve that.

In many cases however, providing more Java heap space will not solve the problem. For example, if your application contains a memory leak, adding more heap will just postpone the java.lang.OutOfMemoryError: Java heap space error. Additionally, increasing the amount of Java heap space also tends to increase the length of GC pauses affecting your application’s throughput or latency.

If you wish to solve the underlying problem with the Java heap space instead of masking the symptoms, you need to figure out which part of your code is responsible for allocating the most memory. In other words, you need to answer these questions:

1. Which objects occupy large portions of heap
2. where these objects are being allocated in source code

At this point, make sure to clear a couple of days in your calendar (or – see an automated way below the bullet list). Here is a rough process outline that will help you answer the above questions:

• Get security clearance in order to perform a heap dump from your JVM. “Dumps” are basically snapshots of heap contents that you can analyze. These snapshot can thus contain confidential information, such as passwords, credit card numbers etc, so acquiring such a dump might not even be possible for security reasons.
• Get the dump at the right moment. Be prepared to get a few dumps, as when taken at a wrong time, heap dumps contain a significant amount of noise and can be practically useless. On the other hand, every heap dump “freezes” the JVM entirely, so don’t take too many of them or your end users start facing performance issues.
• Find a machine that can load the dump. When your JVM-to-troubleshoot uses for example 8GB of heap, you need a machine with more than 8GB to be able to analyze heap contents. Fire up dump analysis software (we recommend Eclipse MAT, but there are also equally good alternatives available).
• Detect the paths to GC roots of the biggest consumers of heap. We have covered this activity in a separate post here. It is especially tough for beginners, but the practice will make you understand the structure and navigation mechanics.
• Next, you need to figure out where in your source code the potentially hazardous large amount of objects is being allocated. If you have good knowledge of your application’s source code you’ll be able to do this in a couple searches.

Alternatively, we suggest Plumbr, the only Java monitoring solution with automatic root cause detection. Among other performance problems it catches all java.lang.OutOfMemoryErrors and automatically hands you the information about the most memory-hungry data structres.

Plumbr takes care of gathering the necessary data behind the scenes – this includes the relevant data about heap usage (only the object layout graph, no actual data), and also some data that you can’t even find in a heap dump. It also does the necessary data processing for you – on the fly, as soon as the JVM encounters an java.lang.OutOfMemoryError. Here is an example java.lang.OutOfMemoryError incident alert from Plumbr:

Without any additional tooling or analysis you can see:

• Which objects are consuming the most memory (271 com.example.map.impl.PartitionContainer instances consume 173MB out of 248MB total heap)
• Where these objects were allocated (most of them allocated in the MetricManagerImpl class, line 304)
• What is currently referencing these objects (the full reference chain up to GC root)

Equipped with this information you can zoom in to the underlying root cause and make sure the data structures are trimmed down to the levels where they would fit nicely into your memory pools.

However, when your conclusion from memory analysis or from reading the Plumbr report are that memory use is legal and there is nothing to change in the source code, you need to allow your JVM more Java heap space to run properly. In this case, alter your JVM launch configuration and add (or increase the value if present) the following:

-Xmx1024m

The above configuration would give the application 1024MB of Java heap space. You can use g or G for GB, m or M for MB, k or K for KB. For example all of the following are equivalent to saying that the maximum Java heap space is 1GB:

    java -Xmx1073741824 com.mycompany.MyClass
    java -Xmx1048576k com.mycompany.MyClass
    java -Xmx1024m com.mycompany.MyClass
    java -Xmx1g com.mycompany.MyClass