Java String & Array limitations and OutOfMemoryError

Q1. What is the maximum possible length of a Java String & how much heap space do you need to store the maximum possible String object?
A1. A Java String internally uses a char array (i.e. char[]) and the indices of an array is an integer. The maximum value of an integer is Integer.MAX_VALUE, which is 2^31 – 1 (or approximately 2 billion). So, you can store a file up to 2 GB in size as a String, and for which you need at least 4 GB memory to store as each char is 2 bytes in Java, plus additional ~4 GB memory for creating the String object, so, in total around 8GB of heap space. So, it is not a good design to read a large file ~2GB into a string for further processing.

Q2. What will be the output if you try to read a 2 GB file into a StringBuilder?

A2. If you run as it is you will get “java.lang.OutOfMemoryError: Java heap space”

Now, if you run it with an 8 GB heap memory, 4 GB to store 2 GB file, and another 4 GB for reading the file into a string.

You will get a slightly different “OutOfMemoryError – Requested array size exceeds VM limit”

When you do “sb.append(line);”, if the underlying array char[] knows that it needs more capacity to satisfy a specific append, it doubles its capacity. This happens each time it runs out of room. The above error indicates that it has gone past the Integer.MAX_VALUE, which is 2^31 – 1.

Q3. How will you fix the above error?
A3. Firstly, it is not a good design to read a 2 GB file into memory. You must read as a stream, and write out as a stream without storing the whole file in memory. You 1) process the contents as you read in chunks 2) split file into smaller sizes, or 3) use a byte array as shown below

The following code will run without throwing “OutOfMemoryError” if run with 3 GB or more heap memory. The byte array will consume 2 GB memory and an additional 1 GB for processing.

Runs if you execute with a 3 GB heap size.

Q4. What happens to the following code?

A4. The above code will throw “Exception in thread “main” java.lang.OutOfMemoryError: Java heap space”. But if you increase the heap memory to 8 GB.

It runs because you need 4 GB to store as a String (i.e. 2 GB * 2 bytes per char) and an additional 4 GB for processing.

Q5. How will you prevent the following code from throwing “java.lang.OutOfMemoryError: Requested array size exceeds VM limit” when the bytes are closer to 2 GB?

A5. The above code can throw the following exception.

It can be prevented by just passing the “bytes” directly as shown below.

Relevant links

1) Java primitives & objects – memory consumption interview Q&As

2) 8 Java FAQ memory management interview Q&As.

3) jvisualvm to sample Java heap memory.