CFSAVECONTENT generates JAVA heap space error

Posted At : January 27, 2008 4:53 PM | Posted By : Dave
Related Categories: ColdFusion

I found this little gem while working on some code for my upcoming presentation to the CF online user group. I was working to crate a text file that contained a million lines of text. The code ran perfectly while doing test runs generating only 10 lines. When I ran it to create the million lines it failed out.

Here is the code I wrote:

<CFPROCESSINGDIRECTIVE SUPPRESSWHITESPACE="YES">
<CFSAVECONTENT VARIABLE="theRes">
<CFLOOP INDEX="i" FROM="1" TO="1000000"><CFOUTPUT>#i#</CFOUTPUT>: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec libero. Suspendisse bibendum. Cras id urna. Morbi tincidunt, orci ac convallis aliquam, lectus turpis varius lorem, eu posuere nunc justo tempus leo. Donec mattis, purus nec placerat bibendum, dui pede condimentum odio, ac blandit ante orci ut diam.
</CFLOOP>
</CFSAVECONTENT>
</CFPROCESSINGDIRECTIVE>

<CFFILE ACTION="WRITE" FILE="#expandPath('.')#\textfile.txt" OUTPUT="#theRes#" NAMECONFLICT="OVERWRITE">

The error was not even a CF error message but a JRun servlet error.

ROOT CAUSE:
java.lang.OutOfMemoryError: Java heap space


javax.servlet.ServletException: ROOT CAUSE:
java.lang.OutOfMemoryError: Java heap space

It appears that there may be a memory max to CFSAVECONTENT. So, I got curious and starting playing with the code. After a bunch of trial and error I was able to figure out the max loop count I could run without error was 294712. This generated a 92mb text file. So, without figuring it out to the byte I would assume that the max is in that area.

Till next time...

--Dave

Comments (6) | Print | Send | del.icio.us | Linking Blogs | Digg it | 2185 Views

Comments
[Add Comment]
Actually I'd say the limit depends on how much space you've allocated to the JVM. What you have to remember is that when you build up strings via concatenation (as you are doing here on each loop iteration), CF (Java, actually) creates an entirely new String instance. So it is very easy to blow the top off the JVM memory if you are concatenating huge strings like this. Instead of doing it this way, use a Java StringBuffer, which appends text to a single StringBuffer instance rather than creating new String instances after each concatenation. If you try this using a StringBuffer you'll be able to build up a much larger string.
# Posted By Brian Kotek | 1/27/08 8:05 PM
And if you're sure your request isn't being acted on by outside threads (via cfthread, or if your data are in a shared scope), you can use StringBuilder, which is supposed to be even faster than StringBuffer. Only works on CF8 though. Or CF7 if you can get it running under java 1.5.

I remember reading something on Luis Majano's site wrt to cfsavecontent use in high-use apps. If I remember correctly, he was saying that one of the problems he had with fusebox's cfsavecontent model for saving the view content was that he'd get instability and out-of-memory errors, and I believe he tracked it down to cfsavecontent. Interesting.

Anyway, good stuff.
# Posted By Marc Esher | 1/28/08 3:09 AM
Approaching the issue slightly differently... why are you generating one huge string, then writing it to file instead of writing it to file straight away?

Get rid of your <CFSAVECONTENT> and put a <CFFILE ACTION="APPEND"> in your loop.

--
Adam
# Posted By Adam Cameron | 1/28/08 6:04 AM
I think we are focusing on the wrong thing here. I don't think it is a matter of what I was doing. I was merely trying to point out the apparent memory cap in CFCONTENT.


--Dave
# Posted By Dave Ferguson | 1/28/08 7:16 AM
@Dave:

As Brian mentioned, I'm sure it's not a limitation of the tag but of the memory allocated to the JVM. This means your mileage will vary.

Something else to keep in mind is that the memory usage of building a string is no where near the physical size of the memory. If you monitor your heap, you'll see that the memory required to generate a 92MB file is significantly higher than 92MBs of RAM. So while the final size of the file written to disk may only be 92MBs, the actually amount of RAM that string consume in the JVM will be higher.
# Posted By Dan G. Switzer, II | 1/28/08 8:20 AM
It *could* be a limit of CFSAVECONTENT, but I rather more think it would be a limit *full stop* ("*period*", whatever you prefer).

All you've demonstrated so far -as far as I can tell - is memory runs out if one uses more of it than is allcoated (the question perhaps is "allocated to what?")

What happens if you swap out the CFSAVECONTENT for simply concatenating the string together:

<cfset theRes = theRes & "#i#: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec libero. Suspendisse bibendum. Cras id urna. Morbi tincidunt, orci ac convallis aliquam, lectus turpis varius lorem, eu posuere nunc justo tempus leo. Donec mattis, purus nec placerat bibendum, dui pede condimentum odio, ac blandit ante orci ut diam.">

My expectation would be you'd get the same error.

But if not, it does actually add some evidence to what you're suggesting. I don't see anything pointing specifically to CFSAVECONTENT here yet.

I'd also try varying your heap size in your JVM: you should see that "max out" figure change along with it.

--
Adam
# Posted By Adam Cameron | 1/29/08 2:59 AM
[Add Comment]