Hashing files and Java heap space

Jan 7

posted by Dave in ColdFusion, Error Messages | 8 Comments Bookmark and Share

So, for those that did not know. The hash() function in ColdFusion can do a cool trick. It can generate a md5 hash of a file. Why would you want to do this? Well, if you want to compare 2 files at the byte level a hash is the best way. IF the files are identical byte for byte the hash will be the same. if one byte in the file changed the hash changes. The change could be something as simple as changing a file attribute or an update to its last modified date.

Here is the code to do it. It is very simple.

view plain print about
1<cfset fileHash = hash(FileRead(myfullFilePath), 'MD5')>
This code would result in something that looks like this: 6168f305888c3d795e67c6de17bf8a21

This works perfectly in about 90+% of all cases. However, in what I was doing it worked in about 5% of all cases. The simple fact is that I was trying to hash files that were in the 250mb+ range. When doing this my server would start generating this lovely error:

view plain print about
1"Error","Thread-26","12/29/09","15:27:48",,"Error invoking CFC for gateway ProcessFile: Java heap space."
2java.lang.OutOfMemoryError: Java heap space
3    at java.util.Arrays.copyOfRange(Arrays.java:3209)
4    at java.lang.String.<init>(String.java:216)
5    at java.lang.StringBuffer.toString(StringBuffer.java:585)
6    at coldfusion.tagext.io.FileUtils.readFile(FileUtils.java:174)

I have not been able to determine the exact file size that causes this. I have seen it with files as small at 50mb. But, I do see it very regularly with files over 100mb. I have also not found a way around it yet. I currently just wrap the code inside a cftry to suppress the error.

Till next time...

--Dave

This entry was posted on January 7, 2010 at 9:09 AM and has received 1673 views. There are currently 8 comments. Print this entry.

Insufficient system resources; huh???

Apr 7

posted by Dave in ColdFusion 8, Error Messages | 2 Comments Bookmark and Share

While working on a project to integrated Adobe LiveCycle and ColdFusion I ran into an odd issue. I was calling a LiveCycle service that returns a PDF document. That document is then written to the file system and then delivered to the client.

Everything was working fine then I started getting a strange error.

view plain print about
1AxisFault
2 faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
3 faultSubcode:
4 faultString: coldfusion.xml.rpc.CFCInvocationException:
5 [coldfusion.tagext.io.FileTag$SingleFileOperationException : An error occurred when performing a file operation write on file \\serverpath\tempMarkup\C1E254134EE109A1D312B01C5A2F7EFA.pdf.][java.io.IOException : Insufficient system resources exist to complete the requested service]
6 faultActor:
7 faultNode:
8 faultDetail:
9 {http://xml.apache.org/axis/}stackTrace:coldfusion.xml.rpc.CFCInvocationException: [coldfusion.tagext.io.FileTag$SingleFileOperationException : An error occurred when performing a file operation write on file \\serverpath\tempMarkup\C1E254134EE109A1D312B01C5A2F7EFA.pdf.][java.io.IOException : Insufficient system resources exist to complete the requested service]
10 at coldfusion.xml.rpc.CFComponentSkeleton.__createCFCInvocationException(CFCo... ''

This error would happen directly after the web service ran at the point where the file would be written. What was actually written was a zero (0) byte file and an error message was displayed. I was sure that the error was not in the web service as the error was after the service ran and returned. But at the same time the error said it was an AxisFault which usually points to web services.

After doing some trial and error, contacting a few people that may have a solution, praying, and doing a quick chant I was able to figure it out. I changed the code to write the file locally then move it to the shared storage. (Notice the UNC path in the error message?).

I have no idea why this threw this error as I was able to move the file from local to remote. I have a feeling it has something to do with Windows and it's ability to write the file to the remote storage.

till next time,

--Dave

This entry was posted on April 7, 2009 at 2:56 PM and has received 1229 views. There are currently 2 comments. Print this entry.