Every install that was used with >512MB eventually crashed with a strange error message and had to be restored from an image to get it going again. Maybe you can run it on >512 but not in a production environment.
What was the error message ?
Tried to use the MaxFileCach entry in system.ini ?
768MB is always safe to use with maxfilechace, all above might be tricky if some components maps soe adresses in the area just below 1GB (ia have a WLAN-adapter that was tricky with 1GB but with 768MB all OK as with 1GB limited to 950MB)