Which version of Design Studio is this? See Poovin's blog re: 1.4 Design Studio 1.4 Client tool - First View where it says " Now SAP has replaced the generic JAVA VM with a custom JRE. " so it is strange that you are experiencing a JAVA VM error
Also see this thread https://scn.sap.com/thread/3665293
Are you using 32 bit or 64 bit Design Studio?