Bo,
Sorry to hear you are having issues running setup. Can you confirm you have the correct RTM release installed at the default location? If the correct version is installed you should have the folder "C:\Program Files\Visual Studio Tools for Applications 2.0 SDK\2008".
Have you tried running one ShapeAppCSharp samples included with the SDK? These samples include a setup file which will correctly register the application. The SDK samples are located at "C:\Program Files\Visual Studio Tools for Applications 2.0 SDK\2008\Visual Studio Tools For Applications\Samples", try extracting the ShapeAppBasicCSharp to C:\ShapeAppSamples, then run the included SetupShapeAppBasicCSharp.js setup file from the command prompt. This should run without errors, if there are errors then you may need to uninstall and re-install VSTA.
As for the testCon sample, can you confirm you are using the TestCon_v_2 sample (the original TestCon sample is for VSTA v 1)? Then confirm the registry settings, they should be something like:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VSTAHostConfig\TestCon\2.0]
"AppName"="TestCon"
"ProjectTemplatesLocation"="C:\\VSTA\\Summit Software\\Samples\\TestCon\\Templates"
"VSTAVersion"="9.0"
You should run the setup command from the Visual Studio 2008 command prompt- this doesn't require any environmental path variables.
I hope this helps, please let me know if you are able to run setup or if you encounter further issues with this.
Thanks,
-Melody