Thanks for the help Chris.
I currently have SQL Prompt 18.104.22.168 installed. (Officially I'm still in the Trial period, but we have a PO that is being worked on for 10 people on our team. I'd like to have this worked out before we get it on everyone's machine.)
When I unload the databases, things continue to work until the next time I switch databases at which point I get the error message again. For the most part, I am able to use the Refactor product in between loads, however with SQL Prompt installed (and Display Suggestions turned off) I have to manually trigger the SQL Server Intellisense which is also cumbersome.
After playing with it a little more, I seem to have the biggest problems with using multiple databases at the same time. As long as I only use objects in the database that I am primarially connected to, it works pretty well. As soon as I try to compare data between databases for test validation, etc... I run into the Outof MemoryException.
With the Databases Unloaded I was able to successfully refactor, but I lost the ability to use snippits and auto insert object/column names through SQL Prompt or Intellisense. Opening a new query window allowed me to start getting suggestions again until the next time it unloaded databases.