Are you using COM interop at all? Note that ANTS Profiler can only tell you about managed memory usage, and not unmanaged, so if you are using interop, even if indirectly, and resources aren't being freed properly (e.g. by calling Dispose()). At the moment this is quite tricky to do, however our new version of the memory profiler will support tracking of objects that implement IDisposable.
Sorry I can't be more help at the moment.