Sounds like a memory leak in Xpa to me what you describe. Did you monitor memory usage of your process ? You can do that f.i. with Windows Performance Monitor (https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/using-performance-monitor-to-find-a-user-mode-memory-leak).
If you find out that you do suffer from such a leak you are however doomed. Magic is not open source. In the past Magic was leaking when you did something assign to Blob variables and in order to avoit this you had to update the Blob variable wiith NULL() - to force Magic to free the memory.
Maybe this bug is back ? First check if you do suffer from a memory leak.

Best regards,

Andreas