As I started my synapse instance with the recommended most simple setup I let it default to sqlite3 database. Now that I federate with large instances it is fastly growing and for sure slowing down everything.
So I should definitely switch to postgres as recommended for better scaling before it gets worse.
Thankfully there's a tutorial [how to port] in the official docs. Of course I have to fit this into my docker-based environment ...