Matrix let-down

I've been trying to use Matrix to replace sites like Discord or Slack. But it seems that if a user creates an invitation-only room in a server, then invited users who are registered on other servers get errors when trying to join. Not very useful error messages either: "Failed to join room". (In my case, I tried creating accounts and rooms at nitro.chat and then at converser.eu, but friends registered at matrix.org don't manage to join).

Quite a let-down. Anyone who's facing the same problem and has maybe managed to solve it?