I have the same issue, we tried with both starting and the same issue, the one who joins get disconnected every few minutes, when testing internet connection upstream and latency are always fine. I think it's more an issue with Epic/Sigames as you start the game through the servers of Epic and then go P2P if I read it right. In FM22 we never had an issue, the only difference is that I started the online carreer through epic client and now we both use steam.
I launched a ticket to let them check the logs of both sides.