I got the error from the subject of this topic recently. I was doing some testing in my lab and I got the error, I was trying to establish RDP via Site To Site IPsec VPN tunnel from my Azure Stack VM.
The solution is simple, you are missing some Windows Updates.
Machine I was trying to connect was just deployed without any patches. Since it was quick demo lab, active for 2-3 days, sheltered from the internet, I usually don’t update since it is waste of hours and I have things that need to be done.
So, I let updates role – KB4103723 for Windows Server 2016 is what you will need
Download it via Windows Updates or Windows Update Catalog.
After I installed update, I was able to connect via RDP perfectly.