I usually allow remote access (SSH, RDP, etc...) through VPN only.
When access from Internet is absolutely required (because it's not possible to have a VPN), then I usually add a firewall rule to allow access only from a list of known WAN IP addresses.
That is certainly the best approach! Also, whenever possible, I run those protocols on IPv6 only, preferably on an address that is not as well in DNS for other services on the host. They cannot viably scan the IPv6 range so this obscurity hides the remote access quite well.
Rob