First, use SSH if it is available, and avoid using telnet. This is because a telnet connection is not encrypted. This means every sniffer between the client and server can easily sniff out the username and password. An SSH connection is encrypted, so sniffers typically cannot sniff out the username and password. Well, at least not easily.
Second, once a machine allows remote access, hackers have one more way to compromise the machine. If you configure a box to have remote CLI access, make sure that you choose relatively difficult passwords. Otherwise, a hacker can just use a password dictionary to hack into the system. With SSH, you can also use the key-pair method to log in, which is often considered most secure than password log in.