So right after the login was completed I was getting the following message:
warning: setlocale: LC_CTYPE: cannot change locale (UTF-8)
It only happened on certain servers, for others everything would work just fine.
This caused a few problems when running some perl scripts that would complain about the locale variables not being set, even though my Terminal app was setup correctly.
To fix this I simply added the following to my ~/.bash_profile: