First, you must let us know, the process through which you had installed the skype on your server which is the current version and which you like to get uninstalled.
If you have installed it through dynamic package, i.e. source file, you will not be able to remove it through either yum, rpm, apt-get or any installation package available for any linux distribution.
Please let us know the type of installation so can guide you