Do I need an ISP to connect to the internet?
yes, or you use your neighbors wifi. Or you start your own ISP.
A server is simply a computer that has a static IP address. I'll give you an example: I'm hosting a website called blunix.com. This website runs on a server with the cloud provider hetzner.de. They have a datacenter somewhere in germany with lots of servers - these are basically just flat computers, not much different from your laptop or workstation. They just don't have a screen and you connect to them remotely using the command line.
You can have a server at home, and you can use a old laptop or workstation and just start calling it a server. When you get a router from your ISP, this router has a public IP address. YOu can setup someting like port forwarding to host a website on your server.
But long story short, in order for anything to be connected to / reachable from the internet, like a server, you need someone to attach you to the internet - thats what the ISP does.
You do not get around paying your ISP, unless you use your neighbors wifi or start your own ISP (which is WAY more expensive than just paying an existing ISP).
So a server does not magically connect you to the internet, its just a computer without a screen that is connected to the internet.