Server Networking (?)

AlphaObeisance

AlphaObeisance

Active Member
Joined
Nov 7, 2022
Messages
205
Reaction score
163
Credits
2,802
Hello everybody,

First and foremost I want to disclaim that I'm not entirely certain where this post should go, so to the moderators I apologize if it is misplaced. Secondly, I assume this will be a server networking inquiry/issue that I discuss here.

I started my first home lab this past fall after having bought a refurbished PowerEdge R730. It came with x2 Xeon processors and it's now got 220GB of RAM and ample storage. I know in the grand scheme of things this may not be that impressive; but it suits my needs at this time as I venture forth to learn systems administration and networking skills.

The issue I'm presented with here is that while I've learned to set up and configure both ESXi and Proxmox with this hardware, and I'm successfully hosting multiple dedicated servers for my family's games as well as other conveniences; I can only seem to provide public access to one of them. I'll elaborate.

While most of the servers I host don't need front end access as games generally connect direct anyway. But I created a cloud server to host my mothers media content after she passed a couple years ago; and I'd wanted to provide a means for my entire family to access said content to revisit her memories. I purchased a domain through GoDaddy and then somewhere along the line ended up on CloudFlare for my SSL and domain management.

I managed to configure public access to said cloud after some wrastling around with CloudFlares data requirements and learning to use a new modem and router I'd just purchased. But then it dawned on me...

"Wouldn't it be nice if I could host a website that redirected traffic to any of my other services; like if I wanted to create a Mattermost server, Nextcloud and Gitea server and have a centeral hub for providing public access to available services?" was the idea that crossed my mind.

Research lead me to NGINX and I couldn't get that figured out; a buddy said to try Apache and stated it was preferred anyway. So I gave apache a try.

I was of the understanding (possibly in ignorance) that using Apache as a reverse-proxy would enable me to basically append my domain with specifics like for example.

www.mywebsite.com
www.mywebsite.com/mattermost
www.mywebsite.com/gitea
www.mywebsite.com/nextcloud

and be directed to each service respectively. Provided of course I configured the router to point to the apache server where the traffic could then be sent to the appropriate server. I'm not sure if there was any logic behind my thinking, as it clearly doesn't work this way. But there in lies the problem.

I'm not sure how to achieve this. It would be pretty neat to host a hub of sorts for my family to land on and choose which services I have to offer. I thought maybe I'd need to learn WordPress and build a landing page for such a thing; but soon realized it woudln't do me any good to build it if I don't know how to configure it.

As I write this I contemplate if I'd still have to port forward for each server individually and not just the apache server; which now that I think of it seems obvious so that the servers can still communicate directly with the traffic. But even so, I'd still be unsure as to how to provide a HUB for any services available from my server.

I do apologize for any lack of appropriate terminology. I'm just a hobbiest learning this stuff on my own through research and having committed the past 5 years to using exclusively Linux. I do truly appreciate any input, guidance, or education that you may provide. I know folks are busy, so I figure for those willing to take the time to contribute to me and my situation; I'd better do the best I can to digest the information so that I can advance in my journey.

Thank you for your time.
 


I am very impressed. any system with 220GB of RAM when I never see things that hold more than 64G is very impressive. how big is the drive? but it says 128G RAM which is still great.

You have enough RAM to power several virtual machines, would that maybe be a good idea for your services? instead of multiple physical computers you have the power to put them all into one and it may make your routing easier. Just throwing out some ideas there and please please please do not use word press. Simple html should do what you need and I will offer to make it for you if you promise never to use word press.
 
APTI said:
I am very impressed. any system with 220GB of RAM when I never see things that hold more than 64G is very impressive. how big is the drive? but it says 128G RAM which is still great.

You have enough RAM to power several virtual machines, would that maybe be a good idea for your services? instead of multiple physical computers you have the power to put them all into one and it may make your routing easier. Just throwing out some ideas there and please please please do not use word press. Simple html should do what you need and I will offer to make it for you if you promise never to use word press.
Click to expand...

I appreciate the kudos. I've learned so much over the past 6 months that I'm contemplating trying to get my Linux+ and Security + certifications in hopes I can land a gig somewhere. But for now, I'm just trying to provide some convenient services for the family with the hardware that I've got.

Yeah the host specs in my signature are actually that of my Host PC that I use as my daily driver/leisure machine. It's pretty mean too, but I seldom use it for any services anymore now that I've got the PowerEdge.

The R730 is supposedly capable of 768GB of RAM but it'll be a while before I can even attempt reaching that haha. For now though it does me well with the 2 Xeon processors and nearly 100TB of storage.

I'm currently running Proxmox (Debian based hypervisor) to host various VM's, each specifically intended for running any given service. i.e. Mattermost, Nextcloud, gitea ect. Even though most of these services don't require much by way of resources, each having a dedicated VM ensures that if I have an issue with one, I don't have to shut all of the others down in the process, plus a VM is a bit more secure than say use of a template. Turnkey templates have come in handy lately too as they're more light weight than VM's for simple services like Gitea which I really only use experimentally at this time.

So all of these services already operate off of the R730. But, I'm unsure how to provide public access to all of said services. Simutaneously.

Since the R730 and all it's VM's are on a single server within the LAN Network, they all share the same public IP. And while I can port forward say nextlcoud to port 80/443 or whatever it requires, and I can point my domain to my public IP so that traffic can then access my Nextcloud server through my domain; I have no idea how to provide access to the rest of my services.

I feel like this could be a very simple issue to resolve, I just seem to be missing a critical component in understanding the networking side of things.

Since each VM has it's own IPV4 I'm unsure how to make that work with the domain. I've no doubt I'm horrible at explaining my situation so I apologize for any redundancy.

Basically this is my current situation.

mywebsite.com > my PublicIP > router > Port 80 to IP for Nextcloud = lands user on nextcloud.

I'm wondering if I can't append to the domain like so

mywebsite.com/mattermost > my PublicIP > Mattermost port to mattermost IPV4 > lands user on mattermost service.

Ect. ect. for each service.

Thus why I was thinking reverse proxy for traffic management would work. At least the way I understood it, I could route traffic from my public IP to the apache server, and then based on the url that was being called, the apache server would direct users to the appropriate service.

Talking about this makes me feel really dumb so I do apologize. I know it's got to be possible lol but this side of things feels like a nasty spiderweb to me.

I'm not sure how to share code and configurations in a code box, but I"m going to try it here with the embedded code function. Apologizes, I don't forum much.

This is the essance of how I set up my reverse proxy. Modueles and Apache are all running. But even when I port forward 80/443 to the Apache server I can't seem to get anything to register. I'm not sure why this feel so difficult to figure out.

<VirtualHost *:80>
ServerName website.com
ServerAlias www.website.com

# Redirect HTTP to HTTPS (optional but recommended)
# Redirect permanent / https://website.com/

ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

<VirtualHost *:443>
ServerName website.com
ServerAlias www.website.com

# Cloudflare Flexible SSL (no SSL configuration needed here)

# Nextcloud
ProxyPass / http://192.168.1.##:80
ProxyPassReverse / http://192.168.1.##:80

# Gitea
ProxyPass /kgit http://192.168.#.##:3000
ProxyPassReverse /kgit http://192.168.#.##:3000

# Mattermost
ProxyPass /kchat http://192.168.#.##:8065
ProxyPassReverse /kchat http://192.168.#.##:8065

ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

A bit off topic but fiigured I'd show you the server itself just for funzies. I've had a lot of fun learning to work with this machine.
photo_2024-05-15_15-16-48.jpg
 
Last edited:
I have not done what you are wanting so if I am not 100% on I apologize. all my services are on separate ports but on the one computer so one IP. With several VM I would think that you should be able to send the traffic thru the router to the computer without issue. Now to get the traffic to the correct VM means you would need to play with the routing tables or you may find software to redirect to the correct VM. While I can't point you to the step by step I hopefully point you to the right tools and you can get it from there.

for future info, what are you using for the VM? I use Qemu AKA gnome boxes.
 
APTI said:
All my services are on separate ports but on the one computer so one IP.
Click to expand...

I think that may be the underlining issue I'm facing is that I just used the default ports during setup; so if say one service and another both use port 80, that's likely an issue.

APTI said:
With several VM I would think that you should be able to send the traffic thru the router to the computer without issue. Now to get the traffic to the correct VM means you would need to play with the routing tables or you may find software to redirect to the correct VM.

for future info, what are you using for the VM? I use Qemu AKA gnome boxes.
Click to expand...

I've theorized that appending the domain with the port like say uh, www.mywebsite.com:#### might work but I haven't actually tried it yet as kids have kept me busy.

I use Proxmox as my hypervisor (Formerly used ESXi/VMware until the Broadcom acquisition) and I have been begrudgingly using Ubuntu Server due to the fact that 99% of the guides out there I'd discovered regarding the setup of services almost alway default to Ubuntu; so I just found it easier to go that route. Though I prefer to use Fedora Server Linux if I'm being honest. And if I'm being really stubborn, I just use Arch Linux for everything lol.
 
AlphaObeisance said:
I think that may be the underlining issue I'm facing is that I just used the default ports during setup; so if say one service and another both use port 80, that's likely an issue.



I've theorized that appending the domain with the port like say uh, www.mywebsite.com:#### might work but I haven't actually tried it yet as kids have kept me busy.

I use Proxmox as my hypervisor (Formerly used ESXi/VMware until the Broadcom acquisition) and I have been begrudgingly using Ubuntu Server due to the fact that 99% of the guides out there I'd discovered regarding the setup of services almost alway default to Ubuntu; so I just found it easier to go that route. Though I prefer to use Fedora Server Linux if I'm being honest. And if I'm being really stubborn, I just use Arch Linux for everything lol.
Click to expand...
The www.mydomain.com:## will work as it redirects to the port specified, however if 2 things have the same port then you may need to change one. Don't forget that you can forward incoming port 123 to 10.10.10.10 on port 99 if you wish. You can use that to deal with duplicate ports or to hide ports so people do not hack.
example I have a mysql server internet accessible. standard port is 3306 and that is the port I use on the LAN but if i go outside I have port 123 forwarded to 3306 so that way I have a non standard port used and much more difficult for people to probe for various services and hack in through them. They try port 3306 and it does nothing so they give up. Keep that in mind when setting up if you have not already.
 
APTI said:
The www.mydomain.com:## will work as it redirects to the port specified, however if 2 things have the same port then you may need to change one. Don't forget that you can forward incoming port 123 to 10.10.10.10 on port 99 if you wish. You can use that to deal with duplicate ports or to hide ports so people do not hack.
example I have a mysql server internet accessible. standard port is 3306 and that is the port I use on the LAN but if i go outside I have port 123 forwarded to 3306 so that way I have a non standard port used and much more difficult for people to probe for various services and hack in through them. They try port 3306 and it does nothing so they give up. Keep that in mind when setting up if you have not already.
Click to expand...

I appreciate this information, though admittedly I must look into it and do some research as it's mostly foreign information to me. I appreciate the bread crumbs though! I'll post updates accordingly. I'm hoping I can set down here in the next evening or so and get some stuff done! I appreciate your chatting with me.

As frustrating as these issues are, it's quite fun to learn this stuff. Even more fun to have people that understand at least the basis of what I'm trying to accomplish! I literally have NOBODY in my day to day life that understands anything technology, let alone linux or networking! Was really starting to feel alone in the world lol
 
I am happy to help, many others on the forum also happy to help. Some may be able to walk you through step by step, I have not done what you are trying so I can only give so much. I understand what you want to do and how though.
 
APTI said:
I am happy to help, many others on the forum also happy to help. Some may be able to walk you through step by step, I have not done what you are trying so I can only give so much. I understand what you want to do and how though.
Click to expand...

It's funny man. When I left Windows 5 years ago I knew nothing of Linux, and only used my Windows machine to surf the web and play games. Now here I am obsessed with learning stuff nobody else seems to understand (in my day to day interactions). Literally the ONLY people who get me are on forums like these, and I'll be honest, this is the only forum I seem to return to; albeit infrequently. I hope to start visiting more often as I reallllly need to start socializing more. With any luck share some nuggets of knowledge I've gained along the way.

I recall building a wordpress website some 10+ years ago but I don't recall how I was able to direct users to each subordinate directory. Granted that's an entirely different ball park as the whole website ran on a single host; and that was looooong before I ever knew anything about computers.

I'll have to do some homework on the use of ports and some common practices. I've tried to keep my home lab setup as close to Enterprise Standard as I can given my nonexistent enterprise experience. Lots of googling about what people tend to do in an enterprise environment. But even that seems to generally boil down to the preferences of the SysAdmin.

Funny thing is that when I started reading up on Apache, I envisioned Apache being an auctioneer of sorts at a podium and when traffic called for a specific service it hollered out xP. Man some of the crap my brain comes up with to memorize stuff is kind of nuts lol
 
I live rural and am the only technology store within 50 miles. My girlfriend understands about 65% of what I say in computers but without that I would have nobody get me. I understand the plight.
 
I've read this 3 or 4 times, and I'm still not sure what you're trying to do.

You just want the same internet domain name, but depending on the URL, you want a different endpoint?
 
dos2unix said:
I've read this 3 or 4 times, and I'm still not sure what you're trying to do.

You just want the same internet domain name, but depending on the URL, you want a different endpoint?
Click to expand...
what he is doing is running various services mostly for gaming on different VM's but all on same server. Different requests will go to different VM to be processed. Different ports (mostly) for different requests. That part is still being worked out. at the moment if I understood right he wants to figure out how to send the network traffic to the correct VM
 
dos2unix said:
I've read this 3 or 4 times, and I'm still not sure what you're trying to do.

You just want the same internet domain name, but depending on the URL, you want a different endpoint?
Click to expand...

Yes. As APTI stated I do run multiple game servers; though that's not the concern as all of the game servers work just fine. Gamer just input my public IP and PW to access my game servers and all is well.

The issue I've run into is that I would like to provide access to multiple services I run off my server machine from the same domain. Apologies for my lack of clarification as I don't really know how to explain my desired outcome.

Best way I could describe is that typically when one goes to a website they get a "home" page. As of right now, navigating to my website directs users to my Nextcloud server.

So if they type in www.mywebsite.com they get the nextcloud server. But I'd like to figure out a way that they could type something like www.mywebsite.com/mattermost, for example, to instead be directed to my mattermost server, or my gitea server or what have you.

If I were building a website through wordpress or something this would be pretty easy because the entire website is contained in the same data structure. But where as each service I have is hosted from a separate VM for each service; I'm not really sure how to get them to all respond to my domain. If this makes any sense.

I was under the impression that by setting up apache as a reverse proxy I could have the apache server manage the traffic accordingly. Here's how I was interpreting how that was done. In my apache reverse-proxy settings under my virtual host I have something like
# Mattermost
ProxyPass / http://192.168.#.##:80
ProxyPassReverse / http://192.168.#.##:80

And I was under the impression that the Proxypass / indicated that just www.mywebsite.com/ would direct traffic to said Mattermost server.

Further more my config has something akin to
# Gitea
ProxyPass /gitea http://192.168.#.##:3000
ProxyPassReverse /kgit http://192.168.#.##:3000

Where the ProxyPass /gitea functions as www.mywebsite.com/gitea and navigating to such would send traffic to my gitea server.

Clearly I misunderstand this; but hopefully it gives some clarity as to how I was interpreting the intended fucntionality of a reverse-proxy.

I do apologize, my networking skills are in their infancy and most of this eludes me to some degree. With any luck I am on the right track though?

I did realize yesterday that even if this is accurate, it still wouldn't work as I do believe I'll still have to port forward the IPV4 and appropriate port for each service VM at the router to provide access. So even if I understood the reverse-proxy, it wouldn't function as intended as I don't have all the services port forwarded.

I would much apprecaite any clarification as to whether or not my understanding of how reverse-proxy works is any bit accurate or not; as I think that's a higher priority than the port forwarding portion at this time.
 
I've been racking my brains over this one for a minute now and I'm "going back" so to speak to see if I comprehend how ports work in the first place.

Services hosted off of my server all register under my public IP, obviously. Each service runs on a port. Various defaults but ultimately the user can use nonstandard ports at will. Am I trackin?

So for public access to my services appropriate Port Forwarding is required obviously at both the System and Router level. Then without domain, services can be accessed simply by PublicIP : Port.

I'm trying to achieve the same effect with my domain. In place of Port, I want to specify an appendage to my domain.

As a result, the effect would translate something like this.

PublicIP : Port < www.mywebsite.com
PublicIP : Port2 <www.mywebsiite.com/service2
PublicIP : Port3 <www.mywebsite.com/service3

Sorry for any confusion. I have a really weird way of trying to resolve problems in my head. But, I figure I made it this far; I'll figure it out eventually lol
 
To me, it sounds like all you need is name based sites.


gameserver1.mysite.com
gameserver2.mysite.com
gameserver3.mysite.com

I'm assuming you only have one public IP address.


You could also do it with redirects, although that can cause latency issues.

You basically have three options.
You can do it port based as @AlphaObeisance mentioned above.
You can do it named based.
Or, you can get three public IP addresses and do it IP based.
 
dos2unix said:
To me, it sounds like all you need is name based sites.


gameserver1.mysite.com
gameserver2.mysite.com
gameserver3.mysite.com

I'm assuming you only have one public IP address.


You could also do it with redirects, although that can cause latency issues.

You basically have three options.
You can do it port based as @AlphaObeisance mentioned above.
You can do it named based.
Or, you can get three public IP addresses and do it IP based.
Click to expand...

Indeed I've only one public IP. These are just small services based off my home network and it'll require manual intervention anytime my ISP changes my IP on me.

I appreciate this info! I Look forward to diving into it today to see if it'll suit my needs; sure sounds like it! Thank you!

The article indicates " The first step is to create a <VirtualHost> block for each different host that you would like to serve. " which is the first issue I have in that I was defining all of the virtual hosts in a single block. I'm not sure why exactly I didn't go straight to apache documentation but this should clear things up! I'll update accordingly once I've figured out everything that I'd FUBAR'd and provide my steps for resolution for any who stumble on my nonsense in the future.
 
Last edited:
Ok it would seem that as suggested, the root of my problem is a lack of understanding how using Apache as a reverse proxy is intended to function.

Initially I had my reverse-proxy configuration and all it's ProxyPass and ProxyPassReverse variables within a single virtual host. After reading the apache documentation that @dos2linux shared I realized this was wrong and that there must be a virtual host for each VM hosting a service that I want to respond to my domain. So now, my reverse-proxy configuration looks more like the following (I have no idea how to share code/configs in a code box, so I apologize). Trying to embed the code seems to embed each line independently and idk why.

<VirtualHost *:443>
# Nextcloud - rcloud
ServerName example.com
ServerAlias www.example.com

ProxyPass / http://192.168.0.14:80
ProxyPassReverse / http://192.168.0.14:80

ErrorLog ${APACHE_LOG_DIR}/rcloud_error.log
CustomLog ${APACHE_LOG_DIR}/rcloud_access.log combined
</VirtualHost>

<VirtualHost *:443>
# Gitea - kgit
ServerName kgit.example.com
ServerAlias www.kgit.example.com

ProxyPass / http://192.168.0.16:3000
ProxyPassReverse / http://192.168.0.16:3000

ErrorLog ${APACHE_LOG_DIR}/kgit_error.log
CustomLog ${APACHE_LOG_DIR}/kgit_access.log combined
</VirtualHost>

<VirtualHost *:443>
ServerName kchat.example.com
ServerAlias www.kchat.example.com

# Mattermost - kchat
ProxyPass / http://192.168.0.15:8065
ProxyPassReverse / http://192.168.0.15:8065

ErrorLog ${APACHE_LOG_DIR}/kchat_error.log
CustomLog ${APACHE_LOG_DIR}/kchat_access.log combined
</VirtualHost>

I went to cloudflare and added DNS records for kgit.example.com, kchat.example.com and so forth but I've got to be missing something yet, as my domain still doesn't register any of my services when I port forward to the apache server.

So as it stands right now. I can set the router firewall settings to port forward say 192.168.0.14:443 and it will provide access to my nextcloud server from www.example.com without issue. It serves up my cloud just as it should.

But when I port forward to say my apache server 192.168.0.9:443 I just get "web server is down".

sudo systemctl status apache2.service indicates that it is actively running. I have sudo ufw allow http/https as well as sudo ufw allow 80/443 just for good measure (I know it's redundant). So the ports are open on the apache server.

Man I feel like I'm so dang close to making this work but it's something stupid I've overlooked. I'm too close to the project lol. Bare with me, I'm really trying to grasp my head around this! It's equally exciting as it is frustrating at this point lol. Starting to feel like a glutton for punishment.
 
You must log in or register to reply here.

Members online

Total: 620 (members: 5, guests: 615)

Latest posts

Top