Finally ready to explore centralized data solution - guidance greatly appreciated

[mark7110]

OK ... so I finally made the switch and got a laptop with Linux Mint and have really been happy so far. My next step is to convert another couple of laptops to other Linux distros to start branching out a bit there. My ideal solution moving forward would be to purchase external hardware for housing centralized data for each individual laptop to connect to while I'm using them. So, my questions are ...

(1) Will it be unreasonably slow to have said laptops working on data files (libreOffice, etc) located on an external hardware solution (or maybe that depends on the quality hardware, connectors and even laptops I look to convert?).
(2) If working on files externally is an unreasonably slow solution, is there a solution that makes more sense where each laptop can grab the external files and bring them local while working, then returning them to the central solution once complete? I know I could always do this manually, but feel I'd inevitably screw that up in forgetting to return everything I'd been working on - I guess I'd want something with a more formal version control structure if I had to go this route.
(3) I'd also want to be able to take backups of the data residing on the external hardware solution ... I presume it's pretty standard for said solution to have USB ports to take backups? As far as any system or application backups for the distros, I would still just take care of that via the USB ports on the laptops themselves - the backups I'm talking about here are just data (libreOffice, music, pictures, etc).

Any guidance or advice here would be greatly appreciated - my personal digging on this has just spun me in circles and I'm hoping that it's just a matter of just getting pointed in the right direction for now. Also, apologies in advance as I feel I'm likely butchering some of the verbiage in trying to get this properly into words.

And, oh yeah, really glad to finally get on this forum .


thanks ... mark

 

[gvisoc]

Seems that what you want (1+2) is to have the same data files available for all your computers, and (3) backups.

(1+2) I'd suggest to try software solutions like Syncthing, that being peer to peer allows you to do that over the network, without the need for buying and managing (maintaining, replacing, powering,...) hardware. And see how you go. That would get you up and running without any further expenses. I use syncthing to synchronize a folder between two computers and a phone and I am quite happy with it. If it doesn't work you can always explore more sophisticated solutions in the line of (3)

(3) There are plenty solutions that are less expensive when you take away the data sync from them. Depending how you go with the first one I'd try to see what is the cheapest NAS that would meet my needs. If peer to peer sync solutions like syncthing didn't meet your needs (or if you just want to tinker with servers) then you can try any NAS that would be able to run things like Nextcloud on Docker.

EDIT: all NASes I know have USB ports and automatic / periodic backup solutions available, both through the USB and to Cloud Services.

 

[osprey]

OK ... so I finally made the switch and got a laptop with Linux Mint and have really been happy so far. My next step is to convert another couple of laptops to other Linux distros to start branching out a bit there. My ideal solution moving forward would be to purchase external hardware for housing centralized data for each individual laptop to connect to while I'm using them. So, my questions are ...

(1) Will it be unreasonably slow to have said laptops working on data files (libreOffice, etc) located on an external hardware solution (or maybe that depends on the quality hardware, connectors and even laptops I look to convert?).
(2) If working on files externally is an unreasonably slow solution, is there a solution that makes more sense where each laptop can grab the external files and bring them local while working, then returning them to the central solution once complete? I know I could always do this manually, but feel I'd inevitably screw that up in forgetting to return everything I'd been working on - I guess I'd want something with a more formal version control structure if I had to go this route.
(3) I'd also want to be able to take backups of the data residing on the external hardware solution ... I presume it's pretty standard for said solution to have USB ports to take backups? As far as any system or application backups for the distros, I would still just take care of that via the USB ports on the laptops themselves - the backups I'm talking about here are just data (libreOffice, music, pictures, etc).

Any guidance or advice here would be greatly appreciated - my personal digging on this has just spun me in circles and I'm hoping that it's just a matter of just getting pointed in the right direction for now. Also, apologies in advance as I feel I'm likely butchering some of the verbiage in trying to get this properly into words.

And, oh yeah, really glad to finally get on this forum ... linux has really rekindled my interest in being a minor league home tek-nerd again .


thanks ... mark

The classical solution to this situation, as I understand it, is linux NFS, the Network File System. This page has links to some relevant documentation if you're interested in following up on this route: https://wiki.debian.org/NFS

 

[theLegionWithin]

the way I read this is a desire for a self hosted cloud based file server (maybe I'm wrong). I did a little bit of searching and found this --> https://www.seafile.com/en/download/ - never heard of it before but it looks like it may have some promise.

it looks like you could use the pro version if you have a very small userbase - but for more than 3x users you've got to pay. the price really isnt that bad though, for something that offers android, iphone, linux, mac, and winOS integration. I didnt read that deep into the documentation but it looks like a neat product.

 

[mark7110]

Seems that what you want (1+2) is to have the same data files available for all your computers, and (3) backups.

(1+2) I'd suggest to try software solutions like Syncthing, that being peer to peer allows you to do that over the network, without the need for buying and managing (maintaining, replacing, powering,...) hardware. And see how you go. That would get you up and running without any further expenses. I use syncthing to synchronize a folder between two computers and a phone and I am quite happy with it. If it doesn't work you can always explore more sophisticated solutions in the line of (3)

(3) There are plenty solutions that are less expensive when you take away the data sync from them. Depending how you go with the first one I'd try to see what is the cheapest NAS that would meet my needs. If peer to peer sync solutions like syncthing didn't meet your needs (or if you just want to tinker with servers) then you can try any NAS that would be able to run things like Nextcloud on Docker.

EDIT: all NASes I know have USB ports and automatic / periodic backup solutions available, both through the USB and to Cloud Services.

Many thanks for the feedback ... I had been thinking about a NAS solution, however a software sync solution had not even crossed my mind. I will definitely be doing some research today on Syncthing. Again, thanks very much for taking the time to provide info.

 

[mark7110]

The classical solution to this situation, as I understand it, is linux NFS, the Network File System. This page has links to some relevant documentation if you're interested in following up on this route: https://wiki.debian.org/NFS

Excellent reading here ... greatly appreciate your taking the time to provide. It is definitely some sort of NFS solution I seek and I'm anxious to read up on this.

 

[mark7110]

the way I read this is a desire for a self hosted cloud based file server (maybe I'm wrong). I did a little bit of searching and found this --> https://www.seafile.com/en/download/ - never heard of it before but it looks like it may have some promise.

it looks like you could use the pro version if you have a very small userbase - but for more than 3x users you've got to pay. the price really isnt that bad though, for something that offers android, iphone, linux, mac, and winOS integration. I didnt read that deep into the documentation but it looks like a neat product.

Exactly correct ... what I had in mind was indeed a self hosted file server (though preferably local and wired - still very unsure on that, though). Seafile definitely look like it provides some robust capabilities & I'm going to be doing some reading on that today. Thanks very much for taking the time to provide that info.

 

[f33dm3bits]

I had been thinking about a NAS solution

I would have a look at a Synology NAS then.

 

[mark7110]

I would have a look at a Synology NAS then.

Checking out their stuff now ... wow - pretty remarkable storage quantities.

 

[MikeWalsh]

@mark7110 :-

Hallo.....and

to Linux.org, the friendliest Linux forum on the web.

My own small home network - just 3 machines - also uses a software solution. This comprises one reasonably powerful HP desktop rig, and a couple of Dell laptops.

All run 'Puppy' Linux....and all multi-boot. The HP desktop boots around a dozen Puppies; each Dell lappie runs 3-4.

On the desktop, I have multiple drives. Puppy is running mostly 'portable' applications; I have one large partition containing nothing BUT 'portable', self-contained apps. These are 'shared' between the occupants of the "kennels", and only 'link-in' to each Puppy via a Menu-entry pointing back to the 'launch' script within each app.

Another large partition on a second drive contains all my 'personal' data; music, documents, vids, images, etc. These also merely 'link-in' to each Pup's 'home' directory via custom scripts.

Both laptops do something similar, just on a smaller scale. These are both Latitude 'workstations', which have a removable bay on the side of the machine. This normally houses an optical drive, but you can also buy an adapter which replaces this & allows installation of a secondary drive.....which is what I've done.

~~~~~~~~~~~~~~~~~~~~~​

When it comes to 'sharing' between the different machines, I used to mess around with SAMBA.....before I realized what a PITA it was to set it all up. These days, my 'sharing' solution is far more simple. I use a Python module called SimpleHTTPServer. This runs on the desktop, making everything on there available to the other two lappies .....and accessed very simply, across the network, via a tab in whatever browser you run (making use of the fact that all browsers are also able to act as file-browsers for the 'local' file-system). This module works in both directions, permitting not just downloading from the desktop, but uploading back to it too.

As I said, it's super-simple. The best solutions usually are!


Mike.

 

[Condobloke]

On Linux Mint, in the Software Manager, there is an app named : Python3-rangehttpserver :

...and is described as : Simplehttpserver with support for Range requests:
RangeHTTPServer is a Python module for running a simple HTTP server with support for range requests. It is suitable for use in local testing, not Internet scale production use.

HTTP range requests ask the server to send only a portion of an HTTP message back to a client and are useful for clients like media players that support random access, data tools that know they need only part of a large file, and download managers that let the user pause and resume the download.

GitHub - danvk/RangeHTTPServer: SimpleHTTPServer with support for Range requests

SimpleHTTPServer with support for Range requests. Contribute to danvk/RangeHTTPServer development by creating an account on GitHub.

github.com

Also in the Software Manager is :Lighttpd

HTH

 

[mark7110]

@mark7110 :-

Hallo.....and

to Linux.org, the friendliest Linux forum on the web.

My own small home network - just 3 machines - also uses a software solution. This comprises one reasonably powerful HP desktop rig, and a couple of Dell laptops.

All run 'Puppy' Linux....and all multi-boot. The HP desktop boots around a dozen Puppies; each Dell lappie runs 3-4.

On the desktop, I have multiple drives. Puppy is running mostly 'portable' applications; I have one large partition containing nothing BUT 'portable', self-contained apps. These are 'shared' between the occupants of the "kennels", and only 'link-in' to each Puppy via a Menu-entry pointing back to the 'launch' script within each app.

Another large partition on a second drive contains all my 'personal' data; music, documents, vids, images, etc. These also merely 'link-in' to each Pup's 'home' directory via custom scripts.

Both laptops do something similar, just on a smaller scale. These are both Latitude 'workstations', which have a removable bay on the side of the machine. This normally houses an optical drive, but you can also buy an adapter which replaces this & allows installation of a secondary drive.....which is what I've done.

~~~~~~~~~~~~~~~~~~~~~​

When it comes to 'sharing' between the different machines, I used to mess around with SAMBA.....before I realized what a PITA it was to set it all up. These days, my 'sharing' solution is far more simple. I use a Python module called SimpleHTTPServer. This runs on the desktop, making everything on there available to the other two lappies .....and accessed very simply, across the network, via a tab in whatever browser you run (making use of the fact that all browsers are also able to act as file-browsers for the 'local' file-system). This module works in both directions, permitting not just downloading from the desktop, but uploading back to it too.

As I said, it's super-simple. The best solutions usually are!


Mike.

Wow, that's quite a system ... curious to see in the long run where time takes me with respect to an evolving system such as yours. Now also curious to learn more about SimpleHTTPServer as it has now been mentioned more than once in this thread as a good option. Thank you very much for taking the time to respond to my request for help and also for detailing your system configuration - it sounds really cool.

 

[mark7110]

On Linux Mint, in the Software Manager, there is an app named : Python3-rangehttpserver :

...and is described as : Simplehttpserver with support for Range requests:
RangeHTTPServer is a Python module for running a simple HTTP server with support for range requests. It is suitable for use in local testing, not Internet scale production use.

HTTP range requests ask the server to send only a portion of an HTTP message back to a client and are useful for clients like media players that support random access, data tools that know they need only part of a large file, and download managers that let the user pause and resume the download.

GitHub - danvk/RangeHTTPServer: SimpleHTTPServer with support for Range requests

SimpleHTTPServer with support for Range requests. Contribute to danvk/RangeHTTPServer development by creating an account on GitHub.

github.com

Also in the Software Manager is :Lighttpd

HTH

Fantastic ... many thanks for the suggestion, summary and link to peruse. I greatly appreciate your taking the time to respond!

 

[MikeWalsh]

@mark7110 :-

Very good suggestion by Condobloke there, though in fact what Brian found was an "additional" module that extends the functionality of the already-present SimpleHTTPServer module further still.

Y'see, SimpleHTTPServer comes built-in to the standard Python libraries of just about all mainstream distros.....because with Python being such an easy-to-learn, and hence widely-used & popular coding language, it tends to come installed by default, OOTB.

This tutorial will explain how you can use SimpleHTTPServer "on-the-fly", to share the contents of ANY directory across the network as & when required:-

An In-Depth Guide on Using Python SimpleHTTPServer – TheLinuxCode

Have you ever needed to quickly share some files or test a web app locally? Setting up a full web server like Apache can be overkill – sometimes you just need something simple that works fast. This is where Python SimpleHTTPServer comes in handy! In this comprehensive tutorial, I‘ll explain...

thelinuxcode.com

All that's involved is a simple one-liner in the terminal...very quick, super simple, and easy to implement. Just be aware that, while HTTP is quite secure enough for a LAN (Local Area Network), i.e., your home network, HTTPS would be required for sharing further afield.....along with additional security-related protocols. SimpleHTTPServer was designed to enable quick file-sharing locally; at this, it excels!

Like all the best Linux apps/utilities, it does ONE thing only.....but it performs it to the very best of its ability.

~~~~~~~~~~~~~~~~~~~~~~​

As for the 'portables' being shared between different Puppies, 'twas a purely selfish move originally.....and yet one that made sense (and was all down to storage space, or rather, the lack of it).

Think about it. If you run half-a-dozen distros, and want to install the same app in all of them, this normally means installing that app half-a-dozen times.....once per distro. That wastes quite a bit of disk space, and if you repeat the process for a dozen other assorted apps, you end up wasting a LOT of storage through multiple duplication.

The 'portables' stop you having to do this. They also permit booting into a different distro, and carrying on from exactly where you left off.....because all your config files'n'stuff are within the portable directory, too.

These days I have 5TB+ of storage space, compared to just 500GB on my previous system. I don't NEED to save space, but I continue to do so.....because it leaves me so much more room for my personal projects.

Well, it makes sense to me, anyway.!


Mike.