The Cost of Convenience: Chasing the "How" in a "Just Use Docker" World

[AlphaObeisance]

I'm going to reiterate a few details here from the previous discussion for the sake of context in hopes to open up a meaningful conversation with a wider range of context than the former post.

For a little context: I’m 36, and about six years ago, I was basically a lobotomized Windows end-user who thought "using a computer" meant browsing the web and gaming. Today, I live out of a terminal on Hyprland, self-host my own infrastructure, and stand a shot at a SysAdmin gig managing enterprise-grade on prem servers.

I got here by doing everything the hard way. I didn't use beginner distros for long; I just brute-forced my way through the Arch Wiki and broke my system until I understood it. But lately, I’ve been feeling like I’m chasing a ghost.

It feels like the entire tech landscape has shifted completely toward convenience. When I'd ask how to build a SQL database from bare metal because I wanted to understand the mechanics, the default answer from the community is almost always, "Why bother? Just deploy a Docker container." Don't get me wrong, I understand the value of efficiency in a production environment. But it feels like we are losing the fundamental "how" of computing. We have a generation of "advanced" users who are really just skilled distro-hoppers, arguing over tribalism and brand loyalty, while the underlying mechanics of the OS become a black box. The base skill of actually reading deep documentation feels like a dying art, replaced by 10-minute YouTube tutorials. (I'm paraphrasing multiple contributions of our previous conversation collectively here).

I know the true greybeards; the folks who have been compiling from source since the 90s, the true veterans, the guys running text-only IRCs, are still out there. I had the privilege of talking to a few of you recently, and it reminded me why I fell in love with this OS in the first place.

So, I’m opening the floor to both the veterans and the newer users who are still hungry to learn:

1. For the veterans: How do you balance the modern demand for convenience (containers, pre-packaged repos) without letting your foundational skills atrophy? Where do you go to find the "meat and potatoes" discussions these days?
2. For the newer/intermediate users: Do you feel the same pressure to just take the easy route? How do you push past the Imposter Syndrome when the systems keep getting more abstracted?

I’d love to hear how the rest of you keep the spirit of the machine alive. I'd prefer to hear from passion rather than consumerism on this one; lord knows there's enough distro discussion to feed humanity for 10 lifetimes over.

It's my hope that this remain an ongoing discussion of relevance rather than being locked because General Linux Topics are considered "unproductive". Mind me not, just trying to keep the spirit alive and start a thread that goes deeper than "I can't connect my wifi on Linux Mint".

If you participate in this thread and grow weary of notifications, do the forum thing and unwatch them. Don't request to have a thread locked because you've grown tired of the discussion.

 

[KGIII]

Well, there's perspective...

To you, I'm an old man. To some of our more aged users, I'm still many years younger than I am.

So, when people give you advice (such as 'just use Docker'), remember that it comes from their perspective. They're not suggesting that you use Docker as a way of gatekeeping. In fact, they're doing the opposite (from their perspective). They are enabling you to use the most easily accomplished and maintained method that they know of.

If you phrased your question differently, you might get different responses.

Maybe you asked it like this: "I'd like to learn about the nitty-gritty database details, and I'm seeking to avoid using Docker. I'm doing this to have a greater understanding of the underlying processes."

Then, your question may inspire different answers that are based on a different perspective.

This being Linux, you'll likely see at least a couple of posts that strongly suggest using Docker, but you'll hopefully get other responses. Remember that not all 'social engineering' is for malicious purposes. Sometimes, you frame a question and your behavior in ways that will help elicit a response more akin to what you're looking to learn.

Knowing how to ask a good question can be hard. Knowing how to ask a question that appeals to a subset of users who are more inclined to skim over your post is even more difficult. So, first, you'll need to understand their perspective. It's perfectly okay to 'appeal to ego'.

For example, "I know that Docker has only been around for a little more than a decade. I think it takes away from learning the fundamentals, even though I'm still young and in the learning phase. How did you deal with managing <foo> and <bar>` before everybody just decided to start telling everybody to 'use Docker'?"

Those are just some basic thoughts on the subject.

Also, as for the age thing, my first use of a programmable computer was right around the year 1970 -- give or take a year. Later, I'd use UNIX (of various types) quite a lot. Of course, I used BASIC along the way. My company used Linux, and I played with Linux on the desktop off and on.

It wasn't until I retired that I turned to exclusively running Linux on the desktop. It took just a day to go through the house, installing Linux on each device. So, even with my UNIX experience, there are people on this site who have used Linux on the desktop for much longer than I have. They will know and remember different things than I will. But, more importantly, they'll have a different perspective.

I stand by it. I believe that social engineering remains one of the most useful hacking tools we have available. In your case, you'd be 'hacking' for knowledge, I suppose. Then again, hacking for knowledge was why we used to do it. We wanted to know what was on the other side of the curtain, what we could learn if we explored deep enough, and what would happen if we pressed the buttons we were told not to. Curiosity can be a strong motivator.

 

[f33dm3bits]

It feels like the entire tech landscape has shifted completely toward convenience. When I'd ask how to build a SQL database from bare metal because I wanted to understand the mechanics, the default answer from the community is almost always, "Why bother? Just deploy a Docker container." Don't get me wrong, I understand the value of efficiency in a production environment. But it feels like we are losing the fundamental "how" of computing.

I prefer podman, I found containers easy useful on the server as for a lot of applications you can just reverse proxy them and you don't have to deal with a load of dependencies and it's if you want to run multiple instances of something by just creating a new container which runs on a different port.

 

[AlphaObeisance]

So, when people give you advice (such as 'just use Docker'), remember that it comes from their perspective. They're not suggesting that you use Docker as a way of gatekeeping. In fact, they're doing the opposite (from their perspective). They are enabling you to use the most easily accomplished and maintained method that they know of.

This is a valid point; but I'd argue that in doing so (innocently mind you), those making suggestion of the easy way fail to acknowledge that in doing so they're assuming a user has pre-existing knowledge of the fundamental systems that make the docker function. For example. A user doesn't want to use GitHub so searches alternatives to find Gitea and GitLab; valid FOSS self hosted alternatives and seeks out to build their own.

Sure, they could easily just make a simple docker-compose.yml and deploy the systems. But what do they do when something goes south with their deployment? They're still going to have to access that container and figure out how to manage it's interior systems to resolve the issues anyway; so wouldn't it be more productive to suggest they learn to compile manually before suggesting the easy road?

What would a user turned onto docker deployments with no former understanding of the systems do if they needed to manually update the SQL database of their Git server in the event of data corruption of sorts? Wouldn't it better serve them to enlighten them to the infrastructure over the wrapper?

Mind you, this is 100% from personal account. This is exactly what I'd encountered. I knew nothing of programming, sysadmin, nothing. And be it my poor choice of articulated inquiry, or pure ignorance; everyone always just said "just use docker!"

If you phrased your question differently, you might get different responses.

Maybe you asked it like this: "I'd like to learn about the nitty-gritty database details, and I'm seeking to avoid using Docker. I'm doing this to have a greater understanding of the underlying processes."

I would agree, but lets be real here my friend, how many users do we see on a daily basis that don't even know how to articulate the fact their wifi isn't connecting or that they're struggling to dual boot a system? Sometimes the words just don't present themselves to folks in such a way that would clarify. I do agree with you though, articulation is key.

This being Linux, you'll likely see at least a couple of posts that strongly suggest using Docker, but you'll hopefully get other responses. Remember that not all 'social engineering' is for malicious purposes. Sometimes, you frame a question and your behavior in ways that will help elicit a response more akin to what you're looking to learn.

Knowing how to ask a good question can be hard. Knowing how to ask a question that appeals to a subset of users who are more inclined to skim over your post is even more difficult. So, first, you'll need to understand their perspective. It's perfectly okay to 'appeal to ego'.

For example, "I know that Docker has only been around for a little more than a decade. I think it takes away from learning the fundamentals, even though I'm still young and in the learning phase. How did you deal with managing <foo> and <bar>` before everybody just decided to start telling everybody to 'use Docker'?"

Those are just some basic thoughts on the subject.

This his generally how I too try to frame inquiries when I resort to discussion based inquiry. Odd how being thorough tends to yield better results; thankfully AI functions in the exact same manner; so I've been able to isolate and minimize human based inquiry significantly the past couple years baha!

Also, as for the age thing, my first use of a programmable computer was right around the year 1970 -- give or take a year. Later, I'd use UNIX (of various types) quite a lot. Of course, I used BASIC along the way. My company used Linux, and I played with Linux on the desktop off and on.

It wasn't until I retired that I turned to exclusively running Linux on the desktop. It took just a day to go through the house, installing Linux on each device. So, even with my UNIX experience, there are people on this site who have used Linux on the desktop for much longer than I have. They will know and remember different things than I will. But, more importantly, they'll have a different perspective.

I stand by it. I believe that social engineering remains one of the most useful hacking tools we have available. In your case, you'd be 'hacking' for knowledge, I suppose. Then again, hacking for knowledge was why we used to do it. We wanted to know what was on the other side of the curtain, what we could learn if we explored deep enough, and what would happen if we pressed the buttons we were told not to. Curiosity can be a strong motivator.

I know nothing if not for "hacking" haha, my entire infrastructure is a beacon of creative and cleaver hacking my way to stability haha. (this genuinely made me chuckle because the utter rediculousness of what I've gone through to get this far brings me a great deal of pride. "I might not be a smart man, but I know what love is") bahaha

I prefer podman, I found containers easy useful on the server as for a lot of applications you can just reverse proxy them and you don't have to deal with a load of dependencies and it's if you want to run multiple instances of something by just creating a new container which runs on a different port.

I guess I'm always a day late and a dollar short. I'd heard of podman some time ago, but I'm a bit resistant to change once I've got it working. 2 years ago I was telling you about how I manually compiled everything and everybody suggested Docker; I can't keep up with all the cool kids toys ya know lol!

 

[CaffeineAddict]

How do you balance the modern demand for convenience (containers, pre-packaged repos) without letting your foundational skills atrophy?

No to docker containers
No to flatpaks/snaps/AUR/copr/AppImage/Software Centers and the rest of same class
No to managed languages or any software made by them
No to convenience
No to distro derivatives
No to anything that doesn't feel native to OS/hardware
No to toolkits or frameworks

Where do you go to find the "meat and potatoes" discussions these days?

I don't go anywhere nowdays, but before that I was hanging on forums for specific programming language because it was very educational and fun to share tips and to talk with like minded people.

 

[AlphaObeisance]

No to docker containers
No to flatpaks/snaps/AUR/copr/AppImage/Software Centers and the rest of same class
No to managed languages or any software made by them
No to convenience
No to distro derivatives
No to anything that doesn't feel native to OS/hardware
No to toolkits or frameworks


I don't go anywhere nowdays, but before that I was hanging on forums for specific programming language because it was very educational and fun to share tips and to talk with like minded people.

Previously, you accredited me for my determination but suggested I learn at least one programming language.


Firstly. Make no mistake, I am NOT a programmer. But, just like I used google to master my Arch Linux installation and get me into the world of Server Administration and the like; so too I have utilized AI to my advantage in not only building scripts (vibe coding I suppose they'd call it), but unlike most; I can't in right mind have AI draw up a code and then just spin it up; I have to at some degree comprehend the processes and functionalities of what make the code work.

Initially I started with Bash, and used AI to teach me how to write functions and so forth. I started building things like basic backup utilities and such. Eventually I started dabbling in Python.

My entire Hyprland experience relies on a custom daemon I've built which had originally started in Bash but I later converted into Python.

I say that I'm "not a programmer" because if you gave me a blank canvas and told me to script/program something; about the only thing I'd probably be able to write from scratch would be in Bash; and only after some google fu so that I could remember the appropriate syntax.

It wasn't until I realized that 99.9% of programmers rely soley on templates to start their projects to I feel a little bit better about my state of "programming" knowledge. Because at the end of the day, you can toss a Bash Script / Python script in front of me and I could probably reverse engineer the crap out of it after staring at it long enough provided it was put together cleanly.

Hopefully this gives me half a tally in the programmer department lol; still a few notches short of a full stack.

 

[f33dm3bits]

I think we should go back to days before the internet, it would be nice and quiet without social media and people just interacting with the world around them where they live not with the entire world. Even though it has it's advantages, it seems to have more disadvantages as the direction society has gone in the last 20 years but that's a whole other topic.

 

[CaffeineAddict]

if you gave me a blank canvas and told me to script/program something; about the only thing I'd probably be able to write from scratch would be in Bash; and only after some google fu so that I could remember the appropriate syntax.

It wasn't until I realized that 99.9% of programmers rely soley on templates to start their projects

It's perfectly standard to have few tabs with docs open all the time while you write something.
I don't think there's a single person who could write much from their head alone, there's just too much to memorize everything.

Templates to start yes, but 99% of work is already existing project and once you're there you understand it better than anyone else and adding little by little is no big problem

 

[KGIII]

Sure, they could easily just make a simple docker-compose.yml and deploy the systems. But what do they do when something goes south with their deployment? They're still going to have to access that container and figure out how to manage it's interior systems to resolve the issues anyway; so wouldn't it be more productive to suggest they learn to compile manually before suggesting the easy road?

They restore from backup or catch it in testing before it goes to production. That's one of the reasons why working on a production server is usually frowned upon. (People still do it all of the time, including me! But, I have backups!)

As for the question, using the whole road thing, we don't first teach people how to drive a Model T before allowing them to get a license to drive on a public road. Sure, there are a few people out there with a Model T. I know of at least two people who daily drive a Model T.

By the way, I'm 99.999% certain that you can't drive a Model T. I could teach you, but it's pretty weird (and kind of pointless).

As for this line in the sand, why not go back further to where you'd be compiling... That has long since been supplanted by the whole packages thing. Sure, you can ask how to do it -- or just read until you figure it out, but it's not something people are going to put a lot of effort into answering.

By the way, if you get said job, you'll want to spend quite a bit of time learning their testing process. If they have you shipping code straight to production, you may not want to work there.

 

[gvisoc]

The problem comes from the judgement attached to the "Why bother? Just...". And it goes both ways. I would never tell someone to not bother if their main or secondary goal is to learn, but I won't judge someone that deploys using docker.

My case is mixed. For example, with the databases: I use a lot of databases to develop at work and on my own projects, and I've been using databases as a programmer for decades. When I deploy on my server I use containers --both made by others and my own. It's more predictable, organised and clean if what you want is to just operate the software. And you have access to all the logs, and you can run your maintenance operations (backups, etc.,...) the same way.

I don't judge anyone that is doing something different than me because that only means that we're in different legs of the trip.

 

[AlphaObeisance]

By the way, if you get said job, you'll want to spend quite a bit of time learning their testing process. If they have you shipping code straight to production, you may not want to work there.

I believe I'll be dealing more hands on prem with the physical hardware and systems administration itself rather than actually programming anything with their LLM. LLM is useless without the hardware and my buddy knows I'm much better with the hardware management than I am development.

The problem comes from the judgement attached to the "Why bother? Just...". And it goes both ways. I would never tell someone to not bother if their main or secondary goal is to learn, but I won't judge someone that deploys using docker.

My case is mixed. For example, with the databases: I use a lot of databases to develop at work and on my own projects, and I've been using databases as a programmer for decades. When I deploy on my server I use containers --both made by others and my own. It's more predictable, organised and clean if what you want is to just operate the software. And you have access to all the logs, and you can run your maintenance operations (backups, etc.,...) the same way.

I don't judge anyone that is doing something different than me because that only means that we're in different legs of the trip.

That is my take actually, though I suppose I didn't convey it properly. Almost all of my servers are hosted using containers now; a choice I choose for the same reason you do. Efficiency and productivity. But 2 years ago when I first started this, all I wanted to do was to learn to build the systems manually; as I figured the only way to truly understand something is to build it yourself.

The missing link here I forgot to mention that should bridge a bit of a gap here is that up until recently (It hasn't even arrived yet), I've had next to no redundancy options. So it was critical to me that I understood how to manage what I had rather than rely on backups and snapshots. Thankfully, I just found a killer deal on a lot of 20 drives to fill my nodes with and have plenty left over for hot swapping in the event of drive failure. So it'll be even less logical now for me to rely on manually compiling everything.

You've got to look at my angle with the understanding that I literally started this journey with 0 knowledge about anything in IT. So in the beginning, not only did I not know how to articulate my intent due to ignorance of the matter; but I didn't know what I should be doing until I got 75% of the way through only to discover I'd have to strip it all down and start over and add something new in. I credit this to being the only reason I know what little I may.

So while "just use docker" is indeed helpful for just getting something working; it only guarantees that when something goes wrong, and we all know it will, the user will be stuck with a dead container and no idea how to fix it. Have mercy on the souls as ignorant as I was with no redundancy options to rely on (at the time, I'm not even sure I understood the concept of redundancy).

I don't think I have a single server today that wasn't deployed using Docker in the homelab. I do very much see where ya'll are coming from. I guess I just kind of figure looking back on when I first started learning; how much more helpful it would have been if I'd of been able to find educational direction rather than simple solutions. I appreciate all who have helped me along the way, obviously I'd not be here without it.

It's just disheartening to me to feel so alone in the 'passion" side of things. Even after 6 years, trying to comprehend how all this stuff works is still baffling and amazing to me. While I aim to be on of the cool kids who can see the world in binary; I know it's unlikely that I'll ever get the opportunity.

Some fancy Batman or Superman; I've always favored those who could tame magic smoke.

I wish I had a better way of explaining things. I've always struggled to connect with people so forgive me if nothing I say makes sense lol.

 

[gvisoc]

I think it's a very healthy approach. And for the areas I don't understand properly I do the same: install them bare metal and rip them apart until I am comfortable enough for my purpose.

My problem is the time: so much stuff to learn and so little time.

 

[AlphaObeisance]

My problem is the time: so much stuff to learn and so little time.

I feel this in my bones. Every time I finally reach that level of "I finally did it!" I literally walk through the door to find another hallway full of doors. A prime example of this is DMZ configuration. When I set out to study networking by improving my LAN, I learned about DMZ setups so bought a couple Flint 2's since they ran OpenWRT. I got all that set up, only to realize I'd out grown the GL-iNet wrapper having spent so much time in LuCI.

At that point of realization I decided to flash the routers with vanilla OpenWRT, then came DNS encryption and DNS hijacking, oh but wait! Now I have to go all the way back to the beginning because I didn't know anything about VLAN configurations; so back I went!

That's the disadvantage of this whole "Solo self found" experience I'm enduring. WIth no formal education, I only know what I've come to realize I needed in my pursuit of knowledge. Try as I may; there's no way I can inform the web of my current skill level, so few ever know where I actually stand in terms of technical comprehension. Which obviously nobody's at fault for, it just is what it is.

I think that's where my frustration really comes from. Being so eager to learn this stuff but having to accept that I've never been "book smart". Self taught guitar, self taught mechanic, self taught systems administrator, network engineer, ect. ect. ect. It's the only way I know, unfortunately it makes it incredibly difficult to relate to people; especially when I lack many of the social skills required to interact with others lol. And nuance is incredibly difficult to convey.

 

[dos2unix]

When we run docker, it usually isnt because of "ease to setup".
I guanaratee you, my ansible script will beat your docker container in time to setup.
If it repeatability,, same thing, I can repeat with ansible.

So then why we do we use it (actually we use podman, less resouces)..
We use it maximize resource usage. My web server is only using 5% of my resources,
lets dump an application on therre, well that bumped me up to 15%, lets dump a DB or there... Ok,, well that bumped me up to 45%, etc... and it isolates the resources from each application from the resources used by another app.
Application isolation, can also help security sometimes, unless I get access to your docker stack,
then all the eggs are in one basket.. so there's pros and cons to all of it.

Also.. when everyone else using calculators, no one knows how to do math.
When everything is just a platform running multiple containers, we forget basic skills.
Thats great I can download a web server or a DB, but do you know how to set one up yourself?
Thats;a rhetorical question, not directed @AlphaObeisance

..and if you dont know the answers to these questions.. do you trust the people you got these comtainers from?
Most (but not all) get vetted by the docker container registry. But there are other places you can get containers from...
do you trust them?

Also are you smart enough to login to the containers, see what processes they are running check the logs, and check whats actually going on.. (it's sending your data to Columbia!) I'm exaggerating ,.. a little.

 

[AlphaObeisance]

all of this ^

Where is my Shia Labeouf applause meme when I need it!

What I'm gathering here is that we're generally all on the same page. My superiors are giving me kudos for finally getting my act together and transitioning to Docker for productivity while simultaneously telling me "sorry dude, but you should migrate to Podman now cuz it's more light weight and got all the new bells and whistles if you wanna keep striving for the production way" (I do btw)?

On the note of Ansible; I've got the course, just haven't gone through it yet. I'll go on a 2 week binge here eventually so I can stop doing it all manually. Then about the time I get ansible done, culture will shift and no doubt suddenly everyone will be using OpenClaw and looking at me like a vintage novelty because I still use a keyboard lol!

 

[osprey]

I got here by doing everything the hard way.

When thinking about where individuals connect with computer technology, one can conceptualise a hierarchy, a set of levels, like rungs on a ladder, each rung being a point at which the individual can enter into the hierarchy. At each level there's a body of knowledge that, once learnt, enables the person to operate or work at that level. At each level the complexity and knowledge of the levels beneath may be quite obscure to the user. For example, if one is functionally only at the level of a desktop user using a browser and office suite applications, then the complexity of the code that produced the desktop applications they use, is generally obscure to them. If one is using higher level languages to write scripts automating functions, then the intricacies of system intialisation and system programming may be obscure to them. But it's also the case that individuals can enter several levels of the hierarchy simultaneously.

The following is one conception of the hierarchy that has come to mind:

1. Machine language:...... binary instructions for the cpu.
2. Assembly language:..... human readable representations of machine language
3. Booting systems:....... firmware, BIOS/UEFI and hardware initialisation.
4. Kernel operation:...... a myriad of functions like memory management,
                           process scheduling, filesystems, device drivers,
                           interprocess communication, system calls.
5. System libraries:...... libc or musl.
6. System programming:.... languages like C, Rust.
7. System initialisation:. init systems.
8. System services:....... display servers, audio servers, network services,
                           databases.
9. Higher level languages: python, java, GUI frameworks, web frameworks.
10. Applications:......... browsers, desktop environments, containers,
                           virtual machines.
11. User applications:.... office suites, media players, text editors, CAD etc.

Beneath these levels sits the actual physical structure that makes it all possible:

1. physics:...... theory
2. electronics:.. semiconductors, transistors and logic gates with digital logic
3. architecture:. cpu and mmu with registers and memory controllers.

I realise the above is entirely contestable, but it seems relevant to the discussion in respect of the matters of "convenience" and "losing the fundamental 'how' of computing" mentioned in post #1.

If the perspective of the user is seen from the point of view of their purpose, then the convenience factor is an advantage for them, and the relevance of the "'how' of computing" is subsumed by that purpose. Do they deserve a critique for their choices? Whilst one may think negatively that such users are taking "the easy route", that's only the case if one attaches a rating of values to the levels, where lower levels appear with greater value, decreasing in value the higher one ascends on the ladder where "easier" is progressively devalued. What would be the point of such value judgements?

In the olden days of computing, the user was often of necessity involved in lower level activities because the higher levels didn't exist. For example, the user needed to be able to program a computer to get a result, as opposed to modern times where an app can achieve the same outcome because it's already programmed to do it.

 

[AlphaObeisance]

all of this!!! ^^^

You single highhandedly just gave me a road map to hyper-fixate and obsess over for like the next 10 years baha!

Literally, the first question that came to mind was "If machines have language, and that language was translated from binary to assembly, who translated the machine language to human readable format and how did they attain the comprehension to speak machine?"

I fear an unhealthy obsession with the study of binary be inevitable at some point in my life because these kinds of goofy questions literally haunt me. It makes me feel like a cave man smacking rocks together because they saw a spark and merely wanted to see if they could do it again.