Why are there no easy ways in Linux to retrieve system information?

Beaver763987450

New Member
Joined
Feb 16, 2024
Messages
3
Reaction score
4
Credits
89
I use Linux for quite some time, but I am somewhat new to the Linux internals, the Linux philosophy of things, and the command line tools in Linux.
I am also a developer and I was interested in creating my own tools for Linux.

The first thing that I wanted to try, was to find a way to programmatically retrieve CPU/RAM/Disk utilization and also retrieving APT package information for using it to build a server monitoring and management tool.

The Windows way for all these things is calling Windows API functions.
E.g. for CPU utilization: https://stackoverflow.com/a/43357695

Trying to find the way to do this in Linux caused a massive headache for me because:
  • There are no OS APIs, as far as I know.
  • The tools that exists in Linux (e.g. APT) often don't provide ways to have input/output served in a machine-readable format. Not to mention that they might not be installed at all...
  • Using virtual files like /proc/stat to retrieve information seems complex/hacky/unreliable/inefficient because, there is a lot of parsing going on (reference this and this).

Perhaps this goes along the same dividing topic, revolving around whether to have central and essential tools, like systemd, to perform actions or let each application do everything from scratch. But wouldn't it be nice if there was one of the below things? :

  • An API to retrieve OS information. I am guessing people in Linux are not so fond of this idea for some reason.
  • Tools (like APT, systemd, etc) that also provide machine-readable input/output options in a consiste.
  • Virtual files (like /proc/stat) where you can at least read data in a binary format, and don't have to parse strings, or remember what each text-column is (since you could read an entire struct with names that are prettier).

Perhaps I am missing a lot of things, since I am pretty new to interacting with Linux in an application-level, but I have seen how firewalls like PfSense interact with the FreeBSD OS, and I am looking at how HTOP retrieves system information ( https://github.com/htop-dev/htop/bl...26e73b03b00005a6096/linux/LinuxMachine.c#L402 ) and I am wondering if this is the best way possible.
Maybe I am too unaccustomed to passing strings of text around, from process to process.
 


Most of these are command line tools, but the information is there.

lshw

hardinfo

dmidecode

lspci

lsusb

inxi

cpuinfo

glxinfo
 
@Beaver763987450, on the matter of system information, you could check this thread, particularly post #55:

Linux is not MS, like Italian is not English despite using the same alphabet.

In any case, the premise of your question is false ... there are numerous ways of easily deriving system information in linux operating systems. It's just a matter of learning ... but I expect you knew that one way or another :) .
 
Last edited:

I made a screenshot of my hardware with hardinfo, it took less than 5 seconds?
Screenshot from 2024-02-16 20-45-31.jpg
 
Last edited:
+1 for Hard Info.

But for some reason, it shows up in the menu and on the desktop icon (if you create one) as "System Profiler and Benchmark." Why; just why?
 
Last edited:
Perhaps I am missing a lot of things
Yes, obviously.

Read post #2

make sure brain is engaged, before putting mouth in gear

or, in another form.....think, before you quack
 
Perhaps I am missing a lot of things, since I am pretty new to interacting with Linux..

I don't have nearly as much knowledge as you about Windows. I did do a lot of poking around and know far more than a lot of people.

Linux Mint is similar enough to Windows that I don't feel completely lost, but a lot of things are VERY different. Commands that work in Windows Power Shell do not work in the Linux terminal. Most of my challenge has been translating what I'm trying to say from Windows to Linux. Actions in Windows are called something different in Linux. If I want to be clear about my question, I need to find out as much I can on my own so I can communicate what I'm trying to say to people who don't use Windows.

Yesterday I was trying to stop processes. In Windows, I would press Ctrl + Alt +Delete to open the Task manager. That doesn't work in Linux and Task Manager is foreign word is Linux. I can 'speak Windows'. I'm learning to 'speak Linux'. As I learn to 'speak Linux' better, things become more apparent. I can then communicate my problem to people who know Linux.

Right now I'm the same place I was when I began with Windows. I look up things every day.

As I said my knowledge isn't anything close to yours. But can say that I had to to understand that Linux is NOT Windows.
 
Yes, obviously.
Or maybe this person was looking for something different or for the wrong thing or in the wrong place? My question to that person would be if the replies to their topic gave them some new insights, if not what can do do to help with that?
 
Hello all, and thanks for all your replies.
I think many people have misunderstood my question.

More specifically:
Neofetch, screenfetch too, altough they display only the most important stuff.
If you want something like CPU-Z, its clone for linux is named CPU-X.

I made a screenshot of my hardware with hardinfo, it took less than 5 seconds?
View attachment 18340
+1 for Hard Info.

But for some reason, it shows up in the menu and on the desktop icon (if you create one) as "System Profiler and Benchmark." Why; just why?

Yes it's a real shame we don't have any way to check System Information...but wait...https://gprivate.com/69idk

m1203.gif
Htop comes to mind - available in most repositories

View attachment 18372
These are all nice tools, and I have already used htop, neofetch, and others like bpytop.
(I have never used hardinfo, which seems really nice, but it is unrelated to what I was asking for in this thread...)
I do not want a graphical tool or a command line tool to view system information myself.
This is not a end-user question, but rather a developer question.
I want a way to gather system information in an application that I will create.
This means that, I would either need to depend on another tool/package that gathers this information and dumps it as text in an easy to parse format (in this case, the post #2 is interesting, although I am not sure how easily and reliably parse-able are the outputs of these tools), or use a virtual file like /proc/stat (the same way htop seems to work internally), or find a better way that I am not aware of.

Seeing that htop uses the /proc/stat method, I am almost convinced this is the "main Linux way".
Unless they have chosen this method only because of compatibility across all Linux and Unix variants.
In any case, I am guessing the authors of htop know what they are doing.
But it seems a bit odd, because this process passes the data through a human-readable form, which is not required for the purpose of a piece of software. That means that, whatever part of the Linux OS feeds data to this /proc/stat file, takes some numeric values (e.g. float, int, double, etc), formats them into a human-readable form (e.g. a text of "0.7535"), dumps them in the file stream, then the application reads the formatted text, and converts it back into a numeric value (e.g. float, int, double, etc) with parsing. And my point is that, there are many conversions being done in this process that are not really needed.

And my question was focusing on two areas:
1) Weather I am missing something big. Since almost all the responses refer to command line tools that dump the information to the console, probably not.
2) Whether it would be of any value to have this /proc/stat file, and others like it, replicated in a binary format, where the information is not converted into human-readable formats, or at least be converted into a standard serialization format like JSON, where you can have labels for each value you read and don't have to guess what each value is, or need to refer to manuals. In essence, this is an OS API architectural question. I feel something like this would be of value, but then again I know little about the internals and the philosophy of Linux.

Most of these are command line tools, but the information is there.

lshw

hardinfo

dmidecode

lspci

lsusb

inxi

cpuinfo

glxinfo
Thanks for this list.
I will check which of those support machine-readable outputs, which is what I am really after.
Otherwise, my application could break on each an every update of these tools, which is not so great. :(
So far, the only tool I found that clearly supports machine-readable output is systemd.
Tools like apt don't appear to support that.
But I haven't search much yet.
I will check the ones above. :)
Although, another problem is that one would have to assume that these tools are installed.
And assumptions when it comes to software are not a good idea.
I guess, that's why htop uses virtual files instead.

@Beaver763987450, on the matter of system information, you could check this thread, particularly post #55:

Linux is not MS, like Italian is not English despite using the same alphabet.

In any case, the premise of your question is false ... there are numerous ways of easily deriving system information in linux operating systems. It's just a matter of learning ... but I expect you knew that one way or another :) .
Yes, I have guessed there are many ways to retrieve system information.
I am trying to find the most reliable and efficient way possible, for software development purposes. :)
Perhaps my question could be posted in StackOverflow.
But these is an architectural part in it, that would probably require an article or a whole discussion to explain.

I don't have nearly as much knowledge as you about Windows. I did do a lot of poking around and know far more than a lot of people.

Linux Mint is similar enough to Windows that I don't feel completely lost, but a lot of things are VERY different. Commands that work in Windows Power Shell do not work in the Linux terminal. Most of my challenge has been translating what I'm trying to say from Windows to Linux. Actions in Windows are called something different in Linux. If I want to be clear about my question, I need to find out as much I can on my own so I can communicate what I'm trying to say to people who don't use Windows.

Yesterday I was trying to stop processes. In Windows, I would press Ctrl + Alt +Delete to open the Task manager. That doesn't work in Linux and Task Manager is foreign word is Linux. I can 'speak Windows'. I'm learning to 'speak Linux'. As I learn to 'speak Linux' better, things become more apparent. I can then communicate my problem to people who know Linux.

Right now I'm the same place I was when I began with Windows. I look up things every day.

As I said my knowledge isn't anything close to yours. But can say that I had to to understand that Linux is NOT Windows.
I think I am getting to know Linux very well in terms of end-user usage.
E.g., I manage headless Linux servers using terminal multiplexers and all the bells and whistles one can use from the terminal.
I don't know all the tools yet, but I know all of the tools I need.
But I don't think I have captured the design philosophy of the internals of Linux, nor the way Linux programs are encouraged to interact with the OS. I have seen some FreeBSD firewalls executing tasks by spawning external tools and passing command-line arguments. I saw tools like htop using the virtual files. I think, there are also some libraries (I think their extension is *.SO in Linux) that one can link to, but perhaps using those is harder for dealing with compatibility, and I haven't seen many examples of using those yet. And I am not sure what capabilities they offer. Perhaps retrieval of system-resource utilization information is not one of them (or management of installed packages for that matter).


The book: The Linux Programming Interface is also a great resource for systems programming.
Nice.
This books seems interesting! :)
From a first look, the method for retrieving CPU information suggested by it appears to be /proc/stat.
But I haven't read it all yet.

But, I am starting to guess that linking against libraries (like *.SO files) is not very much encouraged or preferred.
Probably because of compatibility problems that may appear on each library upgrade.
From a first look, I don't think this book mentions *.SO files at all, but I might be wrong.
Calling methods from a *.SO file could prevent many of the problems I was trying to avoid (converting numerical values to text and back again), but probably it generates new sets of problems.
 
@Beaver763987450 wrote:
I want a way to gather system information in an application that I will create.This means that, I would either need to depend on another tool/package that gathers this information and dumps it as text in an easy to parse format (in this case, the post #2 is interesting, although I am not sure how easily and reliably parse-able are the outputs of these tools), or use a virtual file like /proc/stat (the same way htop seems to work internally), or find a better way that I am not aware of.

It seems what you want is available: outputs that you can parse. That's exactly what is available in the outputs of linux tools, those tools being the command line programs such a mentioned by @dos2unix in post #2 and the link in post #4.

In relation to /proc/stat and the other /proc files, they are generally always available on linux systems because users and applications usually need the information the kernel provides on processes, and in many cases use the /proc filesystem to alter system behaviour. The manpage on proc explains in detail most of the outputs in the /proc files, including all the outputs in /proc/stat, demystifying the arrays of otherwise mind-boggling figures.

And my question was focusing on two areas:
1) Weather I am missing something big. Since almost all the responses refer to command line tools that dump the information to the console, probably not.
2) Whether it would be of any value to have this /proc/stat file, and others like it, replicated in a binary format, where the information is not converted into human-readable formats, or at least be converted into a standard serialization format like JSON, where you can have labels for each value you read and don't have to guess what each value is, or need to refer to manuals. In essence, this is an OS API architectural question. I feel something like this would be of value, but then again I know little about the internals and the philosophy of Linux.

The idea of information being "replicated in a binary format" is specifically the opposite of the intention of the output in most linux tools. By outputting information in text form, it becomes available in all computer systems since they can all generally output text. Since text is universally available, it's available for reading by users to understand what's happening, available for manipulation to alter processes, and the text format makes access to configuration as straight forward as writing text documents. Text is preferred to binary because the user can see what is being configured without that extra layer of software that binary imposes which the user has to trust.

another problem is that one would have to assume that these tools are installed.

As mentioned above, linux systems generally have the necessary tools available. Extra tools that may be desired are straight forward to install. Embedded and minimal systems may have very sparse collections of the tools, but they are special cases and may have their own special tools.

I am trying to find the most reliable and efficient way possible, for software development purposes.
....
I don't know all the tools yet, but I know all of the tools I need.
It's almost certainly the case that linux tools will satisfy your requirements. It's probably just a matter of finding those that do that and becoming familiar with the meaning of their outputs and the ways in which they can be used for your purposes. :)
 
Last edited:
@JasKinasis might have some input in this regard, as well. By mentioning him, I have brought it to his attention.

Cheers

Wizard
 
Using virtual files like /proc/stat to retrieve information seems complex/hacky/unreliable/inefficient because, there is a lot of parsing going on (reference this and this).
Yes, this is the UNIX way.

There are advantages: One common simple means of access - the file API. The info is often human readable without additional software beyond standard tools such as cat, sed, and awk. File-based access permissions and conventions can be can be applied to many things. The various virtual "file-systems" may be in-memory and very fast to access. There is no need to create or revise an API, although the file system-structures and their content often carry the same kind of baggage in a similar way to aged- API's.

Often libraries exist that wrap file systems, such as /proc, and serve up the data in ways that make sense for the targeted programming language. For example, there are probably many /proc wrappers for python.

For better or worse many later add ons have not always followed the UNIX way, so the collective heritage doesn't consistently follow this path. For example, the windowing systems don't expose windows, or info about windows as a virtual-file-system.
 
@Beaver763987450 wrote:
But it seems a bit odd, because this process passes the data through a human-readable form, which is not required for the purpose of a piece of software. That means that, whatever part of the Linux OS feeds data to this /proc/stat file, takes some numeric values (e.g. float, int, double, etc), formats them into a human-readable form (e.g. a text of "0.7535"), dumps them in the file stream, then the application reads the formatted text, and converts it back into a numeric value (e.g. float, int, double, etc) with parsing. And my point is that, there are many conversions being done in this process that are not really needed.

It surely will depend on the "piece of software" whether or not text files in /proc are needed, useful or purposeful. The text files make information available to users and programs. That the text files are created from programs such as C code compiled into binary form is the way of the operating system, but the files in /proc are specifically to provide an interface to the kernel and its data structures. It's about fundamental system functioning that is brought into the user space via the /proc files in text. If you are writing a piece of software that has no purpose for the program or user to access /proc in userspace, then the parsing issue disappears and one can be as binary as one needs to in accomplishing their aims.

The inference, if it is such, that the use of text, eschewing binary, is inefficient in linux, is not the case. Freedom of approach is entirely available for the writing of software.
 
At the end of the day Linux has heaps of ways to check everything...since switching to Linux it's not something I worry about now as I'm not that paranoid as I was in windoze.
animated-smileys-cheeky-125.gif.pagespeed.ce.HvExM_LvTi.gif
 
Thanks all. :)
I think I have some hints on where I should start digging now.

If you are writing a piece of software that has no purpose for the program or user to access /proc in userspace, then the parsing issue disappears and one can be as binary as one needs to in accomplishing their aims.
Regarding this, I was hoping there was a way for user-space programs to avoid this parsing issue.
It could make monitoring programs (like htop) work more efficiently and maybe poll the CPU data more frequently without causing performance issues.
Although, it is nice that kernel-space has no such limitations.
And perhaps the performance penalty of this parsing is not too bad.

But if adding more virtual files to Linux is not a problem, perhaps having differently-formatted virtual files like these would be of some value:
Code:
/proc/stat
/proc/stat.json
/proc/stat.bin

Even as an end-user, I would prefer to read JSON data than to count columns and cross-reference against manuals to figure out what each value means. But it might be because I am lazy. :D
But this will probably require more code in the Linux code-base, and more complexity, so it is not without a cost.

Anyway, I think I will go the htop way for CPU monitoring now. :)
 
Regarding this, I was hoping there was a way for user-space programs to avoid this parsing issue.
It could make monitoring programs (like htop) work more efficiently and maybe poll the CPU data more frequently without causing performance issues.


Anyway, I think I will go the htop way for CPU monitoring now. :)
These observations come to mind in relation to the binary and parsing issues and may be of interest.

The program: htop, is written in C so one can compile it into a binary with multiple build options to adapt it to one's preferences or requirements. One needs to be aware of the dependencies like ncurses and others depending on the nature of the build to have it work on a system.

Another program that covers similar output to htop is the program: glances, which is written in Python and makes use of the python3-psutil package on linux. Information on the psutil component from that package info is thus:
psutil is a module providing an interface for retrieving information on running processes and system utilization (CPU, memory) in a portable way by using Python, implementing many functionalities offered by tools like ps, top and Windows task manager.
so glances looks like it uses the psutil module to do parsing of ps and other tools to produce its output, but when looking at the python, it creates what it calls "clones".

I guess that's an example of alternative approaches available to the software programmer.
 
Last edited:
Top