Process Crash Logs
- Thread starter malonn
- Start date
In most distro's it's /var/log/messages
Usually "dmesg" ( without the quotes ) will give similar output.
When you say "process logs". That makes me think you are looking for a specific process. ( i.e. nginx )
If you aren't looking for generic system processes, then usually they have their own logs.
Some ways to troubleshoot applications...
systemctl status -l nginx
journalctl -u nginx
cat /var/log/nginx/error.log
Depending on your distro, these log files are usually cleaned up/rotated/deleted after a period of time automatically.
( it seems 30 days is the most common limit )
cd /etc/logrotate.d
There are usually a dozen or so files in here, each one has it's own settings per application.
Usually "dmesg" ( without the quotes ) will give similar output.
When you say "process logs". That makes me think you are looking for a specific process. ( i.e. nginx )
If you aren't looking for generic system processes, then usually they have their own logs.
Some ways to troubleshoot applications...
systemctl status -l nginx
journalctl -u nginx
cat /var/log/nginx/error.log
Depending on your distro, these log files are usually cleaned up/rotated/deleted after a period of time automatically.
( it seems 30 days is the most common limit )
cd /etc/logrotate.d
There are usually a dozen or so files in here, each one has it's own settings per application.
Last edited:
If logfiles aren't enough, you can do a system trace. But you need to be somewhat familiar with system calls.
ps -ef | grep nginx
root 1000 1 0 Jun13 ? 00:00:00 nginx: master process /usr/sbin/nginx
nginx 1001 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1002 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1003 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1004 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1005 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1006 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1007 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1008 1000 0 Jun13 ? 00:00:00 nginx: worker process
strace -p 1001 -ff
strace: Process 1001 attached
epoll_wait(11, [{events=EPOLLIN, data={u32=1144422416, u64=139892524220432}}], 512, -1) = 1
accept4(8, {sa_family=AF_INET, sin_port=htons(33642), sin_addr=inet_addr("10.0.0.127")}, [112 => 16], SOCK_NONBLOCK) = 20
epoll_ctl(11, EPOLL_CTL_ADD, 20, {events=EPOLLIN|EPOLLRDHUP|EPOLLET, data={u32=1144423136, u64=139892524221152}}) = 0
epoll_ctl(11, EPOLL_CTL_DEL, 8, 0x7ffe74cfe94c) = 0
epoll_ctl(11, EPOLL_CTL_ADD, 8, {events=EPOLLIN|EPOLLEXCLUSIVE, data={u32=1144422416, u64=139892524220432}}) = 0
epoll_wait(11,
ps -ef | grep nginx
root 1000 1 0 Jun13 ? 00:00:00 nginx: master process /usr/sbin/nginx
nginx 1001 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1002 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1003 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1004 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1005 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1006 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1007 1000 0 Jun13 ? 00:00:00 nginx: worker process
nginx 1008 1000 0 Jun13 ? 00:00:00 nginx: worker process
strace -p 1001 -ff
strace: Process 1001 attached
epoll_wait(11, [{events=EPOLLIN, data={u32=1144422416, u64=139892524220432}}], 512, -1) = 1
accept4(8, {sa_family=AF_INET, sin_port=htons(33642), sin_addr=inet_addr("10.0.0.127")}, [112 => 16], SOCK_NONBLOCK) = 20
epoll_ctl(11, EPOLL_CTL_ADD, 20, {events=EPOLLIN|EPOLLRDHUP|EPOLLET, data={u32=1144423136, u64=139892524221152}}) = 0
epoll_ctl(11, EPOLL_CTL_DEL, 8, 0x7ffe74cfe94c) = 0
epoll_ctl(11, EPOLL_CTL_ADD, 8, {events=EPOLLIN|EPOLLEXCLUSIVE, data={u32=1144422416, u64=139892524220432}}) = 0
epoll_wait(11,
There are number of queries in your post which can't easily be answered with a single response.
The "logs" are generally kept in the /var/log directory. But "crash logs" is an ambiguous expression when considering what constitutes a crash.
If the kernel "crashes" and leaves a core dump, it's usually left in /var/lib/systemd/coredump, and needs specialised tools to analyse it. It's usually a developer's realm rather than a user's.
If a kernel panics, that is "crashes" before it can load up a system, it's messages are on the screen, and may or may not be left in a file depending on how things are configured. Usually such a crash (panic) hasn't got the system up, so it can't write its output to a file on the system, but it's possible to capture the kernel output on a serial connection if its configured and attached to the system.
In relation to deleting or truncating log files, you can find out which files are largest in just the /var/log directory with the command:
Code:
ls -alh /var/logand for the disk usage of the whole /var/log directory and its subdirectories with (as root):
Code:
du -hx --all /var/logThat may be a long list which may be useful depending on what you wish to know. In the example below, the screen output is more economical and may be enough to be informative.
In relation to truncating or deleting log files, it's possible to use configurations rather than brute force deletions as shown in the following example.
It's the directories in /var/log that tend to hold the most data, so the following output shows the size of the directory and its subdirectories on this machine:
Code:
[root@fen ~]# du -hxd1 /var/log
49M ./journal
64K ./wicd
20K ./lightdm
4.0K ./firebird
596K ./apt
16M ./installer
84K ./cups
8.0K ./hp
8.0K ./runit
4.0K ./private
8.0K ./sysstat
4.0K ./speech-dispatcher
48K ./exim4
79M .The journal directory holds the most data (49M). It's controlled by a configuration in the file:
/etc/systemd/journald.conf.
On this machine the configuration: SystemMaxUse=50M, has been written into the journald.conf file to limit the journal size to 50M. If that limit is not set by the user, the journal can grow very large, even gigabytes.
Other configuration to limit log files are through configuring log rotation, as mentioned by dos2unix above.
Last edited:
