Linux Debian 10 deleting /etc/ folder on running some processes

Raheel Hirani

New Member
Joined
Jan 12, 2022
Messages
1
Reaction score
0
Credits
14
Hi Guys,

There is an issue we are facing since a month that as soon as we ran some processes it disconnects our SSH and other sessions after sometime and we are not able to connect to machine.
We checked it on AWS instances running Debian 10 as well as dedicated servers running Debian 10. When we asked the support there they said somehow /etc/ folder is being deleted.

What could be the potential root cause of it?

Please help or suggest what to do in this case..
 


Alexzee

Well-Known Member
Joined
Jun 1, 2019
Messages
2,207
Reaction score
1,075
Credits
11,256
If I had to guess the root cause of it was that it was deleted accidentally.

If you have a backup of /etc you can restore it with a Live cd.
That is if you didn't change anything in the servers configuration.

Otherwise you will have to perform a fresh installation.
 

f33dm3bits

Gold Member
Gold Supporter
Joined
Dec 11, 2019
Messages
4,446
Reaction score
3,158
Credits
32,240
If you already have technical support from AWS why are you asking here, how did AWS technical support verify that?? Who is we? First thing I would do is check the system's syslog/journal. There is no way to guess since you should know more about your system then the people you are asking for help. It could be a script running as soon as you login in that does something.
 

carlarogers

Member
Joined
Jan 1, 2020
Messages
45
Reaction score
10
Credits
399
I deleted /etc before. That was a bad move. I didn't mean to. I cannot remember whether I was able to restore from a backup or start with a new installation. The main thing I remember is it was really super bad and it pays to not do it again.

Root cause is a funny term, because there always is a cause for every root cause, so where the is root cause? The answer comes down to the answer that is of practical use in fixing the problem in a way that reduces the chance for recurrence to as close to zero as possible. It always is a judgement call.

for instance, I will nominate the interest you and your oranization have in running computers as the root cause. Give up computers, the problem is gone forever. Can't do that? Is there a money motive? So money is the root cause...

In my case, I settled on something more practical as the root cause. My fingers. They did it. I could say that for sure becaue no one else had access to the system in a way that could have done that.

I really don't remember how I made the mistake.

There is something else I can add. There is not a common problem that blows away a critical directory. That is a problem a lot of sys admins never experience. If it happens frequently, you won't be able to continue in this line of work. It's one of the worst things that can happen.

I can add something even more. When I did it, i was able to identify immediately what I did that caused it. I was working too fast and did something stupid by mistake. A lot of people set things up to make this kind of mistake impossible to happen. One of the reasons for using sudo is using it always keeps a log of every action take and the userid that took it.

Basically, you and the people you work with have to look at the actions you were taking, the scripts you were working on, whatever it was, and I would think it would be relatively straighnt forward to identify actions that have things going on that if mixed up coiuld have caused thje problem. If you are drawing a blank, it means either you are moron for no knowing your dangerous moves, OR if there are others who work on the system with root permissions, and you are sure it coud not have been you, it still could have been you, but you cannot rule out that one of your colleagues is not owning up to what they were doing.

I always have to assume a screw up like that is something i did, even if I think "no way it was me." The worst is when I am sure it wasn't me, but in honesty, i was doing something that could have gone wrong, but I don't see how I did anything wrong. In cases like that, half the time at least it was me. Sometimes, I know it could not have been me. If you are the only one with access then it was you. If you did nothing to the system when this happened, and others have access, i would start thinking it was someone else, no fessing up. People will do that if the can get away with it. When I was a manager, my best guy deleted a ton of stuff he should not have and of course no one told me. I found out later. People make mistakes and fessing up is not hte best move in all situations. If that guy fessed to me, I might have felt compelled to tell my boss. My boss would have hammered him. There was a good reason he was deleting stuff, he was trying to deal with our system not having enough disk. The disk for that system was this special mirroed fault tolerant rig, in the early 90s, cost $35 K for 1 gig, no lie. So in that environment, where we were squeezed, things were going to happen.

Overall, I am going to guess the shop where this happened is not the tightest running shop in your tri-state area whatever kind of area you have. Dropping /etc is about as bush league as it gets. Don't do it again. Find out how it happened. not possible to guess how it happened, except to say whatever it was that someone was doing, they did it wrong size large.
 
$100 Digital Ocean Credit
Get a free VM to test out Linux!


Top