[info] Encryption as a way to circumvent merging of archives

[rado84]

Some of you have probably noticed that Linux doesn't overwrite an old archive with a new one but it actually merges both files. But, depending on the contents of both files, the result might look like an overwritten file. If the directory and file structure and names inside both archives is the same, you won't see the difference. But if you have an archive like mine which changes often (CLI-SCRIPTS-AND-ALIASES.7z) and you attemt to replace the old archive with the new one, that's when you'll see and feel the difference which results in nothing short of a mess inside the archive - duplicating files and directories with different content and then you have to waste time to figure out which is the latest and which you have to delete.

One way to avoid this merging is to delete the old file first. But there's an easier way - one that will work just like on Windows where the old file is being completely overwritten by the new file. And the reason for that will be because linux can't see what's inside the two archives. When that happens, by default linux replaces the old archive with the new one.

1. GUI version.
Right click on the files/dirs you wanna archive, click "Compress", input a filename. Then click the checkbox which should read something along the line of "encrypt the files in the archive". Then enter whatever password you want. It doesn't have to be anything complex, even the digit "1" is enough for the encryption to work.
The result of that encryption when you open the archive is that you'll be met with a small dialog to enter the password. If you enter the wrong password or don't enter one at all, you'll only see the empty window of the archive manager you're using - no files, no anything.
The same way you can't see what's inside that archive, Linux can't see it as well. As a result, as I already mentioned, if you copy-paste the new archive on top of the already existing one, Linux will overwrite the archive, it won't merge both into one.

2. CLI version.
cd to whatever you wanna archive. In this example I'll use my directory with playlists.

7z a -p1 -mhe -mx -mmt=20 PLAYLISTS.7z PLAYLISTS

7z a - tells p7zip (or 7zip) to create an archive.
-p tells it to use a pasword. And yes, the password must be adhered to the character "p". Otherwise (depending on the p7zip version) you'll have a password " 1" (space1), or p7zip won't accept the password, or will simply skip the creation of one and move on to the next parts of the command.
-mhe - enable file encryption. It can also be used as "-mhe=on", both variants work.
-----------------------------------------
These three flags above are the mandatory. The next two are optional:
-----------------------------------------
-mx (by default -mx and mx=9 are one and the same which means "maximum compression". The number vary from 0 to 9 where 0 means no compression. If you don't include the -mx flag, by default will use standard ("normal") compression, which I think equals 5.
-mmt=20 tells p7zip how many threads to use.

I know it looks like a lot of information and hard but really it is not. If you do it just once or twice, you'll see how easy it is.
The CLI version can be used as an alias which makes it even easier and 5-6 times faster than the GUI verison.

 

[APTI]

I am a big opponent of encrypting drives. Only for real life ultra sensitive stuff. Using encryption can be dangerous to your data and losing it. Doing it for something like that just makes room for more problems. Just my opinion.

 

[rado84]

Errr... which part of my entire post suggests encrypting whole drives? ‍
I was talking about archive files, not drives.

 

[APTI]

Errr... which part of my entire post suggests encrypting whole drives? ‍
I was talking about archive files, not drives.

Same issue, but as I said. old school brain. encryption has caused so much issue for many of my clients. You are free to do as you wish. Just saying that it is a dangerous area to play. Seems that just deleting the old would be safer and easier.maybe in a script to automate things.

 

[rado84]

I've been doing that for nearly 4 years already, so apparently I know what I'm doing, unlike your clients.

 

[APTI]

I've been doing that for nearly 4 years already, so apparently I know what I'm doing, unlike your clients.

ok, wow. no need for the hostility. Not having a great day today so please let me know how I provoked you to have such hostile responses to me.

Thanks.

 

[rado84]

ok, wow. no need for the hostility. Not having a great day today so please let me know how I provoked you to have such hostile responses to me.

Thanks.

No hostility here. There's an old saying among ITs: "Behind every broken computer stands one user who did nothing".

 

[CaffeineAddict]

I am a big opponent of encrypting drives. Only for real life ultra sensitive stuff. Using encryption can be dangerous to your data and losing it.

encryption has caused so much issue for many of my clients. You are free to do as you wish.

I'm aware you're computer guy and having a PC shop, but honestly in addition to fixing their problems I think you should instruct your clients on how to use encryption rather than telling them not to use it.

There are many people, my self included who use encryption on various level without issues, so far I had never had a problem and if I did (I don't recall) it was as simple as restore my backup.

 

[Trml]

I agree with all of you let me explain:

I think @rado84's tip is a smart, easy to remember hack to work around a common problem with a simple task. Still, it is a hack and done manually easy to mistype (e.g. GUI), which is when @APTI is correct - the archive is worth nothing when you mistype the password, yet you rely on it. In the past, email worms liked to use simple encryption for attachments, so they avoid antivirus engines. It's another reason you might run into trouble (when you email the playlists, they might land in spam).

Yet, I also agree with @CaffeineAddict, for the same reason. Encryption takes management, yes, but Linux has all sorts of very proven, well tested encryption methods. On a business level you can create recovery keys with all major methods, you could even establish a key escrow for clients. Obviously, it still depends on the type of data clients need protection for. For example, it might make no big sense to encrypt a gaming PC that resides in a private lounge. Purpose dictates measures. However, if it is business clients, caring for encryption is more or less a must these days in my opinion. Most modern PC storage devices are self-encrypting, transparent to the user. The least thing a business might want to do is secure their respective settings during setup, so that they are in a defined state, non-modifiable by a rogue. Yet, this does not affect any regular to-dos like backups. A business should have a risk assessment whether it needs to encrypt backups. Now, if they had encryption on the file level, like in @rado84's example, or integral to the filesystem, they might not need to extra encrypt individual backups - just execute care for the respective keys. It's a cycle and all depends on what your individual decisions are.

Back to the tip: what I would do is a simple alias that prepends a "mv PLAYLISTS.7z PLAYLISTS.7z.old & " prior to the 7z command - but without file encryption. This way you always keep one backup & a fresh archive.

 

[rado84]

I agree with all of you let me explain:

I think @rado84's tip is a smart, easy to remember hack to work around a common problem with a simple task. Still, it is a hack and done manually easy to mistype (e.g. GUI), which is when @APTI is correct - the archive is worth nothing when you mistype the password, yet you rely on it. In the past, email worms liked to use simple encryption for attachments, so they avoid antivirus engines. It's another reason you might run into trouble (when you email the playlists, they might land in spam).

Yet, I also agree with @CaffeineAddict, for the same reason. Encryption takes management, yes, but Linux has all sorts of very proven, well tested encryption methods. On a business level you can create recovery keys with all major methods, you could even establish a key escrow for clients. Obviously, it still depends on the type of data clients need protection for. For example, it might make no big sense to encrypt a gaming PC that resides in a private lounge. Purpose dictates measures. However, if it is business clients, caring for encryption is more or less a must these days in my opinion. Most modern PC storage devices are self-encrypting, transparent to the user. The least thing a business might want to do is secure their respective settings during setup, so that they are in a defined state, non-modifiable by a rogue. Yet, this does not affect any regular to-dos like backups. A business should have a risk assessment whether it needs to encrypt backups. Now, if they had encryption on the file level, like in @rado84's example, or integral to the filesystem, they might not need to extra encrypt individual backups - just execute care for the respective keys. It's a cycle and all depends on what your individual decisions are.

Back to the tip: what I would do is a simple alias that prepends a "mv PLAYLISTS.7z PLAYLISTS.7z.old & " prior to the 7z command - but without file encryption. This way you always keep one backup & a fresh archive.

It seems you're the only one who understands what the point of this topic is.

1. As I clearly stated in the original post, the password can be as simple as just the digit 1. Just -p1, so you have to press the key "1" - left from "2" in order to decrypt the archive. What's so hard about that? This simple password is only meant to prevent linux from seeing what's inside the file, thus preventing it from merging the old and new archives.
2. This topic has nothing to do with business - it's about a workaround of a function in Linux which personally I find rather stupid and extremely annoying.
3. I also mentioned it's mostly related to archives whose content and dir structure changes often, like my backup of aliases and scripts.

$ clicount
Total number of aliases is: 273.
Total number of files in '/B/CLI/SCRIPTS' is: 207 bash scripts.

This many scripts and aliases most definitely need a proper backup! Preferably one without merging mess of archives.
4. Encryption is bad only in incapable/the wrong hands. I've been encrypting files for a few years already, never had an issue with the password. And it's not a simple password either. Today's supercomputer will need 16,000 years to crack that password...
5. I gave both GUI and CLI variants, so whomever finds this a useful tip can decide which variant to use. I prefer CLI bc CLI doesn't rely on frameworks + it's 5-6 times faster than the GUI. Especially if you append an "exec" in front of the script, it will be completed before you can even say your first name.

I thought someone else might benefit from my finding. Clearly I was wrong. Looks like it's better from now on to keep my findings for myself and let everyone else manage on their own, if they can. Especially expert opinions who have never tried encryption bc they're scared of it.

 

[APTI]

I agree with all of you let me explain:

I think @rado84's tip is a smart, easy to remember hack to work around a common problem with a simple task. Still, it is a hack and done manually easy to mistype (e.g. GUI), which is when @APTI is correct - the archive is worth nothing when you mistype the password, yet you rely on it. In the past, email worms liked to use simple encryption for attachments, so they avoid antivirus engines. It's another reason you might run into trouble (when you email the playlists, they might land in spam).

Yet, I also agree with @CaffeineAddict, for the same reason. Encryption takes management, yes, but Linux has all sorts of very proven, well tested encryption methods. On a business level you can create recovery keys with all major methods, you could even establish a key escrow for clients. Obviously, it still depends on the type of data clients need protection for. For example, it might make no big sense to encrypt a gaming PC that resides in a private lounge. Purpose dictates measures. However, if it is business clients, caring for encryption is more or less a must these days in my opinion. Most modern PC storage devices are self-encrypting, transparent to the user. The least thing a business might want to do is secure their respective settings during setup, so that they are in a defined state, non-modifiable by a rogue. Yet, this does not affect any regular to-dos like backups. A business should have a risk assessment whether it needs to encrypt backups. Now, if they had encryption on the file level, like in @rado84's example, or integral to the filesystem, they might not need to extra encrypt individual backups - just execute care for the respective keys. It's a cycle and all depends on what your individual decisions are.

Back to the tip: what I would do is a simple alias that prepends a "mv PLAYLISTS.7z PLAYLISTS.7z.old & " prior to the 7z command - but without file encryption. This way you always keep one backup & a fresh archive.

Way to go. moderate and middle of the road. Just so you know I am speaking on encryption in general and not singling out any one particular type or method. obviously some are better than others.