Debian Package Management - APT, *.deb, and Friends

Discussion in 'Installation' started by DevynCJohnson, Jan 26, 2014.

  1. DevynCJohnson

    DevynCJohnson Super Moderator Staff Member Staff Writer

    Messages:
    951
    Likes Received:
    848
    Trophy Points:
    93
    Many Linux distros are Debian-based. This group of Linux systems use the Debian installation packages (*.deb files). It helps to understand how to manage applications on such systems. Knowing the apt and dpkg software can greatly help system administrators and well as the average desktop user.


    Advanced Packaging Tool (APT)

    Apt is equivalent to Yum on RedHat-based systems; apt is to Debian as yum is to RedHat. Apt is commonly used to download software from repositories. Repositories are special servers that store installation packages for operating systems. For example, the command "apt-get install firefox" will download Firefox from a repository and install the software. Firefox can then be removed in one of two ways

    Keep configuration files - apt-get remove firefox
    Remove everything - apt-get purge firefox

    NOTE: To manage software, the user must be logged in as Root or use Root privileges via sudo or su.

    The update command used by Apt refreshes the package index by downloading new index files (apt-get update). Apt is also used to download and apply updates to the local software - "apt-get upgrade". To apply updates that require other packages in the newer version, use "apt-get dist-upgrade".

    When packages are being downloaded, they are stored in /var/cache/apt/archives/. Then, after all of the desired packages are in that directory, the installation process can begin one package at a time. Apt cannot install multiple packages at the exact same moment meaning that one is installed at a time. After the installation process, the *.deb files still remain in /var/cache/apt/archives/. To remove the cached Deb files that can no longer be installed on the system, execute "apt-get autoclean". To clear the cache, execute "apt-get clean".

    Pinning is the act of specifying that a particular version of software be installed rather than the mainstream/newest version. This can also refer to specifying a piece of software from a non-default repository (repo). For example, an user may pin Firefox v21 to be the installed version even though Firefox v27 may be released. Or, the user may specify that only Firefox packages from repository "X" can be used. Pinning (also called apt-pinning) can easily be done in the GUI frontends (each frontend has different steps for performing a pin). Without a frontend, pinning is specified in this file - /etc/apt/preferences.

    apt_pin.png

    If a user wanted to compile a program, but needed all of the dependencies for building the application, then use "apt-get build-dep PROGRAM_PACKAGE_NAME" which will install the build dependencies. This will only work if the specified package is in the repositories.

    One feature that makes Apt wonderful is that it can manage and find dependencies. This prevents the user from needing to manually find and install each and every dependency. The command "apt-get check" will see if any broken-dependencies exist on your system. A broken-dependency is the same as an unmet dependency, both of which mean a dependency is missing or there is a conflict among dependencies. If one occurs, there are a few ways to resolve the problem. One method is to use the command "apt-get -f install". Another way is to install the dependencies manually before installing the main software. If the issue results when many packages are being installed at once, then install a few packages at a time.

    Sometimes, users may wish to configure an application an alternative way or hope to fix odd application behavior by reconfiguring the package. This is done using the "dpkg-reconfigure PACKAGE" command. To re-install a package, use this command - sudo apt-get --reinstall install PACKAGE

    If users wish to clean the operating system, the libraries and other software that is no longer needed can be removed via "sudo apt-get autoremove". For illustration, assume many applications are installed along with their dependencies. Later, many of these applications are removed, but the dependencies are left on the system. No application is using this software, so these files are only wasting disk space. Removing the unneeded dependencies will free disk space and it may make the system perform better.

    Various frontends exist for Apt. A frontend is software that provides an interface or an alternate platform for using a particular piece of software. Synaptic, Ubuntu Software Center, and Aptitude are some examples of popular Apt frontends.

    APTonCD is a program that makes an ISO file containing all of the DEB files in the cache directory. This is useful when installing many applications after a fresh Linux install. Then, the user can have a copy of the installed software on a disk (if the ISO is burnt to a disk). This allows the user to install the same software off of the disk onto other Debian systems.


    dpkg

    Dpkg is equivalent to rpm (the command, not the file) on RedHat-based systems. To install a local Deb file, type "dpkg -i FILENAME" and to remove a package, type "dpkg -r PACKAGE".

    To view the contents of a Deb file, use the "dpkg -c FILENAME" command. The output will look similar to executing "ls -l".

    dpkg_c.png

    A Deb file can be extracted like this - dpkg -x FILE TARGET_DIR - However, this will only extract the files. For the control file, hash sum, and other metadata, try this command - dpkg -e FILE TARGET_DIR. Yes, this is the same, but with an "e" as the parameter.


    wpkg

    Debian-based Linux distros are not the only ones that use DEB files. wpkg is a program that allows Deb files to be installed on Windows. With wpkg, Debian files can be used on a Windows system just like on a Debian system. (http://windowspackager.org/documentation/wpkg)

    NOTE: If anyone has tried this, please let us know. I do not believe that Debian software would work on Windows systems, but please feel free to prove me wrong. Do not be afraid; I will like it a lot. I mention this software in this article to inform readers of its existence.


    Deb files (*.deb)

    The Debian software package format is based off of the ar-archive format. The current Debian-file format version is 2.0. The Debian file contains two tar files, one containing the application and the other containing the control information (program info, description, install size, dependencies, etc.). The Deb file may use one of four compression formats - Bzip2, gzip, lzma, and XZ. However, compression is optional.

    To see the actual tar files (the two mentioned above), use this command - ar -x FILE.deb - This command will also pull out a plain-text file stating the version of the Debian format. Users will see that each tar may be compressed. In addition, the tars may each use a different compression format.

    Repositories (Repos)

    As mentioned previously, repositories are a collection of installation packages. There are different types of repositories that offer different packages.

    repo.png

    Stable - The current/mainstream software is found here.

    Proposed - This repo contains updates that are being tested. BEWARE: the software may cause system conflicts and contain bugs.

    Backports - Newer software.

    Testing - Candidates for release are stored here. This software needs some more testing before being deemed stable. BEWARE: the software may cause system conflicts and contain bugs.

    Unstable - Packages with known bugs are stored here. Obviously, this is dangerous software that should only be used by developers on dedicated computers. BEWARE: the software may cause system conflicts and contain bugs.

    Oldstable - This software is stable. This repo is perfect for old systems where security is not a concern.

    Snapshot - These repos contain old versions of software. Users needing some very old software will find this repo useful.


    Experimental - The software here is highly experimental and in the early stages of development.

    Non-free - Proprietary packages or software that does not pass the Debian Free Software Guidelines (DFSG) belong here.

    Contrib - If a piece of software passes the DFSG, but depends on a package from "Non-free", then such a package goes here.

    Within repos are further divisions called ports. For example, free stable software for SPARC processors would come from the SPARC port in "Stable". There are stable ports, unstable ports, and unofficial ports. “Stable” ports contain packages for well supported processors (i386, AMD64, MIPS, etc.). “Unstable” ports are still being developed (hurd-i386). “Unofficial” ports are unsupported except by small third-parties (ppc64, alpha, sh4, etc.).

    The file that stores a list of the repositories used by the system is /etc/apt/sources.list. Lines that are commented are either comments or disabled repos. Here is a partial file -

    Code:
    # deb cdrom:[Ubuntu 13.10 _Saucy Salamander_ - Release amd64 (20131016.1)]/ saucy main restricted
    
    # See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
    # newer versions of the distribution.
    deb http://us.archive.ubuntu.com/ubuntu/ saucy main restricted
    deb-src http://us.archive.ubuntu.com/ubuntu/ saucy main restricted
    
    ## Major bug fix updates produced after the final release of the
    ## distribution.
    deb http://us.archive.ubuntu.com/ubuntu/ saucy-updates main restricted
    deb-src http://us.archive.ubuntu.com/ubuntu/ saucy-updates main restricted
    
    ## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
    ## team. Also, please note that software in universe WILL NOT receive any
    ## review or updates from the Ubuntu security team.
    deb http://us.archive.ubuntu.com/ubuntu/ saucy universe
    deb-src http://us.archive.ubuntu.com/ubuntu/ saucy universe
    deb http://us.archive.ubuntu.com/ubuntu/ saucy-updates universe
    deb-src http://us.archive.ubuntu.com/ubuntu/ saucy-updates universe
    

    Now, Debian-based systems should be easier to understand and manage. For those of you that want to develop or contribute software to the Debian community may find the organization of the repos helpful.

    Attached Files:

    • slide.jpg
      slide.jpg
      File size:
      45.9 KB
      Views:
      123,557
    Lotfi-GHAZOUANI likes this.
  2. labrat

    labrat Active Member

    Messages:
    106
    Likes Received:
    43
    Trophy Points:
    28
    It would be better to omit sudo altogether and use code tags for your commands - would make the article more readable. You could cover the methods for gaining root privileges in a short sentence covering su and sudo.

    It would flow better to start with update and move to upgrade and dist-upgrade. Also your explanations of the latter two options is a little glossed over. I would suggest reading the man pages for apt-get to understand the difference.

    The sources file is also not the index file, just a list of repositories (servers).

    Index files are downloaded from the repository mirror when you issue:
    Code:
    # apt-get update
    wiki debian org / SourcesList

    This is incorrect - "apt-get autoclean" removes obsolete packages/versions (packages which can no longer be downloaded) from the cache.

    Without any expansion on this or practical examples of apt pinning, this section is just acting as padding.

    You should refrain from alluding to "administrators" - the user is more often than not the "administrator" and anyone reading this article is likely to be a user who wants to learn more about package management. dpkg-reconfigure belongs under the dpkg section as a subsection.

    Misleading and over simplified. You would do better to focus on what it does, rather than supposed, unproven advantages. autoremove simply gets rid of orphaned dependencies, if you add --purge it will also get rid of configuration. (aptitude does an autoremove along with removal of a package anyway.)

    Also misleading "better" is subjective. None of those provide a "better platform", just a different approach or UI. Bog standard CLI driven apt, is the most powerful and versatile.

    This has nothing to do with Debian or apt, I would suggest removing it.


    I have no idea where this info is from, but your definitions are way off... and you've mixed terms and confused repositories with branches or distributions.

    In particular backports is not at all what you say it is. In fact you have missed the entire definition of a backport and don't seem to know what a backport actually is. I would suggest reading the Debian wiki on backports.

    Some in depth reading and not to mention hands on experience, is needed before writing a section like that.

    It's not enough to just put together articles and post them online as flawed and full of glaring omissions and as error strewn as this one is. While you continue to put out articles of this quality, you will not be taken seriously.

    I would suggest reading and actually learning how to use a Debian system first, before writing articles about it.
    Last edited: Feb 27, 2014
    DevynCJohnson likes this.
  3. DevynCJohnson

    DevynCJohnson Super Moderator Staff Member Staff Writer

    Messages:
    951
    Likes Received:
    848
    Trophy Points:
    93
    Thank you for the critic. I will fix this soon. Thanks!
  4. labrat

    labrat Active Member

    Messages:
    106
    Likes Received:
    43
    Trophy Points:
    28
    I don't think you understand... you need to pull this down and start from scratch. If you want to redo this and need advice, PM me and I'll help you out.
    DevynCJohnson likes this.
  5. Cyber-Berserker

    Cyber-Berserker Active Member

    Messages:
    242
    Likes Received:
    138
    Trophy Points:
    43
    I finally got around to reading this article.
    1) I agree with all of labrat's criticisms, especially
    and concerning repositories.
    Proposed
    How many distros have such repositories?
    What? This left me shaking my head.
    Again, what? Something for developers to use? Are you aware of Sid (Debian Unstable)?
    Old Stable gains software? The only thing Old Stable will gain is the occasional security update. Unless there is a strange distro or two that add software to their Old Stables.

    Just like putting sudo in front of shell commands gives the wrong impression to newcomers that all Linux systems use Ubuntu's retarded procedure of sudo abuse, this section likewise gives the erroneous impression that all distros have the same repository structure and use the same nomenclature. After replacing the misinformation with facts, the repo section needs to be re-organised.

    2) You should also stop doing this:
    Dpkg is equivalent to rpm (the command, not the file) on RedHat-based systems; dpkg is to Debian as rpm is to RedHat.

    Apt is equivalent to Yum on RedHat-based systems; apt is to Debian as yum is to RedHat.

    Why the repetition?

    3) Apt cannot install multiple packages at the same moment.
    Misleading.
    Code:
    apt-get install openbox obmenu tint2
    All three packages will be installed. Not simultaneously, but in succession in one procedure.
  6. labrat

    labrat Active Member

    Messages:
    106
    Likes Received:
    43
    Trophy Points:
    28
    This was the PM I sent to Devyn 04/03/14:
    No response... and seemingly no interest in correcting the errors.

    Though I know he means well, these types of ill researched articles will not only damage his reputation but also the reputation of this site.
    Last edited: Mar 16, 2014
    Cyber-Berserker likes this.
  7. labrat

    labrat Active Member

    Messages:
    106
    Likes Received:
    43
    Trophy Points:
    28
    No - still wrong. Not even close in fact...

    My suggestion? Just delete this article and start again - but before that actually use a Debian system...

    Or to put it another way - who the fsck writes big articles about software, a distribution or OS they've clearly little to no experience of...?
  8. Cyber-Berserker

    Cyber-Berserker Active Member

    Messages:
    242
    Likes Received:
    138
    Trophy Points:
    43
    I agree that the article needs to be re-written, preferably by someone with more in-depth knowledge, but one point in Devyn's defence is that the article is not just about Debian, but Debian-based systems. This creates a problem for the author, whoever that is, because the material covered by the article is very broad in scope. It includes systems from Debian to Ubuntu to Scientific Linux (I believe). That makes extensive knowledge/research necessary. I good idea would be to replace the article with a few shorter ones. One focusing on APT, including differences in usage between distros; another focusing on repositories, both in general and comparing between distros; etc. And the articles do not need to be all written by the same person. I suggest delegating the composition of the articles to people according to their expertise. For example, an article concerning repositories could be written by someone with experience using several distros. (Debian, Ubuntu, Mint, etc. Using several Buntus is not sufficient experience. The author would need to be familiar with a wide range of systems.)

    Only advice. Take it or leave it.
  9. Cyber-Berserker

    Cyber-Berserker Active Member

    Messages:
    242
    Likes Received:
    138
    Trophy Points:
    43
    Some examples of how the repository section could be cleaned up:

    Stable - Packages thoroughly tested and available for the current release.

    Backports -(Debian's definition copied from the backports web page.)
    For the purposes of this kind of article, it could be summed up as follows:
    Backports are packages taken from the next (future) stable release (often called testing), adjusted and recompiled for usage on the stable release. Because the packages will be present in the next release, they can be easily upgraded when the next Stable release comes out.

    Backports cannot be tested as extensively as the packages in Stable, and are provided on an as-is basis, with risk of incompatibilities with other components in Stable. Use with care!

    Unstable - Where active development of the distribution occurs. Generally, this sytem is run by developers and those who like to live on the edge. Packages with known bugs can be stored here. Obviously, this is dangerous software for those who are not knowledgeable about the system, and therefore, should only be used by those who know what they are doing.

    Oldstable - The previous release. Still receives security updates, but does not receive software updates.

    Non-free -(Copied from the Debian Policy Manual. Could be shortened for continuity with the rest of the article.)
    The non-free archive area contains supplemental packages intended to work with the Debian distribution that do not comply with the DFSG or have other problems that make their distribution problematic. They may not comply with all of the policy requirements in this manual due to restrictions on modifications or other limitations.

    Packages must be placed in non-free if they are not compliant with the DFSG or are encumbered by patents or other legal issues that make their distribution problematic.

    In addition, the packages in non-free
    • must not be so buggy that we refuse to support them, and
    • must meet all policy requirements presented in this manual that it is possible for them to meet.
    Contrib - The contrib archive area contains supplemental packages intended to work with the Debian distribution, but which require software outside of the distribution to either build or function. (Copied from the Debian Policy Manual.)
    DevynCJohnson likes this.
  10. labrat

    labrat Active Member

    Messages:
    106
    Likes Received:
    43
    Trophy Points:
    28
    Irrelevant.

    Debian package management is the same thing irrespective of distribution - apt/dpkg, .deb files, sources.list, etc. It simply takes a little research to find out what each repo does. Also he made no mention of the 'buntu sources - so it's hardly a "Debian-based systems" article.

    In a Debian system there are repositories, every repository is typically split into three sub repositories:

    main

    - Fully DFSG compliant software (open source and source included)

    contrib

    - Free and open source software which requires or installs non-free software

    non-free

    - Non DFSG compliant software - e.g. proprietary drivers, flash, etc.

    As of writing the current repositories are:

    wheezy (stable)

    - The stable distribution - packages here very rarely get version updates, instead bug fixes are backported from later versions.

    - Supported for roughly two years as stable and about one more year once it becomes oldstable (Debian does not use time based releases, so this is just a rough guide).

    - Stable means - "less prone to change". If that's still not clear enough - a stable system does not get a constant stream of updates, so the core system remains the same - this makes it suitable for production servers/workstations.

    wheezy/updates

    - Security updates - important patches for vulnerabilities, etc.

    https://www.debian.org/security/faq

    wheezy-updates

    - Once known as the "volatile" repository - this repository contains updates such as AV databases, tzdata, etc.

    wheezy-proposed-updates

    - The best description is here: https://wiki.debian.org/StableProposedUpdates

    - Or for the lazy - this is means to get the Debian point releases earlier.

    - This can be considered optional.

    wheezy-backports

    - Packages which have been backported from the testing branch.

    - backports is not a complete distribution nor should anyone try to install all packages from backports.

    - More info: http://backports.debian.org/

    jessie (testing)

    - Auto-generated distribution made up of software which migrates from unstable after passing certain criteria (release critical bugs - Debian wiki has more info).

    - Many users run testing as a means to get newer software than the stable release. There is a myth that testing is "stable" and never breaks and you will read about users who have been running it for 10 years with zero problems. These users usually arrive to tell you this life story once testing has broken...

    - Packages can just disappear from testing due to bugs - i.e. testing depends on unstable so if a package can't get through unstable for whatever reason, it will disappear from testing if dependencies cannot be satisfied there.

    jessie/updates

    - Security updates - not as important for testing as it is for stable.

    jessie-updates
    jessie-proposed-updates
    jessie-backports

    - See wheezy/stable - usually only become active once testing freezes and goes stable.

    unstable (sid)

    - This is the first stage for any new version entering the Debian repositories - packages from incoming arrive here after being accepted.

    -Unstable is a complete distribution, so it's possible for more experienced users - or those who are prepared to take the risks - to run a system which is sourced entirely from this repository.

    - It's recommended to run apt-listbugs and apt-listchanges and check thoroughly before saying yes...

    experimental

    - Highly experimental software, e.g. gnome-shell (v3.x) spent a lot of time in here when it was first introduced. If anything is to be installed from here it should only be installed in a testing or unstable system.

    - Not a complete distribution - i.e. you can't run an experimental system

    - Most packages bypass this and go straight to unstable

    testing/unstable/(experimental)

    - These three repositories can be mixed, if you know what you're doing - discussing this is beyond the scope of this forum post. The stable distribution cannot be mixes with any of this unless you actually want a broken system.

    - If something breaks in unstable (it happens), it's often necessary to downgrade to the version in testing and then pin or hold to that version - again beyond the scope...

    squeeze - (oldstable)

    - The previous release - still supported with security updates for roughly one year after the stable release.

    squeeze-updates
    squeeze-proposed-updates

    - See wheezy

    snapshot.debian.org

    - Not a distribution but snapshots of repositories from given dates. Useful to users who need to revert back to a particular version at a particular point in time to overcome a bug (useful when the buggy package has made it through unstable into testing).

    http://snapshot.debian.org/

    The Debian Archive

    - Once oldstable support ends, the old unsupported distribution will be shipped off to the archive along with, sarge, etch, lenny, et al....

    - Unsupported distributions can contain security vulnerabilities.

    wiki: http://www.debian.org/distrib/archive
    howto/more info: http://archive.debian.net/
    Last edited: Mar 18, 2014
  11. DevynCJohnson

    DevynCJohnson Super Moderator Staff Member Staff Writer

    Messages:
    951
    Likes Received:
    848
    Trophy Points:
    93

Share This Page