Linux Containers: Part 3, Tools of the Trade

Discussion in 'Linux Other' started by Eric Hansen, Aug 9, 2013.

  1. Eric Hansen

    Eric Hansen Member

    Messages:
    95
    Likes Received:
    38
    Trophy Points:
    18
    There are a lot of tools that comes with the LXC userspace. For me, these are my options:
    Code:
    ls /usr/bin/lxc-*
    
    /usr/bin/lxc-attach      /usr/bin/lxc-checkpoint  /usr/bin/lxc-destroy  /usr/bin/lxc-kill    /usr/bin/lxc-netstat  /usr/bin/lxc-setcap    /usr/bin/lxc-start-ephemeral  /usr/bin/lxc-version
    /usr/bin/lxc-backup      /usr/bin/lxc-clone      /usr/bin/lxc-execute  /usr/bin/lxc-list    /usr/bin/lxc-ps      /usr/bin/lxc-setuid    /usr/bin/lxc-stop            /usr/bin/lxc-wait
    /usr/bin/lxc-cgroup      /usr/bin/lxc-console    /usr/bin/lxc-freeze  /usr/bin/lxc-ls      /usr/bin/lxc-restart  /usr/bin/lxc-shutdown  /usr/bin/lxc-unfreeze
    /usr/bin/lxc-checkconfig  /usr/bin/lxc-create      /usr/bin/lxc-info    /usr/bin/lxc-monitor  /usr/bin/lxc-restore  /usr/bin/lxc-start    /usr/bin/lxc-unshare
    While it might seem daunting, you likely won’t be using most of these much (if at all), and some aren’t even for containers. For example, the version tool just tells you what version of LXC you’re running (0.7.5 for me if you’re curious). However, some them are also quite useful, and I’ll cover what I can here. I won’t go over every tool, but some of the most beneficial ones will be talked about and others that you can mostly try out for yourself.

    First I would like to go over what tools aren’t supported or otherwise aren’t advisable to run on Ubuntu (and possibly other) host distros:
    Not Supported

    • lxc-attach - Lets you run a command in a running container
    • lxc-checkpoint - Similar to VMware or VirtualBox’s “Snapshot” feature, lets you create a checkpoint or restore point of a container
    • lxc-restart - Restart a checkpointed container
    Not Recommended

    • lxc-setcap - Sets the file capabilities on LXC tools
    • lxc-setuid - Sets or remove setuid bits on LXC tools
    lxc-backup and lxc-restore

    I would like to state that the Ubuntu docs state “[...] lxc-backup and lxc-restore are fragile with respect to customizations and therefore their use is not recommended.” I’m not sure in what way it means “customizations”, so fair warning.

    backup is an interesting program in that it will create a backup of a container (as well as version number) and make a copy of the entire root directory of the container’s directory (i.e.: if the container is linux_org, then the backup will be /var/lib/lxc/linux_org/rootfs.backup1/ if backup is ran with arguments “linux_org” and “1”). backup uses rsync to do the backing up of the filesystem, fair warning on that.

    To back up a container, the command is:
    Code:
    lxc-backup CONTAINER [BACKUP_NUMBER]
    Where “CONTAINER” is the name of the container you want backed up and “BACKUP_NUMBER” is an optional argument used to specify the suffix of the backup directory. If its non-numerical it will be incremented from the highest number found in the container’s directory.


    To restore you use the same parameters but use lxc-restore instead of lxc-backup. It will delete everything from the current rootfs and restore from the specified backup.
    lxc-clone

    This is extremely useful if you wish to have one base template container that you can mirror others from. The syntax is pretty simple:
    Code:
    lxc-clone -o <original container name> -n <new container name>
    So if you have a container currently named “bob” and you want to mirror that container to a new instance/container called “alice”, you would do this:
    Code:
    lxc-clone -o bob -n alice
    This is assuming that “alice” container doesn’t exist already.

    If your container’s a LVM or BTRFS backing store (as mentioned in part 2), you can also pass the “-s” argument to tell lxc-clone you want the new container to be generated from a snapshot of the original one.

    I can’t really tell a difference between using “-s” and not when the container’s backing store is “none” (default), but then again I also don’t use LXC through a LVM or BTRFS either.
    lxc-destroy

    Want to get rid of a container for good for whatever reason? This is the program for you!

    What this does is completely destroy the container and remove it from your system (so make sure you back up any important files you might have laying in one before you use this!). The required argument is “-n <container name>” but you can also pass along “-f” to tell LXC to force a deletion if the container is currently running (default is to abort/error out).
    lxc-shutdown

    Shutting down a container while you’re inside is easy, but if you don’t want to log in to the container just to shut it down you can use the lxc-shutdown command.

    This program takes the container’s name (“-n <container name>”) as a required argument but also accepts “-w” for the program to wait until the container is actually shut down, “-r” to reboot the container instead and “-t <seconds>” to wait <seconds> long before killing the container (not really suggested and the “-t” argument is from running lxc-shutdown -h).
    lxc-list

    This is probably one you’ll be using a lot to make sure containers are running and stopped (frozen containers won’t be listed here for some reason). It takes no arguments and will print everything out neatly for you. At some point you’ll probably notice that some might even be suffixed with a “(auto)”, which basically means that the container is set to automatically boot at load.
    lxc-freeze and lxc-unfreeze

    Suspecting that a container is causing issues in some fashion? lxc-freeze will let you basically pause the container (blocking all processes from running and such) to test, and lxc-unfreeze will start it back up. Both require the typical “-n <container name>” argument (most of these tools will). This is pretty useful also if you’re wanting to test some vulnerability sandboxes ;)
    lxc-info

    This is similar to lxc-list but requires the “-n <container name>” argument. It tells you the state of the container as well as it’s process ID on the host machine. If you’re unsure if the host is frozen or just not plain detected by LXC this will tell you even if it is frozen.
    lxc-ls

    Doesn’t require the “-n” argument and will list all containers that LXC sees (even the frozen ones). So if you see a difference between this and lxc-list, run a lxc-info on the disgruntled containers to investigate further.
    lxc-ps


    Just like “ps” for a normal *nix machine, it sends a request to the container (via passing the “-n” argument) the ps command and returns the output. If you wish to send specific arguments to “ps” itself inside of the container, add a “--” in front, like so:
    Code:
    # lxc-ps -n linux_org -- -ef
    
    CONTAINER  UID        PID  PPID  C STIME TTY          TIME CMD
    linux_org  root      5785  5779  0 14:15 ?        00:00:00 /sbin/init
    linux_org  root      5944  5785  0 14:15 ?        00:00:00 upstart-udev-bridge --daemon
    linux_org  root      5953  5785  0 14:15 ?        00:00:00 /sbin/udevd --daemon
    linux_org  syslog    5961  5785  0 14:15 ?        00:00:00 rsyslogd -c5
    linux_org  root      5968  5785  0 14:15 ?        00:00:00 upstart-socket-bridge --daemon
    linux_org  root      5999  5785  0 14:15 ?        00:00:00 dhclient3 -e IF_METRIC=100 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -1 eth0
    linux_org  root      6021  5785  0 14:15 ?        00:00:00 /usr/sbin/sshd -D
    linux_org  root      6049  5785  0 14:15 pts/10  00:00:00 /sbin/getty -8 38400 tty4
    linux_org  root      6053  5785  0 14:15 pts/8    00:00:00 /sbin/getty -8 38400 tty2
    linux_org  root      6054  5785  0 14:15 pts/9    00:00:00 /sbin/getty -8 38400 tty3
    linux_org  root      6059  5785  0 14:15 ?        00:00:00 cron
    linux_org  root      6079  5785  0 14:15 pts/11  00:00:00 /sbin/getty -8 38400 console
    linux_org  root      6084  5785  0 14:15 pts/7    00:00:00 /bin/login --
    linux_org  eric      6104  6084  0 14:15 pts/7    00:00:00 -bash
    linux_org  root    20920  5953  0 21:51 ?        00:00:00 /sbin/udevd --daemon
    linux_org  root    20921  5953  0 21:51 ?        00:00:00 /sbin/udevd --daemon
    lxc-netstat

    Same thing as lxc-ps but for the netstat command. If you wish to pass netstat arguments don’t use “--”. For example:
    Code:
    # lxc-netstat -n linux_org -ntlup
    
    Active Internet connections (only servers)
    
    Proto Recv-Q Send-Q Local Address          Foreign Address        State      PID/Program name
    tcp        0      0 0.0.0.0:22              0.0.0.0:*              LISTEN      6021/sshd
    tcp6      0      0 :::22                  :::*                    LISTEN      6021/sshd
    udp        0      0 0.0.0.0:68              0.0.0.0:*                          5999/dhclient3
    lxc-cgroup

    This is more of an advanced program but this allows you to tweak LXC’s inner workings of a specific container (these can also be set in the container’s “config” file as well). The one that most people will probably find helpful is setting memory limits, which can be done like so (300M RAM limit):
    Code:
    lxc-cgroup -n linux_org memory.limit_in_bytes 300000000
    Just replace “linux_org” with the container’s name. Please note that LXC, at least at the time of this writing and of version 0.7.5 is
    not able to limit network bandwidth per container. So if you’re looking to use LXC containers as a VPS solution, you will have to implement this in some other way. Its doable, but not built-in, and this is the biggest difference between v-Server and LXC in terms of functionality as far as I’m concerned.

    WIth this program though you can modify a lot of different aspects about containers, so be careful. To learn more about cgroups and what can be set read the official Kernel.org documents (a lot of it to me is dry but there’s not a lot you can go into that’s not dry with this stuff).

    Since lxc-execute doesn’t like to play nice, for some essential information (in this case how much RAM is being used by the container) you can also use lxc-cgroup to poll this too, like so:
    Code:
    # lxc-cgroup -n linux_org memory.usage_in_bytes
    4096
    This is telling me that 4096 bytes of RAM is being used by the container. If a value is not passed to a cgroup then the current value is returned, that is the only difference between getting the RAM and setting the RAM.

    Side note: containers share memory when possible (i.e.: system libraries), which is why the footprint is pretty low.

    These are what I feel to be the essential programs you will be using when administering LXC containers. However if you have any questions I’m open to assisting you.

    Attached Files:

    Last edited by a moderator: Aug 9, 2013
    Primit1v3 likes this.
  2. urgensherpa

    urgensherpa New Member

    Messages:
    1
    Likes Received:
    1
    Trophy Points:
    1
    nice tuto thanks by the way do you have any tutorial on checkpoint and restore the lxc container..live migration? thanks
    DevynCJohnson likes this.
  3. Eric Hansen

    Eric Hansen Member

    Messages:
    95
    Likes Received:
    38
    Trophy Points:
    18
    LXC isn't that mature, and there's no real information on migration. As for CheckPoint, you'd have to be more specific but I don't have access to any CP software or hardware to write up about them.
  4. tedmarcin

    tedmarcin New Member

    Messages:
    2
    Likes Received:
    1
    Trophy Points:
    3
    If i make backup, can't see any versions/revision No. . Whats wrong? Any idea?
  5. Eric Hansen

    Eric Hansen Member

    Messages:
    95
    Likes Received:
    38
    Trophy Points:
    18
    Can you clarify?

    Also, lxc-backup is removed from recent versions of LXC (1.0.0 beta and up I believe).
  6. tedmarcin

    tedmarcin New Member

    Messages:
    2
    Likes Received:
    1
    Trophy Points:
    3
    I have 0.8 from wheezy. I but is see in Debian-Expermental is lxc 1.0 without lxc-backup tool. Have u try it how do new tool lxc-snapshot ?
    DevynCJohnson likes this.
  7. Eric Hansen

    Eric Hansen Member

    Messages:
    95
    Likes Received:
    38
    Trophy Points:
    18
    I have not, but in reading this: http://s3hh.wordpress.com/2013/09/13/announcing-lxc-snapshot/

    It sounds like that unless you have BTRFS support for the container it may not be worth the possible/guaranteed headache until its a bit more fine-tuned.

    Its a great feature, just not something I think should be relied on at the moment.

Share This Page