Linux Containers: Part 3, Tools of the Trade

E

Eric Hansen

Guest
There are a lot of tools that comes with the LXC userspace. For me, these are my options:
Code:
ls /usr/bin/lxc-*

/usr/bin/lxc-attach      /usr/bin/lxc-checkpoint  /usr/bin/lxc-destroy  /usr/bin/lxc-kill    /usr/bin/lxc-netstat  /usr/bin/lxc-setcap    /usr/bin/lxc-start-ephemeral  /usr/bin/lxc-version
/usr/bin/lxc-backup      /usr/bin/lxc-clone      /usr/bin/lxc-execute  /usr/bin/lxc-list    /usr/bin/lxc-ps      /usr/bin/lxc-setuid    /usr/bin/lxc-stop            /usr/bin/lxc-wait
/usr/bin/lxc-cgroup      /usr/bin/lxc-console    /usr/bin/lxc-freeze  /usr/bin/lxc-ls      /usr/bin/lxc-restart  /usr/bin/lxc-shutdown  /usr/bin/lxc-unfreeze
/usr/bin/lxc-checkconfig  /usr/bin/lxc-create      /usr/bin/lxc-info    /usr/bin/lxc-monitor  /usr/bin/lxc-restore  /usr/bin/lxc-start    /usr/bin/lxc-unshare
While it might seem daunting, you likely won’t be using most of these much (if at all), and some aren’t even for containers. For example, the version tool just tells you what version of LXC you’re running (0.7.5 for me if you’re curious). However, some them are also quite useful, and I’ll cover what I can here. I won’t go over every tool, but some of the most beneficial ones will be talked about and others that you can mostly try out for yourself.

First I would like to go over what tools aren’t supported or otherwise aren’t advisable to run on Ubuntu (and possibly other) host distros:
Not Supported

  • lxc-attach - Lets you run a command in a running container
  • lxc-checkpoint - Similar to VMware or VirtualBox’s “Snapshot” feature, lets you create a checkpoint or restore point of a container
  • lxc-restart - Restart a checkpointed container
Not Recommended

  • lxc-setcap - Sets the file capabilities on LXC tools
  • lxc-setuid - Sets or remove setuid bits on LXC tools
lxc-backup and lxc-restore

I would like to state that the Ubuntu docs state “[...] lxc-backup and lxc-restore are fragile with respect to customizations and therefore their use is not recommended.” I’m not sure in what way it means “customizations”, so fair warning.

backup is an interesting program in that it will create a backup of a container (as well as version number) and make a copy of the entire root directory of the container’s directory (i.e.: if the container is linux_org, then the backup will be /var/lib/lxc/linux_org/rootfs.backup1/ if backup is ran with arguments “linux_org” and “1”). backup uses rsync to do the backing up of the filesystem, fair warning on that.

To back up a container, the command is:
Code:
lxc-backup CONTAINER [BACKUP_NUMBER]
Where “CONTAINER” is the name of the container you want backed up and “BACKUP_NUMBER” is an optional argument used to specify the suffix of the backup directory. If its non-numerical it will be incremented from the highest number found in the container’s directory.


To restore you use the same parameters but use lxc-restore instead of lxc-backup. It will delete everything from the current rootfs and restore from the specified backup.
lxc-clone

This is extremely useful if you wish to have one base template container that you can mirror others from. The syntax is pretty simple:
Code:
lxc-clone -o <original container name> -n <new container name>
So if you have a container currently named “bob” and you want to mirror that container to a new instance/container called “alice”, you would do this:
Code:
lxc-clone -o bob -n alice
This is assuming that “alice” container doesn’t exist already.

If your container’s a LVM or BTRFS backing store (as mentioned in part 2), you can also pass the “-s” argument to tell lxc-clone you want the new container to be generated from a snapshot of the original one.

I can’t really tell a difference between using “-s” and not when the container’s backing store is “none” (default), but then again I also don’t use LXC through a LVM or BTRFS either.
lxc-destroy

Want to get rid of a container for good for whatever reason? This is the program for you!

What this does is completely destroy the container and remove it from your system (so make sure you back up any important files you might have laying in one before you use this!). The required argument is “-n <container name>” but you can also pass along “-f” to tell LXC to force a deletion if the container is currently running (default is to abort/error out).
lxc-shutdown

Shutting down a container while you’re inside is easy, but if you don’t want to log in to the container just to shut it down you can use the lxc-shutdown command.

This program takes the container’s name (“-n <container name>”) as a required argument but also accepts “-w” for the program to wait until the container is actually shut down, “-r” to reboot the container instead and “-t <seconds>” to wait <seconds> long before killing the container (not really suggested and the “-t” argument is from running lxc-shutdown -h).
lxc-list

This is probably one you’ll be using a lot to make sure containers are running and stopped (frozen containers won’t be listed here for some reason). It takes no arguments and will print everything out neatly for you. At some point you’ll probably notice that some might even be suffixed with a “(auto)”, which basically means that the container is set to automatically boot at load.
lxc-freeze and lxc-unfreeze

Suspecting that a container is causing issues in some fashion? lxc-freeze will let you basically pause the container (blocking all processes from running and such) to test, and lxc-unfreeze will start it back up. Both require the typical “-n <container name>” argument (most of these tools will). This is pretty useful also if you’re wanting to test some vulnerability sandboxes ;)
lxc-info

This is similar to lxc-list but requires the “-n <container name>” argument. It tells you the state of the container as well as it’s process ID on the host machine. If you’re unsure if the host is frozen or just not plain detected by LXC this will tell you even if it is frozen.
lxc-ls

Doesn’t require the “-n” argument and will list all containers that LXC sees (even the frozen ones). So if you see a difference between this and lxc-list, run a lxc-info on the disgruntled containers to investigate further.
lxc-ps

Just like “ps” for a normal *nix machine, it sends a request to the container (via passing the “-n” argument) the ps command and returns the output. If you wish to send specific arguments to “ps” itself inside of the container, add a “--” in front, like so:
Code:
# lxc-ps -n linux_org -- -ef

CONTAINER  UID        PID  PPID  C STIME TTY          TIME CMD
linux_org  root      5785  5779  0 14:15 ?        00:00:00 /sbin/init
linux_org  root      5944  5785  0 14:15 ?        00:00:00 upstart-udev-bridge --daemon
linux_org  root      5953  5785  0 14:15 ?        00:00:00 /sbin/udevd --daemon
linux_org  syslog    5961  5785  0 14:15 ?        00:00:00 rsyslogd -c5
linux_org  root      5968  5785  0 14:15 ?        00:00:00 upstart-socket-bridge --daemon
linux_org  root      5999  5785  0 14:15 ?        00:00:00 dhclient3 -e IF_METRIC=100 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -1 eth0
linux_org  root      6021  5785  0 14:15 ?        00:00:00 /usr/sbin/sshd -D
linux_org  root      6049  5785  0 14:15 pts/10  00:00:00 /sbin/getty -8 38400 tty4
linux_org  root      6053  5785  0 14:15 pts/8    00:00:00 /sbin/getty -8 38400 tty2
linux_org  root      6054  5785  0 14:15 pts/9    00:00:00 /sbin/getty -8 38400 tty3
linux_org  root      6059  5785  0 14:15 ?        00:00:00 cron
linux_org  root      6079  5785  0 14:15 pts/11  00:00:00 /sbin/getty -8 38400 console
linux_org  root      6084  5785  0 14:15 pts/7    00:00:00 /bin/login --
linux_org  eric      6104  6084  0 14:15 pts/7    00:00:00 -bash
linux_org  root    20920  5953  0 21:51 ?        00:00:00 /sbin/udevd --daemon
linux_org  root    20921  5953  0 21:51 ?        00:00:00 /sbin/udevd --daemon
lxc-netstat

Same thing as lxc-ps but for the netstat command. If you wish to pass netstat arguments don’t use “--”. For example:
Code:
# lxc-netstat -n linux_org -ntlup

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address          Foreign Address        State      PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*              LISTEN      6021/sshd
tcp6      0      0 :::22                  :::*                    LISTEN      6021/sshd
udp        0      0 0.0.0.0:68              0.0.0.0:*                          5999/dhclient3
lxc-cgroup

This is more of an advanced program but this allows you to tweak LXC’s inner workings of a specific container (these can also be set in the container’s “config” file as well). The one that most people will probably find helpful is setting memory limits, which can be done like so (300M RAM limit):
Code:
lxc-cgroup -n linux_org memory.limit_in_bytes 300000000
Just replace “linux_org” with the container’s name. Please note that LXC, at least at the time of this writing and of version 0.7.5 is
not able to limit network bandwidth per container. So if you’re looking to use LXC containers as a VPS solution, you will have to implement this in some other way. Its doable, but not built-in, and this is the biggest difference between v-Server and LXC in terms of functionality as far as I’m concerned.

WIth this program though you can modify a lot of different aspects about containers, so be careful. To learn more about cgroups and what can be set read the official Kernel.org documents (a lot of it to me is dry but there’s not a lot you can go into that’s not dry with this stuff).

Since lxc-execute doesn’t like to play nice, for some essential information (in this case how much RAM is being used by the container) you can also use lxc-cgroup to poll this too, like so:
Code:
# lxc-cgroup -n linux_org memory.usage_in_bytes
4096
This is telling me that 4096 bytes of RAM is being used by the container. If a value is not passed to a cgroup then the current value is returned, that is the only difference between getting the RAM and setting the RAM.

Side note: containers share memory when possible (i.e.: system libraries), which is why the footprint is pretty low.

These are what I feel to be the essential programs you will be using when administering LXC containers. However if you have any questions I’m open to assisting you.
 

Attachments

  • slide.jpg
    slide.jpg
    26.6 KB · Views: 96,330
Last edited:


nice tuto thanks by the way do you have any tutorial on checkpoint and restore the lxc container..live migration? thanks
 
nice tuto thanks by the way do you have any tutorial on checkpoint and restore the lxc container..live migration? thanks
LXC isn't that mature, and there's no real information on migration. As for CheckPoint, you'd have to be more specific but I don't have access to any CP software or hardware to write up about them.
 
If i make backup, can't see any versions/revision No. . Whats wrong? Any idea?
Can you clarify?

Also, lxc-backup is removed from recent versions of LXC (1.0.0 beta and up I believe).
 
I have 0.8 from wheezy. I but is see in Debian-Expermental is lxc 1.0 without lxc-backup tool. Have u try it how do new tool lxc-snapshot ?
 
I have 0.8 from wheezy. I but is see in Debian-Expermental is lxc 1.0 without lxc-backup tool. Have u try it how do new tool lxc-snapshot ?
I have not, but in reading this: http://s3hh.wordpress.com/2013/09/13/announcing-lxc-snapshot/

It sounds like that unless you have BTRFS support for the container it may not be worth the possible/guaranteed headache until its a bit more fine-tuned.

Its a great feature, just not something I think should be relied on at the moment.
 

Members online


Top