三者的关系

严格的说，Container技术并非虚拟化，更准确的说它是一种资源隔离机制。VMM或者叫Hypervisor是标准的虚拟化技术，这种技术通过虚拟层（也就是VMM或叫Hypervisor）把上层虚拟机的指令转换成底层Host操作系统所认识的指令。所以，虚拟化技术可以在主机上虚拟出各种类型的操作系统，而Container技术只能“虚拟”出与其主机操作系统相同或很类似的“虚拟机”。

LXC和Warden以及最近火热的Docker都是基于CGroup在其上封装了一层。所以，我们还是一起学习CGroup吧:)

一、CGroup

（1）CGroup简介及用法

http://linuxaria.com/article/introduction-to-cgroups-the-linux-conrol-group?lang=en

控制资源的三种方法：

First method it’s to use the meta-filesystem of cgroups.

<pre class="bash" name="code" style="margin: 0px 0px 24px; padding: 0px; font-weight: 400; box-sizing: border-box; background-color: rgb(240, 240, 240); overflow-x: auto; font-family: Consolas, Inconsolata, Courier, monospace; font-size: 14px; line-height: 22px; color: rgb(0, 0, 0);">$ echo 2048 >> /sys/fs/cgroup/cpu/Multimedia/cpu.shares</pre>

Second method it’s to use a set of commands

** Notice, the value of memory.memsw.limit_in_bytes must not less than that of memory.limit_in_bytes. See the following validation:**

root@app-node-v-o:/home/test/memleak# echo 1G >> /sys/fs/cgroup/memory/zhxue/memory.limit_in_bytes

root@app-node-v-o:/home/test/memleak# echo 200M >> /sys/fs/cgroup/memory/zhxue/memory.memsw.limit_in_bytes
bash: echo: write error: Invalid argument
root@app-node-v-o:/home/test/memleak# echo 100M >> /sys/fs/cgroup/memory/zhxue/memory.memsw.limit_in_bytes
bash: echo: write error: Invalid argument
root@app-node-v-o:/home/test/memleak# echo 10G >> /sys/fs/cgroup/memory/zhxue/memory.memsw.limit_in_bytes

The cgred Daemon

Cgred is a daemon that moves tasks into cgroups according to parameters set in the /etc/cgrules.conf file. Entries in the /etc/cgrules.conf file can take one of the two forms:

user hierarchies control_group
user:command hierarchies control_group

For example:

<pre style="margin: 1.5em 0px; padding: 1em; font-weight: 400; box-sizing: border-box; background-color: rgb(247, 247, 247); overflow: auto; font-family: "Courier New", Courier, Monaco, "Lucida Console", monospace; font-size: 14px; line-height: 1.5em; color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial; border: 1px solid rgb(221, 221, 221);">maria devices /usergroup/staff</pre>

This entry specifies that any processes that belong to the user named maria access the devices subsystem according to the parameters specified in the /usergroup/staff cgroup. To associate particular commands with particular cgroups, add the command parameter, as follows:

The entry now specifies that when the user named maria uses the ftp command, the process is automatically moved to the /usergroup/staff/ftp cgroup in the hierarchy that contains the devices subsystem.

/etc/cgrules.conf 文件中的条目可包括以下额外符号：

@ - 当在 user 使用前缀时，代表是一个组群而不是单独用户。例如：@admins 是 admins 组群中的所有用户。
* - 代表“所有”。例如：subsystem 字段中的 * 代表所有子系统。
% - 代表与以上行中项目相同的项目。例如：

问题：

1）在ubuntu高版本（13.04）上无法更改memsw值：

a. 直接该文件不行

root@ubuntudev:/sys/fs/cgroup/memory# echo 100M >> zhxue/memory.limit_in_bytes root@ubuntudev:/sys/fs/cgroup/memory# echo 100M >> zhxue/memory.memsw.limit_in_bytes
bash: echo: write error: Operation not supported
root@ubuntudev:/sys/fs/cgroup/memory# cat zhxue/memory.limit_in_bytes
104857600
root@ubuntudev:/sys/fs/cgroup/memory# cat zhxue/memory.memsw.limit_in_bytes
cat: zhxue/memory.memsw.limit_in_bytes: Operation not supported

b. 命令更改也无效：

root@ubuntudev:/sys/fs/cgroup/memory# cgset -r memory.memsw.limit_in_bytes=10MB zhxue
cgset: the group can't be modified

注意到memory.memsw.limit_in_bytes和memory.limit_in_bytes文件属性一模一样，为什么不能改？很奇怪。

(2) CGroup原理——很全面的文章

从底层数据结构到上层文件系统，非常有价值。

Linux Cgroups详解 by 王喆锋

http://files.cnblogs.com/lisperl/cgroups%E4%BB%8B%E7%BB%8D.pdf

(3) CGroup文件及参数详解

https://www.kernel.org/doc/Documentation/cgroups/

在内存这块，CGroup默认分配的内存为8PB，请看：

root@ubuntudev:/sys/fs/cgroup/memory# cgcreate -g memory:zhxue

root@ubuntudev:/sys/fs/cgroup/memory# ll zhxue/
cgroup.clone_children memory.memsw.usage_in_bytes
cgroup.event_control memory.move_charge_at_immigrate
cgroup.procs memory.oom_control
memory.failcnt memory.soft_limit_in_bytes
memory.force_empty memory.stat
memory.limit_in_bytes memory.swappiness
memory.max_usage_in_bytes memory.usage_in_bytes
memory.memsw.failcnt memory.use_hierarchy
memory.memsw.limit_in_bytes notify_on_release
memory.memsw.max_usage_in_bytes tasks

root@ubuntudev:/sys/fs/cgroup/memory# cat zhxue/memory.limit_in_bytes
9223372036854775807

root@ubuntudev:/sys/fs/cgroup/memory# cgset -r memory.limit_in_bytes=10M zhxue

root@ubuntudev:/sys/fs/cgroup/memory# cat zhxue/memory.limit_in_bytes
10485760