summaryrefslogtreecommitdiffstats
path: root/README
blob: 08350045308092e0f785bcf3e2ad0692d4a5d23f (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
Please follow the following steps (TODO: Automate these steps)

0. Update package version in AC_INIT (configure.in)
1. Tag the current sources in trunk to the specified version
1. Run autoconf to generate the configure file again (on changing configure.in)
2. Run configure with --prefix=<desired prefix> default is /usr/local
3. This step should generate
	a. Makefile
	b. libcgroup.spec
4. To build the RPM, carry out the following steps
	a. Create a tag with the same version as specified in step 0
	a. Run mktarball
	b. copy tarball and spec file to correct place in the RPMBUILD
	c. Build an rpm of the spec file


Design
========

After cgroup system has taken shape, its time to have some basic tools
in user space which can enable a user to use the resource management
functionality effictively.

One of the needed functionality is rule based placement of a task.  In general,
there can be either uid or gid or exec based rules. Admin/root will
control/enforce uid/gid based rules and exec based rules can be defined by
user in a config file residing in user's home dir and fully controlled by user.

uid/gid based rules will be defined in /etc/cgrules.conf config file and
this file will be managed by root.

Basic idea is that to begin with provide facility to implement rules
based on uid and gid. So a hierarchy might look like as follows.

                        /mnt/cgroup
                        |       |
                      gid1      gid2
                     |   |
                   uid1 uid2
                  |   |
               proj1  proj2


Admin will write rules to control the resources among users. Then users
can manage their own cgroup at their own (proj1 and proj2) and control
the resources as they want.

Following are the few methods using which tasks can be placed in right
cgroups.

- Use pam_cgroup PAM plugin which will make sure users are placed in right
  cgroup at login time and any tasks launch after login, will continue to run
  in user's cgroup. (PAM module is still on TODO list)

- Use command line tool "cgexec" to launch the task in right cgroup.

- Modify the program and use libcgroup provided APIs for placing a task
  in right cgroup before doing exec().

- Use "cgclassify" tool to classify a already running task.

- May be, a user space daemon can also be implemented which will listen to
  kernel events and place the task in right group based on the rules.
  This method involves few concerns.

	- Reliability of netlink socket. Messages can be dropped.
		- Change the netlink with a cgroup controller which exports the
	  	  events.

	- Delay incurred since the event took place and task was actually placed
  	  in right cgroup.

	- daemon will interfere with container's tasks which is not desired.

HOWTO
=====

Section 1:
----------
To use "cgexec" to place the task in right cgroup.

- make cgexec
- Run a task using cgexec. Following is the cgexec syntax.

 cgexec [-g <list of controllers>:<relative path to cgroup>] command [arguments]

 Note: Cgroup controllers and path are optional. If user does not know the
       right cgroup, cgexec will automatically place the task in right
       cgroup based on /etc/cgrules.conf

Example:
	cgexec -g *:test1 ls
	cgexec -g cpu,memory:test1 ls -l
	cgexec -g cpu,memory:test1 -g swap:test2 ls -l

Section 2
---------
To use "cgclassify" to place task in right cgroup.

- make cgclassify
- Pick a task's pid to be classified, and run
	cgclassify <list of pids>

Example:
--------
	cgclassify 2140 4325

	Note: After classification check out whether tasks 2140 and 4325
	are in the right cgroup or not (Based on rules in /etc/cgrules.conf)

Section 3:
----------
To use a pam plugin which will automatically place the task in right
cgroup upon login.

- Download pam source tree and apply the patch for pam_cgroup module.
- Build pam_cgroup.so
	./configure; make
- Copy pam_cgroup.so to /lib/security/
- Edit /etc/pam.d/su to make use of pam_cgroup.so session module upon
  execution of su.

example:  Add following line to the end.
session                optional        pam_cgroup.so

- Now launch a shell for a user "xyz" using su and the resulting shell
  should be running in the cgroup designated for the user as specified
  by cgrules.conf

  ex. "su test1"

Section 4:
----------
To use cgrulesengd which will move a task to right cgroup based on
rules in /etc/cgrules.conf do following.

- build and install latest libcgroup.so
- build cgrulesengd
	make cgrulesengd
- specify some uid/gid based rules in /etc/cgrules.conf
- Mount some controllers and create an hierarchy of cgroups (matching
  your rules).
- Run cgrulesengd.
	- ./cgrulesengd
- Launch some task or login as a user to the sytem. Daemon should automatically
  place the task in right cgroup.

FAQ
===
Q. Why admin should not control "exec" based rules.
A. Unix file system provides access control only based on uid/gid. So
   even if admin starts putting tasks based on uid/gid, it can't enforce
   it. For example, consider following scenario.

  Lets say an admin creates following cgroup hierarchy.

			/container
			|	|
  		database	browser
		|    |		|     |
	      user1  user2     user1  user2

 Now admin wants to run all the database jobs under /container/database/
 and all the firefox jobs under /container/browser/. Based on which user
 launched it, jobs should go in either user1 or user2 dir.

 Now assume that database subdir has to more cpu resources as compared to
 browser subdir. Then a user, say user2, can always move his browser job
 also to /container/database/user2 dir to get more cpu resources and
 admin will not be able to control that.

 [Note: user2 will control what tasks can be added in /container/database/user2
  and will contol what further subdirs can be created under user2 dir. Root
  should not restrict the control to root only for practical purposes. Its
  something like that till /container/databse, admin controls the resources
  and below that how resources are further subdivided among various projects
  should be controlled by respective user].

In the light of above, it seems to make more sense that admin should enforce
rules only based on uid and gid. Probably later we can have a per user exec
based rules config file (~/.cgrules.conf), which can be parsed by cgrulesd
and then jobs launched by user will be placed in right cgroup based on
combination of rules in /etc/cgrules.conf and ~/cgrules.conf.