Skip to content

Latest commit

 

History

History
39 lines (31 loc) · 2.27 KB

README.md

File metadata and controls

39 lines (31 loc) · 2.27 KB
Raw

Storm-CGroups

purpose

  • Add storm worker to cpu cgroups by crontab, in order to limit the max CPU% usage for each worker in storm. When a worker process die(often trigger by kill or rebalance command), it will automatically remove the unused cgroups.
  • When a worker process CPU% reach the limit, will trigger alarm.

code

.
├── README.md
└── shell
    ├── storm-cpu-cgroup-clearall.sh   # clear all worker process from cgroups
    ├── storm-cpu-cgroup-remove.sh     # if a worker died, will trigger this remove shell
    ├── stormcpualarmd.sh              # alarm shell
    └── stormcpud.sh                   # add worker process to cpu cgroups shell

deploy

  • When deploy on server side, please use crontab with shell script stormcpud.sh and stormcpualarmd.sh.
  • Specify Params: stormcpud.sh can specify max CPU% usage for each worker(default to 50%), stormcpualarmd.sh can specify alarm threshold(default to 80%).

cgroup structure

[liyan@XXXXX]$ lscgroup
cpu:/                                       # CPU root cgroup
cpu:/weibo_storm                            # under CPU root cgroup, will generate weibo_storm cgroup
cpu:/weibo_storm/local-worker-pid-22341     # under weibo_storm cgroup, will generate a certain cgroup named local-worker-pid-${workerpid} for each worker process to limit cpu usage for that worker
cpu:/weibo_storm/local-worker-pid-18547
cpu:/weibo_storm/local-worker-pid-12848
cpu:/weibo_storm/local-worker-pid-12063
cpu:/weibo_storm/local-worker-pid-9842
cpu:/weibo_storm/local-worker-pid-7559

cgroups resources

  1. https://www.kernel.org/doc/Documentation/cgroups/
  2. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/index.html
  3. http://coolshell.cn/articles/17049.html
  4. http://tiewei.github.io/devops/howto-use-cgroup/
  5. http://www.infoq.com/cn/articles/docker-kernel-knowledge-cgroups-resource-isolation