Skip to content

Commit 52ec400

Browse files
labissolabisso
committedFeb 11, 2014
Initial commit of Blockade
·
0.4.00.1.0
0 parents  commit 52ec400

35 files changed

+3561
-0
lines changed
 

‎.gitignore

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
*.pyc
2+
*.egg-info
3+
.*.swp
4+
*.log
5+
*~
6+
build/
7+
dist/
8+
MANIFEST
9+
.DS_Store
10+
.idea
11+
.tox
12+
.coverage
13+
venv
14+
.vagrant/
15+
.blockade/

‎LICENSE

+176
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
2+
Apache License
3+
Version 2.0, January 2004
4+
http://www.apache.org/licenses/
5+
6+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7+
8+
1. Definitions.
9+
10+
"License" shall mean the terms and conditions for use, reproduction,
11+
and distribution as defined by Sections 1 through 9 of this document.
12+
13+
"Licensor" shall mean the copyright owner or entity authorized by
14+
the copyright owner that is granting the License.
15+
16+
"Legal Entity" shall mean the union of the acting entity and all
17+
other entities that control, are controlled by, or are under common
18+
control with that entity. For the purposes of this definition,
19+
"control" means (i) the power, direct or indirect, to cause the
20+
direction or management of such entity, whether by contract or
21+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
22+
outstanding shares, or (iii) beneficial ownership of such entity.
23+
24+
"You" (or "Your") shall mean an individual or Legal Entity
25+
exercising permissions granted by this License.
26+
27+
"Source" form shall mean the preferred form for making modifications,
28+
including but not limited to software source code, documentation
29+
source, and configuration files.
30+
31+
"Object" form shall mean any form resulting from mechanical
32+
transformation or translation of a Source form, including but
33+
not limited to compiled object code, generated documentation,
34+
and conversions to other media types.
35+
36+
"Work" shall mean the work of authorship, whether in Source or
37+
Object form, made available under the License, as indicated by a
38+
copyright notice that is included in or attached to the work
39+
(an example is provided in the Appendix below).
40+
41+
"Derivative Works" shall mean any work, whether in Source or Object
42+
form, that is based on (or derived from) the Work and for which the
43+
editorial revisions, annotations, elaborations, or other modifications
44+
represent, as a whole, an original work of authorship. For the purposes
45+
of this License, Derivative Works shall not include works that remain
46+
separable from, or merely link (or bind by name) to the interfaces of,
47+
the Work and Derivative Works thereof.
48+
49+
"Contribution" shall mean any work of authorship, including
50+
the original version of the Work and any modifications or additions
51+
to that Work or Derivative Works thereof, that is intentionally
52+
submitted to Licensor for inclusion in the Work by the copyright owner
53+
or by an individual or Legal Entity authorized to submit on behalf of
54+
the copyright owner. For the purposes of this definition, "submitted"
55+
means any form of electronic, verbal, or written communication sent
56+
to the Licensor or its representatives, including but not limited to
57+
communication on electronic mailing lists, source code control systems,
58+
and issue tracking systems that are managed by, or on behalf of, the
59+
Licensor for the purpose of discussing and improving the Work, but
60+
excluding communication that is conspicuously marked or otherwise
61+
designated in writing by the copyright owner as "Not a Contribution."
62+
63+
"Contributor" shall mean Licensor and any individual or Legal Entity
64+
on behalf of whom a Contribution has been received by Licensor and
65+
subsequently incorporated within the Work.
66+
67+
2. Grant of Copyright License. Subject to the terms and conditions of
68+
this License, each Contributor hereby grants to You a perpetual,
69+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70+
copyright license to reproduce, prepare Derivative Works of,
71+
publicly display, publicly perform, sublicense, and distribute the
72+
Work and such Derivative Works in Source or Object form.
73+
74+
3. Grant of Patent License. Subject to the terms and conditions of
75+
this License, each Contributor hereby grants to You a perpetual,
76+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77+
(except as stated in this section) patent license to make, have made,
78+
use, offer to sell, sell, import, and otherwise transfer the Work,
79+
where such license applies only to those patent claims licensable
80+
by such Contributor that are necessarily infringed by their
81+
Contribution(s) alone or by combination of their Contribution(s)
82+
with the Work to which such Contribution(s) was submitted. If You
83+
institute patent litigation against any entity (including a
84+
cross-claim or counterclaim in a lawsuit) alleging that the Work
85+
or a Contribution incorporated within the Work constitutes direct
86+
or contributory patent infringement, then any patent licenses
87+
granted to You under this License for that Work shall terminate
88+
as of the date such litigation is filed.
89+
90+
4. Redistribution. You may reproduce and distribute copies of the
91+
Work or Derivative Works thereof in any medium, with or without
92+
modifications, and in Source or Object form, provided that You
93+
meet the following conditions:
94+
95+
(a) You must give any other recipients of the Work or
96+
Derivative Works a copy of this License; and
97+
98+
(b) You must cause any modified files to carry prominent notices
99+
stating that You changed the files; and
100+
101+
(c) You must retain, in the Source form of any Derivative Works
102+
that You distribute, all copyright, patent, trademark, and
103+
attribution notices from the Source form of the Work,
104+
excluding those notices that do not pertain to any part of
105+
the Derivative Works; and
106+
107+
(d) If the Work includes a "NOTICE" text file as part of its
108+
distribution, then any Derivative Works that You distribute must
109+
include a readable copy of the attribution notices contained
110+
within such NOTICE file, excluding those notices that do not
111+
pertain to any part of the Derivative Works, in at least one
112+
of the following places: within a NOTICE text file distributed
113+
as part of the Derivative Works; within the Source form or
114+
documentation, if provided along with the Derivative Works; or,
115+
within a display generated by the Derivative Works, if and
116+
wherever such third-party notices normally appear. The contents
117+
of the NOTICE file are for informational purposes only and
118+
do not modify the License. You may add Your own attribution
119+
notices within Derivative Works that You distribute, alongside
120+
or as an addendum to the NOTICE text from the Work, provided
121+
that such additional attribution notices cannot be construed
122+
as modifying the License.
123+
124+
You may add Your own copyright statement to Your modifications and
125+
may provide additional or different license terms and conditions
126+
for use, reproduction, or distribution of Your modifications, or
127+
for any such Derivative Works as a whole, provided Your use,
128+
reproduction, and distribution of the Work otherwise complies with
129+
the conditions stated in this License.
130+
131+
5. Submission of Contributions. Unless You explicitly state otherwise,
132+
any Contribution intentionally submitted for inclusion in the Work
133+
by You to the Licensor shall be under the terms and conditions of
134+
this License, without any additional terms or conditions.
135+
Notwithstanding the above, nothing herein shall supersede or modify
136+
the terms of any separate license agreement you may have executed
137+
with Licensor regarding such Contributions.
138+
139+
6. Trademarks. This License does not grant permission to use the trade
140+
names, trademarks, service marks, or product names of the Licensor,
141+
except as required for reasonable and customary use in describing the
142+
origin of the Work and reproducing the content of the NOTICE file.
143+
144+
7. Disclaimer of Warranty. Unless required by applicable law or
145+
agreed to in writing, Licensor provides the Work (and each
146+
Contributor provides its Contributions) on an "AS IS" BASIS,
147+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148+
implied, including, without limitation, any warranties or conditions
149+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150+
PARTICULAR PURPOSE. You are solely responsible for determining the
151+
appropriateness of using or redistributing the Work and assume any
152+
risks associated with Your exercise of permissions under this License.
153+
154+
8. Limitation of Liability. In no event and under no legal theory,
155+
whether in tort (including negligence), contract, or otherwise,
156+
unless required by applicable law (such as deliberate and grossly
157+
negligent acts) or agreed to in writing, shall any Contributor be
158+
liable to You for damages, including any direct, indirect, special,
159+
incidental, or consequential damages of any character arising as a
160+
result of this License or out of the use or inability to use the
161+
Work (including but not limited to damages for loss of goodwill,
162+
work stoppage, computer failure or malfunction, or any and all
163+
other commercial damages or losses), even if such Contributor
164+
has been advised of the possibility of such damages.
165+
166+
9. Accepting Warranty or Additional Liability. While redistributing
167+
the Work or Derivative Works thereof, You may choose to offer,
168+
and charge a fee for, acceptance of support, warranty, indemnity,
169+
or other liability obligations and/or rights consistent with this
170+
License. However, in accepting such obligations, You may act only
171+
on Your own behalf and on Your sole responsibility, not on behalf
172+
of any other Contributor, and only if You agree to indemnify,
173+
defend, and hold each Contributor harmless for any liability
174+
incurred by, or claims asserted against, such Contributor by reason
175+
of your accepting any such warranty or additional liability.
176+

‎MANIFEST.in

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
include TODO.md
2+
include README.md
3+
include LICENSE
4+
include MANIFEST.in
5+
exclude .gitignore
6+
recursive-include blockade *
7+
global-exclude *pyc *pyo

‎README.md

+157
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
Blockade
2+
========
3+
4+
Blockade is a utility for testing network failures and partitions in
5+
distributed applications. Blockade uses [Docker](http://www.docker.io)
6+
containers to run application processes and manages the network from
7+
the host system to create various failure scenarios.
8+
9+
A common use is to run a distributed application such as a database
10+
or cluster and create network partitions, then observe the behavior of
11+
the nodes. For example in a leader election system, you could partition
12+
the leader away from the other nodes and ensure that the leader steps
13+
down and that another node emerges as leader.
14+
15+
Blockade features:
16+
17+
* A flexible YAML format to describe the containers in your application
18+
* Support for dependencies between containers, using named links
19+
- A CLI tool for managing and querying the status of your blockade
20+
* Creation of arbitrary partitions between containers
21+
* Giving a container a flaky network connection to others (drop packets)
22+
* Giving a container a slow network connection to others (latency)
23+
* While under partition or network failure control, containers can
24+
freely communicate with the host system -- so you can still grab logs
25+
and monitor the application.
26+
27+
Blockade is written and maintained by the
28+
[Dell Cloud Manager](http://www.enstratius.com) (formerly Enstratius)
29+
team and is used internally to test the behaviors of our software.
30+
We also release a number of other internal components as open source,
31+
most notably [Dasein Cloud](http://dasein.org).
32+
33+
Inspired by the excellent [Jepsen](http://aphyr.com/tags/jepsen) series.
34+
35+
36+
Configuration
37+
-------------
38+
39+
Blockade expects a ``blockade.yaml`` file in the current directory which
40+
describes the containers to launch, how they are linked, and various
41+
parameters for the blockade modes. Example:
42+
43+
44+
```yaml
45+
46+
containers:
47+
c1:
48+
image: my_docker_image
49+
command: /bin/myapp
50+
volumes: {"/opt/myapp": "/opt/myapp_host"}
51+
ports: [80]
52+
environment: {"IS_MASTER": 1}
53+
54+
c2:
55+
image: my_docker_image
56+
command: /bin/myapp
57+
volumes: ["/data"]
58+
links: {c1: master}
59+
60+
c3:
61+
image: my_docker_image
62+
command: /bin/myapp
63+
links: {c1: master}
64+
65+
network:
66+
flaky: 30%
67+
slow: 75ms 100ms distribution normal
68+
```
69+
70+
Blockade stores transient information in a local ``.blockade/`` directory.
71+
This directory will be cleaned up automatically when you run the
72+
``blockade destroy`` command.
73+
74+
75+
Usage
76+
-----
77+
78+
Blockade may be used from the command line manually. The commands are also
79+
intended to be easy to wrap and automate within tests, etc.
80+
81+
Blockade must be run as root (or with sudo).
82+
83+
84+
Commands
85+
--------
86+
87+
``blockade up``
88+
89+
Start the containers and link them together, if necessary.
90+
91+
92+
``blockade destroy``
93+
94+
Destroys all containers and restore networks.
95+
96+
97+
``blockade status``
98+
99+
Print the status of the containers and blockade.
100+
101+
102+
``blockade flaky n1``
103+
104+
``blockade flaky n1 n2``
105+
106+
Make network flaky to one or more containers.
107+
108+
109+
``blockade slow n1``
110+
111+
Make network slow to one or more containers.
112+
113+
114+
``blockade fast n1``
115+
116+
Restore network speed and reliability to one or more containers.
117+
118+
119+
``blockade partition n1,n2``
120+
121+
``blockade partition n1,n2 n3,n4``
122+
123+
Create one or more network partitions. Each partition is specified as a
124+
comma-separated list. Containers may not exist in more than one partition.
125+
Containers not specified are grouped into an implicit partition. Each
126+
partition command replaces any previous partition or block rules.
127+
128+
129+
``blockade join``
130+
131+
Remove all partitions between containers.
132+
133+
License
134+
-------
135+
136+
Blockade is offered under the Apache License 2.0.
137+
138+
Development
139+
-----------
140+
141+
Install test depenedencies with ``pip install blockade[test]``.
142+
143+
You can run integration tests in a Vagrant VM using the included Vagrantfile.
144+
Run ``vagrant up`` and Docker will be installed in your VM and tests run.
145+
You can rerun them with ``vagrant provision``, or SSH into the VM and run
146+
them yourself, from ``/vagrant``.
147+
148+
Blockade documentation is built with [Sphinx](http://sphinx-doc.org) and is
149+
found under ``docs/``. To build:
150+
151+
```
152+
$ pip install -r requirements_docs.txt
153+
$ cd docs/
154+
$ make html
155+
```
156+
157+
HTML output will be under ``docs/_build/html/``.

‎TODO.md

+100
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
TODO
2+
====
3+
4+
Musings of possible features/improvements.
5+
6+
support for DVM/boot2docker for OSX
7+
-----------------------------------
8+
9+
It would be very nice to run natively from OSX. The Docker API calls
10+
from Blockade will already work, perhaps with some minor changes. The
11+
big problem is that Blockade runs ``iptables`` and ``tc`` locally. We
12+
would need to go through Fabric perhaps in this case. Another option
13+
is a Blockade Remote API that runs on the VM and is usable from an
14+
OSX client (similar to how Docker itself works).
15+
16+
17+
support for starting/stopping containers
18+
----------------------------------------
19+
20+
This might be best done by way of a supervisor process within the container.
21+
Otherwise when restarted, we might have a different IP. A better solution
22+
might be to find a way to force the restarted containers IP, or use a full
23+
network overlay such as [Pipework](https://github.com/jpetazzo/pipework).
24+
25+
26+
more flexible partitioning
27+
--------------------------
28+
29+
It could be interesting to allow one way messages. One horror story I heard is
30+
of a NIC failure such that outbound messages worked but not inbound. So the
31+
machine kept sending heartbeats to other nodes.
32+
33+
It might also be valuable to allow complex partitions -- partitions with
34+
overlapping views.
35+
See: http://kellabyte.com/2014/02/09/routing-aware-master-elections/
36+
37+
38+
python-iptables
39+
---------------
40+
41+
Look into using ``python-iptables`` module for iptables interaction. It uses
42+
a library instead of invoking and parsing the iptables binary. I tried it
43+
initially but ran into weird failures and didn't have time to debug.
44+
Preserved code:
45+
46+
```python
47+
48+
def clear_iptables(blockade_id):
49+
"""Remove all iptables rules and chains related to this blockade
50+
"""
51+
# first remove refererences to our custom chains
52+
filter_table = iptc.Table(iptc.Table.FILTER)
53+
forward_chain = iptc.Chain(filter_table, "FORWARD")
54+
for rule in list(forward_chain.rules):
55+
target = rule.target
56+
if target and target.name:
57+
58+
# check if we have a partition chain name as target
59+
try:
60+
parse_partition_index(blockade_id, target.name)
61+
except ValueError:
62+
continue
63+
# and delete the rule if so
64+
forward_chain.delete_rule(rule)
65+
print "done"
66+
67+
# then remove the chains themselves
68+
for chain in filter_table.chains:
69+
if chain.name.startswith(blockade_id):
70+
chain.flush()
71+
filter_table.delete_chain(chain)
72+
73+
74+
def partition_containers(blockade_id, partitions):
75+
if not partitions or len(partitions) == 1:
76+
return
77+
for index, partition in enumerate(partitions, 1):
78+
chain_name = partition_chain_name(blockade_id, index)
79+
filter_table = iptc.Table(iptc.Table.FILTER)
80+
81+
# createe chain for partition and block traffic TO any other partition
82+
chain = filter_table.create_chain(chain_name)
83+
for other in partitions:
84+
if partition is other:
85+
continue
86+
for container in other:
87+
if container.ip_address:
88+
rule = iptc.Rule()
89+
rule.dst = container.ip_address
90+
rule.create_target("DROP")
91+
chain.insert_rule(rule)
92+
93+
# direct traffic FROM any container in the partition to the new chain
94+
forward_chain = iptc.Chain(filter_table, "FORWARD")
95+
for container in partition:
96+
rule = iptc.Rule()
97+
rule.src = container.ip_address
98+
rule.create_target(chain_name)
99+
forward_chain.insert_rule(rule)
100+
```

‎Vagrantfile

+51
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
2+
BOX_NAME = ENV['BOX_NAME'] || "ubuntu"
3+
BOX_URL = ENV['BOX_URL'] || "http://files.vagrantup.com/precise64.box"
4+
VMWARE_BOX_URI = ENV['BOX_URI'] || "http://files.vagrantup.com/precise64_vmware_fusion.box"
5+
script = <<SCRIPT
6+
#!/bin/bash -e
7+
8+
if [ ! -e /vagrant/blockade ]; then
9+
echo "/vagrant/blockade not found. are we in a vagrant blockade environment??"
10+
exit 1
11+
fi
12+
13+
apt-get -y install python-pip python-virtualenv
14+
15+
cd /vagrant
16+
17+
# install into system python
18+
python setup.py install
19+
20+
# and also develop-install into a venv, for dev+test
21+
if [ ! -e /tmp/ve/bin/activate ]; then
22+
rm -fr /tmp/ve
23+
virtualenv /tmp/ve
24+
fi
25+
source /tmp/ve/bin/activate
26+
27+
python setup.py develop
28+
pip install blockade[test]
29+
30+
export BLOCKADE_INTEGRATION_TESTS=1
31+
nosetests blockade --with-coverage
32+
SCRIPT
33+
34+
Vagrant.configure("2") do |config|
35+
36+
config.vm.box = BOX_NAME
37+
config.vm.box_url = BOX_URL
38+
39+
config.vm.provider :virtualbox do |vb, override|
40+
end
41+
42+
config.vm.provider :vmware_fusion do |vb, override|
43+
override.vm.box_url = VMWARE_BOX_URI
44+
end
45+
46+
config.vm.provision "docker",
47+
images: ["ubuntu"]
48+
49+
# kick off the tests automatically
50+
config.vm.provision "shell", inline: script
51+
end

‎blockade/__init__.py

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import blockade.version
18+
__version__ = blockade.version.__version__

‎blockade/cli.py

+260
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import sys
18+
import argparse
19+
import traceback
20+
import errno
21+
import json
22+
23+
import yaml
24+
from clint.textui import puts, puts_err, colored, columns
25+
26+
from .errors import BlockadeError
27+
from .core import Blockade
28+
from .state import BlockadeStateFactory
29+
from .config import BlockadeConfig
30+
from .net import BlockadeNetwork
31+
32+
33+
def load_config(opts):
34+
error = None
35+
paths = (opts.config,) if opts.config else ("blockade.yaml",
36+
"blockade.yml")
37+
try:
38+
for path in paths:
39+
try:
40+
with open(path) as f:
41+
d = yaml.safe_load(f)
42+
return BlockadeConfig.from_dict(d)
43+
except IOError, e:
44+
if e.errno != errno.ENOENT:
45+
raise
46+
except Exception, e:
47+
error = e
48+
raise BlockadeError("Failed to load config (from --config, "
49+
"./blockade.yaml, or ./blockade.yml)" +
50+
(str(error) if error else ""))
51+
52+
53+
def get_blockade(config):
54+
return Blockade(config, BlockadeStateFactory, BlockadeNetwork(config))
55+
56+
57+
def print_containers(containers, to_json=False):
58+
containers = sorted(containers, key=lambda c: c.name)
59+
60+
if to_json:
61+
d = [c.to_dict() for c in containers]
62+
puts(json.dumps(d, indent=2, sort_keys=True, separators=(',', ': ')))
63+
64+
else:
65+
puts(colored.blue(columns(["NODE", 15],
66+
["CONTAINER ID", 15],
67+
["STATUS", 7],
68+
["IP", 15],
69+
["NETWORK", 10],
70+
["PARTITION", 10])))
71+
for container in containers:
72+
partition = container.partition
73+
partition = "" if partition is None else str(partition)
74+
puts(columns([container.name, 15],
75+
[container.container_id[:12], 15],
76+
[container.state, 7],
77+
[container.ip_address or "", 15],
78+
[container.network_state, 10],
79+
[partition, 10]))
80+
81+
82+
def _add_output_options(parser):
83+
parser.add_argument('--json', action='store_true',
84+
help='Output in JSON format')
85+
86+
87+
def _add_container_selection_options(parser):
88+
parser.add_argument('containers', metavar='CONTAINER', nargs='*',
89+
help='Container to select')
90+
parser.add_argument('--all', action='store_true',
91+
help='Select all containers')
92+
93+
94+
def _check_container_selections(opts):
95+
if opts.containers and opts.all:
96+
raise BlockadeError("Either specify individual containers "
97+
"or --all, but not both")
98+
elif not (opts.containers or opts.all):
99+
raise BlockadeError("Specify individual containers or --all")
100+
101+
return (opts.containers or None, opts.all)
102+
103+
104+
def cmd_up(opts):
105+
"""Start the containers and link them together
106+
"""
107+
config = load_config(opts)
108+
b = get_blockade(config)
109+
containers = b.create()
110+
print_containers(containers, opts.json)
111+
112+
113+
def cmd_destroy(opts):
114+
"""Destroy all containers and restore networks
115+
"""
116+
config = load_config(opts)
117+
b = get_blockade(config)
118+
b.destroy()
119+
120+
121+
def cmd_status(opts):
122+
"""Print status of containers and networks
123+
"""
124+
config = load_config(opts)
125+
b = get_blockade(config)
126+
containers = b.status()
127+
print_containers(containers, opts.json)
128+
129+
130+
def cmd_flaky(opts):
131+
"""Make the network flaky for some or all containers
132+
"""
133+
containers, select_all = _check_container_selections(opts)
134+
config = load_config(opts)
135+
b = get_blockade(config)
136+
b.flaky(containers, select_all)
137+
138+
139+
def cmd_slow(opts):
140+
"""Make the network slow for some or all containers
141+
"""
142+
containers, select_all = _check_container_selections(opts)
143+
config = load_config(opts)
144+
b = get_blockade(config)
145+
b.slow(containers, select_all)
146+
147+
148+
def cmd_fast(opts):
149+
"""Restore network speed and reliability for some or all containers
150+
"""
151+
containers, select_all = _check_container_selections(opts)
152+
config = load_config(opts)
153+
b = get_blockade(config)
154+
b.fast(containers, select_all)
155+
156+
157+
def cmd_partition(opts):
158+
"""Partition the network between containers
159+
160+
Replaces any existing partitions outright. Any containers NOT specified
161+
in arguments will be globbed into a single implicit partition. For
162+
example if you have three containers: c1, c2, and c3 and you run:
163+
164+
blockade partition c1
165+
166+
The result will be a partition with just c1 and another partition with
167+
c2 and c3.
168+
"""
169+
partitions = []
170+
for partition in opts.partitions:
171+
names = []
172+
for name in partition.split(","):
173+
name = name.strip()
174+
if name:
175+
names.append(name)
176+
partitions.append(names)
177+
config = load_config(opts)
178+
b = get_blockade(config)
179+
b.partition(partitions)
180+
181+
182+
def cmd_join(opts):
183+
"""Restore full networking between containers
184+
"""
185+
config = load_config(opts)
186+
b = get_blockade(config)
187+
b.join()
188+
189+
190+
def cmd_logs(opts):
191+
"""Fetch the logs of a container
192+
"""
193+
config = load_config(opts)
194+
b = get_blockade(config)
195+
puts(b.logs(opts.container))
196+
197+
198+
_CMDS = (("up", cmd_up), ("destroy", cmd_destroy), ("status", cmd_status),
199+
("logs", cmd_logs), ("flaky", cmd_flaky), ("slow", cmd_slow),
200+
("fast", cmd_fast), ("partition", cmd_partition), ("join", cmd_join))
201+
202+
203+
def setup_parser():
204+
parser = argparse.ArgumentParser(description='Blockade')
205+
parser.add_argument("--config", "-c", metavar="blockade.yaml",
206+
help="Config YAML. Looks in CWD if not specified.")
207+
208+
subparsers = parser.add_subparsers(title="commands")
209+
210+
command_parsers = {}
211+
for command, func in _CMDS:
212+
subparser = subparsers.add_parser(
213+
command,
214+
description=func.__doc__,
215+
formatter_class=argparse.RawDescriptionHelpFormatter)
216+
subparser.set_defaults(func=func)
217+
command_parsers[command] = subparser
218+
219+
# add additional parameters to some commands
220+
_add_output_options(command_parsers["up"])
221+
_add_output_options(command_parsers["status"])
222+
_add_container_selection_options(command_parsers["flaky"])
223+
_add_container_selection_options(command_parsers["slow"])
224+
_add_container_selection_options(command_parsers["fast"])
225+
226+
command_parsers["logs"].add_argument("container", metavar='CONTAINER',
227+
help="Container to fetch logs for")
228+
command_parsers["partition"].add_argument(
229+
'partitions', nargs='+', metavar='PARTITION',
230+
help='Comma-separated partition')
231+
232+
return parser
233+
234+
235+
def main(args=None):
236+
parser = setup_parser()
237+
opts = parser.parse_args(args=args)
238+
239+
rc = 0
240+
241+
try:
242+
opts.func(opts)
243+
except BlockadeError, e:
244+
puts_err(colored.red("\nError:\n") + str(e) + "\n")
245+
rc = 1
246+
247+
except KeyboardInterrupt:
248+
puts_err(colored.red("Caught Ctrl-C. exiting!"))
249+
250+
except:
251+
puts_err(
252+
colored.red("\nUnexpected error! This may be a Blockade bug.\n"))
253+
traceback.print_exc()
254+
rc = 2
255+
256+
sys.exit(rc)
257+
258+
259+
if __name__ == '__main__':
260+
main()

‎blockade/config.py

+148
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import collections
18+
19+
from .errors import BlockadeConfigError
20+
21+
22+
class BlockadeContainerConfig(object):
23+
@staticmethod
24+
def from_dict(name, d):
25+
return BlockadeContainerConfig(
26+
name, d['image'],
27+
command=d.get('command'), links=d.get('links'),
28+
lxc_conf=d.get('lxc_conf'), volumes=d.get('volumes'),
29+
ports=d.get('ports'), environment=d.get('environment'))
30+
31+
def __init__(self, name, image, command=None, links=None, lxc_conf=None,
32+
volumes=None, ports=None, environment=None):
33+
self.name = name
34+
self.image = image
35+
self.command = command
36+
self.links = _dictify(links, "links")
37+
self.lxc_conf = dict(lxc_conf or {})
38+
self.volumes = _dictify(volumes, "volumes")
39+
self.ports = _dictify(ports, "ports")
40+
self.environment = dict(environment or {})
41+
42+
43+
_DEFAULT_NETWORK_CONFIG = {
44+
"flaky": "30%",
45+
"slow": "75ms 100ms distribution normal",
46+
}
47+
48+
49+
class BlockadeConfig(object):
50+
@staticmethod
51+
def from_dict(d):
52+
try:
53+
containers = d['containers']
54+
parsed_containers = {}
55+
for name, container_dict in containers.iteritems():
56+
try:
57+
container = BlockadeContainerConfig.from_dict(
58+
name, container_dict)
59+
parsed_containers[name] = container
60+
61+
except Exception, e:
62+
raise BlockadeConfigError(
63+
"Container '%s' config problem: %s" % (name, e))
64+
65+
network = d.get('network')
66+
if network:
67+
defaults = _DEFAULT_NETWORK_CONFIG.copy()
68+
defaults.update(network)
69+
network = defaults
70+
71+
else:
72+
network = _DEFAULT_NETWORK_CONFIG.copy()
73+
74+
return BlockadeConfig(parsed_containers, network=network)
75+
76+
except KeyError, e:
77+
raise BlockadeConfigError("Config missing value: " + str(e))
78+
79+
except Exception, e:
80+
# TODO log this to some debug stream?
81+
raise BlockadeConfigError("Failed to load config: " + str(e))
82+
83+
def __init__(self, containers, network=None):
84+
self.containers = containers
85+
self.sorted_containers = dependency_sorted(containers)
86+
self.network = network or {}
87+
88+
89+
def _dictify(data, name="input"):
90+
if data:
91+
if isinstance(data, collections.Sequence):
92+
return dict((str(v), str(v)) for v in data)
93+
elif isinstance(data, collections.Mapping):
94+
return dict((str(k), str(v or k)) for k, v in data.items())
95+
else:
96+
raise BlockadeConfigError("invalid %s: need list or map"
97+
% (name,))
98+
else:
99+
return {}
100+
101+
102+
def dependency_sorted(containers):
103+
"""Sort a dictionary or list of containers into dependency order
104+
105+
Returns a sequence
106+
"""
107+
if not isinstance(containers, collections.Mapping):
108+
containers = dict((c.name, c) for c in containers)
109+
110+
# use ordered dict to preserve original order of nondependent containers
111+
d = collections.OrderedDict((name, set(c.links.keys()))
112+
for name, c in containers.iteritems())
113+
sorted_names = _resolve(d)
114+
return [containers[name] for name in sorted_names]
115+
116+
117+
def _resolve(d):
118+
all_keys = set(d.keys())
119+
result = []
120+
resolved_keys = set()
121+
122+
while d:
123+
resolved_keys_count = len(resolved_keys)
124+
for name, links in d.items():
125+
# containers with no links can be started in any order.
126+
# containers whose parent containers have already been resolved
127+
# can be added now too.
128+
if not links or links <= resolved_keys:
129+
result.append(name)
130+
resolved_keys.add(name)
131+
del d[name]
132+
133+
# guard against containers which link to unknown containers
134+
unknown = links - all_keys
135+
if len(unknown) == 1:
136+
raise BlockadeConfigError(
137+
"container %s links to unknown container %s" %
138+
(name, list(unknown)[0]))
139+
elif len(unknown) > 1:
140+
raise BlockadeConfigError(
141+
"container %s links to unknown containers %s" %
142+
(name, unknown))
143+
144+
# if we made no progress this round, we have a circular dep
145+
if len(resolved_keys) == resolved_keys_count:
146+
raise BlockadeConfigError("containers have circular links!")
147+
148+
return result

‎blockade/core.py

+274
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,274 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
from copy import deepcopy
18+
19+
import docker
20+
21+
from .errors import BlockadeError
22+
from .net import NetworkState, BlockadeNetwork
23+
from .state import BlockadeStateFactory
24+
25+
26+
class Blockade(object):
27+
def __init__(self, config, state_factory=None, network=None,
28+
docker_client=None):
29+
self.config = config
30+
self.state_factory = state_factory or BlockadeStateFactory()
31+
self.network = network or BlockadeNetwork(config)
32+
self.docker_client = docker_client or docker.Client()
33+
34+
def create(self):
35+
container_state = {}
36+
for container in self.config.sorted_containers:
37+
veth_device = self.network.new_veth_device_name()
38+
container_state[container.name] = {"veth_device": veth_device}
39+
40+
# generate blockade ID and persist
41+
state = self.state_factory.initialize(container_state)
42+
43+
container_descriptions = []
44+
for container in self.config.sorted_containers:
45+
veth_device = container_state[container.name]['veth_device']
46+
container_id = self._start_container(state.blockade_id, container,
47+
veth_device)
48+
description = self._get_container_description(
49+
state, container.name, container_id)
50+
container_descriptions.append(description)
51+
52+
return container_descriptions
53+
54+
def _start_container(self, blockade_id, container, veth_device):
55+
container_name = docker_container_name(blockade_id, container.name)
56+
volumes = list(container.volumes.values()) or None
57+
response = self.docker_client.create_container(
58+
container.image, command=container.command, name=container_name,
59+
ports=container.ports, volumes=volumes, hostname=container.name,
60+
environment=container.environment)
61+
container_id = response['Id']
62+
63+
links = dict((docker_container_name(blockade_id, link), alias)
64+
for link, alias in container.links.iteritems())
65+
66+
lxc_conf = deepcopy(container.lxc_conf)
67+
lxc_conf['lxc.network.veth.pair'] = veth_device
68+
self.docker_client.start(container_id, lxc_conf=lxc_conf, links=links,
69+
binds=container.volumes)
70+
return container_id
71+
72+
def _get_container_description(self, state, name, container_id,
73+
network_state=True, ip_partitions=None):
74+
try:
75+
container = self.docker_client.inspect_container(container_id)
76+
except docker.APIError, e:
77+
if e.response.status_code == 404:
78+
return Container(name, container_id, ContainerState.MISSING)
79+
else:
80+
raise
81+
82+
state_dict = container.get('State')
83+
if state_dict and state_dict.get('Running'):
84+
container_state = ContainerState.UP
85+
else:
86+
container_state = ContainerState.DOWN
87+
88+
extras = {}
89+
network = container.get('NetworkSettings')
90+
ip = None
91+
if network:
92+
ip = network.get('IPAddress')
93+
if ip:
94+
extras['ip_address'] = ip
95+
96+
if (network_state and name in state.containers
97+
and container_state == ContainerState.UP):
98+
device = state.containers[name]['veth_device']
99+
extras['veth_device'] = device
100+
extras['network_state'] = self.network.network_state(device)
101+
102+
# include partition ID if we were provided a map of them
103+
if ip_partitions and ip:
104+
extras['partition'] = ip_partitions.get(ip)
105+
else:
106+
extras['network_state'] = NetworkState.UNKNOWN
107+
extras['veth_device'] = None
108+
109+
return Container(name, container_id, container_state, **extras)
110+
111+
def destroy(self, force=False):
112+
state = self.state_factory.load()
113+
114+
containers = self._get_docker_containers(state.blockade_id)
115+
for container in containers.values():
116+
container_id = container['Id']
117+
self.docker_client.stop(container_id, timeout=3)
118+
self.docker_client.remove_container(container_id)
119+
120+
self.network.restore(state.blockade_id)
121+
self.state_factory.destroy()
122+
123+
def _get_docker_containers(self, blockade_id):
124+
# look for containers prefixed with our blockade ID
125+
prefix = "/" + blockade_id + "-"
126+
d = {}
127+
for container in self.docker_client.containers(all=True):
128+
for name in container['Names']:
129+
if name.startswith(prefix):
130+
name = name[len(prefix):]
131+
d[name] = container
132+
break
133+
return d
134+
135+
def _get_all_containers(self, state):
136+
containers = []
137+
ip_partitions = self.network.get_ip_partitions(state.blockade_id)
138+
docker_containers = self._get_docker_containers(state.blockade_id)
139+
for name, container in docker_containers.iteritems():
140+
containers.append(self._get_container_description(state, name,
141+
container['Id'], ip_partitions=ip_partitions))
142+
return containers
143+
144+
def status(self):
145+
state = self.state_factory.load()
146+
return self._get_all_containers(state)
147+
148+
def _get_running_containers(self, container_names=None, state=None):
149+
state = state or self.state_factory.load()
150+
containers = self._get_all_containers(state)
151+
152+
running = dict((c.name, c) for c in containers
153+
if c.state == ContainerState.UP)
154+
if container_names is None:
155+
return running.values()
156+
157+
found = []
158+
for name in container_names:
159+
container = running.get(name)
160+
if not container:
161+
raise BlockadeError("Container %s is not found or not running"
162+
% (name,))
163+
found.append(container)
164+
return found
165+
166+
def _get_running_container(self, container_name, state=None):
167+
return self._get_running_containers((container_name,), state)[0]
168+
169+
def flaky(self, container_names=None, include_all=False):
170+
if include_all:
171+
container_names = None
172+
containers = self._get_running_containers(container_names)
173+
for container in containers:
174+
self.network.flaky(container.veth_device)
175+
176+
def slow(self, container_names=None, include_all=False):
177+
if include_all:
178+
container_names = None
179+
containers = self._get_running_containers(container_names)
180+
for container in containers:
181+
self.network.slow(container.veth_device)
182+
183+
def fast(self, container_names=None, include_all=False):
184+
if include_all:
185+
container_names = None
186+
containers = self._get_running_containers(container_names)
187+
for container in containers:
188+
self.network.fast(container.veth_device)
189+
190+
def partition(self, partitions):
191+
state = self.state_factory.load()
192+
containers = self._get_running_containers(state=state)
193+
container_dict = dict((c.name, c) for c in containers)
194+
partitions = expand_partitions(container_dict.keys(), partitions)
195+
196+
container_partitions = []
197+
for partition in partitions:
198+
container_partitions.append([container_dict[c] for c in partition])
199+
200+
self.network.partition_containers(state.blockade_id,
201+
container_partitions)
202+
203+
def join(self):
204+
state = self.state_factory.load()
205+
self.network.restore(state.blockade_id)
206+
207+
def logs(self, container_name):
208+
container = self._get_running_container(container_name)
209+
return self.docker_client.logs(container.container_id)
210+
211+
212+
class Container(object):
213+
ip_address = None
214+
veth_device = None
215+
network_state = NetworkState.NORMAL
216+
partition = None
217+
218+
def __init__(self, name, container_id, state, **kwargs):
219+
self.name = name
220+
self.container_id = container_id
221+
self.state = state
222+
for k, v in kwargs.iteritems():
223+
setattr(self, k, v)
224+
225+
def to_dict(self):
226+
return dict(name=self.name, container_id=self.container_id,
227+
state=self.state, ip_address=self.ip_address,
228+
veth_device=self.veth_device,
229+
network_state=self.network_state,
230+
partition=self.partition)
231+
232+
233+
class ContainerState(object):
234+
UP = "UP"
235+
DOWN = "DOWN"
236+
MISSING = "MISSING"
237+
238+
239+
def docker_container_name(blockade_id, name):
240+
return '-'.join((blockade_id, name))
241+
242+
243+
def expand_partitions(containers, partitions):
244+
"""Validate the partitions of containers. If there are any containers
245+
not in any partition, place them in an new partition.
246+
"""
247+
all_names = frozenset(containers)
248+
partitions = [frozenset(p) for p in partitions]
249+
250+
unknown = set()
251+
overlap = set()
252+
union = set()
253+
254+
for index, partition in enumerate(partitions):
255+
unknown.update(partition - all_names)
256+
union.update(partition)
257+
258+
for other in partitions[index+1:]:
259+
overlap.update(partition.intersection(other))
260+
261+
if unknown:
262+
raise BlockadeError("Partitions have unknown containers: %s" %
263+
list(unknown))
264+
265+
if overlap:
266+
raise BlockadeError("Partitions have overlapping containers: %s" %
267+
list(overlap))
268+
269+
# put any leftover containers in an implicit partition
270+
leftover = all_names.difference(union)
271+
if leftover:
272+
partitions.append(leftover)
273+
274+
return partitions

‎blockade/errors.py

+40
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
18+
class BlockadeError(Exception):
19+
"""Expected error within Blockade
20+
"""
21+
22+
23+
class BlockadeConfigError(BlockadeError):
24+
"""Error in configuration
25+
"""
26+
27+
28+
class AlreadyInitializedError(BlockadeError):
29+
"""Blockade already created in this context
30+
"""
31+
32+
33+
class NotInitializedError(BlockadeError):
34+
"""Blockade not created in this context
35+
"""
36+
37+
38+
class InconsistentStateError(BlockadeError):
39+
"""Blockade state is inconsistent (partially created or destroyed)
40+
"""

‎blockade/net.py

+282
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,282 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import random
18+
import string
19+
import subprocess
20+
21+
from .errors import BlockadeError
22+
23+
24+
class NetworkState(object):
25+
NORMAL = "NORMAL"
26+
SLOW = "SLOW"
27+
FLAKY = "FLAKY"
28+
UNKNOWN = "UNKNOWN"
29+
30+
31+
class BlockadeNetwork(object):
32+
def __init__(self, config):
33+
self.config = config
34+
35+
def new_veth_device_name(self):
36+
chars = string.ascii_letters + string.digits
37+
return "veth" + "".join(random.choice(chars) for _ in range(8))
38+
39+
def network_state(self, device):
40+
return network_state(device)
41+
42+
def flaky(self, device):
43+
flaky_config = self.config.network['flaky'].split()
44+
traffic_control_netem(device, ["loss"] + flaky_config)
45+
46+
def slow(self, device):
47+
slow_config = self.config.network['slow'].split()
48+
traffic_control_netem(device, ["delay"] + slow_config)
49+
50+
def fast(self, device):
51+
traffic_control_restore(device)
52+
53+
def restore(self, blockade_id):
54+
clear_iptables(blockade_id)
55+
56+
def partition_containers(self, blockade_id, partitions):
57+
clear_iptables(blockade_id)
58+
partition_containers(blockade_id, partitions)
59+
60+
def get_ip_partitions(self, blockade_id):
61+
return iptables_get_source_chains(blockade_id)
62+
63+
64+
def parse_partition_index(blockade_id, chain):
65+
prefix = "%s-p" % (blockade_id,)
66+
if chain and chain.startswith(prefix):
67+
try:
68+
return int(chain[len(prefix):])
69+
except ValueError:
70+
pass
71+
raise ValueError("chain %s is not a blockade partition" % (chain,))
72+
73+
74+
def partition_chain_name(blockade_id, partition_index):
75+
return "%s-p%s" % (blockade_id, partition_index)
76+
77+
78+
def iptables_call_output(*args):
79+
cmd = ["iptables", "-n"] + list(args)
80+
try:
81+
output = subprocess.check_output(cmd)
82+
return output.split("\n")
83+
except subprocess.CalledProcessError:
84+
raise BlockadeError("Problem calling '%s'" % " ".join(cmd))
85+
86+
87+
def iptables_call(*args):
88+
cmd = ["iptables"] + list(args)
89+
try:
90+
subprocess.check_call(cmd)
91+
except subprocess.CalledProcessError:
92+
raise BlockadeError("Problem calling '%s'" % " ".join(cmd))
93+
94+
95+
def iptables_get_chain_rules(chain):
96+
if not chain:
97+
raise ValueError("invalid chain")
98+
lines = iptables_call_output("-L", chain)
99+
if len(lines) < 2:
100+
raise BlockadeError("Can't understand iptables output: \n%s" %
101+
"\n".join(lines))
102+
103+
chain_line, header_line = lines[:2]
104+
if not (chain_line.startswith("Chain " + chain) and
105+
header_line.startswith("target")):
106+
raise BlockadeError("Can't understand iptables output: \n%s" %
107+
"\n".join(lines))
108+
return lines[2:]
109+
110+
111+
def iptables_get_source_chains(blockade_id):
112+
"""Get a map of blockade chains IDs -> list of IPs targeted at them
113+
114+
For figuring out which container is in which partition
115+
"""
116+
result = {}
117+
if not blockade_id:
118+
raise ValueError("invalid blockade_id")
119+
lines = iptables_get_chain_rules("FORWARD")
120+
121+
for line in lines:
122+
parts = line.split()
123+
if len(parts) < 4:
124+
continue
125+
try:
126+
partition_index = parse_partition_index(blockade_id, parts[0])
127+
except ValueError:
128+
continue # not a rule targetting a blockade chain
129+
130+
source = parts[3]
131+
if source:
132+
result[source] = partition_index
133+
return result
134+
135+
136+
def iptables_delete_rules(chain, predicate):
137+
if not chain:
138+
raise ValueError("invalid chain")
139+
if not callable(predicate):
140+
raise ValueError("invalid predicate")
141+
142+
lines = iptables_get_chain_rules(chain)
143+
144+
# TODO this is susceptible to check-then-act races.
145+
# better to ultimately switch to python-iptables if it becomes less buggy
146+
for index, line in reversed(list(enumerate(lines, 1))):
147+
line = line.strip()
148+
if line and predicate(line):
149+
iptables_call("-D", chain, str(index))
150+
151+
152+
def iptables_delete_blockade_rules(blockade_id):
153+
def predicate(rule):
154+
target = rule.split()[0]
155+
try:
156+
parse_partition_index(blockade_id, target)
157+
except ValueError:
158+
return False
159+
return True
160+
iptables_delete_rules("FORWARD", predicate)
161+
162+
163+
def iptables_delete_blockade_chains(blockade_id):
164+
if not blockade_id:
165+
raise ValueError("invalid blockade_id")
166+
167+
lines = iptables_call_output("-L")
168+
for line in lines:
169+
parts = line.split()
170+
if len(parts) >= 2 and parts[0] == "Chain":
171+
chain = parts[1]
172+
try:
173+
parse_partition_index(blockade_id, chain)
174+
except ValueError:
175+
continue
176+
# if we are a valid blockade chain, flush and delete
177+
iptables_call("-F", chain)
178+
iptables_call("-X", chain)
179+
180+
181+
def iptables_insert_rule(chain, src=None, dest=None, target=None):
182+
"""Insert a new rule in the chain
183+
"""
184+
if not chain:
185+
raise ValueError("Invalid chain")
186+
if not target:
187+
raise ValueError("Invalid target")
188+
if not (src or dest):
189+
raise ValueError("Need src, dest, or both")
190+
191+
args = ["-I", chain]
192+
if src:
193+
args += ["-s", src]
194+
if dest:
195+
args += ["-d", dest]
196+
args += ["-j", target]
197+
iptables_call(*args)
198+
199+
200+
def iptables_create_chain(chain):
201+
"""Create a new chain
202+
"""
203+
if not chain:
204+
raise ValueError("Invalid chain")
205+
iptables_call("-N", chain)
206+
207+
208+
def clear_iptables(blockade_id):
209+
"""Remove all iptables rules and chains related to this blockade
210+
"""
211+
# first remove refererences to our custom chains
212+
iptables_delete_blockade_rules(blockade_id)
213+
214+
# then remove the chains themselves
215+
iptables_delete_blockade_chains(blockade_id)
216+
217+
218+
def partition_containers(blockade_id, partitions):
219+
if not partitions or len(partitions) == 1:
220+
return
221+
for index, partition in enumerate(partitions, 1):
222+
chain_name = partition_chain_name(blockade_id, index)
223+
224+
# create chain for partition and block traffic TO any other partition
225+
iptables_create_chain(chain_name)
226+
for other in partitions:
227+
if partition is other:
228+
continue
229+
for container in other:
230+
if container.ip_address:
231+
iptables_insert_rule(chain_name, dest=container.ip_address,
232+
target="DROP")
233+
234+
# direct traffic FROM any container in the partition to the new chain
235+
for container in partition:
236+
iptables_insert_rule("FORWARD", src=container.ip_address,
237+
target=chain_name)
238+
239+
240+
def traffic_control_restore(device):
241+
cmd = ["tc", "qdisc", "del", "dev", device, "root"]
242+
243+
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
244+
stderr=subprocess.PIPE)
245+
stdout, stderr = p.communicate()
246+
247+
if p.returncode != 0:
248+
if p.returncode == 2 and stderr:
249+
if "No such file or directory" in stderr:
250+
return
251+
252+
# TODO log error somewhere?
253+
raise BlockadeError("Problem calling traffic control: " +
254+
" ".join(cmd))
255+
256+
257+
def traffic_control_netem(device, params):
258+
try:
259+
cmd = ["tc", "qdisc", "replace", "dev", device,
260+
"root", "netem"] + params
261+
subprocess.check_call(cmd)
262+
263+
except subprocess.CalledProcessError:
264+
# TODO log error somewhere?
265+
raise BlockadeError("Problem calling traffic control: " +
266+
" ".join(cmd))
267+
268+
269+
def network_state(device):
270+
try:
271+
output = subprocess.check_output(
272+
["tc", "qdisc", "show", "dev", device])
273+
# sloppy but good enough for now
274+
if " delay " in output:
275+
return NetworkState.SLOW
276+
if " loss " in output:
277+
return NetworkState.FLAKY
278+
return NetworkState.NORMAL
279+
280+
except subprocess.CalledProcessError:
281+
# TODO log error somewhere?
282+
return NetworkState.UNKNOWN

‎blockade/state.py

+122
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import uuid
18+
import os
19+
import errno
20+
from copy import deepcopy
21+
22+
import yaml
23+
24+
from .errors import AlreadyInitializedError, NotInitializedError, \
25+
InconsistentStateError
26+
27+
BLOCKADE_STATE_DIR = ".blockade"
28+
BLOCKADE_STATE_FILE = ".blockade/state.yml"
29+
BLOCKADE_ID_PREFIX = "blockade-"
30+
BLOCKADE_STATE_VERSION = 1
31+
32+
33+
def _assure_dir():
34+
try:
35+
os.mkdir(BLOCKADE_STATE_DIR)
36+
except OSError, e:
37+
if e.errno != errno.EEXIST:
38+
raise
39+
40+
41+
def _state_delete():
42+
try:
43+
os.remove(BLOCKADE_STATE_FILE)
44+
except OSError, e:
45+
if e.errno not in (errno.EPERM, errno.ENOENT):
46+
raise
47+
48+
try:
49+
os.rmdir(BLOCKADE_STATE_DIR)
50+
except OSError, e:
51+
if e.errno not in (errno.ENOTEMPTY, errno.ENOENT):
52+
raise
53+
54+
55+
def _base_state(blockade_id, containers):
56+
return dict(blockade_id=blockade_id, containers=containers,
57+
version=BLOCKADE_STATE_VERSION)
58+
59+
60+
class BlockadeState(object):
61+
def __init__(self, blockade_id, containers):
62+
self._blockade_id = blockade_id
63+
self._containers = containers
64+
65+
@property
66+
def blockade_id(self):
67+
return self._blockade_id
68+
69+
@property
70+
def containers(self):
71+
return deepcopy(self._containers)
72+
73+
74+
class BlockadeStateFactory(object):
75+
# annoyed with how this ended up structured, and that I called it
76+
# a factory, but fuckit..
77+
78+
@staticmethod
79+
def initialize(containers, blockade_id=None):
80+
if blockade_id is None:
81+
blockade_id = BLOCKADE_ID_PREFIX + uuid.uuid4().hex[:10]
82+
containers = deepcopy(containers)
83+
84+
f = None
85+
path = BLOCKADE_STATE_FILE
86+
_assure_dir()
87+
try:
88+
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
89+
with os.fdopen(os.open(path, flags), "w") as f:
90+
yaml.dump(_base_state(blockade_id, containers), f)
91+
except OSError, e:
92+
if e.errno == errno.EEXIST:
93+
raise AlreadyInitializedError(
94+
"Path %s exists. "
95+
"You may need to destroy a previous blockade." % path)
96+
raise
97+
except Exception:
98+
# clean up our created file
99+
_state_delete()
100+
raise
101+
return BlockadeState(blockade_id, containers)
102+
103+
@staticmethod
104+
def load():
105+
try:
106+
with open(BLOCKADE_STATE_FILE) as f:
107+
state = yaml.safe_load(f)
108+
return BlockadeState(state['blockade_id'], state['containers'])
109+
110+
except (IOError, OSError), e:
111+
if e.errno == errno.ENOENT:
112+
raise NotInitializedError("No blockade exists in this context")
113+
raise InconsistentStateError("Failed to load Blockade state: "
114+
+ str(e))
115+
116+
except Exception, e:
117+
raise InconsistentStateError("Failed to load Blockade state: "
118+
+ str(e))
119+
120+
@staticmethod
121+
def destroy():
122+
_state_delete()

‎blockade/tests/__init__.py

Whitespace-only changes.

‎blockade/tests/test_cli.py

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import unittest
18+
19+
from blockade import cli
20+
21+
22+
class CommandLineTests(unittest.TestCase):
23+
24+
def test_parser(self):
25+
# just make sure we don't have any typos for now
26+
cli.setup_parser()

‎blockade/tests/test_config.py

+233
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import unittest
18+
19+
from blockade.errors import BlockadeConfigError
20+
from blockade.config import BlockadeConfig, BlockadeContainerConfig, \
21+
dependency_sorted
22+
23+
24+
class ConfigTests(unittest.TestCase):
25+
26+
def test_parse_1(self):
27+
containers = {
28+
"c1": {"image": "image1", "command": "/bin/bash"},
29+
"c2": {"image": "image2", "links": ["c1"]}
30+
}
31+
d = dict(containers=containers)
32+
33+
config = BlockadeConfig.from_dict(d)
34+
self.assertEqual(len(config.containers), 2)
35+
self.assertEqual(config.containers["c1"].name, "c1")
36+
self.assertEqual(config.containers["c1"].image, "image1")
37+
self.assertEqual(config.containers["c1"].command, "/bin/bash")
38+
self.assertEqual(config.containers["c2"].name, "c2")
39+
self.assertEqual(config.containers["c2"].image, "image2")
40+
41+
def test_parse_2(self):
42+
containers = {
43+
"c1": {"image": "image1", "command": "/bin/bash"}
44+
}
45+
network = {"flaky": "61%"}
46+
d = dict(containers=containers, network=network)
47+
48+
config = BlockadeConfig.from_dict(d)
49+
# default value should be there
50+
self.assertIn("flaky", config.network)
51+
self.assertEqual(config.network['flaky'], "61%")
52+
# default value should be there
53+
self.assertIn("slow", config.network)
54+
55+
def test_parse_with_volumes_1(self):
56+
containers = {
57+
"c1": {"image": "image1", "command": "/bin/bash",
58+
"volumes": {"/some/mount": "/some/place"}}
59+
}
60+
network = {}
61+
d = dict(containers=containers, network=network)
62+
63+
config = BlockadeConfig.from_dict(d)
64+
# default value should be there
65+
self.assertEqual(len(config.containers), 1)
66+
c1 = config.containers['c1']
67+
self.assertEqual(c1.volumes, {"/some/mount": "/some/place"})
68+
69+
def test_parse_with_volumes_2(self):
70+
containers = {
71+
"c1": {"image": "image1", "command": "/bin/bash",
72+
"volumes": ["/some/mount"]}
73+
}
74+
network = {}
75+
d = dict(containers=containers, network=network)
76+
77+
config = BlockadeConfig.from_dict(d)
78+
# default value should be there
79+
self.assertEqual(len(config.containers), 1)
80+
c1 = config.containers['c1']
81+
self.assertEqual(c1.volumes, {"/some/mount": "/some/mount"})
82+
83+
def test_parse_with_volumes_3(self):
84+
containers = {
85+
"c1": {"image": "image1", "command": "/bin/bash",
86+
"volumes": {"/some/mount": ""}}
87+
}
88+
network = {}
89+
d = dict(containers=containers, network=network)
90+
91+
config = BlockadeConfig.from_dict(d)
92+
# default value should be there
93+
self.assertEqual(len(config.containers), 1)
94+
c1 = config.containers['c1']
95+
self.assertEqual(c1.volumes, {"/some/mount": "/some/mount"})
96+
97+
def test_parse_with_volumes_4(self):
98+
containers = {
99+
"c1": {"image": "image1", "command": "/bin/bash",
100+
"volumes": {"/some/mount": None}}
101+
}
102+
network = {}
103+
d = dict(containers=containers, network=network)
104+
105+
config = BlockadeConfig.from_dict(d)
106+
# default value should be there
107+
self.assertEqual(len(config.containers), 1)
108+
c1 = config.containers['c1']
109+
self.assertEqual(c1.volumes, {"/some/mount": "/some/mount"})
110+
111+
def test_parse_with_env_1(self):
112+
containers = {
113+
"c1": {"image": "image1", "command": "/bin/bash",
114+
"environment": {"HATS": 4, "JACKETS": "some"}}
115+
}
116+
d = dict(containers=containers, network={})
117+
118+
config = BlockadeConfig.from_dict(d)
119+
self.assertEqual(len(config.containers), 1)
120+
c1 = config.containers['c1']
121+
self.assertEqual(c1.environment, {"HATS": 4, "JACKETS": "some"})
122+
123+
def test_parse_with_numeric_port(self):
124+
containers = {
125+
"c1": {"image": "image1", "command": "/bin/bash",
126+
"ports": [10000]}
127+
}
128+
d = dict(containers=containers, network={})
129+
130+
config = BlockadeConfig.from_dict(d)
131+
self.assertEqual(len(config.containers), 1)
132+
c1 = config.containers['c1']
133+
self.assertEqual(c1.ports, {"10000": "10000"})
134+
135+
def test_parse_fail_1(self):
136+
containers = {
137+
"c1": {"image": "image1", "command": "/bin/bash"},
138+
"c2": {"image": "image2", "links": ["c1"]}
139+
}
140+
d = dict(contianers=containers)
141+
with self.assertRaises(BlockadeConfigError):
142+
BlockadeConfig.from_dict(d)
143+
144+
def test_parse_fail_2(self):
145+
containers = {
146+
"c1": {"ima": "image1", "command": "/bin/bash"},
147+
"c2": {"image": "image2", "links": ["c1"]}
148+
}
149+
d = dict(containers=containers)
150+
with self.assertRaises(BlockadeConfigError):
151+
BlockadeConfig.from_dict(d)
152+
153+
def test_link_ordering_1(self):
154+
containers = [BlockadeContainerConfig("c1", "image"),
155+
BlockadeContainerConfig("c2", "image"),
156+
BlockadeContainerConfig("c3", "image")]
157+
ordered = dependency_sorted(containers)
158+
ordered_names = [c.name for c in ordered]
159+
self.assertDependencyLevels(ordered_names, ["c1", "c2", "c3"])
160+
161+
def test_link_ordering_2(self):
162+
containers = [BlockadeContainerConfig("c1", "image"),
163+
BlockadeContainerConfig("c2", "image",
164+
links={"c1": "c1"}),
165+
BlockadeContainerConfig("c3", "image")]
166+
ordered = dependency_sorted(containers)
167+
ordered_names = [c.name for c in ordered]
168+
self.assertDependencyLevels(ordered_names,
169+
["c1", "c3"],
170+
["c2"])
171+
172+
def test_link_ordering_3(self):
173+
containers = [BlockadeContainerConfig("c1", "image"),
174+
BlockadeContainerConfig("c2", "image",
175+
links={"c1": "c1"}),
176+
BlockadeContainerConfig("c3", "image",
177+
links={"c1": "c1"})]
178+
ordered = dependency_sorted(containers)
179+
ordered_names = [c.name for c in ordered]
180+
self.assertDependencyLevels(ordered_names, ["c1"], ["c2", "c3"])
181+
182+
def test_link_ordering_4(self):
183+
containers = [BlockadeContainerConfig("c1", "image"),
184+
BlockadeContainerConfig("c2", "image", links=["c1"]),
185+
BlockadeContainerConfig("c3", "image", links=["c1"]),
186+
BlockadeContainerConfig("c4", "image",
187+
links=["c1", "c3"]),
188+
BlockadeContainerConfig("c5", "image",
189+
links=["c2", "c3"]),
190+
]
191+
ordered = dependency_sorted(containers)
192+
ordered_names = [c.name for c in ordered]
193+
self.assertDependencyLevels(ordered_names, ["c1"], ["c2", "c3"],
194+
["c4", "c5"])
195+
196+
def test_link_ordering_unknown_1(self):
197+
containers = [BlockadeContainerConfig("c1", "image"),
198+
BlockadeContainerConfig("c2", "image", links=["c6"]),
199+
BlockadeContainerConfig("c3", "image", links=["c1"])]
200+
with self.assertRaisesRegexp(BlockadeConfigError, "unknown"):
201+
dependency_sorted(containers)
202+
203+
def test_link_ordering_unknown_2(self):
204+
containers = [BlockadeContainerConfig("c1", "image"),
205+
BlockadeContainerConfig("c2", "image",
206+
links=["c6", "c7"]),
207+
BlockadeContainerConfig("c3", "image", links=["c1"])]
208+
with self.assertRaisesRegexp(BlockadeConfigError, "unknown"):
209+
dependency_sorted(containers)
210+
211+
def test_link_ordering_circular_1(self):
212+
containers = [BlockadeContainerConfig("c1", "image"),
213+
BlockadeContainerConfig("c2", "image", links=["c1"]),
214+
BlockadeContainerConfig("c3", "image", links=["c3"])]
215+
216+
with self.assertRaisesRegexp(BlockadeConfigError, "circular"):
217+
dependency_sorted(containers)
218+
219+
def test_link_ordering_circular_2(self):
220+
containers = [BlockadeContainerConfig("c1", "image"),
221+
BlockadeContainerConfig("c2", "image",
222+
links=["c1", "c3"]),
223+
BlockadeContainerConfig("c3", "image", links=["c2"])]
224+
225+
with self.assertRaisesRegexp(BlockadeConfigError, "circular"):
226+
dependency_sorted(containers)
227+
228+
def assertDependencyLevels(self, seq, *levels):
229+
self.assertEquals(len(seq), sum(len(l) for l in levels))
230+
231+
for level in levels:
232+
self.assertEquals(set(level), set(seq[:len(level)]))
233+
seq = seq[len(level):]

‎blockade/tests/test_core.py

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import unittest
18+
19+
import mock
20+
21+
from blockade.core import Blockade, expand_partitions
22+
from blockade.errors import BlockadeError
23+
from blockade.config import BlockadeContainerConfig, BlockadeConfig
24+
from blockade.state import BlockadeState
25+
26+
27+
class BlockadeCoreTests(unittest.TestCase):
28+
29+
def setUp(self):
30+
self.network = mock.Mock()
31+
32+
self.state_factory = mock.Mock()
33+
self.docker_client = mock.Mock()
34+
35+
def test_create(self):
36+
containers = [BlockadeContainerConfig("c1", "image"),
37+
BlockadeContainerConfig("c2", "image"),
38+
BlockadeContainerConfig("c3", "image")]
39+
config = BlockadeConfig(containers)
40+
41+
self.network.new_veth_device_name.side_effect = ["veth1", "veth2",
42+
"veth3"]
43+
initialize = lambda x: BlockadeState("ourblockadeid", x)
44+
self.state_factory.initialize.side_effect = initialize
45+
self.docker_client.create_container.side_effect = [
46+
{"Id": "container1"},
47+
{"Id": "container2"},
48+
{"Id": "container3"}]
49+
50+
b = Blockade(config, self.state_factory, self.network,
51+
self.docker_client)
52+
b.create()
53+
54+
self.assertEqual(self.state_factory.initialize.call_count, 1)
55+
self.assertEqual(self.docker_client.create_container.call_count, 3)
56+
57+
def test_expand_partitions(self):
58+
containers = ["c1", "c2", "c3", "c4", "c5"]
59+
60+
partitions = expand_partitions(containers, [["c1", "c3"]])
61+
self.assert_partitions(partitions, [["c1", "c3"], ["c2", "c4", "c5"]])
62+
63+
partitions = expand_partitions(containers, [["c1", "c3"], ["c4"]])
64+
self.assert_partitions(partitions, [["c1", "c3"], ["c2", "c5"],
65+
["c4"]])
66+
67+
with self.assertRaisesRegexp(BlockadeError, "unknown"):
68+
expand_partitions(containers, [["c1"], ["c100"]])
69+
70+
with self.assertRaisesRegexp(BlockadeError, "overlap"):
71+
expand_partitions(containers, [["c1"], ["c1", "c2"]])
72+
73+
def assert_partitions(self, partitions1, partitions2):
74+
setofsets1 = frozenset(frozenset(n) for n in partitions1)
75+
setofsets2 = frozenset(frozenset(n) for n in partitions2)
76+
self.assertEqual(setofsets1, setofsets2)

‎blockade/tests/test_integration.py

+150
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
import os
17+
import unittest
18+
import sys
19+
import tempfile
20+
import shutil
21+
import traceback
22+
import json
23+
from StringIO import StringIO
24+
25+
import mock
26+
27+
import blockade.cli
28+
29+
30+
INT_ENV = "BLOCKADE_INTEGRATION_TESTS"
31+
INT_SKIP = (not os.getenv(INT_ENV), "export %s=1 to run" % INT_ENV)
32+
33+
34+
class FakeExit(BaseException):
35+
def __init__(self, rc):
36+
self.rc = rc
37+
38+
39+
def example_config_path(filename):
40+
example_dir = os.path.join(os.path.dirname(__file__), "../..", "examples")
41+
example_dir = os.path.abspath(example_dir)
42+
if not os.path.exists(example_dir):
43+
raise Exception("example config directory not found: %s" % example_dir)
44+
45+
config_path = os.path.join(example_dir, filename)
46+
if not os.path.exists(config_path):
47+
raise Exception("example config not found: %s" % config_path)
48+
return config_path
49+
50+
51+
class IntegrationTests(unittest.TestCase):
52+
"""Integration tests that run the full CLI args down.
53+
54+
Tests that are Linux and Docker only should be decorated with:
55+
@unittest.skipIf(*INT_SKIP)
56+
57+
They will only be run when BLOCKADE_INTEGRATION_TESTS=1 env is set.
58+
"""
59+
60+
sysexit_patch = None
61+
stderr_patch = None
62+
tempdir = None
63+
oldcwd = None
64+
65+
def setUp(self):
66+
self.sysexit_patch = mock.patch("sys.exit")
67+
self.mock_sysexit = self.sysexit_patch.start()
68+
69+
def exit(rc):
70+
raise FakeExit(rc)
71+
72+
self.mock_sysexit.side_effect = exit
73+
74+
self.tempdir = tempfile.mkdtemp()
75+
self.oldcwd = os.getcwd()
76+
os.chdir(self.tempdir)
77+
78+
def tearDown(self):
79+
if self.sysexit_patch:
80+
self.sysexit_patch.stop()
81+
82+
if self.oldcwd:
83+
os.chdir(self.oldcwd)
84+
if self.tempdir:
85+
try:
86+
shutil.rmtree(self.tempdir)
87+
except Exception:
88+
pass
89+
90+
def call_blockade(self, *args):
91+
stdout = StringIO()
92+
stderr = StringIO()
93+
with mock.patch("blockade.cli.puts") as mock_puts:
94+
mock_puts.side_effect = stdout.write
95+
96+
with mock.patch("blockade.cli.puts_err") as mock_puts_err:
97+
mock_puts_err.side_effect = stderr.write
98+
99+
try:
100+
blockade.cli.main(args)
101+
except FakeExit, e:
102+
if e.rc != 0:
103+
raise
104+
return (stdout.getvalue(), stderr.getvalue())
105+
106+
def test_badargs(self):
107+
with mock.patch("sys.stderr"):
108+
with self.assertRaises(FakeExit) as cm:
109+
self.call_blockade("--notarealarg")
110+
111+
self.assertEqual(cm.exception.rc, 2)
112+
113+
@unittest.skipIf(*INT_SKIP)
114+
def test_containers(self):
115+
config_path = example_config_path("sleep/blockade.yaml")
116+
117+
# TODO make this better. so far we just walk through all
118+
# the major operations, but don't really assert anything
119+
# other than exit code.
120+
try:
121+
self.call_blockade("-c", config_path, "up")
122+
123+
self.call_blockade("-c", config_path, "status")
124+
stdout, _ = self.call_blockade("-c", config_path, "status",
125+
"--json")
126+
parsed = json.loads(stdout)
127+
self.assertEqual(len(parsed), 3)
128+
129+
self.call_blockade("-c", config_path, "flaky", "c1")
130+
self.call_blockade("-c", config_path, "slow", "c2", "c3")
131+
self.call_blockade("-c", config_path, "fast", "c3")
132+
133+
# make sure it is harmless for call fast when nothing is slow
134+
self.call_blockade("-c", config_path, "fast", "--all")
135+
136+
with self.assertRaises(FakeExit):
137+
self.call_blockade("-c", config_path, "slow", "notarealnode")
138+
139+
self.call_blockade("-c", config_path, "partition", "c1,c2", "c3")
140+
self.call_blockade("-c", config_path, "join")
141+
142+
stdout, _ = self.call_blockade("-c", config_path, "logs", "c1")
143+
self.assertEquals("I am c1", stdout.strip())
144+
145+
finally:
146+
try:
147+
self.call_blockade("-c", config_path, "destroy")
148+
except Exception:
149+
print "Failed to destroy Blockade!"
150+
traceback.print_exc(file=sys.stdout)

‎blockade/tests/test_net.py

+235
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import unittest
18+
import subprocess
19+
20+
import mock
21+
22+
import blockade.net
23+
from blockade.net import NetworkState, BlockadeNetwork, \
24+
parse_partition_index, partition_chain_name
25+
26+
27+
NORMAL_QDISC_SHOW = "qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap\n"
28+
SLOW_QDISC_SHOW = "qdisc netem 8011: root refcnt 2 limit 1000 delay 50.0ms\n"
29+
FLAKY_QDISC_SHOW = "qdisc netem 8011: root refcnt 2 limit 1000 loss 50%\n"
30+
31+
QDISC_DEL_NOENT = "RTNETLINK answers: No such file or directory"
32+
33+
34+
_IPTABLES_LIST_FORWARD_1 = """Chain FORWARD (policy ACCEPT)
35+
target prot opt source destination
36+
blockade-aa43racd2-p1 all -- 172.17.0.16 anywhere
37+
blockade-4eraffr-p1 all -- 172.17.0.17 anywhere
38+
blockade-e5dcf85cd2-p1 all -- 172.17.0.162 anywhere
39+
blockade-e5dcf85cd2-p1 all -- 172.17.0.164 anywhere
40+
ACCEPT tcp -- 172.17.0.162 172.17.0.164 tcp spt:8000
41+
ACCEPT tcp -- 172.17.0.164 172.17.0.162 tcp dpt:8000
42+
ACCEPT tcp -- 172.17.0.162 172.17.0.163 tcp spt:8000
43+
ACCEPT tcp -- 172.17.0.163 172.17.0.162 tcp dpt:8000
44+
ACCEPT all -- anywhere anywhere
45+
ACCEPT all -- anywhere anywhere
46+
"""
47+
48+
_IPTABLES_LIST_FORWARD_2 = """Chain FORWARD (policy ACCEPT)
49+
target prot opt source destination
50+
"""
51+
52+
_IPTABLES_LIST_1 = """Chain INPUT (policy ACCEPT)
53+
target prot opt source destination
54+
55+
Chain FORWARD (policy ACCEPT)
56+
target prot opt source destination
57+
blockade-e5dcf85cd2-p1 all -- 172.17.0.162 anywhere
58+
blockade-e5dcf85cd2-p1 all -- 172.17.0.164 anywhere
59+
ACCEPT tcp -- 172.17.0.162 172.17.0.164 tcp spt:8000
60+
ACCEPT tcp -- 172.17.0.164 172.17.0.162 tcp dpt:8000
61+
ACCEPT tcp -- 172.17.0.162 172.17.0.163 tcp spt:8000
62+
ACCEPT tcp -- 172.17.0.163 172.17.0.162 tcp dpt:8000
63+
ACCEPT all -- anywhere anywhere
64+
ACCEPT all -- anywhere anywhere
65+
66+
Chain OUTPUT (policy ACCEPT)
67+
target prot opt source destination
68+
69+
Chain blockade-e5dcf85cd2-p1 (2 references)
70+
target prot opt source destination
71+
DROP all -- anywhere 172.17.0.163
72+
73+
Chain blockade-e5dcf85cd2-p2 (0 references)
74+
target prot opt source destination
75+
DROP all -- anywhere 172.17.0.162
76+
DROP all -- anywhere 172.17.0.164
77+
"""
78+
79+
_IPTABLES_LIST_2 = """Chain INPUT (policy ACCEPT)
80+
target prot opt source destination
81+
82+
Chain FORWARD (policy ACCEPT)
83+
target prot opt source destination
84+
ACCEPT tcp -- 172.17.0.162 172.17.0.164 tcp spt:8000
85+
ACCEPT tcp -- 172.17.0.164 172.17.0.162 tcp dpt:8000
86+
ACCEPT tcp -- 172.17.0.162 172.17.0.163 tcp spt:8000
87+
ACCEPT tcp -- 172.17.0.163 172.17.0.162 tcp dpt:8000
88+
ACCEPT all -- anywhere anywhere
89+
ACCEPT all -- anywhere anywhere
90+
91+
Chain OUTPUT (policy ACCEPT)
92+
target prot opt source destination
93+
"""
94+
95+
96+
class NetTests(unittest.TestCase):
97+
def test_iptables_get_blockade_chains(self):
98+
blockade_id = "blockade-e5dcf85cd2"
99+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
100+
mock_check_output = mock_subprocess.check_output
101+
mock_check_output.return_value = _IPTABLES_LIST_FORWARD_1
102+
result = blockade.net.iptables_get_source_chains(blockade_id)
103+
104+
self.assertEqual(mock_subprocess.check_output.call_count, 1)
105+
self.assertEqual(result, {"172.17.0.162": 1, "172.17.0.164": 1})
106+
107+
def test_iptables_delete_blockade_rules_1(self):
108+
blockade_id = "blockade-e5dcf85cd2"
109+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
110+
mock_check_output = mock_subprocess.check_output
111+
mock_check_output.return_value = _IPTABLES_LIST_FORWARD_1
112+
blockade.net.iptables_delete_blockade_rules(blockade_id)
113+
114+
self.assertEqual(mock_subprocess.check_output.call_count, 1)
115+
116+
# rules should be removed in reverse order
117+
expected_calls = [mock.call(["iptables", "-D", "FORWARD", "4"]),
118+
mock.call(["iptables", "-D", "FORWARD", "3"])]
119+
self.assertEqual(mock_subprocess.check_call.call_args_list,
120+
expected_calls)
121+
122+
def test_iptables_delete_blockade_rules_2(self):
123+
blockade_id = "blockade-e5dcf85cd2"
124+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
125+
mock_check_output = mock_subprocess.check_output
126+
mock_check_output.return_value = _IPTABLES_LIST_FORWARD_2
127+
blockade.net.iptables_delete_blockade_rules(blockade_id)
128+
129+
self.assertEqual(mock_subprocess.check_output.call_count, 1)
130+
self.assertEqual(mock_subprocess.check_call.call_count, 0)
131+
132+
def test_iptables_delete_blockade_chains_1(self):
133+
blockade_id = "blockade-e5dcf85cd2"
134+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
135+
mock_subprocess.check_output.return_value = _IPTABLES_LIST_1
136+
blockade.net.iptables_delete_blockade_chains(blockade_id)
137+
138+
self.assertEqual(mock_subprocess.check_output.call_count, 1)
139+
140+
expected_calls = [
141+
mock.call(["iptables", "-F", "blockade-e5dcf85cd2-p1"]),
142+
mock.call(["iptables", "-X", "blockade-e5dcf85cd2-p1"]),
143+
mock.call(["iptables", "-F", "blockade-e5dcf85cd2-p2"]),
144+
mock.call(["iptables", "-X", "blockade-e5dcf85cd2-p2"])]
145+
self.assertEqual(mock_subprocess.check_call.call_args_list,
146+
expected_calls)
147+
148+
def test_iptables_delete_blockade_chains_2(self):
149+
blockade_id = "blockade-e5dcf85cd2"
150+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
151+
mock_subprocess.check_output.return_value = _IPTABLES_LIST_2
152+
blockade.net.iptables_delete_blockade_chains(blockade_id)
153+
154+
self.assertEqual(mock_subprocess.check_output.call_count, 1)
155+
self.assertEqual(mock_subprocess.check_call.call_count, 0)
156+
157+
def test_iptables_insert_rule_1(self):
158+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
159+
blockade.net.iptables_insert_rule("FORWARD", src="192.168.0.1",
160+
target="DROP")
161+
mock_subprocess.check_call.assert_called_once_with(
162+
["iptables", "-I", "FORWARD", "-s", "192.168.0.1",
163+
"-j", "DROP"])
164+
165+
def test_iptables_insert_rule_2(self):
166+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
167+
blockade.net.iptables_insert_rule("FORWARD", src="192.168.0.1",
168+
dest="192.168.0.2",
169+
target="DROP")
170+
mock_subprocess.check_call.assert_called_once_with(
171+
["iptables", "-I", "FORWARD", "-s", "192.168.0.1", "-d",
172+
"192.168.0.2", "-j", "DROP"])
173+
174+
def test_iptables_insert_rule_3(self):
175+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
176+
blockade.net.iptables_insert_rule("FORWARD", dest="192.168.0.2",
177+
target="DROP")
178+
mock_subprocess.check_call.assert_called_once_with(
179+
["iptables", "-I", "FORWARD", "-d", "192.168.0.2",
180+
"-j", "DROP"])
181+
182+
def test_iptables_create_chain(self):
183+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
184+
blockade.net.iptables_create_chain("hats")
185+
mock_subprocess.check_call.assert_called_once_with(
186+
["iptables", "-N", "hats"])
187+
188+
def test_partition_chain_parse(self):
189+
blockade_id = "abc123"
190+
self.assertEqual(partition_chain_name(blockade_id, 1), "abc123-p1")
191+
self.assertEqual(partition_chain_name(blockade_id, 2), "abc123-p2")
192+
193+
index = parse_partition_index(blockade_id,
194+
partition_chain_name(blockade_id, 1))
195+
self.assertEqual(index, 1)
196+
197+
with self.assertRaises(ValueError):
198+
parse_partition_index(blockade_id, "notablockade")
199+
with self.assertRaises(ValueError):
200+
parse_partition_index(blockade_id, "abc123-1")
201+
with self.assertRaises(ValueError):
202+
parse_partition_index(blockade_id, "abc123-p")
203+
with self.assertRaises(ValueError):
204+
parse_partition_index(blockade_id, "abc123-notanumber")
205+
206+
def test_network_already_normal(self):
207+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
208+
mock_process = mock_subprocess.Popen.return_value = mock.Mock()
209+
mock_process.communicate.return_value = "", QDISC_DEL_NOENT
210+
mock_process.returncode = 2
211+
212+
net = BlockadeNetwork(mock.Mock())
213+
214+
# ensure we don't raise an error
215+
net.fast('somedevice')
216+
self.assertIn('somedevice',
217+
mock_subprocess.Popen.call_args[0][0])
218+
219+
def test_network_state_slow(self):
220+
self._network_state(NetworkState.SLOW, SLOW_QDISC_SHOW)
221+
222+
def test_network_state_normal(self):
223+
self._network_state(NetworkState.NORMAL, NORMAL_QDISC_SHOW)
224+
225+
def test_network_state_flaky(self):
226+
self._network_state(NetworkState.FLAKY, FLAKY_QDISC_SHOW)
227+
228+
def _network_state(self, state, output):
229+
with mock.patch('blockade.net.subprocess') as mock_subprocess:
230+
mock_subprocess.check_output.return_value = output
231+
232+
net = BlockadeNetwork(mock.Mock())
233+
self.assertEqual(net.network_state('somedevice'), state)
234+
self.assertIn('somedevice',
235+
mock_subprocess.check_output.call_args[0][0])

‎blockade/tests/test_state.py

+69
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
import os
18+
import shutil
19+
import unittest
20+
import tempfile
21+
22+
from blockade.state import BlockadeStateFactory
23+
from blockade.errors import NotInitializedError
24+
25+
26+
class BlockadeStateTests(unittest.TestCase):
27+
tempdir = None
28+
oldcwd = None
29+
30+
def setUp(self):
31+
self.tempdir = tempfile.mkdtemp()
32+
self.oldcwd = os.getcwd()
33+
os.chdir(self.tempdir)
34+
35+
def tearDown(self):
36+
if self.oldcwd:
37+
os.chdir(self.oldcwd)
38+
if self.tempdir:
39+
try:
40+
shutil.rmtree(self.tempdir)
41+
except Exception:
42+
pass
43+
44+
def test_state_initialize(self):
45+
46+
containers = {"n1": {"a": 1}, "n2": {"a": 4}}
47+
state = BlockadeStateFactory.initialize(containers=containers)
48+
49+
self.assertTrue(os.path.exists(".blockade/state.yml"))
50+
51+
self.assertEqual(state.containers, containers)
52+
self.assertIsNot(state.containers, containers)
53+
self.assertIsNot(state.containers["n2"], containers["n2"])
54+
55+
self.assertRegexpMatches(state.blockade_id, "^blockade-")
56+
57+
state2 = BlockadeStateFactory.load()
58+
self.assertEqual(state2.containers, state.containers)
59+
self.assertIsNot(state2.containers, state.containers)
60+
self.assertIsNot(state2.containers["n2"], state.containers["n2"])
61+
self.assertEqual(state2.blockade_id, state.blockade_id)
62+
63+
BlockadeStateFactory.destroy()
64+
self.assertFalse(os.path.exists(".blockade/state.yml"))
65+
self.assertFalse(os.path.exists(".blockade"))
66+
67+
def test_state_uninitialized(self):
68+
with self.assertRaises(NotInitializedError):
69+
BlockadeStateFactory.load()

‎blockade/version.py

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#
2+
# Copyright (C) 2014 Dell, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
17+
# WARNING: place nothing else into this file. It is directly exec'd by
18+
# setup.py.
19+
20+
__version__ = "0.1.0"

‎docs/.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
_build/

‎docs/Makefile

+177
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line.
5+
SPHINXOPTS =
6+
SPHINXBUILD = sphinx-build
7+
PAPER =
8+
BUILDDIR = _build
9+
10+
# User-friendly check for sphinx-build
11+
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
12+
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
13+
endif
14+
15+
# Internal variables.
16+
PAPEROPT_a4 = -D latex_paper_size=a4
17+
PAPEROPT_letter = -D latex_paper_size=letter
18+
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
19+
# the i18n builder cannot share the environment and doctrees with the others
20+
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
21+
22+
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
23+
24+
help:
25+
@echo "Please use \`make <target>' where <target> is one of"
26+
@echo " html to make standalone HTML files"
27+
@echo " dirhtml to make HTML files named index.html in directories"
28+
@echo " singlehtml to make a single large HTML file"
29+
@echo " pickle to make pickle files"
30+
@echo " json to make JSON files"
31+
@echo " htmlhelp to make HTML files and a HTML help project"
32+
@echo " qthelp to make HTML files and a qthelp project"
33+
@echo " devhelp to make HTML files and a Devhelp project"
34+
@echo " epub to make an epub"
35+
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
36+
@echo " latexpdf to make LaTeX files and run them through pdflatex"
37+
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
38+
@echo " text to make text files"
39+
@echo " man to make manual pages"
40+
@echo " texinfo to make Texinfo files"
41+
@echo " info to make Texinfo files and run them through makeinfo"
42+
@echo " gettext to make PO message catalogs"
43+
@echo " changes to make an overview of all changed/added/deprecated items"
44+
@echo " xml to make Docutils-native XML files"
45+
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
46+
@echo " linkcheck to check all external links for integrity"
47+
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
48+
49+
clean:
50+
rm -rf $(BUILDDIR)/*
51+
52+
html:
53+
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
54+
@echo
55+
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
56+
57+
dirhtml:
58+
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
59+
@echo
60+
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
61+
62+
singlehtml:
63+
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
64+
@echo
65+
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
66+
67+
pickle:
68+
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
69+
@echo
70+
@echo "Build finished; now you can process the pickle files."
71+
72+
json:
73+
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
74+
@echo
75+
@echo "Build finished; now you can process the JSON files."
76+
77+
htmlhelp:
78+
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
79+
@echo
80+
@echo "Build finished; now you can run HTML Help Workshop with the" \
81+
".hhp project file in $(BUILDDIR)/htmlhelp."
82+
83+
qthelp:
84+
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
85+
@echo
86+
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
87+
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
88+
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/blockade.qhcp"
89+
@echo "To view the help file:"
90+
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/blockade.qhc"
91+
92+
devhelp:
93+
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
94+
@echo
95+
@echo "Build finished."
96+
@echo "To view the help file:"
97+
@echo "# mkdir -p $$HOME/.local/share/devhelp/blockade"
98+
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/blockade"
99+
@echo "# devhelp"
100+
101+
epub:
102+
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
103+
@echo
104+
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
105+
106+
latex:
107+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
108+
@echo
109+
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
110+
@echo "Run \`make' in that directory to run these through (pdf)latex" \
111+
"(use \`make latexpdf' here to do that automatically)."
112+
113+
latexpdf:
114+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
115+
@echo "Running LaTeX files through pdflatex..."
116+
$(MAKE) -C $(BUILDDIR)/latex all-pdf
117+
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
118+
119+
latexpdfja:
120+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
121+
@echo "Running LaTeX files through platex and dvipdfmx..."
122+
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
123+
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
124+
125+
text:
126+
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
127+
@echo
128+
@echo "Build finished. The text files are in $(BUILDDIR)/text."
129+
130+
man:
131+
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
132+
@echo
133+
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
134+
135+
texinfo:
136+
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
137+
@echo
138+
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
139+
@echo "Run \`make' in that directory to run these through makeinfo" \
140+
"(use \`make info' here to do that automatically)."
141+
142+
info:
143+
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
144+
@echo "Running Texinfo files through makeinfo..."
145+
make -C $(BUILDDIR)/texinfo info
146+
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
147+
148+
gettext:
149+
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
150+
@echo
151+
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
152+
153+
changes:
154+
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
155+
@echo
156+
@echo "The overview file is in $(BUILDDIR)/changes."
157+
158+
linkcheck:
159+
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
160+
@echo
161+
@echo "Link check complete; look for any errors in the above output " \
162+
"or in $(BUILDDIR)/linkcheck/output.txt."
163+
164+
doctest:
165+
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
166+
@echo "Testing of doctests in the sources finished, look at the " \
167+
"results in $(BUILDDIR)/doctest/output.txt."
168+
169+
xml:
170+
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
171+
@echo
172+
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
173+
174+
pseudoxml:
175+
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
176+
@echo
177+
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

‎docs/commands.rst

+125
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
.. _commands:
2+
3+
=================
4+
Blockade Commands
5+
=================
6+
7+
The Blockade CLI is built to make it easy to manually manage your containers,
8+
and is also easy to wrap in scripts as needed. All commands that produce
9+
output support a ``--json`` flag to output in JSON instead of plain text.
10+
11+
For the most up to date and detailed command help, use the built-in CLI help
12+
system (``blockade --help``).
13+
14+
``up``
15+
------
16+
17+
::
18+
19+
usage: blockade up [--json]
20+
21+
Start the containers and link them together
22+
23+
--json Output in JSON format
24+
25+
``destroy``
26+
-----------
27+
28+
::
29+
30+
usage: blockade destroy
31+
32+
Destroy all containers and restore networks
33+
34+
``status``
35+
----------
36+
37+
::
38+
39+
usage: blockade status [--json]
40+
41+
Print status of containers and networks
42+
43+
optional arguments:
44+
--json Output in JSON format
45+
46+
``logs``
47+
--------
48+
49+
::
50+
51+
usage: blockade logs CONTAINER
52+
53+
Fetch the logs of a container
54+
55+
CONTAINER Container to fetch logs for
56+
57+
``flaky``
58+
---------
59+
60+
::
61+
62+
usage: blockade flaky [--all] [CONTAINER [CONTAINER ...]]
63+
64+
Make the network flaky for some or all containers
65+
66+
CONTAINER Container to select
67+
68+
--all Select all containers
69+
70+
``slow``
71+
--------
72+
73+
::
74+
75+
usage: blockade slow [--all] [CONTAINER [CONTAINER ...]]
76+
77+
Make the network slow for some or all containers
78+
79+
CONTAINER Container to select
80+
81+
--all Select all containers
82+
83+
``fast``
84+
--------
85+
86+
::
87+
88+
usage: blockade fast [--all] [CONTAINER [CONTAINER ...]]
89+
90+
Restore network speed and reliability for some or all containers
91+
92+
CONTAINER Container to select
93+
94+
--all Select all containers
95+
96+
97+
``partition``
98+
-------------
99+
100+
::
101+
102+
usage: blockade partition PARTITION [PARTITION ...]
103+
104+
Partition the network between containers
105+
106+
Replaces any existing partitions outright. Any containers NOT specified
107+
in arguments will be globbed into a single implicit partition. For
108+
example if you have three containers: c1, c2, and c3 and you run:
109+
110+
blockade partition c1
111+
112+
The result will be a partition with just c1 and another partition with
113+
c2 and c3.
114+
115+
116+
PARTITION Comma-separated partition
117+
118+
``join``
119+
--------
120+
121+
::
122+
123+
usage: blockade join
124+
125+
Restore full networking between containers

‎docs/conf.py

+258
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,258 @@
1+
# -*- coding: utf-8 -*-
2+
#
3+
# blockade documentation build configuration file, created by
4+
# sphinx-quickstart on Fri Feb 7 13:21:50 2014.
5+
#
6+
# This file is execfile()d with the current directory set to its
7+
# containing dir.
8+
#
9+
# Note that not all possible configuration values are present in this
10+
# autogenerated file.
11+
#
12+
# All configuration values have a default; values that are commented out
13+
# serve to show the default.
14+
15+
import sys
16+
import os
17+
18+
# If extensions (or modules to document with autodoc) are in another directory,
19+
# add these directories to sys.path here. If the directory is relative to the
20+
# documentation root, use os.path.abspath to make it absolute, like shown here.
21+
#sys.path.insert(0, os.path.abspath('.'))
22+
23+
# -- General configuration ------------------------------------------------
24+
25+
# If your documentation needs a minimal Sphinx version, state it here.
26+
#needs_sphinx = '1.0'
27+
28+
# Add any Sphinx extension module names here, as strings. They can be
29+
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
30+
# ones.
31+
extensions = []
32+
33+
# Add any paths that contain templates here, relative to this directory.
34+
templates_path = ['_templates']
35+
36+
# The suffix of source filenames.
37+
source_suffix = '.rst'
38+
39+
# The encoding of source files.
40+
#source_encoding = 'utf-8-sig'
41+
42+
# The master toctree document.
43+
master_doc = 'index'
44+
45+
# General information about the project.
46+
project = u'blockade'
47+
copyright = u'2014, Dell Cloud Manager'
48+
49+
# The version info for the project you're documenting, acts as replacement for
50+
# |version| and |release|, also used in various other places throughout the
51+
# built documents.
52+
#
53+
# The short X.Y version.
54+
version = '0.1.0'
55+
# The full version, including alpha/beta/rc tags.
56+
release = '0.1.0'
57+
58+
# The language for content autogenerated by Sphinx. Refer to documentation
59+
# for a list of supported languages.
60+
#language = None
61+
62+
# There are two options for replacing |today|: either, you set today to some
63+
# non-false value, then it is used:
64+
#today = ''
65+
# Else, today_fmt is used as the format for a strftime call.
66+
#today_fmt = '%B %d, %Y'
67+
68+
# List of patterns, relative to source directory, that match files and
69+
# directories to ignore when looking for source files.
70+
exclude_patterns = ['_build']
71+
72+
# The reST default role (used for this markup: `text`) to use for all
73+
# documents.
74+
#default_role = None
75+
76+
# If true, '()' will be appended to :func: etc. cross-reference text.
77+
#add_function_parentheses = True
78+
79+
# If true, the current module name will be prepended to all description
80+
# unit titles (such as .. function::).
81+
#add_module_names = True
82+
83+
# If true, sectionauthor and moduleauthor directives will be shown in the
84+
# output. They are ignored by default.
85+
#show_authors = False
86+
87+
# The name of the Pygments (syntax highlighting) style to use.
88+
pygments_style = 'sphinx'
89+
90+
# A list of ignored prefixes for module index sorting.
91+
#modindex_common_prefix = []
92+
93+
# If true, keep warnings as "system message" paragraphs in the built documents.
94+
#keep_warnings = False
95+
96+
97+
# -- Options for HTML output ----------------------------------------------
98+
99+
# The theme to use for HTML and HTML Help pages. See the documentation for
100+
# a list of builtin themes.
101+
html_theme = 'default'
102+
103+
# Theme options are theme-specific and customize the look and feel of a theme
104+
# further. For a list of options available for each theme, see the
105+
# documentation.
106+
#html_theme_options = {}
107+
108+
# Add any paths that contain custom themes here, relative to this directory.
109+
#html_theme_path = []
110+
111+
# The name for this set of Sphinx documents. If None, it defaults to
112+
# "<project> v<release> documentation".
113+
#html_title = None
114+
115+
# A shorter title for the navigation bar. Default is the same as html_title.
116+
#html_short_title = None
117+
118+
# The name of an image file (relative to this directory) to place at the top
119+
# of the sidebar.
120+
#html_logo = None
121+
122+
# The name of an image file (within the static path) to use as favicon of the
123+
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
124+
# pixels large.
125+
#html_favicon = None
126+
127+
# Add any paths that contain custom static files (such as style sheets) here,
128+
# relative to this directory. They are copied after the builtin static files,
129+
# so a file named "default.css" will overwrite the builtin "default.css".
130+
html_static_path = ['_static']
131+
132+
# Add any extra paths that contain custom files (such as robots.txt or
133+
# .htaccess) here, relative to this directory. These files are copied
134+
# directly to the root of the documentation.
135+
#html_extra_path = []
136+
137+
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
138+
# using the given strftime format.
139+
#html_last_updated_fmt = '%b %d, %Y'
140+
141+
# If true, SmartyPants will be used to convert quotes and dashes to
142+
# typographically correct entities.
143+
#html_use_smartypants = True
144+
145+
# Custom sidebar templates, maps document names to template names.
146+
#html_sidebars = {}
147+
148+
# Additional templates that should be rendered to pages, maps page names to
149+
# template names.
150+
#html_additional_pages = {}
151+
152+
# If false, no module index is generated.
153+
#html_domain_indices = True
154+
155+
# If false, no index is generated.
156+
#html_use_index = True
157+
158+
# If true, the index is split into individual pages for each letter.
159+
#html_split_index = False
160+
161+
# If true, links to the reST sources are added to the pages.
162+
#html_show_sourcelink = True
163+
164+
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
165+
#html_show_sphinx = True
166+
167+
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
168+
#html_show_copyright = True
169+
170+
# If true, an OpenSearch description file will be output, and all pages will
171+
# contain a <link> tag referring to it. The value of this option must be the
172+
# base URL from which the finished HTML is served.
173+
#html_use_opensearch = ''
174+
175+
# This is the file name suffix for HTML files (e.g. ".xhtml").
176+
#html_file_suffix = None
177+
178+
# Output file base name for HTML help builder.
179+
htmlhelp_basename = 'blockadedoc'
180+
181+
182+
# -- Options for LaTeX output ---------------------------------------------
183+
184+
latex_elements = {
185+
# The paper size ('letterpaper' or 'a4paper').
186+
#'papersize': 'letterpaper',
187+
188+
# The font size ('10pt', '11pt' or '12pt').
189+
#'pointsize': '10pt',
190+
191+
# Additional stuff for the LaTeX preamble.
192+
#'preamble': '',
193+
}
194+
195+
# Grouping the document tree into LaTeX files. List of tuples
196+
# (source start file, target name, title,
197+
# author, documentclass [howto, manual, or own class]).
198+
latex_documents = [
199+
('index', 'blockade.tex', u'blockade Documentation',
200+
u'Dell Cloud Manager', 'manual'),
201+
]
202+
203+
# The name of an image file (relative to this directory) to place at the top of
204+
# the title page.
205+
#latex_logo = None
206+
207+
# For "manual" documents, if this is true, then toplevel headings are parts,
208+
# not chapters.
209+
#latex_use_parts = False
210+
211+
# If true, show page references after internal links.
212+
#latex_show_pagerefs = False
213+
214+
# If true, show URL addresses after external links.
215+
#latex_show_urls = False
216+
217+
# Documents to append as an appendix to all manuals.
218+
#latex_appendices = []
219+
220+
# If false, no module index is generated.
221+
#latex_domain_indices = True
222+
223+
224+
# -- Options for manual page output ---------------------------------------
225+
226+
# One entry per manual page. List of tuples
227+
# (source start file, name, description, authors, manual section).
228+
man_pages = [
229+
('index', 'blockade', u'blockade Documentation',
230+
[u'Dell Cloud Manager'], 1)
231+
]
232+
233+
# If true, show URL addresses after external links.
234+
#man_show_urls = False
235+
236+
237+
# -- Options for Texinfo output -------------------------------------------
238+
239+
# Grouping the document tree into Texinfo files. List of tuples
240+
# (source start file, target name, title, author,
241+
# dir menu entry, description, category)
242+
texinfo_documents = [
243+
('index', 'blockade', u'blockade Documentation',
244+
u'Dell Cloud Manager', 'blockade', 'One line description of project.',
245+
'Miscellaneous'),
246+
]
247+
248+
# Documents to append as an appendix to all manuals.
249+
#texinfo_appendices = []
250+
251+
# If false, no module index is generated.
252+
#texinfo_domain_indices = True
253+
254+
# How to display URL addresses: 'footnote', 'no', or 'inline'.
255+
#texinfo_show_urls = 'footnote'
256+
257+
# If true, do not generate a @detailmenu in the "Top" node's menu.
258+
#texinfo_no_detailmenu = False

‎docs/config.rst

+128
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
.. _config:
2+
3+
=============
4+
Configuration
5+
=============
6+
7+
The blockade configuration file is conventionally named ``blockade.yaml`` and
8+
is used to describe the containers in your application. Here is an example:
9+
10+
.. code-block:: yaml
11+
12+
containers:
13+
c1:
14+
image: my_docker_image
15+
command: /bin/myapp
16+
volumes: {"/opt/myapp": "/opt/myapp_host"}
17+
ports: [80]
18+
environment: {"IS_MASTER": 1}
19+
20+
c2:
21+
image: my_docker_image
22+
command: /bin/myapp
23+
volumes: ["/data"]
24+
links: {c1: master}
25+
26+
c3:
27+
image: my_docker_image
28+
command: /bin/myapp
29+
links: {c1: master}
30+
31+
network:
32+
flaky: 30%
33+
slow: 75ms 100ms distribution normal
34+
35+
36+
The format is YAML and there are two important sections: ``containers`` and
37+
``network``.
38+
39+
Containers
40+
----------
41+
42+
Containers are described as a map with the key as the Blockade container name
43+
(``c1``, ``c2``, ``c3`` in the example above). This key is used for commands
44+
to manipulate the Blockade and is also used as the hostname of the container.
45+
46+
Each entry in the ``containers`` section is a single Docker container in the
47+
Blockade. Each container parameter controls how the container is launched.
48+
Most are simply pass-throughs to Docker. Many valuable details can be found
49+
in the `Docker run`_ command documentation.
50+
51+
``image``
52+
---------
53+
54+
``image`` is required and specifies the Docker image name to use for the
55+
container. The image must exist in your Docker installation.
56+
57+
``command``
58+
-----------
59+
60+
``command`` is optional and specifies the command to run within the container.
61+
If not specified, a default command must be part of the image you are using.
62+
You may include environment variables in this command, but to do so you must
63+
typically wrap the command in a shell, like ``sh -c "/bin/myapp $MYENV"``.
64+
65+
``volumes``
66+
-----------
67+
68+
``volumes`` is optional and specifies the volumes to mount in the container,
69+
*from the host*. Volumes can be specified as either a map or a list. In map
70+
form, the key is the path *on the host* to expose and the value is the
71+
mountpoint *within the container*. In list form, the host path and container
72+
mountpoint are assumed to be the same. See the `Docker volumes`_ documentation
73+
for details about how this works.
74+
75+
``ports``
76+
---------
77+
78+
``ports`` is optional and specifies ports to expose from the container. Ports
79+
must be exposed in order to use the Docker `named links`_ feature.
80+
81+
``links``
82+
---------
83+
84+
``links`` is optional and specifies links from one container to another. A
85+
dependent container will be given environment variables with the parent
86+
container's IP address and port information. See `named links`_ documentation
87+
for details.
88+
89+
90+
Network
91+
-------
92+
93+
The ``network`` configuration block controls the settings used for network
94+
filter commands like ``slow`` and ``flaky``. If unspecified, defaults will
95+
be used. There are two parameters:
96+
97+
``slow``
98+
--------
99+
100+
``slow`` controls the amount and distribution of delay for network packets
101+
when a container is in the Blockade slow state. It is specified
102+
as an expression understood by the `tc netem`_ traffic control ``delay``
103+
facility. See the man page for details, but the pattern is::
104+
105+
TIME [ JITTER [ CORRELATION ] ]
106+
[ distribution { uniform | normal | pareto | paretonormal } ]
107+
108+
``TIME`` and ``JITTER`` are expressed in milliseconds while ``CORRELATION``
109+
is a percentage.
110+
111+
``flaky``
112+
---------
113+
114+
``flaky`` controls the lossiness of network packets when a contrainer is in
115+
the Blockade flaky state. It is specified as an expression understood by the
116+
`tc netem`_ traffic control ``loss`` facility. See the man page for details,
117+
but the simplified pattern is::
118+
119+
random PERCENT [ CORRELATION ]
120+
121+
``PERCENT`` and ``CORRELATION`` are both expressed as percentages.
122+
123+
124+
125+
.. _Docker run: http://docs.docker.io/en/latest/reference/run/
126+
.. _Docker volumes: http://docs.docker.io/en/latest/use/working_with_volumes
127+
.. _named links: http://docs.docker.io/en/latest/use/working_with_links_names/
128+
.. _tc netem: http://man7.org/linux/man-pages/man8/tc-netem.8.html

0 commit comments

Comments
 (0)
Please sign in to comment.