mkcloud needs clearly defined interface

[aspiers]

Currently mkcloud relies on a bunch of shell environment variables to control its behaviour:

• Some of these are documented via comments near the top of the script.
• A smaller number are documented in docs/mkcloud.md.
• Some are documented in the usage text output when you run ./mkcloud with no arguments. (It's also supposed to be output when you run ./mkcloud help, but that's broken.)
• Many are not documented at all.
• Some of the variables have documentation duplicated in more than one of the three locations, and sometimes the duplicated info is inconsistent.
• These environment variables are mostly (but not all!) lowercase, and are indistinguishable from other "private" shell variables used within the script. One of the causes of this is violation of the common shell coding standard which uses uppercase for environment variables and constants, and lowercase for local variables.
• Some variables are optional and others are mandatory, but it's not always clear which are which.

How did we get ourselves in this mess? I suggest that the answers include (but are not limited to) the following reasons:

• shell code does not encourage clean encapsulation
• the "boiling frog" effect: scripts start as quick small hacks, and gradually grow past the point where they should be treated like proper production code without anyone noticing
• no structured peer review across the whole team for a large part of the code's history

This is not the fault of any one person, and anyway my goal is not to blame people. The priorities are:

1. to learn how we can avoid this in the future, and
2. to figure out how we can clean up the mess.

My suggestions are:

• Consolidate documentation of all variables into docs/mkcloud.md and remove it from the code.
• Stick to universally accepted coding standards (like UPPERCASE for environment variables, lowercase for shell variables).
• Don't push any future changes without peer review.
• Give -1 vote to changes which are missing corresponding updates to docs/mkcloud.md.
• Reach a consensus on https://etherpad.nue.suse.com/p/cloud-dev-language-strategy

Thoughts welcome.

[MaximilianMeister]

I personally had to spend some (and some more) time with @jdsn and @bmwiedemann to find my way around in this 2 main scripts.
I agree with the following:

• documentation can always be improved, but that goes for crowbar and other software too, (lately we had already one round of documentation enhancement)
• the variable casing is somewhat unclear, but it can either be documented and/or improved too.
• review should be done over github workflows (commenting etc) even if it takes more time.

So yet i dont see a big problem nor a mess here, besides the mess in my mkcloud working directory ;)

[aspiers]

@MaximilianMeister One big problem is that I've wasted a lot of time whilst learning mkcloud, because the docs were so bad (and BTW for a long time they did not even exist). And if it happens for me then it can happen for other mkcloud users in the future. It's inefficient to have to rely on @jdsn and @bmwiedemann all the time - if that approach was OK, we wouldn't need to ever bother documenting anything, we'd just ask the authors.

[MaximilianMeister]

i didn't say that this is an approach to go to @jdsn and @bmwiedemann. this should not be needed. i just gave you my background. (and i am sitting next to them, so it was easy for me to reach them)
your many recent enhancements and the ones from others about readability and documentation make it easier for the next ones working with it. we should definitely push it further

[aspiers]

OK we are agreed :)

[aspiers]

This is being addressed gradually thanks to productive discussions during the Cloud team's workshop in March 2015:

• there is now an mkcloud refactoring epic
• we reached a consensus on https://etherpad.nue.suse.com/p/cloud-dev-language-strategy

[MaximilianMeister]

two of the gradual refactoring steps are #335 and #343

[aspiers]

@vuntz, @sidthe Fixing this mess becomes more urgent now that

a) the team is growing faster than ever
b) mkcloud is starting to be used externally