some fiddling about in 11

Author: hjwp

bibliography.asciidoc

@@ -8,16 +8,13 @@ TODO remove this chapter entirely,
 or remove links from text (they dont work),
 and strip down to just the most important books.
 
-- [[[lpthw]]] Zed A. Shaw, 'Learn Python the Hard Way': http://learnpythonthehardway.org/ 
-- [[[iwp]]] Al Sweigart, 'Invent Your Own Computer Games with Python': http://inventwithpython.com
 - [[[tddbe]]] Kent Beck, 'Test Driven Development: By Example', Addison-Wesley
 - [[[refactoring]]] Martin Fowler, 'Refactoring', Addison-Wesley  
 - [[[seceng]]] Ross Anderson, 'Security Engineering, Second Edition',
   Addison-Wesley: http://www.cl.cam.ac.uk/~rja14/book.html 
 - [[[jsgoodparts]]] Douglas Crockford, 
 http://oreil.ly/SuXjXq['JavaScript: The Good Parts'], O'Reilly
 - [[[twoscoops]]] Daniel Greenfeld and Audrey Roy, 'Two Scoops of Django', http://twoscoopspress.com/products/two-scoops-of-django-1-6
-- [[[mockfakestub]]] Emily Bache, 'Mocks, Fakes and Stubs', https://leanpub.com/mocks-fakes-stubs 
 - [[[GOOSGBT]]] Steve Freeman and Nat Pryce, 'Growing
   Object-Oriented Software Guided by Tests', Addison-Wesley
 

chapter_09_docker.asciidoc

@@ -667,6 +667,8 @@ Ah, we forgot that we need to install Django.
 
 === Virtualenv and requirements.txt
 
+// TODO: move to next chapter
+
 Just like on our own machine,
 a virtualenv is useful in a deployed environment to make
 sure we have full control over the packages installed for a particular

chapter_10_production_readiness.asciidoc

@@ -38,7 +38,7 @@ it's not designed for real-life workloads.
 Instead, we'll use the popular Gunicorn Python/WSGI server.
 
 ((("DEBUG settings")))
-In addition, several options in 'settings.py' are currently unacceptable.
+In addition, several options in _settings.py_ are currently unacceptable.
 `DEBUG=True`, is strongly recommended against for production,
 we'll want to set a unique `SECRET_KEY`,
 and as we'll see, other things will come up.
@@ -58,8 +58,6 @@ an ORM, all sorts of middleware, the admin site...
 which I guess is what you'd want next if you already had a pony...
 
 //001
-cmdg
-
 
 [subs="specialcharacters,quotes"]
 ----

chapter_11_ansible.asciidoc

@@ -81,14 +81,15 @@ We can separate out "deployment" into two tasks:
 - _Provisioning_ a new server to be able to host the code
 - _Deploying_ a new version of the code to an existing server
 
-Ultimately, infrastructure-as-code tools can let you automate both of these,
-but for the purposes of this book, we can live with manual provisioning.
+Infrastructure-as-code tools can let you automate both of these,
+but the provisioning parts tend to be quite vendor-specific,
+so for the purposes of this book, we can live with manual provisioning.
 
-I should probably stress once more that deployment is something that varies a lot,
-and that as a result there are few universal best practices for how to do it.
-So, rather than trying to remember the specifics of what I'm doing here,
-you should be trying to understand the rationale,
-so that you can apply the same kind of thinking in the specific future circumstances you encounter.
+NOTE: I should probably stress once more that deployment is something that varies a lot,
+  and that as a result there are few universal best practices for how to do it.
+  So, rather than trying to remember the specifics of what I'm doing here,
+  you should be trying to understand the rationale,
+  so that you can apply the same kind of thinking in the specific future circumstances you encounter.
 
 
 ==== Choosing Where to Host Our Site
@@ -136,7 +137,7 @@ any solution should be fine, as long as:
 
 I'm recommending Ubuntu as a distro because it's popular and I'm used to it.
 If you know what you're doing, you can probably get away with using
-something else, but you're on your own.
+something else, but I won't be able to help you as much if you get stuck.
 
 ((("Linux servers")))
 If you've never started a Linux server before and you have absolutely no idea
@@ -158,7 +159,7 @@ NOTE: Some people get to this chapter, and are tempted to skip the domain bit,
 In these instructions, I'm assuming that you have a nonroot user account set up,
 and that it has "sudo" privileges,
 so whenever we need to do something that requires root access, we use sudo,
-(or "become" in ansible terminology),
+(or "become" in Ansible terminology),
 and I'm explicit about that in the various instructions that follow.
 
 My user is called "elspeth", but you can call yours whatever you like!
@@ -168,6 +169,7 @@ See the guide linked above if you need tips on creating a sudo user.
 
 .General Server Debugging Tips
 *******************************************************************************
+// TODO: good advice but not quite sure it's phrased quite right for the new version of the chapter.
 
 The most important lesson to remember from this chapter is,
 as always but more than ever, to work incrementally,
@@ -237,32 +239,34 @@ rather than specifying a procedural series of steps to be followed one by one.
 Take a look at the instructions here:
 https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html
 
-----
-pipx install --include-deps ansible
-pipx inject ansible docker
-
-# or
+The simplest thing to do is to install them into the virtualenv
+on our local machine:
 
-pip install ansible docker
+[subs="specialcharacters,quotes"]
+----
+$ *pip install ansible*
+# we also need the Docker SDK for the ansible/docker integration to work:
+$ *pip install docker*
 ----
 
 // TODO: consider introducing an explicit requirements.dev.txt here,
 // with -r requirements.txt and put ansible, docker, and selenium in there.
+// or, maybe get that in place in the previous chapter, keep this one shorter.
 
 
 ==== A First Cut of an Ansible Playbook
 
 Let's dip our toes into Ansible,
 and see if we can get it to run a simple "hello world" container on our server.
 
-Here's what's called a "playbook" in ansible terminology.
+Here's what's called a "playbook" in Ansible terminology.
 It's in a format called YAML (Yet Another Markup Language),
 which, if you've never come across before,
-you will soon develop a love-hatefootnote:[
+you will soon develop a love-hate relationshipfootnote:[
 The "love" part is that yaml is very easy to _read_ and scan through at a glance.
 The "hate" part is that the actual syntax is surprisingly fiddly to get right:
 the difference between lists and key/value maps is subtle and I can never quite remember it honestly.]
-relationship with.
+for.
 
 
 [role="sourcecode"]
@@ -292,20 +296,20 @@ relationship with.
 ----
 ====
 
-<1> An ansible playbook is a series of "tasks"
+<1> An Ansible playbook is a series of "tasks"
   (so in that sense it's still quite sequential and procedural),
   but the individual tasks themselves are quite declarative.
   Each one usually has a human-readable `name` attribute.
 
-<2> Each tasks uses an ansible "module" to do its work.
-  The next few use the `builtin.apt` module which provides
-  a wrapper around the `apt` Debian & Ubuntu package management tool.
+<2> Each tasks uses an Ansible "module" to do its work.
+  This one uses the `builtin.apt` module which provides a wrapper
+  around the `apt` Debian & Ubuntu package management tool.
 
 <3> Each module then provides a bunch of parameters which control how it works.
     Here we specify the `name` of the package we want to install ("docker")
     and tell it update its cache first, which is required on a fresh server.
 
-Most ansible modules have pretty good documentation,
+Most Ansible modules have pretty good documentation,
 check out the `builtin.apt` one for example.
 I often skip to the
 https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_module.html#examples[Examples section].
@@ -400,6 +404,14 @@ SSHing in to check things worked is a key server debugging skill!
 It's something we want to practice on our staging server,
 because ideally we'll want to avoid doing it on production machines.
 
+Let's move on to trying to get our actual docker container running on the server.
+As we go through, you'll see that we're going to work through very similar issues
+to the ones we've already figured our way through in the last couple of chapters:
+
+* Configuration
+* Networking
+* And the database.
+
 
 === Getting our image onto the server
 
@@ -421,7 +433,7 @@ so we're going to "simulate" this process by doing it manually.
 
 It turns out you can "export" a container image to an archive format,
 manually copy that to the server, and then re-import it.
-In ansible config, it looks like this:
+In Ansible config, it looks like this:
 
 [role="sourcecode"]
 .infra/ansible-provision.yaml (ch11l002)
@@ -550,17 +562,19 @@ This means we don't have a dependency on having run `docker build` locally.
       delegate_to: 127.0.0.1
 
     - name: Export container image locally
+      [...]
 ----
 ====
 
-<1> Having this step also allows us to work around an issue
-  with compatility between Apple's new ARM-based chips and
-  our server's x86/amd64 architecture.
+<1> I needed this `platform` attribute to work around an issue
+  with compatibility between Apple's new ARM-based chips and our server's
+  x86/amd64 architecture.
   You could also use this `platform:` to cross-build docker images
   for a rasbperry pi from a regular PC, or vice-versa.
+  It does no harm in any case.
 
 
-In any case, let's see if it works!
+Now let's see if it works!
 
 [subs="specialcharacters,quotes"]
 ----
@@ -596,25 +610,25 @@ Ah woops, we need to set those environment variables on the server too.
 === Using an env File to Store Our Environment Variables
 
 When we run our container manually locally, we can pass in environment variables with the `-e` flag.
-But we don't want to hard-code secrets like SECRET_KEY into our ansible files
+But we don't want to hard-code secrets like SECRET_KEY into our Ansible files
 and commit them to our repo!
 
-Instead, we can use ansible to automate the creation of a secret key,
-and then save it to a file on the server, where it will be relatively secure
-(better than saving it to version contorl and pushing it to GitHub in any case!)
+Instead, we can use Ansible to automate the creation of a secret key,
+and then save it to a file on the server, where it _ill be _relatively_ secure
+(better than saving it to version control and pushing it to GitHub in any case!)
 
-We can use a so-called "env file" to store environment variables,
-which are essentially a list of key-value pairs using shell syntax,
+We can use a so-called "env file" to store environment variables.
+Env files are essentially a list of key-value pairs using shell syntax,
 a bit like you'd use with `export`.
 
 One extra subtlety is that we want to vary the actual contents of the env file,
 depending on where we're deploying to.
 Each server should get its own unique secret key,
-adn we want different config for staging and prod, for example.
+and we want different config for staging and prod, for example.
 
 So, just as we inject variables into our html templates in Django,
 we can use a templating language called "jinja2" to have variables in our env file.
-It's a common tool in ansible scripts, and the syntax is very similar to Django's.
+It's a common tool in Ansible scripts, and the syntax is very similar to Django's.
 
 Here's what our template for the env file will looks like:
 
@@ -632,7 +646,7 @@ DJANGO_ALLOWED_HOST="{{ host }}"
 And here's how we use it in the provisioning script:
 
 
-[role="sourcecode"]
+[role="sourcecode small-code"]
 .infra/ansible-provision.yaml (ch11l004)
 ====
 [source,yaml]
@@ -647,7 +661,7 @@ And here's how we use it in the provisioning script:
         force: false  # do not recreate file if it already exists. <2>
       vars:  # <3>
         host: "{{ inventory_hostname }}"  # <4>
-        secret_key: "{{ lookup('password', '/dev/null length=32 chars=ascii_letters,digits') }}"
+        secret_key: "{{ lookup('password', '/dev/null length=32 chars=ascii_letters,digits') }}"  # <5>
 
     - name: Run container
       community.docker.docker_container:
@@ -667,14 +681,19 @@ And here's how we use it in the provisioning script:
 
 <3> The `vars` section defines the variables we'll inject into our template.
 
-<4> We actually use a built-in ansible variable called `inventory_hostname`.
+<4> We actually use a built-in Ansible variable called `inventory_hostname`.
     This variable woul actually be available in the template already,
     but I'm renaming it for clarity.
 
+<5> This `lookup('password')` thing I copy-pasted from Stackoverflow.
+    Come on there's no shame in that.
+    
 
 NOTE: Using an env file to store secrets is definitely better than committing
     it to version control, but it's maybe not the state of the art either.
-    TODO: mention other secret management tools. vault
+    You'll probably come across more advanced alternatives from various cloud providers,
+    or Hashicorp's "vault" tool.
+
 
 
 .Idempotency and Declarative Configuration
@@ -685,8 +704,8 @@ meaning that, as much as possible, you specify the desired state that you want,
 rather than specifying a series of steps to get there.
 
 This concept goes along with the idea of "idempotency",
-which means that you get the same result when you run something once,
-as when you run it multiple times.
+which is wanting to get the same result when you run something for the first time,
+vs running it again on later occations.
 
 An example is the `apt` module that we used to install docker.
 It doesn't crash if docker is already installed, and in fact,
@@ -749,6 +768,7 @@ skipped=0    rescued=0    ignored=0
 
 Looks good!  What do our tests think?
 
+
 ==== More debugging
 
 We run our tests as usual and run into a new problem:
@@ -901,6 +921,9 @@ Here's how
       community.docker.docker_container_exec:  # <3>
         container: superlists
         command: ./manage.py migrate
+
+    - name: Run container
+      [...]
 ----
 ====
 
@@ -955,11 +978,15 @@ Podman and Systemd into the mix, should things not go according to plan:
 
 - You can get detailed info on the Container using
   `docker inspect superlists`.
+  This is a good place to go check on environment variables,
+  port mappings, and exactly which image was running, for example.
 
 - And you can inspect the image with
   `docker image inspect superlists`.
-((("debugging", "Docker")))
+  You might need this to check the exact image hash,
+  to make sure it's the same one you built locally.
 
+((("debugging", "Docker")))
 
 *******************************************************************************
 
@@ -985,7 +1012,6 @@ Rewire the FT runner to be able to test against the local VM.
 Having a Vagrant config file is particularly helpful when working
 in a team--it helps new developers to spin up servers that look exactly
 like yours.((("", startref="ansible29")))
-////
 
 
 
@@ -1045,6 +1071,8 @@ $ *git log --graph --oneline --decorate*
 [...]
 ----
 
+////
+
 
 Anyway, you now have a live website!  Tell all your friends!  Tell your mum, if
 no one else is interested!  And, in the next chapter, it's back to coding
@@ -1057,26 +1085,26 @@ Further Reading
 
 
 ((("automated deployment", "additional resources")))
-There's no such thing as the One True Way in deployment,
-and I'm no grizzled expert in any case.
+There's no such thing as the One True Way in deployment;
 I've tried to set you off on a reasonably sane path,
 but there's plenty of things you could do differently,
-and lots, lots more to learn besides.q
+and lots, lots more to learn besides.
 Here are some resources I used for inspiration:
 
 
 * http://12factor.net/[The 12-factor App] by the Heroku team
 
 * http://hynek.me/talks/python-deployments[Solid Python Deployments for Everybody] by Hynek Schlawack
 
-* The deployment chapter of <<twoscoops,Two Scoops of Django>> by Dan
-  Greenfeld and Audrey Roy
+* The deployment chapter of
+  https://www.feldroy.com/books/two-scoops-of-django-3-x[Two Scoops of Django]
+  by Dan Greenfeld and Audrey Roy
 
 
 
 
 [role="pagebreak-before less_space"]
-.Automated Deployments
+.Automated Deployment Recap
 *******************************************************************************
 
 TODO Maybe recap the key steps of any deployment:
@@ -1102,6 +1130,10 @@ Automating provisioning::
   brand new servers.
   This will involve interacting with the API of your hosting provider.
 
+////
+
+TODO: find a place for this
+
 Security::
   A serious discussion of server security is beyond the scope of this book,
   and I'd warn against running your own servers
@@ -1116,5 +1148,6 @@ Security::
   wild place!
   ((("security issues and settings", "server security")))
   ((("Platform-As-A-Service (PaaS)")))
+////
 
 *******************************************************************************