Merge pull request #3150 from freedomofpress/docs-upgrade-to-0.6

[0.6.0] Upgrade docs for SecureDrop 0.6

docs/hardware.rst

@@ -262,6 +262,8 @@ also be possible to print and cut out labels using adhesive-backed paper and som
 -  :download:`Secure Viewing Station TAILS USB Drive Label <images/labels/usb_svs.png>`
 -  :download:`File Transfer USB Drive Label <images/labels/usb_file_transfer.png>`
 
+.. _Specific Hardware Recommendations:
+
 Specific Hardware Recommendations
 ---------------------------------
 

docs/index.rst

@@ -84,6 +84,7 @@ anonymous sources.
    :name: upgradetoc
    :maxdepth: 2
 
+   upgrade/0.5.x_to_0.6.rst
    upgrade/0.4.x_to_0.5.rst
    upgrade/0.3.x_to_0.4.rst
    upgrade_to_tails_2x.rst

docs/upgrade/0.5.x_to_0.6.rst

@@ -0,0 +1,220 @@
+Upgrade from 0.5.x to 0.6.x
+===========================
+
+Updating the Tails workstations
+-------------------------------
+
+All Tails drives should be updated to Tails 3.6,
+`released concurrently <https://blog.torproject.org/tails-36-out>`__ with
+SecureDrop 0.6. For the *Admin Workstation* and *Journalist Workstation*, you
+will be prompted on boot about this upgrade. All you need to do is accept the
+upgrade. To upgrade the *Secure Viewing Station* to Tails 3.6, you will need to
+perform a `manual Tails upgrade <https://tails.boum.org/upgrade/tails/index.en.html>`__.
+
+For the *Journalist Workstations* and the *Admin Workstation*, you will also
+need to update the SecureDrop code using the following manual method:
+
+.. code:: sh
+
+   cd ~/Persistent/securedrop
+   git checkout 0.6
+   gpg --recv-key "2224 5C81 E3BA EB41 38B3 6061 310F 5612 00F4 AD77"
+   git tag -v 0.6 # Output should include "Good signature"
+   ./securedrop-admin setup
+
+From this point forward, you will be able to use the command 
+``securedrop-admin check_for_updates`` to check whether updates are available,
+and ``securedrop-admin update`` to check for updates and to apply them in a 
+single step.
+
+Troubleshooting Linux kernel issues
+-----------------------------------
+
+The latest Linux kernel (version 4.4.115) will be automatically installed on your
+SecureDrop servers within 24 hours of the 0.6 release (13th of March at
+22:00 UTC). If you are using hardware other than 
+:ref:`our official hardware recommendations <Specific Hardware Recommendations>`,
+you face moderate risk of a SecureDrop outage.
+
+If you are experiencing an outage as part of the 0.6 release (e.g. you
+cannot reach the source interface, you cannot SSH into the servers
+etc.), you will have to perform the steps outlined below.
+
+First, you need to physically access each server. Power down the server
+(safely if possible), attach required peripherals (keyboard, monitor),
+and power the server back up. If you did not configure Google
+Authenticator for console access, you will need to use single user mode
+in order to login to each server.
+
+Boot into single user mode
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. |GRUB in default state| image:: ../images/0.5.x_to_0.6/grub-in-default-state.png
+.. |GRUB in edit mode| image:: ../images/0.5.x_to_0.6/grub-in-edit-mode.png
+
+To access single user mode, you will have to edit the boot options for
+the new kernel. You can do so using the GRUB bootloader, pictured below:
+
+|GRUB in default state|
+
+Press any key quickly just once. You will only have about 2 to 3 seconds
+before Ubuntu starts booting. If you miss that window, just log in normally
+and reboot safely, provided you can log in. Do not unplug or forcibly 
+shut down the server.
+
+Once you hit a key, you will be able to interact with the menu with the
+up (⬆) and down (⬇) keys. Select “Ubuntu” as shown above, and press “e”
+to edit the boot options. In the line that begins “linux”, replace the
+word “quiet” with “single”. Note that the word "quiet" may be wrapped, as in the
+screenshot below:
+
+|GRUB in edit mode|
+
+Press the “F10” key to boot.
+
+Test the new kernel
+~~~~~~~~~~~~~~~~~~~
+
+Observe the boot process. It is possible that the system will fail to
+boot completely; if so, the log information will help us to understand
+what is happening.
+
+Provided that you can log in, check if you have network access. Try a
+command such as ``sudo host freedom.press``. If you don’t have network
+access, it is most likely due to the upgraded kernel missing a network
+driver for your hardware.
+
+If everything appears to be operating normally, the outage may not be
+kernel-related. In that case, please still follow the steps at the end of
+this document to send us log information along with an issue report,
+and we will help you investigate.
+
+If you are experiencing network issues or other kernel problems, we
+recommend that you roll back to an older kernel, and that you report the
+issue to us immediately. 
+
+Compare the behavior of the old kernel
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. |GRUB with advanced options selected| image:: ../images/0.5.x_to_0.6/grub-with-advanced-options-selected.png
+
+Reboot the server in a safe way with ``sudo reboot``. After the BIOS screen,
+you can select a new kernel from the GRUB boot menu by selecting
+**Advanced Options for Ubuntu**, pictured below.
+
+|GRUB with advanced options selected|
+
+Under **Advanced Options**, choose the option with kernel version ``3.14.x-grsec``.
+As before, you may need to edit the kernel options to enter single user
+mode. The boot process should proceed normally. Wait until you get a
+login prompt and log in.
+
+Once you are logged in, check to see if you have network access. If you do, then  
+your instance is having an issue with the 4.x kernel. In that case, we need to 
+temporarily set an older kernel as the default.
+
+Roll back to the old kernel
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. important:: The older kernel has reached end of life, so it
+  is of critical importance for the near term security of your instance
+  that we work together to resolve any compatibility issues. Rolling back to an 
+  older version is only a stopgap measure to avoid a prolonged outage of your
+  SecureDrop instance.
+
+Inspect the file ``/boot/grub/grub.cfg``. You should find a ``menuentry`` line
+with the same text that you selected during boot, e.g.:
+
+.. code:: sh
+
+  submenu 'Advanced options for Ubuntu'…
+
+    menuentry 'Ubuntu, with Linux 3.14.xx-grsec…
+
+Take note of its position among the other submenu entries (it will most likely 
+be third). Then edit the GRUB configuration:
+
+.. code:: sh
+
+  sudo vim /etc/grub/default
+
+Make a backup of the file or take a note of the current value of 
+``GRUB_DEFAULT`` somewhere, so you can restore the previous behavior easily at a 
+later point.
+
+Once you have done so, set the ``GRUB_DEFAULT`` variable to point to the index 
+of the  menu and submenu. Note that the index starts at 0, so for a typical 
+setup, the line in ``/etc/grub/default`` would look like this:
+
+.. code:: sh
+
+  GRUB_DEFAULT=”1>2”
+
+The “1” means the second entry of the main menu (“Advanced options”),
+the “2” means the third entry of the submenu. Again, update these
+numbers consistent with your configuration. 
+
+
+.. caution:: Ensure that you have chosen the right index for the main menu
+  and the submenu, and double-check that you are beginning the count at 0, not
+  1; otherwise, you may boot into the wrong kernel.
+
+This change still has to be applied to take effect on the next boot:
+
+.. code:: sh
+
+  sudo update-grub2
+
+Now you can reboot into the old, working kernel.
+
+.. code:: sh
+
+  sudo reboot
+
+The server should come up automatically. From here on, you should be
+able to perform all administrative tasks via SSH as usual. If you want
+additional confirmation of the kernel version, the command 
+``uname -a`` should display ``3.14.79-grsec``.
+
+Please notify us of the compatibility issue so we can help you resolve it ASAP.
+
+Report compatibility issues
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you have encountered issues with the kernel upgrade, it is important
+that you report them to us so that we may incorporate any necessary
+changes to our updated kernel, and so that we can work with you to
+switch back to the new kernel as soon as possible.
+
+Run the following commands via SSH from the *Admin Workstation*:
+
+.. code:: sh
+
+  cd ~/Persistent/securedrop/
+  source .venv/bin/activate
+  cd install_files/ansible-base
+  ansible all -b -m setup > server-facts.log
+
+Please also send us a copy of ``/var/log/syslog`` and ``/var/log/dmesg`` for
+analysis.
+
+You can share ``server-facts.log``, ``syslog`` and ``dmesg`` with us as follows:
+
+-  If you are a member of our Support Portal, please create a new issue
+   and attach the files to it.
+-  Alternatively, email us at [email protected] 
+   (`GPG encrypted <https://securedrop.org/sites/default/files/fpf-email.asc>`__) 
+   with the subject “SecureDrop kernel facts” and the files attached.
+
+Once we get your information, we can try to provide assistance to
+resolve compatibility issues.
+
+Getting support
+---------------
+
+Should you require further support with your SecureDrop installation or upgrade,
+we are happy to help!
+
+-  Community support is available at https://forum.securedrop.club
+-  Paid support options are provided by Freedom of the Press Foundation.
+   Contact [email protected] for more information.