diff --git a/docs/hardware.rst b/docs/hardware.rst index 7cc372c637..7a45c360e3 100644 --- a/docs/hardware.rst +++ b/docs/hardware.rst @@ -262,6 +262,8 @@ also be possible to print and cut out labels using adhesive-backed paper and som - :download:`Secure Viewing Station TAILS USB Drive Label ` - :download:`File Transfer USB Drive Label ` +.. _Specific Hardware Recommendations: + Specific Hardware Recommendations --------------------------------- diff --git a/docs/images/0.5.x_to_0.6/grub-in-default-state.png b/docs/images/0.5.x_to_0.6/grub-in-default-state.png new file mode 100644 index 0000000000..f90f25bca5 Binary files /dev/null and b/docs/images/0.5.x_to_0.6/grub-in-default-state.png differ diff --git a/docs/images/0.5.x_to_0.6/grub-in-edit-mode.png b/docs/images/0.5.x_to_0.6/grub-in-edit-mode.png new file mode 100644 index 0000000000..3d90de42b0 Binary files /dev/null and b/docs/images/0.5.x_to_0.6/grub-in-edit-mode.png differ diff --git a/docs/images/0.5.x_to_0.6/grub-with-advanced-options-selected.png b/docs/images/0.5.x_to_0.6/grub-with-advanced-options-selected.png new file mode 100644 index 0000000000..8b22fe939c Binary files /dev/null and b/docs/images/0.5.x_to_0.6/grub-with-advanced-options-selected.png differ diff --git a/docs/index.rst b/docs/index.rst index f52c69ee27..187f273f66 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -84,6 +84,7 @@ anonymous sources. :name: upgradetoc :maxdepth: 2 + upgrade/0.5.x_to_0.6.rst upgrade/0.4.x_to_0.5.rst upgrade/0.3.x_to_0.4.rst upgrade_to_tails_2x.rst diff --git a/docs/upgrade/0.5.x_to_0.6.rst b/docs/upgrade/0.5.x_to_0.6.rst new file mode 100644 index 0000000000..cec058d894 --- /dev/null +++ b/docs/upgrade/0.5.x_to_0.6.rst @@ -0,0 +1,220 @@ +Upgrade from 0.5.x to 0.6.x +=========================== + +Updating the Tails workstations +------------------------------- + +All Tails drives should be updated to Tails 3.6, +`released concurrently `__ with +SecureDrop 0.6. For the *Admin Workstation* and *Journalist Workstation*, you +will be prompted on boot about this upgrade. All you need to do is accept the +upgrade. To upgrade the *Secure Viewing Station* to Tails 3.6, you will need to +perform a `manual Tails upgrade `__. + +For the *Journalist Workstations* and the *Admin Workstation*, you will also +need to update the SecureDrop code using the following manual method: + +.. code:: sh + + cd ~/Persistent/securedrop + git checkout 0.6 + gpg --recv-key "2224 5C81 E3BA EB41 38B3 6061 310F 5612 00F4 AD77" + git tag -v 0.6 # Output should include "Good signature" + ./securedrop-admin setup + +From this point forward, you will be able to use the command +``securedrop-admin check_for_updates`` to check whether updates are available, +and ``securedrop-admin update`` to check for updates and to apply them in a +single step. + +Troubleshooting Linux kernel issues +----------------------------------- + +The latest Linux kernel (version 4.4.115) will be automatically installed on your +SecureDrop servers within 24 hours of the 0.6 release (13th of March at +22:00 UTC). If you are using hardware other than +:ref:`our official hardware recommendations `, +you face moderate risk of a SecureDrop outage. + +If you are experiencing an outage as part of the 0.6 release (e.g. you +cannot reach the source interface, you cannot SSH into the servers +etc.), you will have to perform the steps outlined below. + +First, you need to physically access each server. Power down the server +(safely if possible), attach required peripherals (keyboard, monitor), +and power the server back up. If you did not configure Google +Authenticator for console access, you will need to use single user mode +in order to login to each server. + +Boot into single user mode +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. |GRUB in default state| image:: ../images/0.5.x_to_0.6/grub-in-default-state.png +.. |GRUB in edit mode| image:: ../images/0.5.x_to_0.6/grub-in-edit-mode.png + +To access single user mode, you will have to edit the boot options for +the new kernel. You can do so using the GRUB bootloader, pictured below: + +|GRUB in default state| + +Press any key quickly just once. You will only have about 2 to 3 seconds +before Ubuntu starts booting. If you miss that window, just log in normally +and reboot safely, provided you can log in. Do not unplug or forcibly +shut down the server. + +Once you hit a key, you will be able to interact with the menu with the +up (⬆) and down (⬇) keys. Select “Ubuntu” as shown above, and press “e” +to edit the boot options. In the line that begins “linux”, replace the +word “quiet” with “single”. Note that the word "quiet" may be wrapped, as in the +screenshot below: + +|GRUB in edit mode| + +Press the “F10” key to boot. + +Test the new kernel +~~~~~~~~~~~~~~~~~~~ + +Observe the boot process. It is possible that the system will fail to +boot completely; if so, the log information will help us to understand +what is happening. + +Provided that you can log in, check if you have network access. Try a +command such as ``sudo host freedom.press``. If you don’t have network +access, it is most likely due to the upgraded kernel missing a network +driver for your hardware. + +If everything appears to be operating normally, the outage may not be +kernel-related. In that case, please still follow the steps at the end of +this document to send us log information along with an issue report, +and we will help you investigate. + +If you are experiencing network issues or other kernel problems, we +recommend that you roll back to an older kernel, and that you report the +issue to us immediately. + +Compare the behavior of the old kernel +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. |GRUB with advanced options selected| image:: ../images/0.5.x_to_0.6/grub-with-advanced-options-selected.png + +Reboot the server in a safe way with ``sudo reboot``. After the BIOS screen, +you can select a new kernel from the GRUB boot menu by selecting +**Advanced Options for Ubuntu**, pictured below. + +|GRUB with advanced options selected| + +Under **Advanced Options**, choose the option with kernel version ``3.14.x-grsec``. +As before, you may need to edit the kernel options to enter single user +mode. The boot process should proceed normally. Wait until you get a +login prompt and log in. + +Once you are logged in, check to see if you have network access. If you do, then +your instance is having an issue with the 4.x kernel. In that case, we need to +temporarily set an older kernel as the default. + +Roll back to the old kernel +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. important:: The older kernel has reached end of life, so it + is of critical importance for the near term security of your instance + that we work together to resolve any compatibility issues. Rolling back to an + older version is only a stopgap measure to avoid a prolonged outage of your + SecureDrop instance. + +Inspect the file ``/boot/grub/grub.cfg``. You should find a ``menuentry`` line +with the same text that you selected during boot, e.g.: + +.. code:: sh + + submenu 'Advanced options for Ubuntu'… + + menuentry 'Ubuntu, with Linux 3.14.xx-grsec… + +Take note of its position among the other submenu entries (it will most likely +be third). Then edit the GRUB configuration: + +.. code:: sh + + sudo vim /etc/grub/default + +Make a backup of the file or take a note of the current value of +``GRUB_DEFAULT`` somewhere, so you can restore the previous behavior easily at a +later point. + +Once you have done so, set the ``GRUB_DEFAULT`` variable to point to the index +of the menu and submenu. Note that the index starts at 0, so for a typical +setup, the line in ``/etc/grub/default`` would look like this: + +.. code:: sh + + GRUB_DEFAULT=”1>2” + +The “1” means the second entry of the main menu (“Advanced options”), +the “2” means the third entry of the submenu. Again, update these +numbers consistent with your configuration. + + +.. caution:: Ensure that you have chosen the right index for the main menu + and the submenu, and double-check that you are beginning the count at 0, not + 1; otherwise, you may boot into the wrong kernel. + +This change still has to be applied to take effect on the next boot: + +.. code:: sh + + sudo update-grub2 + +Now you can reboot into the old, working kernel. + +.. code:: sh + + sudo reboot + +The server should come up automatically. From here on, you should be +able to perform all administrative tasks via SSH as usual. If you want +additional confirmation of the kernel version, the command +``uname -a`` should display ``3.14.79-grsec``. + +Please notify us of the compatibility issue so we can help you resolve it ASAP. + +Report compatibility issues +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you have encountered issues with the kernel upgrade, it is important +that you report them to us so that we may incorporate any necessary +changes to our updated kernel, and so that we can work with you to +switch back to the new kernel as soon as possible. + +Run the following commands via SSH from the *Admin Workstation*: + +.. code:: sh + + cd ~/Persistent/securedrop/ + source .venv/bin/activate + cd install_files/ansible-base + ansible all -b -m setup > server-facts.log + +Please also send us a copy of ``/var/log/syslog`` and ``/var/log/dmesg`` for +analysis. + +You can share ``server-facts.log``, ``syslog`` and ``dmesg`` with us as follows: + +- If you are a member of our Support Portal, please create a new issue + and attach the files to it. +- Alternatively, email us at securedrop@freedom.press + (`GPG encrypted `__) + with the subject “SecureDrop kernel facts” and the files attached. + +Once we get your information, we can try to provide assistance to +resolve compatibility issues. + +Getting support +--------------- + +Should you require further support with your SecureDrop installation or upgrade, +we are happy to help! + +- Community support is available at https://forum.securedrop.club +- Paid support options are provided by Freedom of the Press Foundation. + Contact securedrop@freedom.press for more information.