Test and commit

Test changes

To preview what is going to change, use test=True, for example:

./run.py 'docs' state.apply test=True

To preview changes to a Pillar file, run, for example:

./run.py 'docs' pillar.items

To compare Jinja2 output after refactoring but before committing, use script/diff to compare a full state or one SLS file, for example:

./script/diff docs
./script/diff docs zip

If you get the error, An Exception occurred while executing state.show_highstate: 'list' object has no attribute 'values', run state.apply test=True as above. You might have conflicting IDs.

Using a virtual machine

  1. Create a virtual machine

  2. Get the virtual machine’s IP address

    • If using VirtualBox, run (replacing VMNAME):

      VBoxManage guestproperty get VMNAME "/VirtualBox/GuestInfo/Net/0/V4/IP"

  3. Update the relevant target in salt-config/roster to point to the virtual machine’s IP address

  4. In the relevant Pillar file, set https to False

  5. Edit /etc/hosts to map the virtual machine’s IP address to the service’s hostname

  6. Deploy to the virtual machine and test

Note that Python errors that occur on the virtual machine might still be reported to Sentry. The server_name tag in any error reports is expected to be different, but the error reports might still confuse other developers who don’t know to check that tag.

Review code

For context, for other repositories, work is done on a branch and tested on a local machine before a pull request is made, which is then tested on continuous integration, reviewed and approved before merging.

However, for this repository, in some cases, it’s impossible to test changes to server configurations, for example: if SSL certificates are involved (without additional configuration), or if external services are involved. In other cases, it’s too much effort to setup a test environment in which to test changes.

In such cases, the same process is followed as in other repositories, but without the benefit of tests.

In entirely uncontroversial or time-sensitive cases, work is done on the default branch, deployed to servers, and committed to the default branch once successful. In cases where the changes require trial and error, the general approach is discussed in a GitHub issue, and then work is done on the default branch as above. Developers can always request informal reviews from colleagues.

Take extra care when making larger changes or when making changes to higher-priority services like the OCDS documentation, Data Review Tool or Kingfisher.