Increase verbosity

./ --log-level=trace TARGET FUNCTION

Salt hangs inexplicably

If you haven’t previously connected to a server using SSH, then ./ will log a TRACE-level message like:

The authenticity of host ' (' can't be established.
ECDSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

You can also experience this issue if anyone changes the hostnames in the salt-config/roster file.

Unless you increase verbosity, you won’t see this message, and ./ will appear to hang.

To fix this, simply connect to the server using SSH, for example:

curl --silent --connect-timeout 1 || true

Then, re-run the ./ command.


It is not recommended to use the -i (--ignore-host-keys) option, as this disables strict host key checking, allowing for man-in-the-middle attacks.

Watch Salt activity

If you want to check whether a deployment is simply slow or actually stalled, perform these steps:

  1. Find the server’s IP or fully-qualified domain name in the roster:

    cat salt-config/roster
  2. Open a secondary terminal to connect to the server as root, for example:

    curl --silent --connect-timeout 1 || true
  3. Watch the processes on the server:

    watch -n 1 pstree
  4. Look at the lines below these:

|      |-sshd---bash---watch---watch---sh---pstree

Then, once the deployment is done:

  1. Stop watching the processes, e.g. with Ctrl-C
  2. Disconnect from the server, e.g. with Ctrl-D

Restart a service

Services should restart normally. To manually restart a service, run, for example:

./ TARGET service.restart uwsgi

Check git revision

To check which branch is deployed, run, for example:

./ covid19 git.current_branch /home/covid19admin/covid19admin

To check which commit is deployed, run, for example:

./ covid19 git.revision /home/covid19admin/covid19admin