Reference#

Monitoring#

Prometheus#

Servers are monitored by Prometheus. Salt is used to:

  • Install a Node Exporter service on each server, to export hardware and OS metrics like disk space used, memory used, etc.

  • Set up a Prometheus server to collect metrics from all servers, and to email alerts if metrics are out of bounds

Read the user guide to learn how to use Prometheus.

DMARC Analyzer#

OCP’s DMARC policy (dig TXT _dmarc.open-contracting.org) sends aggregate and forensic reports to DMARC Analyzer.

Google Postmaster Tools#

Google Postmaster Tools can be used to debug deliverability issues from AWS to GMail.

Sentry#

Application errors are reported to Sentry, which notifies individual email addresses. All Salt-managed, OCP-authored services report errors to Sentry.

See the Software Development Handbook for access to Sentry.

Hosting#

All servers (not services) are managed by Dogsbody Technology (sysadmin@dogsbody.com). Servers are hosted by:

  • Hetzner for hardware servers, including Kingfisher and Registry

  • Linode for VPS servers provisioned after August 2021

    • Network status: The relevant systems are: Regions: EU-West (London), Backups: EU-West (London) Backups.

    • Access: The ‘opencontractingpartnership’ and ‘opencontracting-dogsbody’ users have full access.

    • Backups: It is configured to have one daily backup and two weekly backups. Dogsbody also configured daily and weekly backups to Google Cloud Platform.

  • Bytemark for VPS servers provisioned before August 2021.

Unmanaged services are:

Administrative access#

See the Software Development Handbook for access to third-party services.

The staff of the following organizations have had administrative roles:

The ssh.root lists in Pillar files and the ssh.admin list in the pillar/common.sls file give people access to servers. All people should belong to the above organizations.

Root access#

Server owners (OCP) and server managers (Dogsbody) should have root access to all servers. Otherwise, only developers who are reasonably expected to deploy to a server should have root access to that server.

If a developer did not deploy (and was not granted root access) to a server within the last six months, their root access to that server should be revoked.

If a developer intends to deploy to a server, anyone with root access can grant that developer root access to that server.

Root access should be routinely reviewed.

Redash#

There should be a minimum of two admin members from OCP only.

Users should belong to a single group. Non-admin staff of OCP should belong to the unrestricted group.

Redmine CRM#

There should be a minimum of two Administrator roles from OCP only.

See the process documentation for access to Redmine CRM.