Servers are monitored by Prometheus. Salt is used to:
Install a Node Exporter service on each server, to export hardware and OS metrics like disk space used, memory used, etc.
Set up a Prometheus server to collect metrics from all servers, and to email alerts if metrics are out of bounds
Read the user guide to learn how to use Prometheus.
Google Postmaster Tools#
Application errors are reported to Sentry, which notifies individual email addresses. All Salt-managed, OCP-authored services report errors to Sentry.
See the Software Development Handbook for access to Sentry.
Hetzner for hardware servers, including Kingfisher and Registry
Linode for VPS servers provisioned after August 2021
Network status: The relevant systems are: Regions: EU-West (London), Backups: EU-West (London) Backups.
Access: The ‘opencontractingpartnership’ and ‘opencontracting-dogsbody’ users have full access.
Backups: It is configured to have one daily backup and two weekly backups. Dogsbody also configured daily and weekly backups to Google Cloud Platform.
Unmanaged services are:
See the Software Development Handbook for access to third-party services.
The staff of the following organizations have had administrative roles:
ssh.root lists in Pillar files and the
ssh.admin list in the
pillar/common.sls file give people access to servers. All people should belong to the above organizations.
Server owners (OCP) and server managers (Dogsbody) should have root access to all servers. Otherwise, only developers who are reasonably expected to deploy to a server should have root access to that server.
If a developer did not deploy (and was not granted root access) to a server within the last six months, their root access to that server should be revoked.
If a developer intends to deploy to a server, anyone with root access can grant that developer root access to that server.
Root access should be routinely reviewed.
There should be a minimum of two admin members from OCP only.
Users should belong to a single group. Non-admin staff of OCP should belong to the unrestricted group.
There should be a minimum of two Administrator roles from OCP only.
See the process documentation for access to Redmine CRM.