Amazon Web Services (AWS)

Our default region is us-east-1 (N. Virginia). For large data transfer operations (like backups), use the closest region: for example, us-west-2 (London) for Linode servers in the London datacenter.

Simple Email Service (SES)

Reference: Setting up Amazon Simple Email Service

Note

Dedicated IP addresses for Amazon SES are available. However, a dedicated IP address would take a long time to cultivate a sending reputation with our low volume. The shared IP addresses have good reputation. As describe below, SPF, DKIM and Return-Path are configured to improve deliverability.

Verify a domain

  1. Go to SES’ Identities:

    1. Click Create identity

    2. Check Domain

    3. Enter the domain in Domain

    4. Expand Advanced DKIM settings

    5. Check Easy DKIM

    6. Check RSA_2048_BIT

    7. Click Create identity

  2. Go to GoDaddy’s DNS Management:

    1. Add the three CNAME records. Add the MX record if none exists.

      Note

      Omit .open-contracting.org from hostnames. GoDaddy appends it automatically.

    2. Add or update the SPF record

  3. Wait for the domain’s Identity status to become “Verified” on SES’ Identities

    Note

    AWS will notify you by email. Last time, it took a few minutes.

Reference: Creating (and verifying) a domain identity

Verify an email address

  1. Go to SES’ Identities:

    1. Click Create identity

    2. Check Email address

    3. Enter the email address in Email address

    4. Click Create identity

  2. If the domain’s MX record points to AWS, go to SES’ Email receiving:

    1. Click Create rule set

    2. Enter a name in Rule set name (email-address-verification, for example)

    3. Click Create rule set

    4. Click the rule set’s name

    5. Click Create rule

    6. Enter a rule name in Rule name (deliver-to-s3-bucket, for example)

    7. Click Next

    8. Click Next

    9. Select “Deliver to S3 bucket” from the Add new action dropdown

    10. Click Create S3 bucket

    11. Enter a bucket name in Bucket name (ocp-aws-verification, for example)

    12. Click Create bucket

    13. Click Next

    14. Click Create rule

    15. Click Set as active

  3. If the domain’s MX record points to AWS, go to S3 (otherwise, check your email):

    1. Click the bucket name

    2. Click the long alphanumeric string (if there is none, double-check the earlier steps)

    3. Click Download

    4. Copy the URL in the downloaded file

    5. Open the URL in a web browser

  4. Check that the email address’ Identity status is “Verified” on SES’ Identities

  5. If the domain’s MX record points to AWS, cleanup:

    1. Set the rule set as inactive

Reference: Creating (and verifying) an email address identity

Use a MAIL FROM domain

Note

This optional step improves email deliverability. Also known as the Return-Path address.

  1. Refer to Using a custom MAIL FROM domain

  2. Check that the identity’s MAIL FROM configuration is “Successful”

Create SMTP credentials

Note

You only need to do this once per AWS region.

  1. Go to SES’ SMTP Settings:

    1. Click Create SMTP credentials

    2. Enter a user name in IAM User Name:

    3. Click Create

    4. Click Download Credentials

    5. Click Close

Reference: Obtaining Amazon SES SMTP credentials

Set up basic notifications

  1. Go to SNS’ Topics:

    1. Click Create topic

    2. Set Type to Standard

    3. Enter a hyphenated address in Name (data-open-contracting-org, for example)

    4. Click Create topic

  2. Click Create subscription:

    1. Select “Email” from the Protocol dropdown

    2. Enter an email address in Endpoint

    3. Click Create subscription

  3. Click the email address on SES’ Identities:

    1. Click the Notifications tab

    2. Click Edit in the Feedback notifications section

    3. Select the created topic from the Bounce feedback dropdown

    4. Check the Include original email headers box

    5. Select the created topic from the Complaint feedback dropdown

    6. Check the Include original email headers box

    7. Click Save changes

Reference: Configuring Amazon SNS notifications for Amazon SES

Set up advanced notifications

  1. Go to SES’ Configuration sets:

    1. Click Create set

    2. Enter a name in Configuration set name (credere, for example)

    3. Click Create set

  2. Click the configuration set’s name

  3. Click the Event destinations tab

  4. Click Add destination:

    1. Check:

      • Rendering failures, if using email templates

      • Rejects

      • Delivery delays

      Do not check, to avoid unnecessary notifications:

      • Sends

      • Deliveries (same as Delivery feedback above)

      • Hard bounces (same as Bounce feedback above)

      • Complaints (same as Complaint feedback above)

      • Subscriptions

    2. Click Next

    3. Check Amazon SNS

    4. Enter a name in Name (credere-noreply-open-contracting-org, for example)

    5. Select the SNS topic for basic notifications from the SNS topic dropdown

    6. Click Next

    7. Click Add destination

  5. Go to SNS’ Subscriptions:

    1. Click Create subscription

    2. Select the SNS topic from the Topic ARN dropdown

    3. Select “Email” from the Protocol dropdown

    4. Enter the subscriber’s email address in Endpoint

    5. Click Create subscription

  6. Wait for the email to confirm the subscription

Check DMARC compliance

Check DMARC compliance, sending the email using SES.

Note

SES adds an extra DKIM signature (“The extra DKIM signature, which contains d=amazonses.com, is automatically added by Amazon SES. You can ignore it”). It is not aligned, but according to RFC 7489, “a single email can contain multiple DKIM signatures, and it is considered to be a DMARC ‘pass’ if any DKIM signature is aligned and verifies.”

Debug delivery issues

Bounces and complaints are sent to the subscribed address. The relevant properties of the notification message are:

See also

Disable account-level suppression list

Note

This optional step can negatively affect sender reputation.

Reference: Disabling the account-level suppression list

Move out of sandbox

Note

You only need to do this once per AWS account.

Reference: Moving out of the Amazon SES sandbox

Relational Database Service (RDS)

Note

This configuration is for data analysis, where it is acceptable for the data to be lost.

  1. Go to RDS’ Databases

  2. Click Create database

    1. Set Engine type to “PostgreSQL”

    2. Set Version to the latest version

    3. Set Templates to “Free tier”

    4. Check Auto generate a password

    5. Set DB instance class to “db.t3.micro”

    6. Uncheck Enable storage autoscaling

    7. Set Public access to “Yes”

    8. Add “postgresql-anywhere” to Existing VPC security groups

    9. Remove “default” from Existing VPC security groups

    10. Expand Additional configuration

    11. Uncheck Enable automated backups

    12. Uncheck Enable encryption

    13. Uncheck Turn on Performance Insights

    14. Click Create database

  3. Wait for the database to be created

  4. Click View connection details

Simple Storage Service (S3)

Create bucket

  1. Go to Amazon S3 Buckets

  2. Select the nearest region to the server from the top-right dropdown

  3. Click the Create bucket button

    1. Enter a Bucket name (ocp-registry-backup, for example)

    2. Click the Create bucket button

If the bucket is for file or MySQL backups:

Warning

Do not create lifecycle rules when using pgBackRest, which manages lifecycle itself.

  1. Click the created bucket

  2. Click the Management tab

  3. Click the Create lifecycle rule button

    1. Lifecycle rule name: delete-after-30-days

    2. Choose a rule scope: Apply to all objects in the bucket

    3. Check I acknowledge that this rule will apply to all objects in the bucket.

    4. Check Expire current versions of objects

    5. Check Delete expired object delete markers or incomplete multipart uploads

    6. Days after object creation: 30

    7. Check Delete incomplete multipart uploads

    8. Number of days: 7

  4. Click Create rule

Reference: Creating a bucket

Identity and Access Management (IAM)

Create a backup policy

  1. Go to IAM Policies

  2. Click Create policy

    1. Click the JSON tab

    2. Paste the appropriate content below, replacing BUCKET_NAME and/or PREFIX:

      See also

      pgBackRest sample Amazon S3 policy

      {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME"
            ],
            "Condition": {
                "StringEquals": {
                    "s3:prefix": [
                        "",
                        "PREFIX"
                    ],
                    "s3:delimiter": [
                        "/"
                    ]
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME"
            ],
            "Condition": {
                "StringLike": {
                    "s3:prefix": [
                        "PREFIX/*"
                    ]
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME/PREFIX/*"
            ]
        }
    ]
}

      {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME/*"
            ]
        }
    ]
}

    3. Click the Next button

    4. Enter a Policy name (registry-backup, for example)

    5. Click the Create policy button

Create a backup user

Note

If a policy is relevant to many users, instead of attaching policies directly, create a group, attach the policy to the group, and add the user to the group.

  1. Go to IAM Users

  2. Click the Create user button

    1. Enter a User name (registry-backup, for example)

    2. Click the Next button

    3. Click the Attach existing policies directly radio button

    4. Search for and check the policy above

    5. Click the Next button

    6. Click the Create user button

  3. Click the created user

  4. Click the Security credentials tab

  5. Click the Create access key button

    1. Check the Command Line Interface (CLI) radio button

    2. Check the I understand the above recommendation and want to proceed to create an access key. box

    3. Click the Next button

    4. Click the Create access key button

    5. Copy the Access key and Secret access key

    6. Click the Done button

Reference: Creating an IAM user in your AWS account