Google Workspace¶

Email¶

Use Google Postmaster Tools to debug deliverability issues from AWS to Gmail.

These services send email from open-contracting.org:

These services send email from noreply.open-contracting.org:

Amazon Web Services (AWS)

These services send email from payments.open-contracting.org:

Trolley (using SendGrid)

Servers relay email through Amazon Web Services (AWS), from noreply.open-contracting.org.

Check DNS configuration¶

Google Admin Toolbox Check MX should report no problems (all green).
MXToolBox Domain Health Report should report no errors (only warnings).

Check DMARC compliance¶

Send an email to ping@tools.mxtoolbox.com and check the results (all green).

Similar tools include:

Monitor DMARC reports¶

The DMARC policies send aggregate reports to:

$ dig TXT _dmarc.open-contracting.org
v=DMARC1; p=none; rua=mailto:re+tvgueigvygp@dmarc.postmarkapp.com,mailto:dmarc_agg@vali.email;

$ dig TXT _dmarc.noreply.open-contracting.org
v=DMARC1; p=none; rua=mailto:re+jbvvmcsfauo@dmarc.postmarkapp.com,mailto:dmarc_agg@vali.email;

$ dig TXT _dmarc.open-spending.eu
v=DMARC1; p=quarantine; rua=mailto:re+wtazrnx9nxe@dmarc.postmarkapp.com,mailto:dmarc_agg@vali.email;

$ dig TXT dream-office.org
v=DMARC1; p=none; rua=mailto:re+yjzbqifwsvu@dmarc.postmarkapp.com,mailto:dmarc_agg@vali.email;

DMARC compliance should be over 95%, and DKIM alignment should be over 90%. Failures should be 3% or less.

Note

Mailchimp is not SPF aligned; therefore, we have no target for SPF alignment. It sends mail from mcsv.net, mcdlv.net, mailchimpapp.net and rsgsv.net.

Note

Tools might report a “DKIM invalid” warning due to AWS SES using null DKIM records.

Sending domains with volumes of less than 10 can be ignored. For google.com:

SPF misalignment with calendar-server.bounces.google.com can be ignored.
Google Groups rewrites the From header only if the DMARC policy is “reject” or “quarantine”.

Delete user¶

One-time setup

Install Google Apps Manager (GAM). In a bash shell:
```
bash <(curl -s -S -L https://gam-shortn.appspot.com/gam-install)
```
- Select only these scopes (gam oauth create):
  - 0) Calendar API
  - 32) Directory API - Groups
  - 39) Directory API - Users
- Limit its Domain-wide Delegation to the scopes used when impersonating users with gam user and gam all:
  - https://www.googleapis.com/auth/drive
  - https://www.googleapis.com/auth/calendar

Install Got Your Back (GYB). In a bash shell:

bash <(curl -s -S -L https://gyb-shortn.jaylee.us/gyb-install)

Perform the global steps once, and repeat the other steps for each user to be deleted.

Setup (global)¶

Configure the administrator, OCP Archive shared drive, and GAM and GYB projects:

set admin jmckinney
set shareddrive 0AKb5W5k2WH46Uk9PVA
set gamproject gam-project-9yro6
set gybproject gyb-project-haj-zu2-x36

Enable the service accounts:

gcloud iam service-accounts enable $gamproject@$gamproject.iam.gserviceaccount.com --project $gamproject
gcloud iam service-accounts enable $gybproject@$gybproject.iam.gserviceaccount.com --project $gybproject

Write the shortcuts in active users’ My Drive and shared drives:

gam all users_na_ns print filelist corpora alldrives \
  query "mimeType='application/vnd.google-apps.shortcut' and not trashed" \
  fields id,name,owners,driveid,parents,shortcutdetails > google-shortcuts.csv

Calendar (global)¶

A secondary calendar is a calendar that a user creates in addition to their default calendar. It is deleted along with its creator, even if other users are owners.

To verify that no calendar in use by active users was created by an archived user, we review all active users’ calendar lists, due to limitations of the Calendar API.

Write the calendar lists of active users:

gam all users_na_ns print calendars > google-calendars.csv

Report the secondary calendars that are owned by archived users:
```
uv run manage.py google-calendar google-calendars.csv
```
Transfer all reported calendars to an active user, before deleting the archived users.

Setup¶

Configure the user to delete and its retention start date. For example:

set user data-tools
set retentionstartdate 2026-08-08

Gmail¶

Unarchive the user:

gam update user $user@open-contracting.org archived off

Backup the user’s mail:

gyb --email $user@open-contracting.org --service-account --action backup \
  --local-folder $user-$retentionstartdate --fast-incremental

Attention

If errors are logged, re-run the command to backup missed messages.

Compress the backup:

tar czf $user-$retentionstartdate-gmail.tar.gz $user-$retentionstartdate

Upload the backup to the OCP Archive shared drive:

gam user $admin@open-contracting.org add drivefile \
  localfile $user-$retentionstartdate-gmail.tar.gz teamdriveparentid $shareddrive

Delete the local files:

rm -rf $user-$retentionstartdate $user-$retentionstartdate-gmail.tar.gz

Groups¶

List the groups of which the user is an owner, along with all owners:

gam print groups member $user@open-contracting.org role owner

If the user is the sole owner of a group, add another owner, replacing GROUP and USER:
```
gam update group GROUP@open-contracting.org add owner USER@open-contracting.org
```
Note

If the new owner is already a member or manager of the group, use update, instead of add.

Drive¶

List the user’s Forms, Sites and Apps Script in Drive, whose deletion could break things:

gam user $user@open-contracting.org print filelist \
  showownedby me fields id,name,mimetype,modifiedtime \
  query "mimeType='application/vnd.google-apps.form' or mimeType='application/vnd.google-apps.site' or mimeType='application/vnd.google-apps.script'"

Write the user’s files in Drive:

gam user $user@open-contracting.org print filelist \
  query "'me' in owners and not trashed" \
  fields id,name,mimetype,modifiedtime > google-drive-$user.csv

Report the user’s files that have shortcuts:
```
uv run manage.py google-drive google-drive-$user.csv google-shortcuts.csv
```
For each shared drive, it prints commands to run, in order to move those files next to the shortcuts in that shared drive, and to delete those shortcuts.

If a folder has shortcuts from active users’ My Drive, notify those users of the new folder (folders can’t be moved, only recreated).

Re-run the previous step then this step.
You may review the google-drive-$user.csv file, and move any other in-use files to shared drives. Replace FILE_ID, and replace FOLDER_ID with a shared drive or one of its folders:
```
gam user $user@open-contracting.org move drivefile FILE_ID \
  shareddriveparentid FOLDER_ID duplicatefiles uniquename summary showpermissionmessages
```
To move several, change FILE_ID to: ids FILE_ID_1,FILE_ID_2

Move the remaining files to the OCP Archive shared drive:

Make the user a Manager of the shared drive:

gam add drivefileacl $shareddrive user $user@open-contracting.org role manager

Create a folder named after the user, and configure it as the destination:

set folder ( \
  gam user $admin@open-contracting.org create drivefile \
    shareddriveparentid $shareddrive mimetype gfolder \
    drivefilename $user-$retentionstartdate returnidonly \
)

Move the user’s My Drive into the folder, preserving the hierarchy:
```
gam user $user@open-contracting.org move drivefile \
  root mergewithparentretain \
  shareddriveparentid $folder createshortcutsfornonmovablefiles \
  duplicatefiles uniquename summary showpermissionmessages
```
Attention

If you see this fragment, those files might be in use.
```
is not a member of this shared drive
```
Note
- Folders are recreated, and therefore change IDs.
- Files owned by other users are replaced by shortcuts.
- duplicatefiles uniquename renames files that have the same name as a file at the destination. Otherwise, the default is to delete the file in the destination, if it is older.

Move remaining files (those in other users’ folders or with no parent folder), with no hierarchy:

gam user $user@open-contracting.org move drivefile \
  query "'me' in owners and not trashed and mimeType != 'application/vnd.google-apps.folder'" \
  shareddriveparentid $folder duplicatefiles uniquename summary showpermissionmessages

Confirm that no files remain:

gam user $user@open-contracting.org print filelist \
  query "'me' in owners and not trashed and mimeType != 'application/vnd.google-apps.folder'" \
  fields id,name,parents

Remove the user from the shared drive:
```
gam delete drivefileacl $shareddrive $user@open-contracting.org
```
Note

Retry on 409 Conflict. Google might still be moving files.

Deletion¶

Delete the user:

gam delete user $user@open-contracting.org

Tip

You can undelete within 20 days, replacing USER:

gam undelete user USER@open-contracting.org

Teardown (global)¶

Disable the service accounts:

gcloud iam service-accounts disable $gamproject@$gamproject.iam.gserviceaccount.com --project $gamproject
gcloud iam service-accounts disable $gybproject@$gybproject.iam.gserviceaccount.com --project $gybproject

Delete temporary files:

rm -f google-calendars.csv google-drive-*.csv google-shortcuts.csv

Orphaned files¶

An orphaned file has no parent folder, reachable only by search or shortcut.

A file is orphaned when the folder that contained it is moved to a shared drive by a user who doesn’t own the file, in which case the folder is recreated in the shared drive with a shortcut to the file.

To review orphaned files, replacing USER:

gam user USER@open-contracting.org print filelist select orphans excludetrashed \
  fields id,name,mimetype,parents

Note

If moving files, move only those whose parents.0.id is empty or isn’t listed (i.e. the parent is outside your My Drive).