Uploaded image for project: 'GPII - Global Public Inclusive Infrastructure'
  1. GPII - Global Public Inclusive Infrastructure
  2. GPII-3858

Improve docker image workflow (Separate GCR instances)

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: APCP Infrastructure
    • Labels:
    • Story Points:
      5

      Description

      Problem

      Before GPII-3641, we pulled all images from public repos (mostly Docker Hub).

      After GPII-3641, we pull all images from a single production GCR instance (gcr.io/gpii-common-prd)[1].

      Because Docker tags are mutable, it is possible with either of these arrangements to deploy an image to production that has not moved through the pipeline.[2]

      [1] We also have a staging GCR instance (gcr.io/gpii2test-common-stg), but nothing uses it.

      [2] A scenario:

      • Pod uses couchdb:2.2, which is sha256:11111
      • couchb:2.2 changes upstream to sha256:22222
      • Pod crashes and restarts, pulls couchdb:2.2 which is now sha256:22222

      Mitigations

      This problem is mitigated by a few factors:

      • For "internal" components (e.g. gpii/universal), we use sha256 directly rather than a tag. These components are not susceptible to this problem.
      • In practice, I do not expect projects to change the tag of a published image, so this problem should be rare.

      A solution

      Here's one way it could work:

      1. gpii-version-updater runs as today, but pushes new images to the common-stg GCR and writes to versions.common-stg.yml.
      2. dev environments (including CI dev environments like dev-doe and dev-gitlab-runner) get image information from versions.common-stg.yml.
      3. Once dev environments pass, a pipeline step pushes image to the common-prd GCR and writes to versions.common-prd.yml.
        • This will require granting gcr_uploader privileges to the CI worker, probably via ansible-gpii-ci-worker (there is code to do something similar for i46 in ansible-gpii-version-updater).
      4. stg and prd get image information from versions.common-prd.yml.

        Attachments

          Activity

            People

            Assignee:
            devops-triage DevOps Triage
            Reporter:
            tyler Tyler Roscoe (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: