Consolidate CircleCI workflows and optimize test organization

[mmorel-35]

Description

Consolidate CircleCI workflows and optimize test organization

Changes

• Workflows: 8 → 2

  • main: Single CI pipeline with dependency chain
  • docker: Image builds/publishing (unchanged filters)

• Test organization

  lint (quality gate)
    ↓ requires
  ├─ regular tests (parallel by platform)
  │  ├─ test-linux
  │  ├─ test-linux-arm64  
  │  ├─ test-mac-arm64
  │  └─ test-windows
  └─ bazel tests (parallel, matrix by platform × version)
     ├─ test-bazel-linux [7.x, 8.x]
     ├─ test-bazel-mac-arm64 [7.x, 8.x]
     └─ test-bazel-windows [7.7.1, 8.5.1]

• Reusable commands

  • Extracted setup-docker command
  • Applied to build-docker-image-x64 and publish-docker-image-x64

[mmorel-35]

@sbc100

With that organisation you can have "main" job being the only one required to apply to all jobs that used to be required individually now

[sbc100]

I think I would rather have a little more fine-grained info available in the PR without having to click though.

For example, its kind of kind of see just the windows jobs failing when you break windows. I know one can always click through.. and I don't feel too strongly about this.

@brendandahl @dschuff @juj @kripken what do you folks think?

[sbc100]

Thanks for working on this BTW! I really appreciate the improvements.

[sbc100]

Also, I do think we want to run the docker build as part of CI (even if we don't run the docker publish).

I also think it is nice to have the lint job be super fast so you don't need to wait for it to fail.

Its also nice if the lint job can fail but the other jobs still report green (then you know all you need to do is fix the lint errors and your are done).

[sbc100]

Would you mind splitting this up into 3 changes:

1. Add caching to the lint CI job (how much time does this actually save?)
2. Extract setup-docker steps.
3. Re-organize the OS testing

[mmorel-35]

Would you mind splitting this up into 3 changes

I will.

[sbc100]

I think maybe I'd rather keep the bazel jobs separate, so they can pass and fail on their own.

It also increases the parallelism doesn't it?

[mmorel-35]

I don't think it changes parallelism as circle ci also execute jobs in parallel .
But it is faster to view when not grouping