Skip to content

Commit 06fbdd2

Browse files
mergify[bot]fastfadingviolets
mergify[bot]
and
fastfadingviolets
authored
ci: First run at interchaintests. (#3205) (#3243)
* First run at interchaintests. These include tests for different transaction types, and consumer chain launches. They are triggered by the docker-push workflow and will test the pushed image. They can also be triggered manually and will try to test an image on the branch supplied. * Upgrade interchaintest to v8 and use upstream * Add jq to the docker image. This is used by some features of interchaintest and is just generally useful to have. (cherry picked from commit bb6f13f) Co-authored-by: violet <[email protected]>
1 parent 7c7b31f commit 06fbdd2

File tree

19 files changed

+4370
-3
lines changed

19 files changed

+4370
-3
lines changed

‎.github/workflows/docker-push.yml‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ jobs:
1616
build-and-push-image:
1717
runs-on: ubuntu-latest
1818
permissions:
19-
contents: read
19+
contents: write
2020
packages: write
2121

2222
steps:
@@ -54,3 +54,14 @@ jobs:
5454
tags: ${{ steps.meta.outputs.tags }}
5555
labels: ${{ steps.meta.outputs.labels }}
5656
if: ${{ env.NEW_COMMIT_COUNT > 0 }}
57+
58+
- name: Dispatch Interchain test
59+
uses: peter-evans/repository-dispatch@v3
60+
with:
61+
event-type: image-pushed
62+
client-payload: |
63+
{
64+
"tag_name": "${{ fromJson(steps.meta.outputs.json)['labels']['org.opencontainers.image.version'] }}",
65+
"ref_name": "${{ fromJson(steps.meta.outputs.json)['labels']['org.opencontainers.image.revision'] }}"
66+
}
67+
if: ${{ env.NEW_COMMIT_COUNT > 0 }}

‎.github/workflows/interchain-test.yml‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
name: Interchain Test
2+
on:
3+
repository_dispatch:
4+
types: [image-pushed]
5+
workflow_dispatch:
6+
7+
jobs:
8+
prepare-matrix:
9+
runs-on: ubuntu-latest
10+
steps:
11+
# We need to figure out a) a ref to clone based on a docker image, and b) which tag to test
12+
# If the event is a registry_package, this comes from the pushed image; for a workflow_dispatch, it's the branch/tag that the user supplied
13+
- name: Get metadata
14+
id: get-metadata
15+
run: |
16+
if [[ "${{ github.event_name }}" == 'repository_dispatch' ]]; then
17+
echo "ref_name=${{ github.event.client_payload.ref_name }}" | tee -a $GITHUB_OUTPUT
18+
echo "tag_name=${{ github.event.client_payload.tag_name }}" | tee -a $GITHUB_OUTPUT
19+
else
20+
echo "ref_name=${{ github.ref_name }}" | tee -a $GITHUB_OUTPUT
21+
echo "tag_name=${{ github.ref_name }}" | tee -a $GITHUB_OUTPUT
22+
fi
23+
- name: Check out repository code
24+
uses: actions/checkout@v3
25+
with:
26+
ref: ${{ steps.get-metadata.outputs.ref_name }}
27+
- name: Setup go
28+
uses: actions/setup-go@v5
29+
- name: Prepare matrix
30+
id: generate-matrix
31+
run: |
32+
cd ./tests/interchain
33+
echo "matrix=$(go run ./matrix_tool/main.go ${{ steps.get-metadata.outputs.tag_name }})" | tee -a $GITHUB_OUTPUT
34+
outputs:
35+
matrix: ${{ steps.generate-matrix.outputs.matrix }}
36+
ref_name: ${{ steps.get-metadata.outputs.ref_name }}
37+
test:
38+
needs: prepare-matrix
39+
runs-on: ubuntu-latest
40+
name: "${{ matrix.previous_version }} -> ${{ matrix.test_version }} test ${{ matrix.test_name }}"
41+
strategy:
42+
matrix:
43+
${{fromJson(needs.prepare-matrix.outputs.matrix)}}
44+
fail-fast: false
45+
max-parallel: 10
46+
steps:
47+
- name: Check out repository code
48+
uses: actions/checkout@v3
49+
with:
50+
ref: ${{ needs.prepare-matrix.outputs.ref_name }}
51+
- name: Setup go
52+
uses: actions/setup-go@v5
53+
- name: Run test
54+
env:
55+
TEST_DOCKER_REGISTRY: "ghcr.io/${{ github.repository_owner }}"
56+
TEST_OLD_GAIA_IMAGE_VERSION: "${{ matrix.previous_version }}"
57+
TEST_NEW_GAIA_IMAGE_VERSION: "${{ matrix.test_version }}"
58+
TEST_UPGRADE_NAME: "${{ matrix.upgrade_name }}"
59+
run: |
60+
# This docker pull/tag is a quick hack only necessary for v19, since there were no official v18 images built.
61+
# Once we're testing 19 -> 20 this can be removed
62+
docker pull "ghcr.io/hyphacoop/gaia:v18.1.0" && docker tag "ghcr.io/hyphacoop/gaia:v18.1.0" "ghcr.io/${{ github.repository_owner }}/gaia:v18.1.0"
63+
cd ./tests/interchain
64+
go test -v ./... -failfast -p 1 -timeout 5h -run="^${{ matrix.test_name }}"

‎.github/workflows/test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ jobs:
4646
- name: test & coverage report creation
4747
if: env.GIT_DIFF
4848
run: |
49-
go test -v -coverprofile=profile.out -covermode=atomic -coverpkg=./... $(go list ./... | grep -v -e '/tests/e2e')
49+
go test -v -coverprofile=profile.out -covermode=atomic -coverpkg=./... $(go list ./... | grep -v -e '/tests/e2e' | grep -v -e '/tests/interchain')
5050
- uses: actions/upload-artifact@v4
5151
if: env.GIT_DIFF
5252
with:

‎Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ RUN echo "Ensuring binary is statically linked ..." \
2424
&& file /src/app/build/gaiad | grep "statically linked"
2525

2626
FROM alpine:$IMG_TAG
27-
RUN apk add --no-cache build-base
27+
RUN apk add --no-cache build-base jq
2828
RUN addgroup -g 1025 nonroot
2929
RUN adduser -D nonroot -u 1025 -G nonroot
3030
ARG IMG_TAG

‎tests/interchain/README.md‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# Interchain tests for gaia.
2+
3+
4+
These tests use [interchaintest](https://github.com/strangelove-ventures/interchaintest/) to
5+
create, upgrade, and test chains.
6+
7+
They dockerize the validators, so they depend on a `gaia` docker image being built.
8+
You can build a docker image using the [docker-push](../../.github/workflows/docker-push.yml) workflow.
9+
`docker-push` runs nightly on the `main` branch, and for all new releases, but you can also
10+
[run it manually on any branch](https://github.com/cosmos/gaia/actions/workflows/docker-push.yml)
11+
12+
Once the `gaia` image is built, the `docker-push` action workflow automatically invoke the
13+
[interchain-test](../../.github/workflows/interchain-test.yml) workflow.
14+
15+
Read on to learn how these tests work.
16+
17+
## Upgrade testing
18+
19+
The tests will make sure it's possible to upgrade from a previous version of
20+
`gaia` to the current version being tested. It does so by starting a chain from genesis
21+
on the previous version, then upgrading it to the current version.
22+
23+
## Version selection
24+
25+
The `interchain-test` workflow will start by selecting versions to test upgrading from.
26+
27+
The [`matrix_tool`](./matrix_tool/main.go) tool will take the tag of the image
28+
being tested (e.g. `v18.0.0`, or `main`, or `some-feature-branch`), and figure
29+
out a corresponding semver. If the tag is already a valid semver, that's the
30+
version. Otherwise, it will take the major version from the module line in `go.mod`,
31+
and append `.999.0`. Given that semver, it'll figure out:
32+
33+
* The previous rc (if the current version is itself an rc)
34+
* The previous minor version (if applicable)
35+
* The previous major version
36+
37+
For instance, for `v15.1.0-rc1`, we'll test upgrading from:
38+
* `v15.1.0-rc0`
39+
* `v15.0.0`
40+
* `v14.2.0`
41+
42+
The workflow will then test upgrading from each of those three to the current
43+
version. When it's a major upgrade, it will do so via governance proposal,
44+
otherwise it'll simply stop the old image and start the new one.
45+
46+
## Test Suites
47+
48+
Each of the *_test.go files in this directory contains a test suite. These
49+
share some common scaffolding (a `SetupSuite`) to create and upgrade a chain,
50+
and then run a set of tests on that chain.
51+
52+
So, for instance, a transactions suite:
53+
54+
```go
55+
type TxSuite struct {
56+
*chainsuite.Suite
57+
}
58+
```
59+
60+
It extends `chainsuite.Suite,` so its SetupSuite will create and upgrade a
61+
chain (more on this later). The individual `Test*` methods then run the gaia
62+
version being tested:
63+
64+
```go
65+
func (s *TxSuite) TestBankSend() {
66+
balanceBefore, err := s.Chain.GetBalance(s.GetContext(), s.Chain.ValidatorWallets[1].Address, chainsuite.Uatom)
67+
s.Require().NoError(err)
68+
69+
_, err = s.Chain.Validators[0].ExecTx(
70+
s.GetContext(),
71+
s.Chain.ValidatorWallets[0].Moniker,
72+
"bank", "send",
73+
s.Chain.ValidatorWallets[0].Address, s.Chain.ValidatorWallets[1].Address, txAmountUatom(),
74+
)
75+
s.Require().NoError(err)
76+
77+
balanceAfter, err := s.Chain.GetBalance(s.GetContext(), s.Chain.ValidatorWallets[1].Address, chainsuite.Uatom)
78+
s.Require().NoError(err)
79+
s.Require().Equal(balanceBefore.Add(sdkmath.NewInt(txAmount)), balanceAfter)
80+
}
81+
```
82+
83+
Because of how testify works, we have to instantiate each suite to run it.
84+
This is also where we tell the suite to run an upgrade on Setup:
85+
86+
```go
87+
func TestTransactions(t *testing.T) {
88+
txSuite := TxSuite{chainsuite.NewSuite(chainsuite.SuiteConfig{UpgradeOnSetup: true})}
89+
suite.Run(t, &txSuite)
90+
}
91+
```
92+
93+
Of course, we can also parameterize the test suites themselves. This enables us
94+
to write tests once and run them a bunch of times on different configurations:
95+
96+
```go
97+
type ConsumerLaunchSuite struct {
98+
*chainsuite.Suite
99+
OtherChain string
100+
OtherChainVersion string
101+
ShouldCopyProviderKey [chainsuite.ValidatorCount]bool
102+
}
103+
104+
func TestICS40ChainLaunch(t *testing.T) {
105+
s := &ConsumerLaunchSuite{
106+
Suite: chainsuite.NewSuite(chainsuite.SuiteConfig{}),
107+
OtherChain: "ics-consumer",
108+
OtherChainVersion: "v4.0.0",
109+
ShouldCopyProviderKey: noProviderKeysCopied(),
110+
}
111+
suite.Run(t, s)
112+
}
113+
114+
func TestICS33ConsumerAllKeysChainLaunch(t *testing.T) {
115+
s := &ConsumerLaunchSuite{
116+
Suite: chainsuite.NewSuite(chainsuite.SuiteConfig{}),
117+
OtherChain: "ics-consumer",
118+
OtherChainVersion: "v3.3.0",
119+
ShouldCopyProviderKey: allProviderKeysCopied(),
120+
}
121+
suite.Run(t, s)
122+
}
123+
```
124+
125+
Notice also how `UpgradeOnSetup` isn't set here: the ConsumerLaunchSuite needs
126+
to be handed a pre-upgrade chain so it can make sure that a consumer chain that
127+
launched before the upgrade keeps working after the upgrade.
128+
129+
130+
## Writing new tests
131+
132+
All you need to start writing new tests is a test suite as described above.
133+
The suite will have an `s.Chain` that you can test. Check out utilities in
134+
[`chainsuite/chain.go`](./chainsuite/chain.go) and
135+
[`chain_ics.go`](./chainsuite/chain_ics.go) for some convenience methods.
136+
137+
In addition, the s.Chain object extends the `interchaintest` chain object, so
138+
check out [the docs](https://pkg.go.dev/github.com/strangelove-ventures/interchaintest/v7) to
139+
see what else is available.

0 commit comments

Comments
 (0)