Skip to content

perf(trie): batch storage proof jobs at worker level (experiment) #20210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yongkangc wants to merge 16 commits into from
Closed

perf(trie): batch storage proof jobs at worker level (experiment) #20210

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
9a55cb9
Add bench compare latency stats
yongkangc Nov 20, 2025
e7bd168
chore(deps): weekly `cargo update` (#20174)
github-actions[bot] Dec 7, 2025
0b07657
ci: run on ubuntu instead of reth runner (#20196)
shekhirin Dec 8, 2025
9265700
feat: parallelize recovery (#20169)
klkvr Dec 8, 2025
3094cd6
fix: make inserted blocks part of fcu canonical (#20164)
mattsse Dec 8, 2025
0cbd2ae
feat(primitives-traits): add recover_transactions_ref to avoid clonin…
forkfury Dec 8, 2025
c0d73fa
fix: trace filter range off by one (#20199)
mattsse Dec 8, 2025
12fdca7
perf: avoid duplicate storage get call (#20180)
mattsse Dec 8, 2025
b6dca7b
fix: set merkle changesets distance minimum to 128 (#20200)
joshieDo Dec 8, 2025
a430fb6
test(era): complete int tests with roundtrip mainnet era files (#20064)
lean-apple Dec 8, 2025
388507b
fix(rpc): return error if toBlock exceeds current head (#20202)
futreall Dec 8, 2025
88dfd15
feat: add StorageSettings for StoragesHistory in RocksDB (#20154)
0xKarl98 Dec 8, 2025
bb4285c
feat: add helper method to eth validator (#20206)
klkvr Dec 8, 2025
394b0d5
perf(trie): batch proof jobs at worker level
yongkangc Dec 9, 2025
1a5d9a3
perf(trie): use Arc for proof sharing and add batch validation
yongkangc Dec 9, 2025
9bd5a3e
fix: strengthen batch invariants and prevent blinded node starvation
yongkangc Dec 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/bench.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ env:
name: bench
jobs:
codspeed:
runs-on:
group: Reth
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/compact.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ env:
name: compact-codec
jobs:
compact-codec:
runs-on:
group: Reth
runs-on: ubuntu-latest
strategy:
matrix:
bin:
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ concurrency:
jobs:
test:
name: e2e-testsuite
runs-on:
group: Reth
runs-on: ubuntu-latest
env:
RUST_BACKTRACE: 1
timeout-minutes: 90
Expand All @@ -43,4 +42,3 @@ jobs:
--exclude 'op-reth' \
--exclude 'reth' \
-E 'binary(e2e_testsuite)'

9 changes: 3 additions & 6 deletions .github/workflows/hive.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,7 @@ jobs:
prepare-hive:
if: github.repository == 'paradigmxyz/reth'
timeout-minutes: 45
runs-on:
group: Reth
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Checkout hive tests
Expand Down Expand Up @@ -179,8 +178,7 @@ jobs:
- prepare-reth
- prepare-hive
name: run ${{ matrix.scenario.sim }}${{ matrix.scenario.limit && format(' - {0}', matrix.scenario.limit) }}
runs-on:
group: Reth
runs-on: ubuntu-latest
permissions:
issues: write
steps:
Expand Down Expand Up @@ -247,8 +245,7 @@ jobs:
notify-on-error:
needs: test
if: failure()
runs-on:
group: Reth
runs-on: ubuntu-latest
steps:
- name: Slack Webhook Action
uses: rtCamp/action-slack-notify@v2
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/integration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ jobs:
test:
name: test / ${{ matrix.network }}
if: github.event_name != 'schedule'
runs-on:
group: Reth
runs-on: ubuntu-latest
env:
RUST_BACKTRACE: 1
strategy:
Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/kurtosis-op.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ on:

push:
tags:
- '*'
- "*"

env:
CARGO_TERM_COLOR: always
Expand All @@ -32,8 +32,7 @@ jobs:
strategy:
fail-fast: false
name: run kurtosis
runs-on:
group: Reth
runs-on: ubuntu-latest
needs:
- prepare-reth
steps:
Expand Down Expand Up @@ -83,12 +82,10 @@ jobs:
kurtosis service logs -a op-devnet op-cl-2151908-2-op-node-op-reth-op-kurtosis
exit 1


notify-on-error:
needs: test
if: failure()
runs-on:
group: Reth
runs-on: ubuntu-latest
steps:
- name: Slack Webhook Action
uses: rtCamp/action-slack-notify@v2
Expand Down
10 changes: 4 additions & 6 deletions .github/workflows/kurtosis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ on:

push:
tags:
- '*'
- "*"

env:
CARGO_TERM_COLOR: always
Expand All @@ -30,8 +30,7 @@ jobs:
strategy:
fail-fast: false
name: run kurtosis
runs-on:
group: Reth
runs-on: ubuntu-latest
needs:
- prepare-reth
steps:
Expand All @@ -54,13 +53,12 @@ jobs:
- name: Run kurtosis
uses: ethpandaops/kurtosis-assertoor-github-action@v1
with:
ethereum_package_args: '.github/assets/kurtosis_network_params.yaml'
ethereum_package_args: ".github/assets/kurtosis_network_params.yaml"

notify-on-error:
needs: test
if: failure()
runs-on:
group: Reth
runs-on: ubuntu-latest
steps:
- name: Slack Webhook Action
uses: rtCamp/action-slack-notify@v2
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/prepare-reth.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ jobs:
prepare-reth:
if: github.repository == 'paradigmxyz/reth'
timeout-minutes: 45
runs-on:
group: Reth
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- run: mkdir artifacts
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/stage.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,7 @@ jobs:
name: stage-run-test
# Only run stage commands test in merge groups
if: github.event_name == 'merge_group'
runs-on:
group: Reth
runs-on: ubuntu-latest
env:
RUST_LOG: info,sync=error
RUST_BACKTRACE: 1
Expand Down
5 changes: 2 additions & 3 deletions .github/workflows/sync-era.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ concurrency:
jobs:
sync:
name: sync (${{ matrix.chain.bin }})
runs-on:
group: Reth
runs-on: ubuntu-latest
env:
RUST_LOG: info,sync=error
RUST_BACKTRACE: 1
Expand Down Expand Up @@ -64,4 +63,4 @@ jobs:
${{ matrix.chain.bin }} stage unwind num-blocks 100 --chain ${{ matrix.chain.chain }}
- name: Run stage unwind to block hash
run: |
${{ matrix.chain.bin }} stage unwind to-block ${{ matrix.chain.unwind-target }} --chain ${{ matrix.chain.chain }}
${{ matrix.chain.bin }} stage unwind to-block ${{ matrix.chain.unwind-target }} --chain ${{ matrix.chain.chain }}
5 changes: 2 additions & 3 deletions .github/workflows/sync.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ concurrency:
jobs:
sync:
name: sync (${{ matrix.chain.bin }})
runs-on:
group: Reth
runs-on: ubuntu-latest
env:
RUST_LOG: info,sync=error
RUST_BACKTRACE: 1
Expand Down Expand Up @@ -63,4 +62,4 @@ jobs:
${{ matrix.chain.bin }} stage unwind num-blocks 100 --chain ${{ matrix.chain.chain }}
- name: Run stage unwind to block hash
run: |
${{ matrix.chain.bin }} stage unwind to-block ${{ matrix.chain.unwind-target }} --chain ${{ matrix.chain.chain }}
${{ matrix.chain.bin }} stage unwind to-block ${{ matrix.chain.unwind-target }} --chain ${{ matrix.chain.chain }}
9 changes: 3 additions & 6 deletions .github/workflows/unit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ concurrency:
jobs:
test:
name: test / ${{ matrix.type }} (${{ matrix.partition }}/${{ matrix.total_partitions }})
runs-on:
group: Reth
runs-on: ubuntu-latest
env:
RUST_BACKTRACE: 1
strategy:
Expand Down Expand Up @@ -65,8 +64,7 @@ jobs:

state:
name: Ethereum state tests
runs-on:
group: Reth
runs-on: ubuntu-latest
env:
RUST_LOG: info,sync=error
RUST_BACKTRACE: 1
Expand Down Expand Up @@ -100,8 +98,7 @@ jobs:

doc:
name: doc tests
runs-on:
group: Reth
runs-on: ubuntu-latest
env:
RUST_BACKTRACE: 1
timeout-minutes: 30
Expand Down
Loading
Loading