Repository navigation
Conversation
`SubgraphInstance::new_host` computed keccak256 over the full module bytes for every data source before looking up `module_cache`. Data sources created from the same template share one `Arc` of module bytes, so on a subgraph with millions of dynamic data sources this rehashed the same bytes millions of times at every runner start. Memoize the hash by the identity of the `Arc` holding the bytes, keeping a clone of the `Arc` so its address cannot be reused while the entry exists. The `module_cache` key is still the keccak256 of the module contents.
1 of 3 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6731.
SubgraphInstance::new_hostcomputed keccak256 over the full module bytes forevery data source before looking up
module_cache. Data sources created fromthe same template share one
Arcof module bytes, so on a subgraph withmillions of dynamic data sources this rehashed the same bytes millions of times
at every runner start. On the subgraph from #6731 (Uniswap v3 on Base,
~1.77M data sources created from one template, 66 KB module), startup goes
from 238 s to 11 s (measurements below).
What changes
core/src/subgraph/context/instance/module_cache.rswith a smallModuleCache<R>that replaces theHashMap<[u8; 32], Sender<T::Req>>fieldof
SubgraphInstance:senders: HashMap<[u8; 32], Sender<R>>— unchanged semantics: keccak256of the module contents → channel to the mapping thread;
hashes: HashMap<usize, (Arc<Vec<u8>>, [u8; 32])>— memo of the hash,keyed by
Arc::as_ptrof the module bytes;sender(&module_bytes, spawn)hashes (memoized), returns the cached senderor calls
spawnand caches its sender.new_hostcallsself.module_cache.sender(&module_bytes, |bytes| T::spawn_mapping(…))instead of hashing inline. Everything else in
new_hostis unchanged.Why this is safe
only avoids recomputing a hash; it never decides which module a data source
gets. Equal bytes in a different
Arcare hashed again and map to the samesender, as before.
Arc, so theallocation cannot be freed and its address cannot be reused for other bytes
while the entry exists. A hit also checks
Arc::ptr_eqagainst the storedArc.Arc<Vec<u8>>and never mutated inplace (
Arc::make_mutwould allocate a new buffer, hence a new address).ModuleCacheis a plain field used through&mut self, like the map it replaces.spawn_mapping; a failed spawn caches nothing and can be retried; a datasource without a runtime is still skipped; a rebuilt
SubgraphInstancestarts with an empty cache.
Arcclone)per distinct module
Arc. Templates and manifest data sources share theirmapping's
Arc, so this is bounded by the number of mappings in themanifest; the retained modules are the ones the senders already keep alive.
Tests
Unit tests in
module_cache.rs(no database needed):two_templates_two_hashes— distinct modules get distinct, correct keccak256hashes and distinct senders;
sources_of_one_template_share_hash_and_sender— clones of oneArcarehashed once and share the sender;
many_sources_hash_once_per_template— 100,000 lookups over two templatescompute exactly two hashes and spawn two modules;
equal_unshared_bytes_give_same_hash_and_module— equal bytes in anotherArcare rehashed and reuse the existing module;dropped_bytes_are_not_confused_with_new_ones— 64 short-livedArcs allhash to their correct keccak256.
Before / after
Same subgraph (1,770,112 data sources), same machine, database and build
settings, one startup at a time, measured from the start of the runner
build to the first block processed. The two builds differ only by this change.
Data source count at start)Start processing block)Measured with this exact patch applied on a v0.45.0-based build; the code
path is unchanged on master.
After the change, startup on this subgraph is dominated by creating the hosts
(~6 s) and reading the data sources from the store (~3.5 s).
Checks run locally on this branch:
cargo fmt --all -- --check,cargo clippy --all-targetswith-D warnings(no warnings),cargo check --release,cargo test -p graph-core module_cache(5/5), the workspace unit tests (all pass except twogndformatter/codegen tests that fail identically on master in my environment) and the runner tests (16/16).