Conversation
Before loading a kind, the generator filtered all the tasks loaded so far to find the ones belonging to the kind's dependencies (copying them first when loading kinds in parallel). This is proportional to the number of kinds times the number of tasks, and in parallel mode it happens on the main thread, delaying the submission of newly unblocked kinds. Now loaded tasks are also grouped by kind, so the tasks of each kind dependency are looked up directly. They are still passed in the order in which kinds were loaded. With 150 kinds of 270 tasks each (40,500 tasks), each depending on three other kinds, gathering the kind dependency tasks takes 0.01s in total instead of 0.42s.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before loading a kind, the generator filtered all the tasks loaded so far to find the ones belonging to the kind's dependencies (copying them first when loading kinds in parallel). This is proportional to the number of kinds times the number of tasks, and in parallel mode it happens on the main thread, delaying the submission of newly unblocked kinds.
Now loaded tasks are also grouped by kind, so the tasks of each kind dependency are looked up directly. They are still passed in the order in which kinds were loaded.
With 150 kinds of 270 tasks each (40,500 tasks), each depending on three other kinds, gathering the kind dependency tasks takes 0.01s in total instead of 0.42s.