47 lines
2.6 KiB
Markdown
47 lines
2.6 KiB
Markdown
|
# nixpkgs lib profiling
|
||
|
|
|
||
|
|
This is a repository for profiling nixpkgs lib functions.
|
||
|
|
The goal is to create alternative implementations of functions that might have
|
||
|
|
different time/space complexity characteristics, and measure if they are faster
|
||
|
|
despite potential overhead.
|
||
|
|
|
||
|
|
I am personally especially interested in functions that rely on some type of `fold`
|
||
|
|
and accumulates a collection (either a list or an attrset). Because the lists and attrsets
|
||
|
|
will be reallocated on every iteration (this is only a suspicion, I have not verified this),
|
||
|
|
the time complexity of these functions might be O(n^2) while they really could've been O(n).
|
||
|
|
For functions like `mergeAttrsList`, this has been somewhat mitigated by folding with a binary
|
||
|
|
merge function, reducing the time complexity to O(n log n). But I'm hoping to find a more general
|
||
|
|
solution that could be applied to other functions as well, without too much overhead.
|
||
|
|
|
||
|
|
## Some interesting builtins
|
||
|
|
|
||
|
|
| Name | Expected complexity | Comment |
|
||
|
|
|------------------|---------------------|---------|
|
||
|
|
| `hasAttr` | O(1) | this is the only real kind of lookup table we have with low overhead. I need to verify that it is actually amortized O(1) |
|
||
|
|
| `catAttrs` | O(n) | could this fix the O(n^2) fold collection resizing problem? |
|
||
|
|
| `concatMap` | O(n * f) | could this be used with attr lookups to fix the O(n^2) fold collection resizing problem? |
|
||
|
|
| `concatLists` | O(n) | could this be used with attr lookups to fix the O(n^2) fold collection resizing problem? |
|
||
|
|
| `filter` | O(n * f) | could potentially remove many items at once without realloc? |
|
||
|
|
| `genList` | O(n * f) | can seemingly allocate an arbitrarily sized list? |
|
||
|
|
| `intersectAttrs` | O(n log m) (known) | could be used as a shorthand for binary merge? |
|
||
|
|
| `listToAttrs` | O(n) | could potentially remove many items at once without realloc? |
|
||
|
|
| `partition` | O(n * f) | could potentially remove many items at once without realloc? |
|
||
|
|
| `genericClosure` | O(n * f) | We might be able to use this functions uniqueness checking for something, and then use `catAttrs` on the result |
|
||
|
|
|
||
|
|
## evaluation speed measurement
|
||
|
|
|
||
|
|
For now, I'm using this:
|
||
|
|
|
||
|
|
```
|
||
|
|
# Alternatively drop NIX_COUNT_CALLS if too verbose
|
||
|
|
NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix eval .#sometest
|
||
|
|
```
|
||
|
|
|
||
|
|
Could alternatively create something with
|
||
|
|
|
||
|
|
```
|
||
|
|
nix eval --trace-function-calls .#sometest 2>function-trace.txt
|
||
|
|
```
|
||
|
|
|
||
|
|
source: https://discourse.nixos.org/t/nix-flamegraph-or-profiling-tool/33333
|