diff --git a/doc/rfc/index.md b/doc/rfc/index.md index 37ff46ec5..751ffa09b 100644 --- a/doc/rfc/index.md +++ b/doc/rfc/index.md @@ -15,6 +15,7 @@ Design documents and technical proposals, grouped by scope. Shared/cross-cutting ## SubmitQueue +- [Tango Conflict Analysis](submitqueue/tango-conflict-analysis.md) - How four per-batch target-signature designs for a Tango-backed conflict analyzer scale with in-flight batches, and whether cached signatures miss conflicts as the base advances - [Orchestrator Workflow](submitqueue/workflow.md) - Queue-driven controller pipeline from gateway entry through batching, scoring, build, land, and conclude - [Gateway History APIs](submitqueue/history-api.md) - Request lifecycle history exposed through separate request ID and change ID endpoints - [Build Runner](submitqueue/build-runner.md) - Vendor-agnostic BuildRunner interface, provider-neutral BuildStatus lifecycle, and how the orchestrator wires it into the build stage diff --git a/doc/rfc/submitqueue/tango-conflict-analysis.md b/doc/rfc/submitqueue/tango-conflict-analysis.md new file mode 100644 index 000000000..ab9cd4c12 --- /dev/null +++ b/doc/rfc/submitqueue/tango-conflict-analysis.md @@ -0,0 +1,273 @@ +# Tango-backed conflict analysis for SubmitQueue + +**Status:** research, not an implementation. **Updated:** October 1, 2026. **SubmitQueue base:** `origin/main@6c4b769c`. **Tango inspected:** `uber/tango@50b3695a9909f19b78a3b1b35c095c9a8331d9db`. **go-code measured:** `c5655f3f7e87f`. + +## Question + +A Tango-backed `conflict.Analyzer` stores one target signature per batch and compares a new batch against every in-flight batch. This research asks which signature representation still works as the number of in-flight batches `B` grows, measured at the **worst case**: a controller instance with nothing cached (after a restart, deploy or partition rebalance) must fetch and decode all `B` signatures for one admission. A process-local cache of the immutable signatures is possible even in a stateless controller and would remove most of this cost in steady state; it is an optimization of any design below, not a design of its own, and is deliberately excluded. Today go-code runs fewer than 100 in-flight batches; the goal is to choose a design for growth, and to find where each design stops working. + +## Vocabulary + +- **`B`**: number of in-flight batches the new batch is compared against. +- **`T`**: total affected targets across those `B` batches. +- **Cold admission**: one dependency-analysis call on a controller with nothing cached, which fetches and decodes all `B` signatures, then checks the candidate against them. +- **Signature**: the immutable per-`(batch, base)` record a design stores to represent a batch's affected targets. +- **TangoSnapshot** (baseline): the signature is the Tango changed-targets response for the batch's affected targets, including their direct dependencies and the response's ID→name metadata, stored as raw or gzip protobuf. +- **NameKey**: the signature is the sorted list of the batch's affected target labels. +- **ID64**: the signature is the sorted list of 64-bit hashes of those labels; a shared ID→label dictionary, stored separately, recovers names. + +## Findings + +Costs per affected target, fitted from the measured cases (B = 100 to 10,000 on the full go-code graph): + +| Design | Stored (bytes/target) | Decoded heap (bytes/target) | Serial decode (µs/target) | +| --- | ---: | ---: | ---: | +| TangoSnapshot raw (gzip) | 535 (132) | 880 | 1.5 (4.1) | +| NameKey | 101 | 123 | 0.08 | +| ID64 | 8 | 8 | 0.005 | + +- **TangoSnapshot as the conflict signature breaks on a single tail request, at any B.** The largest observed request (2.88M targets) is about 1.4 GiB raw and 2.4 GiB of heap by itself. At the 99th-percentile draw, 10 in-flight batches already exceed a 1 s cold load, and about 55 exceed 1 GiB of heap. At the median draw it holds below 1 s only up to about B ≈ 80–130, below 10 s up to about 570–890, and below 1 GiB of heap up to about 630. It is viable for today's go-code load (fewer than 100 in flight) only if a multi-second, multi-GiB cold start is acceptable whenever a tail request is in flight. +- **NameKey** is about 5× smaller than raw TangoSnapshot in stored bytes (1.3× smaller than gzip) and 7× smaller in heap. It holds 1 GiB of heap to about B ≈ 3,700 (median) or 1,100 (p99), and 10 s of cold load to about 1,450 (object store) or 3,700 (KV point reads). +- **ID64** is a further 12–15× smaller. Its heap stays below 1 GiB until about B ≈ 50,000. Its cold-load limit comes from round trips, not bytes: under the object-store profile, `ceil(B/32) × 150 ms` alone exceeds 1 s at B ≈ 200 and 10 s at B ≈ 2,100. Under KV point reads it holds 1 s to about B ≈ 2,100 (median) or 1,000 (p99), and 10 s to about 20,000. +- **The conflict rule needs the new-edge rule, and caching signatures across base advances is a trade-off.** Signatures computed once against the newest base miss conflicts unless they also inherit from structural dependencies; see [Conflict rule and base drift](#conflict-rule-and-base-drift). +- **The in-memory check is never the bottleneck.** At B = 10,000 it takes 32 ms (ID64) to 292 ms (TangoSnapshot) of CPU, against seconds to minutes of loading. + +## What exists today + +- SubmitQueue calls `conflict.Analyzer.Analyze(ctx, batch, inFlight)` from the dependency-analysis stage, whose messages are partitioned by queue and consumed in order ([`dependencyanalysis.go`](../../../submitqueue/orchestrator/controller/dependencyanalysis/dependencyanalysis.go)). Before the call, [`ListByStates`](../../../submitqueue/orchestrator/core/batch/list.go) hydrates every in-flight `Batch` by key at concurrency 16; the speculation stage repeats that listing ([`run.go`](../../../submitqueue/orchestrator/controller/speculate/run.go)). Signatures are not loaded today, so every design below adds its own reads on top of this. +- Tango's `GetChangedTargets` returns `NEW`/`DELETED`/`CHANGED` targets with old/new detail and the distance from a directly changed seed; `GetChangedTargetGraph` is unimplemented. Response IDs are local to one response and the ID→name metadata arrives last, so signatures must be built from resolved labels. Request `OutputConfig{MaxDistance: -1}` explicitly: an unset distance means `0`, not unlimited. See the [wire contract](https://github.com/uber/tango/blob/50b3695a9909f19b78a3b1b35c095c9a8331d9db/proto/tango.proto) and [output filtering](https://github.com/uber/tango/blob/50b3695a9909f19b78a3b1b35c095c9a8331d9db/controller/output_filter.go). +- Tango's compared-targets cache key omits the computation strategy and `SeedAttributes` policy ([cache keys](https://github.com/uber/tango/blob/50b3695a9909f19b78a3b1b35c095c9a8331d9db/core/cachekey/cachekey.go)). Fix or isolate it before treating Tango as a conflict oracle. +- Tango applies GitHub PR URIs only ([request parser](https://github.com/uber/tango/blob/50b3695a9909f19b78a3b1b35c095c9a8331d9db/core/workspace/request.go)); `phab://` queues need an adapter or the conservative `all` analyzer. + +## Prior art: the production SubmitQueue + +The legacy production SubmitQueue (`java-code`, `dev-platform/cicd/submitqueue/`) does not call Tango. Its Java `TargetAnalyzer` Thrift service diffs name→hash target maps plus direct edges between the speculation base and base plus the requests. `ChangesRetrieverImpl.processTargetStats` turns that diff into the `targetsChanged`/`targetsAdded`/`targetsRemoved` counts behind the Hive table used below. Its `ConflictAnalyzerImpl` is pairwise on one common base, and `FastTargetComparer` flags a conflict when any of three pairs intersects: + +- A's changed targets with B's changed targets; +- the destinations of A's **new edges** with B's changed targets; +- the destinations of B's new edges with A's changed targets. + +Removed edges are ignored. This replaced the "union graph" algorithm of [Keeping Master Green at Scale](https://dl.acm.org/doi/10.1145/3302424.3303970) (EuroSys 2019), which its Javadoc says needed too much memory and failed on cycles in the union of three graphs. A code comment there puts a target-analyzer RPC at about 3 minutes on average. Two things were not confirmed from source: whether that service's target hashes are transitive, and which entry point production calls. The new-edge rule is exactly the gap the next section finds in a plain affected-set overlap. + +## Conflict rule and base drift + +**Proposed premise:** compute each batch's changed targets once against the newest base at admission, cache the result, and never recompute it. Because every base advance is a landed batch that the queue itself analyzed, this should introduce no extra missed conflicts. **The premise is false as stated.** The [simulation](../../../tool/tangograph-eval/basedrift.go) tests it on 200,000 random small DAGs, each with up to three batches landing between admissions. Ground truth for a pair of batches allowed to run independently is a target affected by both in the first graph that contains both changes. Signatures follow Tango: the reverse closure, in the changed graph, of edited, rewired and new targets. + +| Comparison signature | Missed / independent pairs, same base (pairs) | Missed / independent pairs, across a base advance (pairs) | False conflicts (pairs) | Mean signature size (targets) | +| --- | ---: | ---: | ---: | ---: | +| `affected` | 28,529 / 189,645 (15.04%) | 15,862 / 102,737 (15.44%) | 4,868 | 6.53 | +| `affected+added-deps` | 1,451 / 152,898 (0.95%) | 1,416 / 86,644 (1.63%) | 9,854 | 7.35 | +| `affected+added-deps+inherit-structural` | 0 / 135,393 (0.00%) | 0 / 62,040 (0.00%) | 49,319 | 11.09 | +| `affected+added-deps+inherit-all` | 0 / 131,053 (0.00%) | 0 / 56,248 (0.00%) | 59,608 | 12.43 | + +Source: [simulation result](tango-eval/base-drift-simulation-20261001.json), 200,000 trials on 40-node random DAGs. + +Two mechanisms produce the misses: + +1. **A new dependency edge is invisible to the other change.** A makes `X` depend on `Y` and B edits `Y`. A's signature has `X` and its dependents; B's signature, computed on a graph where `X` does not yet depend on `Y`, does not. The fix is the production rule above: add each change's new-edge destinations (and the dependencies of its new targets) to its signature. +2. **A cached signature goes stale when a structural batch it depends on lands.** L makes `Z` depend on `X`, A edits `X`, and, with rule 1, A correctly depends on L. L lands. C now edits `Z` on the new base, where `Z` depends on `X`, but A's cached signature was computed before that edge existed and has no `Z`. C runs independently of A, and `Z` with both changes is never built. The same staleness also lets two batches that both depend on one structural batch miss each other on a shared base. + +Inheritance closes both cases in every trial: a batch's comparison signature becomes its own signature united with the comparison signatures of the dependencies that add edges or targets, recorded once at admission. Any dependency path a landed structural batch created runs through one of its new edges. That edge's destination is in the batch's signature and its dependents are in its affected set, so inheriting the signature covers every target the path reaches. The cost is more false conflicts (about 5× in the toy model) and larger signatures (+51% mean), both growing along dependency chains of structural batches. + +**Trade-off:** either accept a possible conflict undercount, which rule 1 alone shrinks from about 15% to about 1–2% of truly conflicting independent pairs in the toy model, or pay the false conflicts of inheritance. Recomputing signatures after every structural land is the third option, but it costs one Tango call per in-flight batch per structural land. The toy rates show only that misses exist; they do not estimate production rates. Commits that bypass the queue (direct pushes, reverts, bots) break the "landed batches only" premise under every rule. + +## Workload: the real distribution of changed targets + +The [summary query](tango-eval/hive-go-diff-targets-20260930.sql) and the [histogram query](tango-eval/hive-go-diff-target-histogram-20260930.sql) read `rawdata_user.kafka_hp_submitqueue_request_feature_event_nodedup` (`TIER_THREE`) for the `go` queue, September 1–29, 2026. They keep single-diff requests (`stackheight = 1`) and the latest event per diff, and use `targetschanged + targetsadded + targetsremoved` as the affected-target proxy. The checked-in [histogram](tango-eval/hive-go-diff-target-histogram-20260930.csv) holds one row per distinct count (4,953 rows) and reproduces the summary query's 58,876 diffs and 148,766,796 targets exactly. It contains no diff IDs. + +| Statistic | Affected targets per request | +| --- | ---: | +| Requests | 58,876 | +| Zero targets | 5,942 (10.1%) | +| Median / p90 / p95 | 42 / 1,026 / 3,274 | +| p99 / p99.9 / max | 32,314 / 410,642 / 2,884,044 | +| Mean | 2,526.8 | +| Requests with ≥100,000 targets | 233 (0.40%), carrying 66.1% of all targets | + +**One batch is one request** throughout; multi-request batches are not modeled. Each batch's target count is drawn from this histogram, and batch `i` always draws the same count, so a smaller `B` is a prefix of a larger one. Totals over many batches converge on `B × 2,527`, but at small `B` they are dominated by whether a tail request is in flight. Results are therefore reported at the 50th and 99th percentile of that total. Each drawn count is turned into a **closure-shaped** set on the complete go-code graph: the reverse-dependency closure of a random source file, widened with the closures of dependencies of targets already in the set until the count is reached. Labels, dependencies and set shape come from the real graph; which targets a real diff touches does not. The candidate batch is fixed at the mean, 2,527 targets. + +## Design details + +All three store an immutable signature per `(batch, base)` and differ in the data structure. + +1. **TangoSnapshot (baseline): the Tango response itself.** Each batch stores a default-field `GetChangedTargets`-like response for its affected targets: optimized targets with their real direct dependencies plus the response-local ID→name and rule-type metadata, as raw protobuf or per-message gzip. This is the option to beat. The response is fetched anyway, and its dependency metadata is what conflict relaxation in the speculator could use, so if it is small enough, one artifact serves both. It is modeled, not captured: a real response also carries old/new detail and may be larger. +2. **NameKey: sorted canonical labels.** One blob of varint-length-prefixed UTF-8 labels per batch. Names are directly available to relaxation. +3. **ID64: sorted 64-bit target IDs plus an ID→name dictionary.** One blob of 8-byte IDs per batch. Conflict detection does not need the dictionary: a 64-bit collision only adds a false conflict, which is safe, and for 3 million labels the chance of any collision is about `n²/2⁶⁵ ≈ 2×10⁻⁷`. Relaxation may need names, so the dictionary stays in the design: an append-only store keyed by ID whose row can hold every label hashing to that ID. IDs are then a pure function of the label, with no read-before-write registration. The dictionary must be written before the batch is announced to speculation, but the conflict check never reads it. + +## Measured results + +The [benchmark](../../../tool/tangograph-eval/loadbench.go) reads the complete go-code Bazel graph (2,969,283 targets), builds the workload above, and for each `B` encodes and then actually decodes every signature in Go. It measures live heap after GC and times one candidate check against all `B` decoded batches. Raw numbers are in the [result JSON](tango-eval/go-code-load-histogram-20261001.json); the run took 13 min 48 s wall time and peaked at 35.2 GB (35,166,136 KiB) RSS for all cases together ([timing log](tango-eval/go-code-load-histogram-20261001-timing.txt)). + +Measured cases use one fixed draw per `B` (the 'Total targets' column), not the median draw, so small-`B` rows reflect whichever tail requests that draw contains. Use the scale model below for percentiles. + +### 1. Serialized size of all in-flight signatures + +Application payload in MiB. The ID64 dictionary is the exact minimal size for the labels in all `B` batches and the candidate: an 8-byte ID, a varint length and the label per distinct target. It is stored, not fetched. Store overhead (keys, row headers, checksums) is excluded. + +| `B` (batches) | Total targets (targets) | Largest batch (targets) | TangoSnapshot raw (MiB) | TangoSnapshot gzip (MiB) | NameKey (MiB) | ID64 signatures (MiB) | ID64 dictionary (MiB) | ID64 total (MiB) | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | 177,555 | 149,002 | 105.0 | 25.8 | 17.5 | 1.4 | 16.8 | 18.1 | +| 250 | 1,128,207 | 831,151 | 422.3 | 102.9 | 108.0 | 8.6 | 85.5 | 94.1 | +| 500 | 1,723,599 | 831,151 | 730.4 | 177.8 | 165.4 | 13.2 | 87.4 | 100.6 | +| 1,000 | 2,627,040 | 831,151 | 1,272.9 | 311.2 | 251.5 | 20.0 | 87.8 | 107.9 | +| 2,500 | 5,192,014 | 831,151 | 2,774.0 | 675.3 | 497.2 | 39.6 | 88.4 | 128.0 | +| 5,000 | 8,806,334 | 831,151 | 4,875.6 | 1,179.4 | 845.9 | 67.2 | 90.5 | 157.7 | +| 10,000 | 21,208,724 | 2,856,456 | 10,686.5 | 2,659.0 | 2,037.7 | 161.8 | 292.3 | 454.1 | + +**Interpretation.** Stored size scales with total targets, not with `B`: about 535 bytes per target for raw TangoSnapshot, 132 gzipped, 101 for NameKey and 8 for ID64. At `B = 10,000` that is about 10.4 GiB, 2.6 GiB, 2.0 GiB and 162 MiB. A few tail requests dominate every total: in the `B = 100` row a single 149,002-target batch is 84% of all targets. The ID64 dictionary grows with *distinct* labels, not batches, so it plateaus near 90 MiB from `B = 250` to `5,000`. It jumps at `B = 10,000` when a 2.86M-target request registers most of the graph. It is bounded by the whole graph's labels, about 300 MiB, so ID64's total stays the smallest of the three. + +### 2. Go heap with all signatures decoded + +Live heap after GC, in MiB, for `B` decoded signatures plus the candidate. The graph, workload and serialized buffers are excluded. + +| `B` (batches) | Total targets (targets) | Largest batch (targets) | TangoSnapshot (MiB) | NameKey (MiB) | ID64 (MiB) | +| ---: | ---: | ---: | ---: | ---: | ---: | +| 100 | 177,555 | 149,002 | 166.9 | 21.6 | 1.4 | +| 250 | 1,128,207 | 831,151 | 732.2 | 132.4 | 8.7 | +| 500 | 1,723,599 | 831,151 | 1,227.9 | 202.7 | 13.3 | +| 1,000 | 2,627,040 | 831,151 | 2,092.5 | 308.2 | 20.3 | +| 2,500 | 5,192,014 | 831,151 | 4,499.6 | 609.3 | 40.3 | +| 5,000 | 8,806,334 | 831,151 | 7,884.8 | 1,036.1 | 68.6 | +| 10,000 | 21,208,724 | 2,856,456 | 17,677.6 | 2,494.6 | 164.6 | + +**Interpretation.** Decoded heap is what a cold controller must hold at once. TangoSnapshot needs about 880 bytes per target, 1.6× its raw bytes, because response-local ID→name maps and dependency lists are materialized. That is 2 GiB at `B = 1,000` and 17 GiB at `B = 10,000`. NameKey's 123 bytes per target (string headers plus label bytes) reaches 2.4 GiB at `B = 10,000`. ID64 stays at about 8 bytes per target: 165 MiB at `B = 10,000`. A single tail request (2.88M targets) costs about 2.4 GiB as TangoSnapshot, 340 MiB as NameKey and 22 MiB as ID64. + +### 3. Fetch and decode latency + +Time for one cold admission, in seconds: `ceil(B/P) × first byte + bytes / bandwidth + measured serial decode CPU`. These are planning assumptions, not measurements of a chosen backend. **Object store** means 150 ms to first byte, `P = 32` parallel GETs and 100 MiB/s per controller (S3 Standard-like, after [AWS guidance](https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html)). **KV point read** means 5 ms, `P = 16` and 100 MiB/s, which suits signatures kept in a key-value or SQL store. Decode is measured on one goroutine; spreading it over cores would shorten it at the cost of concurrent heap. + +| `B` (batches) | Store profile | TangoSnapshot raw (s) | TangoSnapshot gzip (s) | NameKey (s) | ID64 (s) | +| ---: | --- | ---: | ---: | ---: | ---: | +| 100 | object-store | 1.93 | 1.79 | 0.79 | 0.61 | +| 100 | kv-point-read | 1.37 | 1.22 | 0.23 | 0.05 | +| 250 | object-store | 6.97 | 5.70 | 2.37 | 1.29 | +| 250 | kv-point-read | 5.85 | 4.58 | 1.25 | 0.17 | +| 500 | object-store | 12.24 | 10.22 | 4.17 | 2.54 | +| 500 | kv-point-read | 10.00 | 7.98 | 1.93 | 0.30 | +| 1,000 | object-store | 21.52 | 18.07 | 7.50 | 5.01 | +| 1,000 | kv-point-read | 17.04 | 13.59 | 3.01 | 0.53 | +| 2,500 | object-store | 47.71 | 40.70 | 17.17 | 12.28 | +| 2,500 | kv-point-read | 36.65 | 29.63 | 6.11 | 1.21 | +| 5,000 | object-store | 86.02 | 72.80 | 32.70 | 24.25 | +| 5,000 | kv-point-read | 64.04 | 50.82 | 10.72 | 2.26 | +| 10,000 | object-store | 186.73 | 161.50 | 69.21 | 48.69 | +| 10,000 | kv-point-read | 142.90 | 117.67 | 25.38 | 4.87 | + +**Interpretation.** +- **TangoSnapshot and NameKey** are bound by bytes (transfer plus decode). Gzip makes TangoSnapshot slightly faster, despite costlier decompression, because at 100 MiB/s transfer outweighs decode. At `B = 1,000` TangoSnapshot takes 14–22 s and NameKey 3–7.5 s. +- **ID64** is bound by round trips. Behind the object store, `ceil(B/32) × 150 ms` is 47 of its 49 s at `B = 10,000`. Behind 5 ms point reads, the same signatures load in 4.9 s at `B = 10,000` and 0.5 s at `B = 1,000`. For ID64 the store's per-request latency and parallelism matter far more than the encoding. + +### 4. Checking one candidate + +Local CPU time for one candidate, in milliseconds, against all `B` decoded batches, stopping at the first shared target in each batch. "Matches" is the number of in-flight batches the candidate would depend on. + +| `B` (batches) | Matches (batches) | TangoSnapshot (ms) | NameKey (ms) | ID64 (ms) | +| ---: | ---: | ---: | ---: | ---: | +| 100 | 31 | 0.70 | 1.31 | 0.32 | +| 250 | 72 | 5.44 | 6.95 | 0.64 | +| 500 | 146 | 17.59 | 8.19 | 1.66 | +| 1,000 | 312 | 34.70 | 15.19 | 3.04 | +| 2,500 | 814 | 74.59 | 40.08 | 8.54 | +| 5,000 | 1,709 | 122.61 | 73.69 | 23.64 | +| 10,000 | 3,424 | 291.98 | 151.73 | 31.98 | + +**Interpretation.** The check is never the bottleneck: at `B = 10,000` it is 32 ms (ID64) to 292 ms (TangoSnapshot), against 5–187 s of loading. ID64 is fastest because it compares fixed-width integers, while TangoSnapshot pays a map lookup per target. About a third of in-flight batches overlap the mean-sized candidate. That rate reflects the closure-shaped synthetic sets, not observed diffs, so the dependency list itself (`Ω(matches)`) can be as costly as the check. + +## Where each design stops working + +The [scale model](../../../tool/tangograph-eval/scale.go) fits each measured cost as a per-target ratio, `Σ cost / Σ T` over the measured cases (`T` is total targets across the batches). It does not use a two-term `a·B + b·T` fit: `B` and `T` rise together across the cases, so a per-batch term cannot be identified. Round trips are still charged per batch in the latency formula. It then draws `T` by Monte Carlo from the histogram, 400 trials per `B`. Below is the first `B` at which a budget is exceeded for the median draw and for the 99th-percentile draw. The budgets are illustrative and should be replaced by the admission SLO and controller memory limit once those are set. "—" means not exceeded up to `B = 100,000`. Beyond the largest measured `B` (10,000) these are extrapolations. + +| Budget | TangoSnapshot raw (first `B`, median / p99) | TangoSnapshot gzip (first `B`, median / p99) | NameKey (first `B`, median / p99) | ID64 (first `B`, median / p99) | +| --- | ---: | ---: | ---: | ---: | +| Cold load > 1 s (object-store) | 81 / ≤10 | 94 / ≤10 | 161 / 34 | 196 / 133 | +| Cold load > 10 s (object-store) | 574 / 55 | 664 / 55 | 1,450 / 633 | 2,142 / 1,943 | +| Cold load > 60 s (object-store) | 3,014 / 1,030 | 3,489 / 1,381 | 8,397 / 6,908 | 12,406 / 12,406 | +| Cold load > 1 s (kv-point-read) | 120 / ≤10 | 133 / ≤10 | 450 / 55 | 2,142 / 1,030 | +| Cold load > 10 s (kv-point-read) | 732 / 55 | 890 / 55 | 3,664 / 1,136 | 20,208 / 17,457 | +| Cold load > 60 s (kv-point-read) | 3,664 / 1,136 | 4,676 / 1,678 | 21,219 / 15,834 | — / — | +| Decoded heap > 1 GiB | 633 / 55 | 633 / 55 | 3,664 / 1,136 | 53,619 / 42,012 | +| Decoded heap > 4 GiB | 2,249 / 450 | 2,249 / 450 | 14,362 / 9,721 | — / — | +| Decoded heap > 16 GiB | 8,397 / 4,453 | 8,397 / 4,453 | 56,300 / 46,318 | — / — | + +**Interpretation.** At the 99th-percentile draw, TangoSnapshot exceeds 1 s with 10 batches and 1 GiB with about 55: one tail request in flight is enough. NameKey stays within 10 s and 1 GiB into the low thousands. ID64's heap is not a constraint below about 50,000 batches. Its load limit is set by the store: about 200 batches for 1 s behind an object store, about 2,000 behind KV point reads. + +Representative modeled points (median draw / 99th-percentile draw): + +| `B` (batches) | Design | Decoded heap (MiB) | Cold load, object store (s) | Cold load, KV point read (s) | +| ---: | --- | ---: | ---: | ---: | +| 1,000 | TangoSnapshot raw | 1,683 / 6,171 | 18.1 / 53.7 | 13.7 / 49.2 | +| 1,000 | NameKey | 236 / 865 | 6.9 / 12.5 | 2.4 / 8.0 | +| 1,000 | ID64 | 16 / 57 | 5.0 / 5.4 | 0.5 / 0.9 | +| 5,000 | TangoSnapshot raw | 10,403 / 18,929 | 106.0 / 173.6 | 84.0 / 151.6 | +| 5,000 | NameKey | 1,458 / 2,653 | 36.5 / 47.1 | 14.5 / 25.1 | +| 5,000 | ID64 | 96 / 175 | 24.6 / 25.4 | 2.6 / 3.4 | +| 10,000 | TangoSnapshot raw | 20,681 / 34,992 | 210.9 / 324.4 | 167.1 / 280.5 | +| 10,000 | NameKey | 2,899 / 4,905 | 72.6 / 90.4 | 28.8 / 46.6 | +| 10,000 | ID64 | 191 / 324 | 49.0 / 50.3 | 5.1 / 6.5 | +| 20,000 | TangoSnapshot raw | 42,097 / 57,413 | 427.5 / 548.9 | 340.0 / 461.4 | +| 20,000 | NameKey | 5,900 / 8,047 | 146.0 / 165.0 | 58.5 / 77.5 | +| 20,000 | ID64 | 389 / 531 | 97.8 / 99.3 | 10.3 / 11.8 | +| 50,000 | TangoSnapshot raw | 106,413 / 131,715 | 1,078.1 / 1,278.6 | 859.2 / 1,059.8 | +| 50,000 | NameKey | 14,915 / 18,461 | 366.6 / 398.0 | 147.7 / 179.2 | +| 50,000 | ID64 | 985 / 1,219 | 244.8 / 247.2 | 25.9 / 28.4 | + +## Recommendation + +Use **ID64 signatures stored as one record per `(queue, batch)` in a low-latency key-value or SQL store**, not an object store. The latency profile matters more than the encoding: the same 8-byte IDs hit a 1 s cold load at B ≈ 200 behind 150 ms object GETs, but at B ≈ 2,000 behind 5 ms point reads. + +Keep the ID→name dictionary so that conflict relaxation can recover labels. For relaxation that needs Tango's dependency metadata, store the TangoSnapshot response as a separate per-batch record and read it **only for the batches relaxation actually examines**, typically the matched pairs. It is then never loaded for all `B` batches on the conflict-check path. This keeps the baseline's metadata available while moving its size off the path that scales with `B`. + +Add the warm-controller cache as an optimization once the store is chosen; it does not change the cold-start limits above. Beyond about B ≈ 20,000 (KV, 10 s) even ID64 cold loads become long. That is the point at which to revisit the persistent target→batch posting index, which reads postings for the candidate's targets instead of every in-flight signature. + +Implementation shape: + +1. Add `submitqueue/orchestrator/extension/conflict/tango/` implementing the shared `conflict.Analyzer`, with a Tango streaming client, a `changeset.Resolver`, a base-revision resolver and a key-oriented signature store injected at construction, and per-queue routing in `service/submitqueue/orchestrator/server/`. +2. Make one `GetChangedTargets` call per new batch against the newest base, with `MaxDistance: -1` and hashes, tags and attributes omitted. Treat a partial stream, unknown ID or cancellation as a failed analysis, never as an empty signature. +3. Form the signature from affected targets **plus new-edge destinations and dependencies of new targets**. Either record the comparison signature with inheritance from structural dependencies or accept the documented undercount; this is a policy decision. +4. Persist the signature (and, for ID64, the dictionary rows) before the `Creating → Created` promotion and before announcing to speculation. Keep each write key-local and idempotent on redelivery. +5. Shadow it against `all` and `pathoverlap`, recording per-batch `K`, in-flight `B`, cold-start frequency, fetch/decode latency, Tango latency, conflict pairs, and how often a land is structural. + +## Full-graph footprint + +For scale: the complete `//external:all-targets + deps(//...:all-targets)` query at `c5655f3f7e87f` produced the [measurement](tango-eval/go-code-full-20260928.json) below. No design here stores the full graph per batch. At 378.50 MiB, `B = 100` copies would already be 37 GiB. + +| Property | Value | +| --- | ---: | +| Bazel `streamed_proto` input | 2,799,274,578 bytes | +| Targets (external) | 2,969,283 (430,553) | +| Dependency edges | 14,519,552 | +| Modeled default Tango graph protobuf (gzip) | 378.50 MiB (62.87 MiB) | +| Go heap, Tango-shaped ID graph | 1.099 GiB | +| Sorted 64-bit fingerprints, all targets | 22.65 MiB | + +## Reproduce + +Capture the graph from a go-code checkout; use only a complete stream. + +```sh +bazel query --order_output=no --proto:locations --noproto:default_values --output=streamed_proto \ + '//external:all-targets + deps(//...:all-targets)' > /tmp/sq-go-code-full-20260928.streamed_proto +``` + +From this repository: + +```sh +GOMEMLIMIT=300GiB go run ./tool/tangograph-eval -benchmark-load \ + -input /tmp/sq-go-code-full-20260928.streamed_proto -go-code-revision c5655f3f7e87f \ + -target-histogram doc/rfc/submitqueue/tango-eval/hive-go-diff-target-histogram-20260930.csv \ + -candidate-targets 2527 -benchmark-batches 100,250,500,1000,2500,5000,10000 > load.json +go run ./tool/tangograph-eval -simulate-base-drift 200000 > drift.json +go run ./tool/tangograph-eval -input /tmp/sq-go-code-full-20260928.streamed_proto \ + -query '//external:all-targets + deps(//...:all-targets)' -go-code-revision c5655f3f7e87f \ + -tango-revision 50b3695a9909f19b78a3b1b35c095c9a8331d9db > footprint.json +``` + +The raw Bazel output is not committed: it is 2.8 GB and contains internal labels. Its SHA-256 is `99f0cfd8e2541d381ca845dcaac01b4166cb451817ca998202932dbf3009c14f`. + +## Limitations + +- The Hive counts come from the legacy target analyzer, not Tango. They describe requests, not multi-request batches. +- Affected sets are closure-shaped but synthetic. Real match rates depend on which targets real diffs touch, which this data does not show. +- TangoSnapshot is modeled from the Bazel graph, not captured from a Tango server; old/new detail would make it larger. +- Store latencies are assumptions. No blob store, KV store or Tango RPC was benchmarked. +- The base-drift simulation runs on toy DAGs. It shows which rules can miss conflicts, not how often production would. diff --git a/doc/rfc/submitqueue/tango-eval/base-drift-simulation-20261001.json b/doc/rfc/submitqueue/tango-eval/base-drift-simulation-20261001.json new file mode 100644 index 000000000..a902ab034 --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/base-drift-simulation-20261001.json @@ -0,0 +1,67 @@ +{ + "trials": 200000, + "base_nodes": 40, + "notes": [ + "Toy random DAGs, not go-code: rates show whether a rule can miss conflicts, not how often production would.", + "Signatures follow Tango's GetChangedTargets: reverse closure, in the changed graph, of edited targets, rewired targets and new targets.", + "A pair is independent when neither batch transitively depends on the other; truth is a shared affected target in the first graph containing both changes." + ], + "rules": [ + { + "rule": { + "name": "affected", + "added_deps": false, + "inherit_dependency_signatures": false, + "inherit_structural_only": false + }, + "same_base_independent_pairs": 189645, + "same_base_missed_conflicts": 28529, + "cross_base_independent_pairs": 102737, + "cross_base_missed_conflicts": 15862, + "false_conflicts": 4868, + "mean_comparison_signature_targets": 6.530785180519154 + }, + { + "rule": { + "name": "affected+added-deps", + "added_deps": true, + "inherit_dependency_signatures": false, + "inherit_structural_only": false + }, + "same_base_independent_pairs": 152898, + "same_base_missed_conflicts": 1451, + "cross_base_independent_pairs": 86644, + "cross_base_missed_conflicts": 1416, + "false_conflicts": 9854, + "mean_comparison_signature_targets": 7.352695695211527 + }, + { + "rule": { + "name": "affected+added-deps+inherit-structural", + "added_deps": true, + "inherit_dependency_signatures": true, + "inherit_structural_only": true + }, + "same_base_independent_pairs": 135393, + "same_base_missed_conflicts": 0, + "cross_base_independent_pairs": 62040, + "cross_base_missed_conflicts": 0, + "false_conflicts": 49319, + "mean_comparison_signature_targets": 11.093508509465586 + }, + { + "rule": { + "name": "affected+added-deps+inherit-all", + "added_deps": true, + "inherit_dependency_signatures": true, + "inherit_structural_only": false + }, + "same_base_independent_pairs": 131053, + "same_base_missed_conflicts": 0, + "cross_base_independent_pairs": 56248, + "cross_base_missed_conflicts": 0, + "false_conflicts": 59608, + "mean_comparison_signature_targets": 12.430846297192007 + } + ] +} diff --git a/doc/rfc/submitqueue/tango-eval/go-code-full-20260928.json b/doc/rfc/submitqueue/tango-eval/go-code-full-20260928.json new file mode 100644 index 000000000..675a5f55b --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/go-code-full-20260928.json @@ -0,0 +1,55 @@ +{ + "input": "/tmp/sq-go-code-full-20260928.streamed_proto", + "bazel_streamed_proto_input_bytes": 2799274578, + "bazel_query": "//external:all-targets + deps(//...:all-targets)", + "go_code_revision": "c5655f3f7e87f", + "tango_revision": "50b3695a9909f19b78a3b1b35c095c9a8331d9db", + "go_version": "go1.26.3", + "max_protobuf_message_bytes": 4250000, + "sample": { + "bazel_query_targets_scanned": 2969283, + "targets": 2969283, + "rule_targets": 997235, + "source_files": 1887433, + "generated_files": 84568, + "package_groups": 47, + "external_targets": 430553, + "unique_packages_in_query": 353238, + "included_edges": 14519552, + "edges_to_targets_outside_scope": 4812, + "excluded_main_repo_edges": 160, + "excluded_external_repo_edges": 4652, + "go_heap_parsed_graph_bytes": 2465486896, + "go_heap_compact_graph_bytes": 1180570080, + "go_heap_sorted_fingerprints_bytes": 24119248, + "sorted_fingerprints_raw_bytes": 23754264, + "tango_default": { + "protobuf_bytes": 396890158, + "gzip_best_speed_bytes": 65922174, + "messages": 94, + "largest_message_bytes": 4249995 + }, + "tango_all_fields_model": { + "protobuf_bytes": 624477967, + "gzip_best_speed_bytes": 168236138, + "messages": 147, + "largest_message_bytes": 4249995 + }, + "tango_names_only_model": { + "protobuf_bytes": 335244866, + "gzip_best_speed_bytes": 44226073, + "messages": 80, + "largest_message_bytes": 4249995 + }, + "synthetic_source_change_overlap": { + "eligible_main_repo_source_files": 1533785, + "sampled_source_file_seeds": 64, + "source_seed_pairs": 2016, + "full_reverse_closure_overlap_pairs": 4, + "one_hop_overlap_pairs": 0, + "median_full_closure_targets": 2, + "p95_full_closure_targets": 225, + "max_full_closure_targets": 1615 + } + } +} diff --git a/doc/rfc/submitqueue/tango-eval/go-code-load-histogram-20261001-timing.txt b/doc/rfc/submitqueue/tango-eval/go-code-load-histogram-20261001-timing.txt new file mode 100644 index 000000000..ff4993919 --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/go-code-load-histogram-20261001-timing.txt @@ -0,0 +1,23 @@ + Command being timed: "/tmp/tangograph-eval -benchmark-load -input /tmp/sq-go-code-full-20260928.streamed_proto -go-code-revision c5655f3f7e87f -target-histogram doc/rfc/submitqueue/tango-eval/hive-go-diff-target-histogram-20260930.csv -candidate-targets 2527 -benchmark-batches 100,250,500,1000,2500,5000,10000" + User time (seconds): 1325.01 + System time (seconds): 77.42 + Percent of CPU this job got: 169% + Elapsed (wall clock) time (h:mm:ss or m:ss): 13:48.32 + Average shared text size (kbytes): 0 + Average unshared data size (kbytes): 0 + Average stack size (kbytes): 0 + Average total size (kbytes): 0 + Maximum resident set size (kbytes): 35166136 + Average resident set size (kbytes): 0 + Major (requiring I/O) page faults: 0 + Minor (reclaiming a frame) page faults: 17919234 + Voluntary context switches: 1424629 + Involuntary context switches: 41491 + Swaps: 0 + File system inputs: 0 + File system outputs: 112 + Socket messages sent: 0 + Socket messages received: 0 + Signals delivered: 0 + Page size (bytes): 4096 + Exit status: 0 diff --git a/doc/rfc/submitqueue/tango-eval/go-code-load-histogram-20261001.json b/doc/rfc/submitqueue/tango-eval/go-code-load-histogram-20261001.json new file mode 100644 index 000000000..a049358b2 --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/go-code-load-histogram-20261001.json @@ -0,0 +1,1613 @@ +{ + "input": "/tmp/sq-go-code-full-20260928.streamed_proto", + "bazel_streamed_proto_input_bytes": 2799274578, + "go_code_revision": "c5655f3f7e87f", + "go_version": "go1.26.3", + "graph_targets": 2969283, + "source_file_seeds": 1533786, + "histogram_requests": 58876, + "histogram_mean_targets": 2526.781642774645, + "notes": [ + "One new batch versus B in-flight batches; one batch is one request. Each batch's target count is drawn from the Hive per-request histogram (batch i always draws the same count, so smaller B is a prefix of larger B).", + "Affected sets are closure-shaped synthetic sets on the complete go-code graph: the reverse-dependency closure of a random source file, widened by climbing to a dependency until the drawn count is reached. They are not observed diffs.", + "ID64 stores sorted uint64 target IDs; NameKey stores sorted varint-prefixed UTF-8 labels; TangoSnapshot stores default-field GetTargetGraphResponse-shaped protobuf for the affected targets with their real direct dependencies in the response metadata, raw or per-message gzip/best-speed.", + "Go heap is measured after GC on all B decoded signatures plus the candidate, excluding the shared graph, workload indices, serialized buffers and ID64's dictionary held in storage.", + "Deserialize time is serial single-goroutine Go CPU for all B blobs; check time is local Go CPU for one candidate against all B decoded batches." + ], + "cases": [ + { + "in_flight_batches": 100, + "total_batch_targets": 177555, + "median_batch_targets": 34, + "max_batch_targets": 149002, + "candidate_targets": 2527, + "id_registry_entries": 157288, + "id_registry_minimal_bytes": 17572035, + "id64": { + "batch_blob_bytes": 1420440, + "max_single_batch_blob_bytes": 1192016, + "decoded_go_heap_bytes": 1460392, + "deserialize_milliseconds": 1.10812, + "check_microseconds": 319.114629984051, + "matching_batches": 31 + }, + "namekey": { + "batch_blob_bytes": 18356758, + "max_single_batch_blob_bytes": 15444347, + "decoded_go_heap_bytes": 22700024, + "deserialize_milliseconds": 16.259201, + "check_microseconds": 1314.0653790849674, + "matching_batches": 31 + }, + "tango_snapshot": { + "raw_batch_blob_bytes": 110111157, + "gzip_batch_blob_bytes": 27007012, + "decoded_go_heap_bytes": 175054592, + "raw_deserialize_milliseconds": 284.178867, + "gzip_deserialize_milliseconds": 928.707633, + "check_microseconds": 700.9578461538462, + "matching_batches": 31 + } + }, + { + "in_flight_batches": 250, + "total_batch_targets": 1128207, + "median_batch_targets": 38, + "max_batch_targets": 831151, + "candidate_targets": 2527, + "id_registry_entries": 832221, + "id_registry_minimal_bytes": 89665731, + "id64": { + "batch_blob_bytes": 9025656, + "max_single_batch_blob_bytes": 6649208, + "decoded_go_heap_bytes": 9096584, + "deserialize_milliseconds": 5.257342, + "check_microseconds": 639.3606166134185, + "matching_batches": 72 + }, + "namekey": { + "batch_blob_bytes": 113203407, + "max_single_batch_blob_bytes": 82894949, + "decoded_go_heap_bytes": 138875248, + "deserialize_milliseconds": 92.194151, + "check_microseconds": 6947.268724137931, + "matching_batches": 72 + }, + "tango_snapshot": { + "raw_batch_blob_bytes": 442763617, + "gzip_batch_blob_bytes": 107934164, + "decoded_go_heap_bytes": 767743424, + "raw_deserialize_milliseconds": 1546.66784, + "gzip_deserialize_milliseconds": 3469.78557, + "check_microseconds": 5442.542243243243, + "matching_batches": 72 + } + }, + { + "in_flight_batches": 500, + "total_batch_targets": 1723599, + "median_batch_targets": 39, + "max_batch_targets": 831151, + "candidate_targets": 2527, + "id_registry_entries": 845798, + "id_registry_minimal_bytes": 91655873, + "id64": { + "batch_blob_bytes": 13788792, + "max_single_batch_blob_bytes": 6649208, + "decoded_go_heap_bytes": 13932160, + "deserialize_milliseconds": 5.930734, + "check_microseconds": 1660.6582975206613, + "matching_batches": 146 + }, + "namekey": { + "batch_blob_bytes": 173482346, + "max_single_batch_blob_bytes": 82894949, + "decoded_go_heap_bytes": 212558648, + "deserialize_milliseconds": 117.94745, + "check_microseconds": 8186.6921600000005, + "matching_batches": 146 + }, + "tango_snapshot": { + "raw_batch_blob_bytes": 765929522, + "gzip_batch_blob_bytes": 186478740, + "decoded_go_heap_bytes": 1287503088, + "raw_deserialize_milliseconds": 2535.467949, + "gzip_deserialize_milliseconds": 6041.633127, + "check_microseconds": 17590.712750000002, + "matching_batches": 146 + } + }, + { + "in_flight_batches": 1000, + "total_batch_targets": 2627040, + "median_batch_targets": 40, + "max_batch_targets": 831151, + "candidate_targets": 2527, + "id_registry_entries": 850830, + "id_registry_minimal_bytes": 92108906, + "id64": { + "batch_blob_bytes": 21016320, + "max_single_batch_blob_bytes": 6649208, + "decoded_go_heap_bytes": 21303016, + "deserialize_milliseconds": 12.965979, + "check_microseconds": 3040.473393939394, + "matching_batches": 312 + }, + "namekey": { + "batch_blob_bytes": 263674500, + "max_single_batch_blob_bytes": 82894949, + "decoded_go_heap_bytes": 323208568, + "deserialize_milliseconds": 182.227514, + "check_microseconds": 15193.300928571427, + "matching_batches": 312 + }, + "tango_snapshot": { + "raw_batch_blob_bytes": 1334758663, + "gzip_batch_blob_bytes": 326331330, + "decoded_go_heap_bytes": 2194150784, + "raw_deserialize_milliseconds": 3991.475172, + "gzip_deserialize_milliseconds": 10161.646615, + "check_microseconds": 34701.05383333333, + "matching_batches": 312 + } + }, + { + "in_flight_batches": 2500, + "total_batch_targets": 5192014, + "median_batch_targets": 39, + "max_batch_targets": 831151, + "candidate_targets": 2527, + "id_registry_entries": 855213, + "id_registry_minimal_bytes": 92642012, + "id64": { + "batch_blob_bytes": 41536112, + "max_single_batch_blob_bytes": 6649208, + "decoded_go_heap_bytes": 42239392, + "deserialize_milliseconds": 32.321787, + "check_microseconds": 8535.270541666667, + "matching_batches": 814 + }, + "namekey": { + "batch_blob_bytes": 521330120, + "max_single_batch_blob_bytes": 82894949, + "decoded_go_heap_bytes": 638872048, + "deserialize_milliseconds": 351.862969, + "check_microseconds": 40075.9066, + "matching_batches": 814 + }, + "tango_snapshot": { + "raw_batch_blob_bytes": 2908708672, + "gzip_batch_blob_bytes": 708055385, + "decoded_go_heap_bytes": 4718199712, + "raw_deserialize_milliseconds": 8121.403348, + "gzip_deserialize_milliseconds": 22093.803663, + "check_microseconds": 74588.15733333334, + "matching_batches": 814 + } + }, + { + "in_flight_batches": 5000, + "total_batch_targets": 8806334, + "median_batch_targets": 40, + "max_batch_targets": 831151, + "candidate_targets": 2527, + "id_registry_entries": 872380, + "id_registry_minimal_bytes": 94919655, + "id64": { + "batch_blob_bytes": 70450672, + "max_single_batch_blob_bytes": 6649208, + "decoded_go_heap_bytes": 71939632, + "deserialize_milliseconds": 25.314694, + "check_microseconds": 23643.63922222222, + "matching_batches": 1709 + }, + "namekey": { + "batch_blob_bytes": 886950100, + "max_single_batch_blob_bytes": 82894949, + "decoded_go_heap_bytes": 1086451624, + "deserialize_milliseconds": 691.928778, + "check_microseconds": 73687.56766666667, + "matching_batches": 1709 + }, + "tango_snapshot": { + "raw_batch_blob_bytes": 5112463982, + "gzip_batch_blob_bytes": 1236702725, + "decoded_go_heap_bytes": 8267814752, + "raw_deserialize_milliseconds": 13713.883966, + "gzip_deserialize_milliseconds": 37459.182918, + "check_microseconds": 122608.7415, + "matching_batches": 1709 + } + }, + { + "in_flight_batches": 10000, + "total_batch_targets": 21208724, + "median_batch_targets": 41, + "max_batch_targets": 2856456, + "candidate_targets": 2527, + "id_registry_entries": 2857355, + "id_registry_minimal_bytes": 306475453, + "id64": { + "batch_blob_bytes": 169669792, + "max_single_batch_blob_bytes": 22851648, + "decoded_go_heap_bytes": 172616808, + "deserialize_milliseconds": 123.044387, + "check_microseconds": 31983.78614285714, + "matching_batches": 3424 + }, + "namekey": { + "batch_blob_bytes": 2136662598, + "max_single_batch_blob_bytes": 283519384, + "decoded_go_heap_bytes": 2615727600, + "deserialize_milliseconds": 1880.892051, + "check_microseconds": 151732.2155, + "matching_batches": 3424 + }, + "tango_snapshot": { + "raw_batch_blob_bytes": 11205573238, + "gzip_batch_blob_bytes": 2788124331, + "decoded_go_heap_bytes": 18536265608, + "raw_deserialize_milliseconds": 32914.078754, + "gzip_deserialize_milliseconds": 87958.863152, + "check_microseconds": 291976.636, + "matching_batches": 3424 + } + } + ], + "scale_model": { + "blob_store_profiles": [ + { + "name": "object-store", + "first_byte_ms": 150, + "parallel_gets": 32, + "mib_per_second": 100 + }, + { + "name": "kv-point-read", + "first_byte_ms": 5, + "parallel_gets": 16, + "mib_per_second": 100 + } + ], + "monte_carlo_trials": 400, + "fitted_designs": [ + { + "name": "TangoSnapshot raw", + "blob_bytes": { + "per_batch": 0, + "per_target": 535.4490757797312 + }, + "decoded_heap_bytes": { + "per_batch": 0, + "per_target": 879.6788261242503 + }, + "deserialize_milliseconds": { + "per_batch": 0, + "per_target": 0.0015443414683817991 + }, + "check_microseconds": { + "per_batch": 0, + "per_target": 0.013400936369409026 + } + }, + { + "name": "TangoSnapshot gzip", + "blob_bytes": { + "per_batch": 0, + "per_target": 131.6734308657514 + }, + "decoded_heap_bytes": { + "per_batch": 0, + "per_target": 879.6788261242503 + }, + "deserialize_milliseconds": { + "per_batch": 0, + "per_target": 0.00411403168492311 + }, + "check_microseconds": { + "per_batch": 0, + "per_target": 0.013400936369409026 + } + }, + { + "name": "NameKey", + "blob_bytes": { + "per_batch": 0, + "per_target": 100.66838491676907 + }, + "decoded_heap_bytes": { + "per_batch": 0, + "per_target": 123.29822675620352 + }, + "deserialize_milliseconds": { + "per_batch": 0, + "per_target": 8.157192400166282e-05 + }, + "check_microseconds": { + "per_batch": 0, + "per_target": 0.007271457738270583 + } + }, + { + "name": "ID64", + "blob_bytes": { + "per_batch": 0, + "per_target": 8 + }, + "decoded_heap_bytes": { + "per_batch": 0, + "per_target": 8.139004337688087 + }, + "deserialize_milliseconds": { + "per_batch": 0, + "per_target": 5.039783157931779e-06 + }, + "check_microseconds": { + "per_batch": 0, + "per_target": 0.001708672751452221 + } + } + ], + "points": [ + { + "in_flight_batches": 100, + "total_targets_p50_p99": [ + 118858, + 3065590 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 60.69412827398995, + 1565.4252359577047 + ], + "heap_mib_p50_p99": [ + 99.71319762752165, + 2571.806538179627 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 1.3904986209888235, + 20.988570121633607 + ], + [ + 0.8254986209888235, + 20.423570121633606 + ] + ], + "check_ms_p50_p99": [ + 1.5928084949952181, + 41.081776524696615 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 14.925423284379464, + 384.95707791112795 + ], + "heap_mib_p50_p99": [ + 99.71319762752165, + 2571.806538179627 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 1.2382398108503856, + 17.061505172094716 + ], + [ + 0.6732398108503856, + 16.496505172094718 + ] + ], + "check_ms_p50_p99": [ + 1.5928084949952181, + 41.081776524696615 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 11.41094483798727, + 294.31151782703216 + ], + "heap_mib_p50_p99": [ + 13.97607863978275, + 360.47154518275255 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 0.7238049241228623, + 3.7931812527705793 + ], + [ + 0.15880492412286235, + 3.2281812527705793 + ] + ], + "check_ms_p50_p99": [ + 0.8642709238553649, + 22.291308127864916 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 0.9068145751953125, + 23.388595581054688 + ], + "heap_mib_p50_p99": [ + 0.9225709701241785, + 23.794985110829565 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 0.6096671642985386, + 0.849335864661671 + ], + [ + 0.044667164298538585, + 0.28433586466167093 + ] + ], + "check_ms_p50_p99": [ + 0.2030894258921081, + 5.238090100124415 + ] + } + ] + }, + { + "in_flight_batches": 200, + "total_targets_p50_p99": [ + 284205, + 3416844 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 145.12758691976404, + 1744.790994533081 + ], + "heap_mib_p50_p99": [ + 238.42727735389954, + 2866.483038873375 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 2.94018543621909, + 23.774683825522352 + ], + [ + 1.9551854362190895, + 22.789683825522353 + ] + ], + "check_ms_p50_p99": [ + 3.808613120867892, + 45.78890902819701 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 35.688636225891955, + 429.0652963762832 + ], + "heap_mib_p50_p99": [ + 238.42727735389954, + 2866.483038873375 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 2.5761147372724924, + 19.397657442202252 + ], + [ + 1.591114737272492, + 18.412657442202253 + ] + ], + "check_ms_p50_p99": [ + 3.808613120867892, + 45.78890902819701 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 27.28505929495845, + 328.0336065221337 + ], + "heap_mib_p50_p99": [ + 33.4186292030781, + 401.7742217088446 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 1.3460337416104773, + 4.609054604314874 + ], + [ + 0.36103374161047713, + 3.6240546043148743 + ] + ], + "check_ms_p50_p99": [ + 2.066584646505191, + 24.84543674426341 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 2.1683120727539062, + 26.068450927734375 + ], + "heap_mib_p50_p99": [ + 2.2059876707006865, + 26.52140439720489 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 1.073115452299939, + 1.327904662121824 + ], + [ + 0.08811545229993906, + 0.342904662121824 + ] + ], + "check_ms_p50_p99": [ + 0.4856133393264785, + 5.838268238763013 + ] + } + ] + }, + { + "in_flight_batches": 500, + "total_targets_p50_p99": [ + 930740, + 4908888 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 475.2768257057447, + 2506.694357591832 + ], + "heap_mib_p50_p99": [ + 780.8230119961594, + 4118.19918958227 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 8.590148635339123, + 35.04794287796012 + ], + [ + 6.3501486353391225, + 32.807942877960116 + ] + ], + "check_ms_p50_p99": [ + 12.472787516463757, + 65.78369573255554 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 116.87634376906344, + 616.4265868146102 + ], + "heap_mib_p50_p99": [ + 780.8230119961594, + 4118.19918958227 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 7.39785728811597, + 28.759586637884937 + ], + [ + 5.15785728811597, + 26.519586637884938 + ] + ], + "check_ms_p50_p99": [ + 12.472787516463757, + 65.78369573255554 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 89.35555703872075, + 471.2770716641509 + ], + "heap_mib_p50_p99": [ + 109.44232136828315, + 577.2182328651489 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 3.369477822932515, + 7.513198155510184 + ], + [ + 1.129477822932515, + 5.273198155510184 + ] + ], + "check_ms_p50_p99": [ + 6.767836575317962, + 35.694771633903606 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 7.100982666015625, + 37.45184326171875 + ], + "heap_mib_p50_p99": [ + 7.224366090068636, + 38.10258934519291 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 2.47570055443657, + 2.799258163683761 + ], + [ + 0.23570055443656965, + 0.5592581636837609 + ] + ], + "check_ms_p50_p99": [ + 1.5903300766866402, + 8.38768316553079 + ] + } + ] + }, + { + "in_flight_batches": 1000, + "total_targets_p50_p99": [ + 2006378, + 7356034 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 1024.5449502609115, + 3756.3148562472143 + ], + "heap_mib_p50_p99": [ + 1683.2048833861552, + 6171.176294374536 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 18.143982249258052, + 53.723376911498576 + ], + [ + 13.65898224925805, + 49.23837691149858 + ] + ], + "check_ms_p50_p99": [ + 26.887343910982146, + 98.57774356520936 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 251.94804656368694, + 923.7234443141144 + ], + "heap_mib_p50_p99": [ + 1683.2048833861552, + 6171.176294374536 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 15.573783129569527, + 44.30019139451283 + ], + [ + 11.088783129569528, + 39.81519139451283 + ] + ], + "check_ms_p50_p99": [ + 26.887343910982146, + 98.57774356520936 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 192.62202529195528, + 706.2149640777974 + ], + "heap_mib_p50_p99": [ + 235.9226699854451, + 864.9692040999821 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 6.889884366654161, + 12.46219548717962 + ], + [ + 2.4048843666541613, + 7.977195487179622 + ] + ], + "check_ms_p50_p99": [ + 14.589292833995856, + 53.48909035228151 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 15.307449340820312, + 56.12208557128906 + ], + "heap_mib_p50_p99": [ + 15.573424572984647, + 57.09723723810296 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 4.963186203461048, + 5.398293671975264 + ], + [ + 0.47818620346104795, + 0.9132936719752641 + ] + ], + "check_ms_p50_p99": [ + 3.4282434177132046, + 12.569054854556088 + ] + } + ] + }, + { + "in_flight_batches": 2000, + "total_targets_p50_p99": [ + 4741648, + 11910855 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 2421.2942497947797, + 6082.207013603583 + ], + "heap_mib_p50_p99": [ + 3977.897020849609, + 9992.339081321867 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 40.985666132817414, + 88.66649743641851 + ], + [ + 32.16066613281742, + 79.84149743641852 + ] + ], + "check_ms_p50_p99": [ + 63.54252313413557, + 159.61660996025734 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 595.4256631066594, + 1495.6885742134948 + ], + "heap_mib_p50_p99": [ + 3977.897020849609, + 9992.339081321867 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 34.91154674181889, + 73.4085206066598 + ], + [ + 26.086546741818893, + 64.5835206066598 + ] + ], + "check_ms_p50_p99": [ + 63.54252313413557, + 159.61660996025734 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 455.2212200201304, + 1143.4998853948816 + ], + "heap_mib_p50_p99": [ + 557.5530913372983, + 1400.553990030537 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 14.388997550499939, + 21.856590212803642 + ], + [ + 5.5639975504999395, + 13.031590212803641 + ] + ], + "check_ms_p50_p99": [ + 34.478693041755236, + 86.60927875916887 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 36.1759033203125, + 90.87261199951172 + ], + "heap_mib_p50_p99": [ + 36.80447925547604, + 92.45157290513404 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 9.835655910934365, + 10.418754246420685 + ], + [ + 1.010655910934366, + 1.5937542464206846 + ] + ], + "check_ms_p50_p99": [ + 8.10192473457792, + 20.351753384998442 + ] + } + ] + }, + { + "in_flight_batches": 5000, + "total_targets_p50_p99": [ + 12400741, + 22563317 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 6332.364375528164, + 11521.8231527091 + ], + "heap_mib_p50_p99": [ + 10403.317724181044, + 18928.978168515532 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 106.02462232024402, + 173.613697634435 + ], + [ + 84.039622320244, + 151.628697634435 + ] + ], + "check_ms_p50_p99": [ + 166.18154107452168, + 302.369575399805 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 1557.2053077197925, + 2833.3562479987463 + ], + "heap_mib_p50_p99": [ + 10403.317724181044, + 18928.978168515532 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 90.13909446772303, + 144.70976353495172 + ], + [ + 68.15409446772301, + 122.72476353495172 + ] + ], + "check_ms_p50_p99": [ + 166.18154107452168, + 302.369575399805 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 1190.5313188945388, + 2166.187935595588 + ], + "heap_mib_p50_p99": [ + 1458.1578977231502, + 2653.1381376629847 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 36.466865491361695, + 47.052412535505304 + ], + [ + 14.481865491361692, + 25.067412535505305 + ] + ], + "check_ms_p50_p99": [ + 90.1714641047393, + 164.0682060007022 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 94.61014556884766, + 172.14444732666016 + ], + "heap_mib_p50_p99": [ + 96.25404814676905, + 175.1355504375757 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 24.558598501326152, + 25.385158698270278 + ], + [ + 2.5735985013261504, + 3.4001586982702774 + ] + ], + "check_ms_p50_p99": [ + 21.188808244516366, + 38.55332494027868 + ] + } + ] + }, + { + "in_flight_batches": 10000, + "total_targets_p50_p99": [ + 24651891, + 41710147 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 12588.340999767945, + 21299.0376108043 + ], + "heap_mib_p50_p99": [ + 20681.139504073115, + 34991.77279513352 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 210.9043475430075, + 324.35508577244366 + ], + [ + 167.07934754300751, + 280.53008577244367 + ] + ], + "check_ms_p50_p99": [ + 330.35842267660706, + 558.9550259056967 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 3095.6259396539112, + 5237.692029385404 + ], + "heap_mib_p50_p99": [ + 20681.139504073115, + 34991.77279513352 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 179.32492006380997, + 270.9237866346547 + ], + [ + 135.49992006380995, + 227.09878663465463 + ] + ], + "check_ms_p50_p99": [ + 330.35842267660706, + 558.9550259056967 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 2366.7011757986406, + 4004.3765384016233 + ], + "heap_mib_p50_p99": [ + 2898.725935446942, + 4904.54403194483 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 72.62791393713567, + 90.39614232519841 + ], + [ + 28.80291393713568, + 46.571142325198416 + ] + ], + "check_ms_p50_p99": [ + 179.25518357495295, + 303.2935711675535 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 188.0790023803711, + 318.2231674194336 + ], + "heap_mib_p50_p99": [ + 191.34697702523604, + 323.75246749745156 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 48.955030208876686, + 50.342441770559795 + ], + [ + 5.130030208876681, + 6.5174417705597945 + ] + ], + "check_ms_p50_p99": [ + 42.12201442347025, + 71.2689916379666 + ] + } + ] + }, + { + "in_flight_batches": 20000, + "total_targets_p50_p99": [ + 50179803, + 68436214 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 25624.01689449213, + 34946.54420486822 + ], + "heap_mib_p50_p99": [ + 42097.19676798452, + 57412.994762332906 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 427.48491959305073, + 548.9043252679332 + ], + [ + 339.98491959305073, + 461.40432526793325 + ] + ], + "check_ms_p50_p99": [ + 672.4563470324802, + 917.1093491772591 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 6301.256962945486, + 8593.779652445573 + ], + "heap_mib_p50_p99": [ + 42097.19676798452, + 57412.994762332906 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 363.2038691146546, + 461.2365493166343 + ], + [ + 275.7038691146546, + 373.7365493166343 + ] + ], + "check_ms_p50_p99": [ + 672.4563470324802, + 917.1093491772591 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 4817.504619075435, + 6570.208676527386 + ], + "heap_mib_p50_p99": [ + 5900.459984660742, + 8047.164756877965 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 146.01830926748877, + 165.0345604126434 + ], + [ + 58.51830926748875, + 77.5345604126434 + ] + ], + "check_ms_p50_p99": [ + 364.88031682924344, + 497.6310378682416 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 382.84151458740234, + 522.1268768310547 + ], + "heap_mib_p50_p99": [ + 389.4935934842431, + 531.199114418936 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 97.83131047190176, + 99.31617244702036 + ], + [ + 10.331310471901759, + 11.816172447020364 + ] + ], + "check_ms_p50_p99": [ + 85.74086205934042, + 116.935094074353 + ] + } + ] + }, + { + "in_flight_batches": 50000, + "total_targets_p50_p99": [ + 126844165, + 157003552 + ], + "designs": [ + { + "name": "TangoSnapshot raw", + "blob_mib_p50_p99": [ + 64772.215764333465, + 80172.9267239904 + ], + "heap_mib_p50_p99": [ + 106413.0078963382, + 131714.53506536264 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 1078.0628616750978, + 1278.6463632767422 + ], + [ + 859.2378616750979, + 1059.8213632767422 + ] + ], + "check_ms_p50_p99": [ + 1699.8305839958196, + 2103.9946101232013 + ] + }, + { + "name": "TangoSnapshot gzip", + "blob_mib_p50_p99": [ + 15928.27452740809, + 19715.496396970182 + ], + "heap_mib_p50_p99": [ + 106413.0078963382, + 131714.53506536264 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 915.573659131696, + 1077.5225515431748 + ], + [ + 696.748659131696, + 858.697551543175 + ] + ], + "check_ms_p50_p99": [ + 1699.8305839958196, + 2103.9946101232013 + ] + }, + { + "name": "NameKey", + "blob_mib_p50_p99": [ + 12177.655436197441, + 15073.102956806153 + ], + "heap_mib_p50_p99": [ + 14915.14264952783, + 18461.47494890727 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 366.5734769494088, + 397.9881113797966 + ], + [ + 147.74847694940883, + 179.16311137979665 + ] + ], + "check_ms_p50_p99": [ + 922.3419851437206, + 1141.6446931263679 + ] + }, + { + "name": "ID64", + "blob_mib_p50_p99": [ + 967.7441787719727, + 1197.842041015625 + ], + "heap_mib_p50_p99": [ + 984.5592585996851, + 1218.6551959614155 + ], + "load_seconds_p50_p99_by_profile": [ + [ + 244.76670887416861, + 247.2196842672613 + ], + [ + 25.941708874168647, + 28.394684267261315 + ] + ], + "check_ms_p50_p99": [ + 216.73516841620952, + 268.2676911836119 + ] + } + ] + } + ], + "crossovers": [ + { + "design": "TangoSnapshot raw", + "metric": "load_seconds", + "profile": "object-store", + "limit": 1, + "first_in_flight_exceeding_at_p50": 81, + "first_in_flight_exceeding_at_p99": 10 + }, + { + "design": "TangoSnapshot raw", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 1, + "first_in_flight_exceeding_at_p50": 120, + "first_in_flight_exceeding_at_p99": 10 + }, + { + "design": "TangoSnapshot raw", + "metric": "load_seconds", + "profile": "object-store", + "limit": 10, + "first_in_flight_exceeding_at_p50": 574, + "first_in_flight_exceeding_at_p99": 55 + }, + { + "design": "TangoSnapshot raw", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 10, + "first_in_flight_exceeding_at_p50": 732, + "first_in_flight_exceeding_at_p99": 55 + }, + { + "design": "TangoSnapshot raw", + "metric": "load_seconds", + "profile": "object-store", + "limit": 60, + "first_in_flight_exceeding_at_p50": 3014, + "first_in_flight_exceeding_at_p99": 1030 + }, + { + "design": "TangoSnapshot raw", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 60, + "first_in_flight_exceeding_at_p50": 3664, + "first_in_flight_exceeding_at_p99": 1136 + }, + { + "design": "TangoSnapshot raw", + "metric": "heap_mib", + "limit": 1024, + "first_in_flight_exceeding_at_p50": 633, + "first_in_flight_exceeding_at_p99": 55 + }, + { + "design": "TangoSnapshot raw", + "metric": "heap_mib", + "limit": 4096, + "first_in_flight_exceeding_at_p50": 2249, + "first_in_flight_exceeding_at_p99": 450 + }, + { + "design": "TangoSnapshot raw", + "metric": "heap_mib", + "limit": 16384, + "first_in_flight_exceeding_at_p50": 8397, + "first_in_flight_exceeding_at_p99": 4453 + }, + { + "design": "TangoSnapshot gzip", + "metric": "load_seconds", + "profile": "object-store", + "limit": 1, + "first_in_flight_exceeding_at_p50": 94, + "first_in_flight_exceeding_at_p99": 10 + }, + { + "design": "TangoSnapshot gzip", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 1, + "first_in_flight_exceeding_at_p50": 133, + "first_in_flight_exceeding_at_p99": 10 + }, + { + "design": "TangoSnapshot gzip", + "metric": "load_seconds", + "profile": "object-store", + "limit": 10, + "first_in_flight_exceeding_at_p50": 664, + "first_in_flight_exceeding_at_p99": 55 + }, + { + "design": "TangoSnapshot gzip", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 10, + "first_in_flight_exceeding_at_p50": 890, + "first_in_flight_exceeding_at_p99": 55 + }, + { + "design": "TangoSnapshot gzip", + "metric": "load_seconds", + "profile": "object-store", + "limit": 60, + "first_in_flight_exceeding_at_p50": 3489, + "first_in_flight_exceeding_at_p99": 1381 + }, + { + "design": "TangoSnapshot gzip", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 60, + "first_in_flight_exceeding_at_p50": 4676, + "first_in_flight_exceeding_at_p99": 1678 + }, + { + "design": "TangoSnapshot gzip", + "metric": "heap_mib", + "limit": 1024, + "first_in_flight_exceeding_at_p50": 633, + "first_in_flight_exceeding_at_p99": 55 + }, + { + "design": "TangoSnapshot gzip", + "metric": "heap_mib", + "limit": 4096, + "first_in_flight_exceeding_at_p50": 2249, + "first_in_flight_exceeding_at_p99": 450 + }, + { + "design": "TangoSnapshot gzip", + "metric": "heap_mib", + "limit": 16384, + "first_in_flight_exceeding_at_p50": 8397, + "first_in_flight_exceeding_at_p99": 4453 + }, + { + "design": "NameKey", + "metric": "load_seconds", + "profile": "object-store", + "limit": 1, + "first_in_flight_exceeding_at_p50": 161, + "first_in_flight_exceeding_at_p99": 34 + }, + { + "design": "NameKey", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 1, + "first_in_flight_exceeding_at_p50": 450, + "first_in_flight_exceeding_at_p99": 55 + }, + { + "design": "NameKey", + "metric": "load_seconds", + "profile": "object-store", + "limit": 10, + "first_in_flight_exceeding_at_p50": 1450, + "first_in_flight_exceeding_at_p99": 633 + }, + { + "design": "NameKey", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 10, + "first_in_flight_exceeding_at_p50": 3664, + "first_in_flight_exceeding_at_p99": 1136 + }, + { + "design": "NameKey", + "metric": "load_seconds", + "profile": "object-store", + "limit": 60, + "first_in_flight_exceeding_at_p50": 8397, + "first_in_flight_exceeding_at_p99": 6908 + }, + { + "design": "NameKey", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 60, + "first_in_flight_exceeding_at_p50": 21219, + "first_in_flight_exceeding_at_p99": 15834 + }, + { + "design": "NameKey", + "metric": "heap_mib", + "limit": 1024, + "first_in_flight_exceeding_at_p50": 3664, + "first_in_flight_exceeding_at_p99": 1136 + }, + { + "design": "NameKey", + "metric": "heap_mib", + "limit": 4096, + "first_in_flight_exceeding_at_p50": 14362, + "first_in_flight_exceeding_at_p99": 9721 + }, + { + "design": "NameKey", + "metric": "heap_mib", + "limit": 16384, + "first_in_flight_exceeding_at_p50": 56300, + "first_in_flight_exceeding_at_p99": 46318 + }, + { + "design": "ID64", + "metric": "load_seconds", + "profile": "object-store", + "limit": 1, + "first_in_flight_exceeding_at_p50": 196, + "first_in_flight_exceeding_at_p99": 133 + }, + { + "design": "ID64", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 1, + "first_in_flight_exceeding_at_p50": 2142, + "first_in_flight_exceeding_at_p99": 1030 + }, + { + "design": "ID64", + "metric": "load_seconds", + "profile": "object-store", + "limit": 10, + "first_in_flight_exceeding_at_p50": 2142, + "first_in_flight_exceeding_at_p99": 1943 + }, + { + "design": "ID64", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 10, + "first_in_flight_exceeding_at_p50": 20208, + "first_in_flight_exceeding_at_p99": 17457 + }, + { + "design": "ID64", + "metric": "load_seconds", + "profile": "object-store", + "limit": 60, + "first_in_flight_exceeding_at_p50": 12406, + "first_in_flight_exceeding_at_p99": 12406 + }, + { + "design": "ID64", + "metric": "load_seconds", + "profile": "kv-point-read", + "limit": 60, + "first_in_flight_exceeding_at_p50": 0, + "first_in_flight_exceeding_at_p99": 0 + }, + { + "design": "ID64", + "metric": "heap_mib", + "limit": 1024, + "first_in_flight_exceeding_at_p50": 53619, + "first_in_flight_exceeding_at_p99": 42012 + }, + { + "design": "ID64", + "metric": "heap_mib", + "limit": 4096, + "first_in_flight_exceeding_at_p50": 0, + "first_in_flight_exceeding_at_p99": 0 + }, + { + "design": "ID64", + "metric": "heap_mib", + "limit": 16384, + "first_in_flight_exceeding_at_p50": 0, + "first_in_flight_exceeding_at_p99": 0 + } + ] + } +} diff --git a/doc/rfc/submitqueue/tango-eval/hive-go-diff-tail-20260930.sql b/doc/rfc/submitqueue/tango-eval/hive-go-diff-tail-20260930.sql new file mode 100644 index 000000000..944fef107 --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/hive-go-diff-tail-20260930.sql @@ -0,0 +1,28 @@ +-- Cross-check the broad-diff contribution independently of the mean query. +-- Grain and filters intentionally match hive-go-diff-targets-20260930.sql. +WITH single_diff_events AS ( + SELECT msg.diffids[1] AS diff_id, + msg.targetschanged + msg.targetsadded + msg.targetsremoved AS affected_targets, + ROW_NUMBER() OVER ( + PARTITION BY msg.diffids[1] + ORDER BY ts DESC, datestr DESC + ) AS latest_event + FROM rawdata_user.kafka_hp_submitqueue_request_feature_event_nodedup + WHERE datestr BETWEEN '2026-09-01' AND '2026-09-29' + AND msg.queueid = 'go' + AND CARDINALITY(msg.diffids) = 1 + AND msg.stackheight = 1 + AND msg.diffids[1] IS NOT NULL + AND msg.targetschanged IS NOT NULL + AND msg.targetsadded IS NOT NULL + AND msg.targetsremoved IS NOT NULL + AND (hadoop_isdeleted = false OR hadoop_isdeleted IS NULL) +) +SELECT COUNT(*) AS unique_diffs, + SUM(affected_targets) AS total_affected_targets, + COUNT_IF(affected_targets >= 100000) AS broad_diffs, + SUM(IF(affected_targets >= 100000, affected_targets, 0)) AS broad_affected_targets, + AVG(IF(affected_targets < 100000, CAST(affected_targets AS DOUBLE), NULL)) AS avg_below_100000 +FROM single_diff_events +WHERE latest_event = 1 +LIMIT 10; diff --git a/doc/rfc/submitqueue/tango-eval/hive-go-diff-target-histogram-20260930.csv b/doc/rfc/submitqueue/tango-eval/hive-go-diff-target-histogram-20260930.csv new file mode 100644 index 000000000..7836eea91 --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/hive-go-diff-target-histogram-20260930.csv @@ -0,0 +1,4954 @@ +affected_targets,diffs +0,5942 +1,80 +2,1598 +3,799 +4,1056 +5,1291 +6,866 +7,789 +8,641 +9,724 +10,778 +11,637 +12,703 +13,651 +14,581 +15,650 +16,660 +17,551 +18,648 +19,591 +20,558 +21,612 +22,525 +23,506 +24,492 +25,428 +26,504 +27,457 +28,474 +29,429 +30,419 +31,384 +32,430 +33,365 +34,351 +35,352 +36,329 +37,298 +38,325 +39,295 +40,305 +41,266 +42,259 +43,267 +44,240 +45,218 +46,216 +47,254 +48,184 +49,220 +50,194 +51,200 +52,191 +53,211 +54,199 +55,201 +56,195 +57,177 +58,210 +59,383 +60,311 +61,177 +62,197 +63,224 +64,140 +65,174 +66,178 +67,141 +68,150 +69,162 +70,171 +71,147 +72,126 +73,112 +74,141 +75,129 +76,161 +77,110 +78,123 +79,122 +80,148 +81,143 +82,122 +83,111 +84,138 +85,93 +86,101 +87,100 +88,98 +89,103 +90,100 +91,93 +92,103 +93,91 +94,103 +95,114 +96,107 +97,97 +98,74 +99,84 +100,89 +101,95 +102,77 +103,67 +104,72 +105,84 +106,68 +107,79 +108,97 +109,78 +110,77 +111,63 +112,78 +113,60 +114,59 +115,54 +116,68 +117,72 +118,79 +119,62 +120,61 +121,72 +122,72 +123,76 +124,62 +125,83 +126,71 +127,57 +128,72 +129,70 +130,62 +131,75 +132,76 +133,59 +134,64 +135,71 +136,57 +137,55 +138,61 +139,72 +140,57 +141,58 +142,72 +143,64 +144,61 +145,65 +146,57 +147,61 +148,196 +149,53 +150,44 +151,61 +152,42 +153,59 +154,55 +155,35 +156,57 +157,39 +158,40 +159,39 +160,39 +161,33 +162,38 +163,41 +164,37 +165,36 +166,31 +167,42 +168,37 +169,39 +170,39 +171,44 +172,40 +173,29 +174,42 +175,28 +176,31 +177,26 +178,27 +179,36 +180,28 +181,38 +182,36 +183,39 +184,28 +185,43 +186,26 +187,36 +188,30 +189,39 +190,48 +191,43 +192,40 +193,36 +194,35 +195,35 +196,27 +197,34 +198,22 +199,24 +200,29 +201,25 +202,39 +203,29 +204,23 +205,32 +206,29 +207,26 +208,30 +209,22 +210,30 +211,23 +212,31 +213,32 +214,25 +215,28 +216,38 +217,23 +218,34 +219,21 +220,27 +221,38 +222,24 +223,22 +224,27 +225,28 +226,35 +227,34 +228,23 +229,20 +230,24 +231,25 +232,26 +233,19 +234,22 +235,20 +236,27 +237,37 +238,22 +239,32 +240,28 +241,35 +242,22 +243,36 +244,32 +245,21 +246,28 +247,18 +248,26 +249,26 +250,19 +251,31 +252,28 +253,31 +254,23 +255,35 +256,35 +257,24 +258,25 +259,25 +260,30 +261,26 +262,17 +263,25 +264,35 +265,22 +266,32 +267,20 +268,26 +269,21 +270,41 +271,20 +272,23 +273,28 +274,18 +275,20 +276,20 +277,15 +278,24 +279,26 +280,18 +281,24 +282,51 +283,23 +284,25 +285,31 +286,24 +287,23 +288,26 +289,18 +290,57 +291,27 +292,20 +293,20 +294,23 +295,28 +296,25 +297,20 +298,26 +299,26 +300,18 +301,20 +302,28 +303,26 +304,28 +305,26 +306,23 +307,25 +308,20 +309,22 +310,21 +311,15 +312,10 +313,19 +314,18 +315,15 +316,17 +317,18 +318,20 +319,19 +320,24 +321,21 +322,26 +323,21 +324,18 +325,25 +326,20 +327,16 +328,21 +329,17 +330,32 +331,21 +332,38 +333,27 +334,20 +335,15 +336,21 +337,19 +338,15 +339,19 +340,22 +341,15 +342,24 +343,19 +344,15 +345,19 +346,19 +347,20 +348,16 +349,19 +350,25 +351,12 +352,22 +353,37 +354,20 +355,27 +356,18 +357,46 +358,27 +359,27 +360,21 +361,18 +362,19 +363,12 +364,16 +365,18 +366,23 +367,18 +368,10 +369,22 +370,9 +371,19 +372,14 +373,18 +374,16 +375,24 +376,17 +377,12 +378,21 +379,12 +380,21 +381,15 +382,22 +383,13 +384,19 +385,12 +386,8 +387,10 +388,13 +389,13 +390,14 +391,11 +392,13 +393,15 +394,7 +395,18 +396,21 +397,24 +398,16 +399,12 +400,21 +401,14 +402,23 +403,17 +404,11 +405,24 +406,12 +407,11 +408,11 +409,18 +410,8 +411,16 +412,10 +413,6 +414,12 +415,14 +416,11 +417,9 +418,5 +419,15 +420,11 +421,9 +422,12 +423,7 +424,20 +425,10 +426,11 +427,9 +428,14 +429,17 +430,18 +431,10 +432,14 +433,9 +434,23 +435,14 +436,15 +437,8 +438,9 +439,11 +440,8 +441,9 +442,25 +443,16 +444,16 +445,14 +446,11 +447,13 +448,7 +449,6 +450,7 +451,22 +452,8 +453,18 +454,11 +455,15 +456,11 +457,7 +458,4 +459,20 +460,15 +461,17 +462,12 +463,13 +464,9 +465,18 +466,7 +467,17 +468,10 +469,10 +470,6 +471,10 +472,13 +473,11 +474,10 +475,10 +476,12 +477,12 +478,11 +479,10 +480,11 +481,8 +482,7 +483,7 +484,11 +485,6 +486,9 +487,7 +488,8 +489,7 +490,12 +491,14 +492,12 +493,9 +494,3 +495,1 +496,8 +497,5 +498,10 +499,38 +500,5 +501,6 +502,12 +503,10 +504,8 +505,16 +506,11 +507,13 +508,10 +509,3 +510,8 +511,12 +512,12 +513,5 +514,11 +515,5 +516,11 +517,11 +518,11 +519,11 +520,10 +521,10 +522,7 +523,7 +524,7 +525,10 +526,7 +527,8 +528,9 +529,11 +530,11 +531,5 +532,5 +533,9 +534,9 +535,5 +536,10 +537,7 +538,6 +539,5 +540,19 +541,5 +542,17 +543,16 +544,12 +545,6 +546,17 +547,14 +548,18 +549,17 +550,11 +551,13 +552,17 +553,6 +554,4 +555,12 +556,9 +557,12 +558,14 +559,9 +560,7 +561,7 +562,7 +563,6 +564,8 +565,9 +566,9 +567,4 +568,15 +569,14 +570,11 +571,7 +572,10 +573,7 +574,11 +575,9 +576,7 +577,7 +578,8 +579,10 +580,7 +581,15 +582,7 +583,15 +584,7 +585,11 +586,6 +587,8 +588,9 +589,5 +590,7 +591,5 +592,6 +593,8 +594,3 +595,1 +596,8 +597,6 +598,8 +599,4 +600,1 +601,6 +602,6 +603,6 +604,6 +605,10 +606,9 +607,10 +608,3 +609,6 +610,12 +611,6 +612,3 +613,9 +614,4 +615,5 +616,5 +617,11 +618,8 +619,5 +620,8 +621,3 +622,6 +623,1 +624,9 +625,3 +626,7 +627,9 +628,4 +629,5 +630,5 +631,7 +632,5 +633,4 +634,4 +635,2 +636,6 +637,6 +638,15 +639,8 +640,6 +641,5 +642,6 +643,6 +644,8 +645,9 +646,11 +647,7 +648,10 +649,9 +650,5 +651,8 +652,7 +653,9 +654,6 +655,7 +656,9 +657,15 +658,17 +659,13 +660,10 +661,14 +662,9 +663,9 +664,7 +665,13 +666,3 +667,10 +668,7 +669,7 +670,8 +671,8 +672,4 +673,11 +674,10 +675,5 +676,2 +677,3 +678,4 +679,9 +680,9 +681,7 +682,7 +683,3 +684,9 +685,10 +686,7 +687,11 +688,7 +689,7 +690,6 +691,7 +692,5 +693,8 +694,5 +695,11 +696,7 +697,9 +698,8 +699,6 +700,6 +701,11 +702,6 +703,8 +704,11 +705,13 +706,6 +707,11 +708,7 +709,6 +710,4 +711,6 +712,11 +713,7 +714,13 +715,2 +716,7 +717,6 +718,9 +719,8 +720,2 +721,5 +722,4 +723,5 +724,6 +725,10 +726,4 +727,7 +728,8 +729,5 +730,10 +731,14 +732,6 +733,5 +734,6 +735,3 +736,6 +737,8 +738,7 +739,8 +740,13 +741,11 +742,8 +743,10 +744,11 +745,12 +746,4 +747,7 +748,10 +749,6 +750,7 +751,3 +752,8 +753,3 +754,2 +755,3 +756,1 +757,3 +758,5 +759,3 +760,5 +761,6 +762,5 +763,5 +764,3 +765,4 +766,8 +767,2 +768,3 +769,4 +770,3 +771,3 +772,4 +773,3 +774,3 +775,4 +776,6 +777,4 +778,17 +779,5 +780,9 +781,3 +782,4 +783,6 +784,13 +785,5 +786,15 +787,10 +788,15 +789,6 +790,8 +791,5 +792,5 +793,10 +794,25 +795,3 +796,3 +797,4 +798,11 +799,4 +800,3 +801,3 +802,12 +803,2 +804,2 +805,6 +806,4 +807,4 +808,4 +809,1 +810,4 +811,3 +812,1 +813,6 +814,2 +815,7 +816,7 +817,5 +818,2 +819,3 +820,1 +821,6 +822,3 +823,3 +824,4 +825,5 +826,4 +827,5 +828,5 +829,15 +830,6 +831,5 +832,4 +833,2 +834,3 +835,3 +836,5 +837,2 +838,8 +839,3 +840,2 +841,2 +842,1 +843,1 +844,5 +845,5 +846,5 +847,5 +848,4 +849,2 +850,8 +851,4 +852,2 +853,1 +854,7 +855,4 +856,3 +857,4 +858,6 +859,1 +860,1 +861,8 +862,4 +863,2 +864,2 +865,6 +866,2 +867,3 +868,1 +869,4 +870,2 +871,6 +872,2 +873,2 +874,6 +875,3 +876,1 +877,4 +878,5 +879,7 +880,2 +881,2 +882,5 +883,5 +884,3 +885,2 +886,2 +887,1 +888,1 +889,5 +890,4 +891,2 +892,1 +893,3 +894,2 +895,2 +897,4 +899,4 +900,2 +901,5 +902,3 +903,3 +904,3 +905,3 +906,2 +907,1 +908,1 +909,2 +910,3 +912,5 +913,1 +915,3 +917,2 +918,1 +919,5 +920,3 +921,3 +922,3 +923,3 +924,3 +925,4 +927,1 +928,6 +929,2 +930,2 +931,3 +932,3 +933,6 +934,3 +935,2 +936,3 +937,5 +938,4 +939,3 +940,3 +941,2 +942,2 +943,2 +944,2 +946,3 +947,1 +948,3 +949,4 +950,4 +951,5 +952,3 +953,3 +954,4 +955,2 +956,4 +957,2 +959,4 +960,2 +961,2 +962,4 +963,2 +964,3 +965,3 +966,2 +967,2 +968,2 +969,3 +970,4 +971,1 +972,6 +973,4 +974,1 +975,4 +976,4 +977,3 +978,3 +979,3 +980,4 +981,4 +982,7 +983,3 +984,2 +985,4 +986,5 +987,4 +988,2 +989,6 +990,3 +991,2 +992,4 +993,3 +994,3 +995,4 +996,4 +997,2 +998,3 +999,2 +1000,2 +1001,2 +1002,6 +1003,4 +1004,2 +1005,2 +1006,4 +1007,7 +1008,4 +1009,5 +1010,3 +1011,2 +1012,2 +1013,14 +1014,2 +1015,4 +1016,2 +1017,4 +1018,2 +1019,5 +1020,2 +1021,2 +1022,3 +1023,3 +1024,5 +1025,4 +1026,4 +1027,1 +1028,3 +1029,3 +1030,1 +1031,4 +1032,3 +1033,4 +1034,1 +1035,3 +1036,5 +1037,1 +1038,2 +1040,12 +1041,4 +1042,3 +1043,2 +1044,3 +1045,3 +1046,2 +1047,1 +1048,1 +1049,6 +1050,1 +1051,3 +1052,3 +1053,1 +1054,1 +1055,1 +1056,5 +1058,4 +1059,5 +1060,4 +1061,4 +1062,6 +1063,3 +1064,3 +1065,10 +1066,3 +1067,6 +1068,2 +1069,2 +1070,3 +1071,9 +1072,6 +1073,5 +1074,8 +1075,3 +1076,5 +1077,1 +1078,3 +1079,1 +1080,2 +1081,1 +1082,7 +1083,2 +1084,3 +1085,4 +1086,2 +1087,2 +1088,1 +1089,3 +1090,4 +1092,2 +1093,3 +1094,1 +1095,1 +1096,3 +1098,3 +1099,2 +1100,4 +1101,16 +1102,3 +1103,5 +1104,7 +1105,4 +1106,6 +1107,5 +1108,6 +1109,4 +1110,9 +1111,6 +1112,4 +1113,4 +1114,11 +1115,3 +1116,3 +1117,5 +1118,5 +1119,5 +1120,6 +1121,2 +1122,1 +1123,2 +1124,2 +1125,2 +1126,3 +1127,3 +1128,2 +1129,7 +1132,4 +1133,4 +1134,5 +1135,4 +1136,6 +1137,2 +1138,2 +1139,1 +1140,5 +1141,2 +1142,4 +1143,3 +1144,4 +1145,5 +1146,1 +1147,1 +1148,3 +1149,4 +1151,4 +1152,4 +1153,4 +1154,3 +1155,1 +1156,3 +1157,2 +1158,6 +1159,3 +1160,2 +1161,3 +1164,1 +1165,1 +1166,3 +1167,4 +1168,3 +1169,3 +1170,4 +1171,4 +1172,2 +1173,7 +1174,3 +1175,7 +1176,5 +1177,5 +1178,5 +1179,1 +1180,3 +1181,3 +1182,5 +1183,11 +1184,2 +1185,4 +1186,1 +1189,2 +1190,1 +1191,2 +1192,2 +1194,1 +1196,2 +1198,4 +1199,12 +1200,2 +1201,4 +1202,5 +1203,2 +1204,3 +1205,2 +1206,1 +1207,2 +1208,1 +1209,5 +1211,1 +1212,2 +1213,5 +1214,3 +1215,2 +1216,3 +1217,2 +1218,3 +1219,2 +1220,6 +1221,4 +1222,3 +1223,1 +1224,1 +1225,4 +1226,3 +1227,1 +1228,3 +1229,3 +1230,2 +1231,3 +1232,2 +1233,1 +1235,1 +1236,3 +1237,1 +1239,4 +1240,1 +1242,1 +1243,4 +1244,6 +1245,7 +1246,2 +1247,2 +1248,3 +1249,3 +1250,3 +1251,5 +1252,2 +1253,3 +1254,2 +1255,4 +1256,6 +1257,4 +1258,4 +1259,14 +1260,4 +1261,1 +1262,3 +1263,2 +1264,7 +1265,1 +1267,7 +1268,2 +1269,1 +1270,1 +1271,1 +1273,2 +1274,2 +1275,4 +1276,2 +1277,5 +1278,2 +1280,3 +1281,1 +1282,2 +1283,2 +1284,4 +1286,4 +1287,3 +1288,2 +1289,4 +1290,3 +1291,1 +1292,4 +1293,2 +1294,5 +1295,2 +1296,1 +1297,3 +1298,1 +1299,1 +1300,2 +1301,2 +1302,2 +1304,1 +1307,3 +1308,1 +1309,1 +1310,1 +1311,5 +1312,1 +1313,2 +1314,3 +1317,1 +1318,2 +1319,3 +1320,3 +1323,8 +1324,3 +1325,2 +1326,2 +1327,6 +1328,2 +1329,3 +1331,4 +1332,3 +1333,5 +1334,1 +1335,2 +1336,1 +1338,3 +1339,1 +1340,4 +1341,1 +1342,4 +1343,2 +1344,1 +1345,3 +1347,3 +1348,1 +1349,1 +1350,1 +1352,2 +1353,1 +1354,3 +1356,2 +1357,3 +1359,1 +1360,3 +1361,1 +1363,1 +1364,1 +1365,1 +1366,4 +1367,2 +1368,2 +1370,1 +1371,3 +1372,4 +1374,3 +1375,1 +1376,1 +1377,3 +1378,2 +1379,2 +1380,3 +1381,3 +1382,3 +1383,1 +1384,3 +1385,3 +1386,2 +1387,1 +1388,2 +1389,2 +1390,1 +1391,2 +1392,3 +1393,1 +1395,2 +1396,2 +1397,1 +1400,3 +1401,5 +1402,2 +1403,1 +1404,2 +1405,3 +1406,1 +1407,2 +1408,3 +1410,1 +1412,3 +1414,2 +1415,1 +1416,1 +1417,3 +1418,1 +1419,2 +1420,1 +1421,2 +1422,3 +1423,1 +1424,2 +1425,4 +1426,2 +1427,2 +1428,2 +1429,2 +1430,2 +1431,3 +1432,1 +1433,2 +1434,1 +1435,1 +1437,1 +1438,5 +1439,1 +1440,2 +1442,1 +1443,1 +1444,1 +1445,1 +1447,1 +1448,2 +1450,1 +1452,1 +1453,4 +1454,1 +1455,2 +1456,1 +1458,2 +1459,1 +1460,1 +1461,3 +1463,2 +1467,1 +1468,2 +1470,1 +1471,1 +1472,1 +1474,10 +1475,14 +1476,10 +1477,6 +1478,6 +1479,2 +1481,1 +1482,2 +1483,3 +1486,2 +1487,4 +1488,1 +1489,1 +1491,2 +1492,2 +1493,1 +1494,2 +1496,3 +1498,1 +1499,4 +1500,2 +1501,2 +1502,2 +1503,2 +1507,1 +1508,1 +1509,1 +1510,1 +1511,3 +1512,1 +1514,3 +1516,1 +1517,2 +1518,1 +1520,4 +1521,2 +1522,2 +1523,1 +1524,1 +1525,2 +1526,2 +1527,2 +1528,1 +1530,2 +1531,2 +1532,2 +1533,1 +1534,2 +1536,1 +1539,2 +1540,1 +1541,2 +1543,1 +1545,1 +1546,1 +1548,1 +1549,1 +1551,1 +1553,5 +1554,1 +1556,2 +1557,1 +1558,3 +1560,2 +1562,2 +1566,1 +1567,1 +1568,1 +1569,1 +1570,1 +1571,1 +1572,1 +1574,3 +1577,1 +1580,6 +1582,1 +1583,2 +1584,1 +1586,3 +1587,3 +1588,1 +1589,1 +1590,1 +1591,1 +1592,1 +1594,2 +1596,2 +1597,1 +1598,1 +1599,3 +1600,1 +1601,2 +1602,3 +1603,2 +1604,1 +1605,2 +1606,3 +1607,4 +1608,3 +1609,5 +1610,3 +1611,3 +1612,4 +1613,5 +1614,8 +1615,6 +1616,1 +1617,9 +1618,1 +1619,2 +1620,1 +1621,4 +1622,1 +1623,1 +1624,2 +1625,1 +1626,4 +1627,1 +1628,6 +1630,1 +1631,2 +1632,1 +1633,6 +1634,1 +1635,1 +1636,1 +1638,1 +1640,1 +1641,1 +1642,2 +1645,4 +1646,1 +1647,3 +1648,2 +1649,1 +1650,2 +1651,1 +1653,3 +1654,1 +1655,2 +1656,4 +1657,3 +1658,1 +1660,2 +1661,1 +1662,3 +1664,1 +1665,2 +1666,1 +1671,1 +1673,5 +1674,1 +1675,3 +1676,1 +1677,1 +1678,2 +1680,1 +1681,2 +1682,4 +1687,1 +1688,1 +1689,1 +1691,2 +1692,1 +1693,2 +1694,1 +1695,1 +1696,1 +1697,1 +1698,1 +1699,3 +1700,2 +1701,5 +1702,1 +1703,2 +1704,1 +1705,1 +1706,2 +1707,1 +1708,1 +1710,1 +1712,2 +1714,2 +1715,3 +1716,3 +1718,2 +1719,2 +1720,1 +1721,1 +1722,4 +1723,1 +1724,2 +1726,2 +1728,1 +1729,1 +1730,3 +1731,2 +1732,2 +1733,2 +1734,2 +1735,1 +1736,5 +1737,2 +1738,1 +1739,2 +1740,2 +1741,3 +1742,5 +1743,5 +1744,1 +1745,2 +1746,1 +1747,1 +1748,2 +1749,2 +1750,2 +1751,2 +1753,1 +1754,1 +1755,2 +1756,2 +1757,1 +1758,3 +1759,3 +1760,2 +1761,2 +1762,1 +1763,2 +1764,1 +1766,4 +1767,1 +1768,1 +1769,1 +1770,1 +1771,1 +1772,3 +1774,2 +1775,2 +1777,3 +1779,2 +1780,1 +1781,3 +1782,2 +1783,1 +1785,2 +1789,5 +1790,3 +1791,5 +1792,4 +1793,1 +1794,3 +1795,4 +1796,3 +1797,1 +1798,5 +1799,3 +1800,4 +1802,1 +1803,1 +1805,2 +1806,1 +1807,3 +1809,2 +1810,1 +1811,1 +1812,2 +1813,2 +1814,4 +1817,1 +1819,2 +1820,1 +1824,1 +1825,1 +1826,3 +1827,1 +1833,3 +1844,3 +1845,1 +1846,1 +1847,1 +1848,1 +1853,2 +1854,1 +1855,2 +1859,2 +1861,2 +1862,1 +1865,1 +1867,1 +1875,2 +1881,2 +1882,1 +1884,2 +1889,1 +1890,1 +1891,3 +1892,2 +1895,1 +1897,1 +1898,1 +1899,1 +1902,1 +1906,1 +1908,1 +1911,1 +1916,1 +1918,1 +1922,1 +1925,1 +1926,1 +1927,1 +1928,1 +1930,2 +1931,1 +1932,2 +1934,1 +1935,1 +1937,2 +1941,1 +1943,1 +1945,2 +1953,1 +1954,1 +1957,1 +1961,1 +1967,1 +1972,1 +1976,1 +1981,1 +1982,1 +1984,2 +1985,1 +1988,1 +1993,2 +1996,1 +1998,1 +1999,1 +2002,1 +2007,1 +2012,1 +2014,1 +2016,1 +2017,2 +2018,2 +2020,1 +2022,1 +2023,2 +2025,2 +2026,2 +2028,4 +2029,2 +2030,2 +2031,2 +2032,1 +2033,4 +2034,4 +2035,5 +2036,1 +2037,6 +2039,2 +2040,1 +2041,4 +2042,2 +2043,3 +2044,1 +2045,1 +2046,1 +2049,2 +2050,1 +2051,1 +2052,6 +2053,1 +2054,1 +2055,3 +2057,1 +2059,1 +2060,1 +2061,1 +2062,2 +2063,3 +2064,2 +2065,1 +2066,4 +2067,2 +2068,1 +2069,2 +2071,1 +2075,1 +2076,2 +2079,1 +2080,1 +2082,2 +2083,1 +2084,1 +2085,2 +2086,2 +2088,1 +2089,1 +2093,1 +2094,1 +2096,1 +2098,1 +2100,1 +2101,1 +2103,2 +2104,2 +2105,1 +2108,2 +2111,3 +2112,3 +2113,4 +2114,4 +2117,1 +2119,1 +2120,1 +2121,3 +2125,2 +2126,3 +2127,2 +2129,3 +2131,5 +2133,2 +2134,2 +2135,2 +2136,1 +2138,2 +2139,2 +2140,1 +2141,1 +2143,2 +2144,2 +2147,2 +2148,3 +2150,1 +2151,1 +2152,1 +2153,3 +2154,1 +2155,1 +2156,1 +2157,1 +2158,2 +2159,2 +2160,1 +2161,1 +2162,1 +2163,1 +2165,1 +2167,2 +2169,2 +2170,2 +2171,4 +2172,1 +2173,3 +2176,1 +2179,1 +2180,1 +2181,3 +2182,1 +2183,3 +2184,2 +2185,3 +2186,1 +2187,1 +2188,4 +2190,4 +2191,1 +2192,3 +2194,4 +2197,2 +2198,1 +2202,1 +2203,1 +2205,2 +2207,1 +2209,3 +2210,3 +2213,1 +2214,1 +2215,1 +2216,2 +2217,2 +2218,1 +2219,1 +2220,2 +2221,1 +2224,2 +2225,1 +2226,1 +2228,2 +2229,1 +2230,1 +2232,3 +2233,1 +2234,1 +2236,1 +2239,1 +2240,2 +2242,6 +2245,2 +2246,1 +2247,2 +2249,1 +2251,2 +2252,2 +2253,1 +2254,4 +2255,6 +2256,2 +2257,5 +2258,2 +2259,2 +2260,3 +2261,3 +2262,3 +2263,2 +2264,5 +2265,8 +2266,3 +2267,2 +2268,3 +2269,3 +2271,5 +2272,3 +2273,9 +2274,3 +2275,6 +2276,1 +2277,3 +2278,2 +2279,5 +2280,3 +2281,6 +2282,3 +2283,5 +2284,3 +2285,4 +2286,3 +2287,6 +2288,4 +2289,3 +2290,2 +2291,2 +2292,2 +2293,1 +2295,1 +2297,3 +2299,2 +2301,1 +2302,2 +2303,3 +2304,1 +2305,1 +2306,1 +2307,1 +2308,2 +2309,1 +2310,1 +2312,2 +2313,2 +2314,1 +2315,1 +2316,1 +2317,2 +2326,1 +2327,1 +2329,2 +2331,3 +2332,1 +2333,1 +2334,1 +2335,2 +2336,2 +2337,3 +2338,1 +2339,1 +2341,1 +2342,1 +2343,2 +2344,1 +2346,1 +2349,1 +2351,1 +2352,2 +2356,3 +2357,1 +2358,2 +2361,1 +2362,1 +2363,1 +2364,1 +2365,1 +2366,2 +2367,1 +2369,2 +2372,4 +2373,1 +2376,1 +2378,1 +2379,4 +2381,1 +2382,1 +2384,1 +2385,1 +2386,4 +2388,1 +2389,1 +2390,2 +2391,1 +2392,1 +2394,1 +2396,1 +2397,1 +2400,1 +2402,4 +2404,1 +2405,1 +2407,1 +2409,3 +2410,1 +2411,1 +2412,1 +2413,1 +2416,2 +2417,2 +2418,1 +2428,1 +2429,2 +2434,1 +2435,1 +2436,1 +2440,2 +2441,1 +2446,1 +2447,1 +2453,1 +2454,1 +2456,1 +2459,1 +2460,4 +2464,1 +2470,1 +2472,1 +2473,1 +2478,1 +2479,1 +2485,1 +2486,1 +2491,2 +2493,1 +2494,2 +2497,1 +2500,1 +2501,1 +2503,1 +2504,1 +2507,2 +2508,1 +2510,1 +2512,1 +2520,1 +2521,2 +2524,3 +2532,3 +2533,3 +2536,1 +2537,2 +2538,1 +2539,1 +2542,1 +2543,6 +2544,1 +2545,1 +2550,1 +2551,1 +2552,2 +2553,1 +2556,1 +2558,1 +2571,1 +2572,1 +2573,3 +2574,1 +2575,1 +2576,3 +2579,1 +2580,1 +2583,1 +2585,1 +2587,1 +2588,2 +2591,1 +2592,1 +2596,1 +2597,1 +2598,2 +2601,1 +2605,1 +2606,1 +2609,1 +2610,1 +2612,1 +2613,1 +2614,3 +2615,1 +2617,1 +2622,1 +2624,4 +2626,1 +2627,3 +2630,6 +2631,3 +2632,2 +2633,1 +2634,2 +2635,2 +2636,3 +2637,2 +2638,2 +2639,2 +2640,2 +2641,2 +2642,3 +2643,3 +2644,4 +2645,6 +2646,3 +2647,2 +2648,3 +2649,6 +2650,4 +2652,3 +2653,3 +2654,4 +2656,1 +2658,5 +2659,1 +2660,2 +2661,2 +2662,4 +2663,3 +2664,4 +2665,1 +2666,1 +2667,1 +2669,2 +2670,3 +2672,2 +2675,1 +2676,4 +2677,2 +2678,3 +2679,1 +2680,2 +2681,1 +2682,1 +2683,1 +2687,3 +2688,2 +2690,2 +2692,1 +2694,1 +2695,1 +2698,2 +2699,1 +2702,1 +2703,2 +2705,1 +2706,2 +2709,1 +2712,1 +2713,3 +2714,2 +2717,1 +2718,2 +2719,3 +2720,2 +2721,3 +2722,1 +2723,1 +2724,2 +2725,3 +2726,1 +2727,2 +2728,3 +2729,3 +2730,2 +2735,1 +2736,1 +2737,1 +2741,2 +2743,2 +2745,2 +2746,2 +2747,1 +2748,2 +2751,1 +2753,1 +2755,1 +2757,1 +2758,1 +2762,1 +2763,3 +2765,1 +2767,2 +2768,1 +2769,1 +2770,1 +2771,2 +2773,1 +2775,1 +2778,2 +2780,1 +2786,1 +2788,1 +2789,1 +2792,4 +2793,1 +2796,2 +2798,1 +2803,1 +2804,2 +2805,2 +2806,1 +2810,1 +2812,2 +2813,1 +2818,2 +2822,1 +2825,1 +2828,1 +2829,1 +2830,1 +2832,1 +2835,1 +2840,1 +2841,1 +2843,1 +2845,1 +2847,1 +2849,2 +2851,1 +2852,1 +2857,1 +2867,2 +2868,1 +2871,1 +2874,2 +2877,1 +2882,1 +2883,2 +2885,1 +2887,1 +2891,3 +2898,1 +2905,2 +2906,1 +2909,2 +2911,1 +2915,1 +2917,1 +2918,2 +2924,1 +2925,3 +2926,1 +2928,2 +2933,1 +2941,1 +2950,1 +2951,1 +2952,2 +2956,1 +2957,1 +2960,1 +2962,1 +2963,2 +2969,1 +2975,1 +2977,2 +2983,2 +2986,1 +2987,1 +2994,1 +2996,1 +2997,2 +2998,1 +3003,2 +3009,1 +3011,1 +3015,1 +3022,1 +3023,1 +3027,2 +3029,1 +3030,1 +3031,1 +3035,1 +3036,2 +3037,1 +3040,1 +3041,3 +3042,1 +3043,2 +3048,1 +3050,1 +3052,1 +3053,1 +3054,1 +3060,1 +3063,2 +3066,2 +3067,1 +3069,1 +3072,1 +3075,1 +3079,1 +3089,1 +3094,2 +3096,1 +3100,2 +3101,1 +3104,1 +3108,1 +3110,1 +3111,1 +3116,1 +3117,1 +3118,1 +3121,1 +3123,1 +3124,2 +3129,2 +3132,1 +3136,1 +3139,1 +3140,1 +3143,1 +3150,2 +3151,1 +3155,2 +3158,1 +3163,1 +3166,1 +3168,1 +3173,1 +3174,1 +3179,1 +3182,1 +3183,2 +3190,1 +3197,1 +3199,1 +3200,1 +3207,1 +3209,1 +3210,1 +3212,1 +3214,3 +3215,4 +3216,2 +3217,3 +3219,1 +3221,2 +3222,2 +3223,2 +3224,1 +3226,1 +3227,1 +3228,4 +3230,1 +3231,4 +3232,2 +3235,1 +3238,1 +3240,1 +3241,1 +3243,1 +3244,1 +3250,1 +3257,1 +3269,1 +3270,1 +3271,1 +3274,2 +3275,1 +3282,1 +3289,1 +3290,2 +3291,2 +3293,1 +3294,2 +3295,1 +3296,2 +3297,3 +3298,1 +3299,1 +3300,1 +3302,2 +3304,1 +3305,1 +3306,3 +3311,1 +3312,1 +3315,1 +3318,2 +3320,1 +3323,1 +3325,1 +3327,1 +3328,1 +3330,1 +3332,1 +3335,1 +3336,2 +3338,2 +3339,1 +3341,1 +3343,1 +3345,1 +3349,1 +3350,3 +3352,4 +3354,2 +3355,1 +3357,2 +3358,1 +3359,1 +3361,1 +3363,1 +3364,3 +3365,1 +3366,1 +3367,1 +3368,2 +3369,1 +3372,1 +3374,1 +3379,1 +3383,1 +3384,1 +3387,2 +3388,2 +3391,2 +3392,1 +3393,1 +3395,2 +3401,1 +3402,2 +3405,1 +3407,4 +3408,1 +3409,1 +3411,2 +3414,1 +3417,1 +3419,1 +3425,1 +3429,1 +3430,2 +3436,3 +3438,1 +3439,1 +3443,1 +3446,2 +3447,2 +3448,1 +3449,4 +3450,1 +3452,1 +3453,1 +3454,1 +3455,2 +3457,1 +3459,1 +3465,1 +3467,1 +3469,2 +3471,1 +3475,2 +3481,1 +3484,1 +3485,1 +3488,1 +3490,10 +3494,1 +3497,1 +3501,3 +3503,1 +3512,1 +3514,2 +3519,1 +3523,1 +3524,1 +3526,3 +3527,1 +3530,1 +3531,1 +3532,1 +3534,2 +3535,1 +3537,2 +3538,1 +3539,2 +3541,2 +3554,1 +3558,1 +3559,1 +3565,3 +3566,1 +3568,1 +3569,1 +3572,1 +3573,1 +3577,1 +3578,2 +3581,2 +3587,1 +3589,1 +3590,1 +3593,1 +3598,2 +3600,1 +3601,1 +3609,1 +3611,2 +3615,2 +3622,1 +3625,1 +3626,1 +3629,1 +3637,1 +3642,1 +3643,1 +3645,1 +3649,1 +3651,1 +3652,1 +3656,1 +3662,1 +3668,1 +3669,1 +3670,1 +3672,1 +3673,1 +3674,1 +3676,1 +3677,1 +3681,1 +3683,1 +3686,1 +3687,1 +3688,1 +3696,2 +3697,1 +3701,1 +3703,1 +3707,1 +3709,1 +3711,1 +3712,1 +3713,3 +3715,2 +3716,1 +3718,1 +3722,1 +3728,2 +3734,1 +3735,1 +3738,1 +3741,1 +3745,1 +3747,1 +3752,1 +3758,2 +3763,2 +3766,1 +3768,1 +3771,2 +3772,1 +3774,1 +3776,1 +3783,1 +3784,1 +3785,1 +3790,1 +3798,1 +3799,1 +3800,1 +3807,1 +3813,1 +3815,1 +3818,1 +3819,1 +3823,1 +3824,1 +3827,1 +3830,1 +3831,1 +3834,1 +3836,1 +3841,1 +3844,2 +3853,1 +3854,1 +3857,1 +3859,1 +3860,2 +3868,1 +3879,1 +3880,1 +3881,1 +3884,1 +3886,1 +3888,1 +3890,1 +3894,1 +3899,1 +3905,1 +3906,2 +3915,1 +3926,1 +3930,1 +3936,2 +3938,2 +3941,1 +3943,1 +3955,1 +3956,1 +3960,1 +3967,1 +3971,1 +3973,1 +4007,1 +4012,1 +4015,1 +4016,1 +4019,3 +4020,1 +4022,1 +4025,1 +4034,1 +4035,1 +4040,1 +4048,1 +4050,2 +4052,1 +4054,1 +4056,1 +4057,2 +4059,1 +4062,1 +4063,1 +4064,1 +4068,1 +4069,2 +4070,1 +4071,1 +4074,1 +4075,1 +4076,3 +4079,1 +4081,1 +4082,1 +4083,1 +4087,1 +4091,1 +4092,1 +4093,2 +4095,1 +4098,1 +4101,1 +4105,1 +4107,1 +4112,1 +4117,1 +4118,2 +4120,2 +4121,2 +4122,1 +4123,1 +4126,3 +4127,1 +4128,1 +4129,1 +4135,2 +4138,1 +4139,1 +4140,1 +4142,1 +4147,1 +4149,1 +4152,2 +4154,1 +4156,2 +4158,1 +4159,1 +4161,1 +4163,1 +4164,1 +4165,2 +4166,1 +4167,1 +4168,1 +4169,1 +4171,1 +4173,1 +4176,1 +4178,2 +4181,2 +4183,2 +4184,2 +4186,1 +4187,2 +4188,1 +4192,1 +4195,2 +4196,1 +4200,1 +4209,1 +4210,1 +4211,2 +4212,1 +4213,2 +4214,1 +4216,1 +4218,1 +4219,1 +4222,1 +4223,3 +4224,2 +4225,1 +4226,1 +4227,1 +4229,1 +4230,3 +4231,1 +4233,1 +4238,1 +4239,1 +4241,1 +4243,3 +4244,1 +4245,1 +4247,1 +4249,1 +4251,1 +4252,1 +4253,2 +4255,1 +4256,1 +4265,1 +4267,1 +4268,2 +4270,1 +4281,1 +4282,1 +4283,2 +4286,1 +4292,2 +4298,2 +4301,1 +4304,1 +4305,2 +4308,2 +4309,1 +4310,1 +4311,2 +4312,2 +4314,1 +4315,1 +4316,2 +4319,1 +4320,2 +4323,1 +4325,2 +4326,1 +4331,1 +4336,1 +4337,1 +4339,2 +4340,1 +4341,1 +4342,1 +4345,1 +4349,3 +4350,1 +4351,2 +4353,3 +4355,1 +4358,1 +4359,1 +4362,1 +4363,1 +4364,1 +4365,2 +4366,1 +4367,1 +4368,1 +4370,1 +4373,1 +4375,2 +4376,1 +4377,1 +4378,3 +4381,1 +4387,1 +4391,2 +4396,1 +4397,1 +4398,2 +4399,1 +4401,2 +4402,1 +4404,1 +4407,1 +4408,2 +4409,1 +4412,1 +4414,1 +4415,1 +4416,1 +4417,4 +4420,1 +4422,1 +4424,2 +4425,1 +4426,2 +4427,2 +4428,1 +4431,1 +4436,1 +4440,1 +4441,1 +4442,1 +4447,2 +4448,1 +4450,1 +4451,1 +4453,1 +4457,3 +4458,1 +4459,1 +4462,1 +4464,1 +4466,1 +4467,1 +4468,3 +4472,4 +4479,1 +4482,1 +4484,3 +4488,1 +4493,1 +4494,1 +4495,1 +4500,2 +4501,1 +4503,3 +4504,1 +4505,1 +4506,2 +4507,1 +4509,1 +4515,1 +4516,3 +4517,1 +4518,2 +4522,2 +4524,2 +4525,1 +4528,1 +4539,3 +4542,1 +4543,2 +4550,1 +4551,4 +4552,1 +4553,1 +4559,3 +4565,1 +4566,1 +4569,1 +4571,1 +4572,1 +4574,1 +4575,2 +4576,1 +4578,1 +4580,1 +4583,1 +4585,1 +4586,1 +4587,2 +4588,1 +4590,1 +4591,1 +4592,1 +4594,3 +4596,1 +4597,2 +4598,1 +4599,2 +4600,1 +4604,1 +4606,2 +4607,1 +4608,1 +4609,1 +4610,1 +4613,1 +4617,1 +4618,1 +4621,1 +4625,1 +4630,1 +4642,1 +4651,1 +4657,1 +4660,1 +4661,1 +4670,1 +4674,2 +4675,2 +4676,1 +4677,2 +4679,1 +4681,2 +4682,2 +4683,4 +4687,1 +4689,1 +4691,3 +4695,1 +4699,1 +4705,1 +4706,1 +4712,3 +4717,1 +4718,1 +4719,2 +4723,1 +4724,4 +4729,1 +4731,1 +4732,1 +4733,2 +4737,1 +4738,1 +4743,1 +4745,1 +4746,1 +4749,1 +4750,4 +4751,1 +4753,1 +4755,1 +4761,1 +4762,2 +4764,1 +4772,1 +4773,1 +4780,1 +4786,1 +4787,1 +4788,1 +4789,1 +4792,2 +4794,1 +4795,1 +4798,1 +4801,1 +4812,1 +4813,2 +4823,1 +4825,3 +4831,1 +4833,2 +4834,1 +4839,1 +4843,1 +4859,1 +4863,1 +4869,1 +4873,1 +4876,1 +4879,1 +4881,1 +4883,1 +4889,1 +4890,1 +4891,3 +4892,1 +4893,1 +4896,1 +4897,1 +4903,1 +4904,2 +4907,1 +4909,1 +4910,2 +4913,1 +4914,1 +4917,1 +4919,1 +4920,1 +4921,4 +4922,2 +4926,1 +4927,1 +4930,1 +4932,1 +4935,1 +4936,1 +4940,1 +4941,1 +4944,1 +4945,1 +4946,1 +4947,1 +4951,1 +4953,2 +4956,2 +4959,1 +4968,1 +4971,1 +4972,1 +4973,1 +4975,1 +4977,1 +4980,1 +4983,1 +4984,3 +4986,1 +4987,2 +4988,2 +4989,2 +4991,2 +4992,1 +4993,1 +4996,1 +4997,1 +4999,1 +5002,1 +5007,1 +5009,1 +5015,3 +5017,2 +5019,1 +5020,1 +5023,2 +5024,3 +5025,1 +5026,2 +5027,2 +5028,2 +5030,1 +5034,2 +5037,5 +5038,1 +5041,3 +5043,1 +5046,1 +5049,1 +5051,1 +5053,1 +5059,1 +5060,1 +5065,1 +5066,2 +5069,1 +5070,1 +5072,1 +5073,1 +5075,1 +5077,1 +5080,1 +5081,1 +5083,2 +5087,1 +5089,1 +5092,2 +5093,1 +5094,3 +5095,2 +5096,1 +5102,1 +5103,1 +5104,1 +5108,1 +5113,1 +5115,2 +5118,1 +5120,1 +5126,3 +5128,1 +5129,4 +5132,1 +5137,2 +5139,1 +5141,1 +5145,1 +5149,2 +5150,1 +5158,1 +5159,1 +5164,2 +5172,1 +5178,1 +5185,1 +5187,1 +5195,1 +5211,2 +5227,1 +5231,1 +5232,1 +5233,1 +5243,1 +5254,1 +5257,1 +5264,1 +5267,1 +5268,1 +5269,1 +5282,1 +5283,1 +5286,1 +5291,1 +5292,1 +5296,1 +5297,1 +5298,1 +5299,1 +5303,1 +5304,1 +5306,1 +5307,1 +5309,2 +5312,1 +5324,1 +5344,1 +5345,1 +5349,1 +5353,1 +5358,1 +5360,1 +5362,1 +5364,2 +5365,1 +5366,2 +5368,1 +5369,2 +5371,1 +5373,1 +5384,1 +5392,1 +5393,1 +5397,1 +5398,1 +5421,1 +5430,1 +5432,1 +5435,1 +5443,1 +5448,1 +5453,3 +5455,1 +5464,1 +5473,1 +5474,1 +5478,1 +5479,1 +5486,1 +5487,1 +5495,1 +5496,3 +5498,1 +5501,1 +5503,1 +5507,1 +5512,1 +5518,2 +5520,1 +5525,1 +5529,1 +5537,2 +5539,2 +5543,1 +5544,1 +5545,1 +5546,1 +5549,1 +5550,2 +5552,1 +5553,1 +5555,1 +5559,1 +5561,2 +5570,1 +5573,1 +5574,1 +5577,1 +5578,1 +5579,1 +5580,1 +5581,1 +5582,2 +5589,1 +5594,1 +5596,1 +5598,2 +5599,1 +5603,1 +5607,2 +5609,2 +5610,1 +5612,1 +5616,1 +5617,1 +5622,1 +5667,1 +5669,1 +5670,1 +5695,1 +5703,1 +5705,1 +5707,1 +5725,1 +5726,1 +5727,1 +5728,1 +5756,1 +5763,1 +5769,1 +5770,1 +5781,1 +5790,1 +5791,1 +5808,1 +5816,1 +5827,1 +5829,1 +5831,1 +5833,1 +5838,1 +5843,1 +5848,1 +5853,1 +5854,1 +5860,1 +5861,1 +5864,1 +5868,1 +5871,1 +5882,1 +5885,1 +5886,1 +5899,1 +5906,2 +5911,1 +5944,1 +5976,1 +5998,1 +6005,1 +6016,1 +6023,2 +6035,1 +6044,1 +6047,1 +6054,1 +6063,1 +6066,1 +6069,1 +6083,2 +6084,2 +6097,1 +6114,1 +6115,2 +6132,1 +6137,1 +6138,1 +6139,1 +6140,1 +6155,2 +6160,1 +6170,1 +6177,1 +6179,2 +6184,1 +6188,1 +6191,1 +6193,1 +6199,1 +6200,1 +6204,1 +6206,1 +6208,1 +6212,1 +6218,2 +6224,1 +6230,1 +6237,2 +6244,1 +6246,1 +6253,1 +6259,1 +6283,1 +6291,1 +6294,3 +6301,1 +6303,3 +6304,1 +6308,1 +6313,1 +6315,1 +6316,2 +6318,1 +6324,1 +6345,1 +6347,1 +6352,1 +6353,1 +6354,1 +6357,1 +6358,1 +6360,1 +6364,1 +6366,1 +6370,1 +6372,1 +6378,1 +6379,2 +6382,1 +6396,1 +6398,1 +6401,1 +6406,2 +6407,1 +6408,1 +6411,1 +6419,1 +6426,1 +6430,1 +6435,1 +6438,1 +6439,1 +6441,1 +6448,1 +6453,2 +6454,1 +6455,2 +6457,1 +6466,1 +6479,1 +6480,1 +6481,2 +6487,1 +6489,1 +6490,1 +6491,1 +6503,1 +6506,1 +6508,1 +6514,1 +6520,1 +6534,1 +6536,1 +6551,1 +6555,1 +6557,1 +6558,1 +6563,1 +6564,1 +6570,1 +6571,1 +6574,1 +6575,1 +6579,2 +6584,1 +6590,1 +6591,1 +6592,1 +6595,1 +6609,2 +6613,1 +6617,1 +6618,1 +6625,1 +6630,1 +6632,2 +6637,1 +6640,1 +6645,1 +6657,1 +6658,1 +6662,1 +6663,1 +6675,1 +6692,1 +6697,1 +6698,1 +6707,1 +6708,1 +6710,1 +6713,1 +6714,1 +6715,1 +6718,1 +6727,1 +6734,1 +6737,1 +6738,2 +6745,1 +6751,1 +6753,1 +6755,1 +6757,2 +6762,1 +6766,1 +6774,1 +6780,1 +6785,1 +6804,1 +6808,1 +6810,2 +6813,1 +6830,1 +6837,1 +6851,1 +6876,1 +6893,1 +6904,1 +6906,1 +6910,1 +6926,2 +6927,1 +6937,1 +6942,1 +6949,1 +6950,1 +6952,1 +6956,1 +6959,1 +6965,1 +6966,1 +6970,3 +6973,1 +6986,1 +6992,1 +6998,1 +7002,1 +7004,1 +7007,1 +7008,1 +7012,1 +7018,1 +7022,1 +7023,1 +7031,1 +7039,1 +7040,1 +7041,1 +7042,2 +7043,2 +7044,1 +7045,2 +7047,1 +7050,1 +7053,1 +7054,1 +7055,1 +7058,1 +7063,1 +7069,1 +7070,1 +7071,1 +7072,1 +7073,2 +7076,1 +7078,1 +7094,1 +7095,1 +7097,1 +7100,1 +7108,1 +7131,1 +7142,2 +7144,1 +7146,1 +7152,1 +7153,1 +7160,1 +7165,1 +7171,2 +7178,2 +7179,1 +7180,1 +7190,1 +7193,1 +7198,1 +7201,1 +7218,1 +7220,1 +7222,1 +7229,1 +7242,1 +7246,1 +7250,1 +7265,1 +7269,1 +7293,1 +7304,1 +7305,1 +7340,1 +7347,1 +7349,1 +7350,1 +7353,1 +7368,1 +7372,1 +7389,1 +7391,1 +7396,1 +7419,2 +7420,1 +7426,1 +7430,1 +7435,2 +7448,1 +7456,1 +7463,1 +7466,1 +7476,1 +7478,1 +7480,1 +7484,1 +7489,1 +7494,1 +7499,1 +7503,2 +7504,1 +7513,1 +7516,1 +7534,1 +7538,1 +7546,1 +7556,1 +7558,1 +7565,2 +7587,1 +7595,1 +7599,1 +7602,1 +7608,1 +7621,1 +7638,2 +7641,1 +7645,1 +7649,2 +7651,1 +7674,1 +7676,2 +7677,1 +7690,1 +7696,1 +7701,1 +7703,1 +7705,1 +7707,1 +7710,1 +7713,1 +7714,1 +7720,1 +7733,1 +7735,1 +7743,1 +7744,1 +7780,1 +7802,1 +7812,1 +7817,1 +7820,1 +7832,1 +7851,1 +7866,1 +7878,3 +7879,1 +7880,1 +7890,2 +7894,2 +7908,1 +7910,1 +7916,1 +7923,1 +7925,1 +7944,1 +7945,1 +7954,1 +7956,1 +7980,1 +8003,1 +8010,1 +8023,1 +8048,2 +8049,1 +8061,1 +8096,1 +8101,1 +8139,1 +8160,1 +8196,1 +8197,1 +8202,1 +8221,2 +8223,3 +8228,1 +8233,2 +8237,1 +8238,1 +8241,1 +8252,2 +8262,1 +8266,1 +8271,1 +8273,1 +8279,1 +8294,1 +8302,1 +8306,1 +8308,1 +8319,1 +8320,1 +8321,1 +8332,2 +8335,1 +8344,1 +8355,1 +8378,1 +8380,1 +8403,1 +8426,1 +8437,1 +8456,1 +8457,1 +8470,1 +8482,1 +8485,1 +8491,1 +8492,1 +8503,1 +8506,2 +8507,1 +8510,1 +8520,1 +8523,1 +8534,1 +8560,1 +8561,1 +8565,1 +8574,1 +8599,1 +8616,1 +8633,1 +8634,1 +8646,1 +8653,1 +8659,1 +8666,1 +8667,1 +8671,2 +8672,1 +8692,1 +8694,1 +8695,1 +8721,3 +8722,1 +8730,1 +8738,1 +8742,1 +8758,1 +8783,1 +8785,1 +8787,1 +8849,1 +8869,1 +8874,1 +8887,1 +8893,1 +8914,1 +8921,1 +8937,2 +8973,1 +8992,1 +9004,1 +9008,1 +9009,1 +9013,1 +9023,1 +9029,1 +9031,1 +9056,1 +9058,2 +9059,1 +9061,1 +9062,1 +9069,1 +9074,1 +9075,1 +9077,1 +9078,1 +9094,1 +9098,1 +9103,1 +9104,1 +9110,1 +9123,1 +9124,1 +9158,1 +9186,1 +9191,2 +9212,1 +9215,2 +9223,1 +9242,1 +9248,1 +9250,1 +9252,1 +9255,1 +9260,1 +9283,1 +9287,1 +9291,1 +9298,2 +9308,1 +9312,1 +9321,1 +9330,1 +9356,1 +9357,1 +9362,1 +9366,1 +9386,1 +9393,1 +9407,1 +9482,1 +9497,1 +9514,1 +9536,1 +9542,2 +9543,1 +9565,1 +9572,1 +9576,1 +9607,1 +9659,1 +9662,2 +9672,1 +9679,1 +9683,1 +9721,1 +9799,1 +9806,1 +9807,1 +9823,1 +9839,2 +9861,1 +9884,1 +9933,1 +9937,1 +9938,1 +9952,1 +9953,1 +9961,1 +9964,1 +9970,1 +9980,1 +9981,1 +9982,1 +9990,1 +9996,1 +9998,1 +10000,1 +10007,1 +10010,1 +10027,1 +10028,1 +10037,1 +10055,1 +10062,2 +10063,2 +10066,1 +10069,1 +10072,1 +10074,1 +10080,1 +10083,2 +10095,2 +10104,1 +10119,1 +10125,1 +10132,1 +10134,2 +10139,1 +10141,1 +10146,3 +10172,1 +10174,1 +10197,1 +10201,1 +10202,1 +10205,1 +10210,1 +10212,1 +10216,2 +10228,1 +10234,1 +10278,1 +10290,1 +10296,1 +10330,1 +10332,1 +10335,1 +10354,1 +10394,1 +10397,1 +10425,1 +10433,1 +10491,1 +10505,1 +10506,3 +10509,1 +10513,1 +10537,1 +10541,1 +10560,1 +10561,1 +10575,1 +10576,3 +10577,1 +10578,1 +10579,1 +10582,1 +10585,1 +10588,1 +10618,2 +10620,1 +10621,1 +10627,1 +10629,1 +10632,1 +10645,1 +10647,1 +10668,1 +10680,1 +10681,2 +10685,1 +10729,1 +10733,1 +10738,1 +10740,1 +10776,1 +10881,1 +10893,1 +10991,1 +11033,1 +11056,2 +11065,1 +11074,1 +11081,1 +11085,1 +11121,1 +11176,1 +11177,1 +11201,1 +11240,1 +11244,1 +11278,1 +11308,1 +11365,1 +11383,1 +11407,1 +11410,1 +11427,1 +11481,1 +11487,1 +11492,1 +11535,1 +11571,1 +11597,1 +11633,1 +11653,1 +11655,1 +11662,1 +11718,1 +11743,1 +11757,1 +11793,1 +11803,1 +11809,2 +11825,1 +11830,1 +11835,1 +11840,1 +11863,1 +11986,1 +12060,1 +12062,1 +12076,1 +12097,1 +12105,1 +12112,1 +12116,1 +12151,1 +12153,1 +12182,1 +12192,1 +12238,1 +12300,1 +12322,1 +12545,1 +12568,1 +12569,1 +12577,1 +12579,1 +12583,1 +12594,1 +12606,1 +12627,1 +12629,1 +12631,1 +12638,1 +12641,1 +12643,1 +12652,1 +12672,1 +12690,1 +12700,1 +12773,1 +12798,2 +12818,1 +12844,1 +12884,1 +12933,1 +12946,1 +12961,1 +12987,1 +13204,1 +13226,1 +13242,1 +13259,1 +13327,1 +13333,1 +13342,1 +13358,1 +13371,1 +13406,1 +13442,1 +13443,2 +13468,1 +13474,2 +13487,1 +13490,1 +13496,1 +13498,1 +13531,1 +13541,1 +13547,1 +13550,1 +13553,2 +13556,1 +13561,1 +13562,1 +13563,2 +13570,1 +13571,1 +13582,1 +13610,1 +13612,1 +13645,1 +13746,1 +13800,1 +13820,1 +13828,1 +13829,1 +13831,2 +13836,1 +13841,1 +13911,1 +13915,1 +13918,1 +13920,1 +13928,1 +13984,1 +13988,2 +14002,1 +14007,1 +14017,1 +14021,1 +14047,1 +14050,1 +14085,1 +14142,1 +14173,1 +14181,1 +14190,1 +14192,1 +14194,1 +14200,1 +14202,1 +14212,1 +14217,1 +14218,1 +14230,1 +14239,1 +14398,1 +14409,1 +14437,1 +14471,1 +14565,1 +14593,1 +14762,1 +14832,1 +14841,1 +14865,1 +14866,2 +14885,1 +14886,1 +14891,1 +14906,1 +14947,1 +15034,1 +15037,1 +15043,1 +15065,1 +15088,1 +15138,2 +15139,1 +15172,1 +15184,1 +15244,1 +15257,1 +15262,1 +15263,2 +15271,1 +15286,1 +15300,1 +15348,1 +15349,1 +15363,1 +15367,1 +15385,1 +15388,1 +15399,1 +15421,1 +15536,1 +15573,2 +15574,1 +15580,1 +15582,1 +15584,1 +15610,1 +15629,1 +15639,2 +15641,1 +15649,1 +15670,1 +15678,1 +15688,1 +15707,1 +15724,1 +15729,1 +15749,1 +15778,1 +15786,1 +15794,1 +15804,1 +15811,1 +15823,1 +15825,1 +15842,1 +15851,1 +15857,1 +15860,2 +15861,1 +15864,1 +15870,1 +15873,1 +15874,1 +15877,1 +15880,1 +15882,1 +15889,1 +15896,1 +15901,1 +15904,1 +15934,1 +15939,1 +15952,1 +15956,1 +15957,1 +15958,1 +15962,2 +15976,1 +15985,1 +16018,1 +16046,1 +16057,1 +16059,1 +16062,1 +16066,1 +16113,2 +16117,1 +16128,1 +16140,2 +16144,1 +16160,2 +16166,1 +16187,1 +16206,1 +16215,1 +16216,1 +16217,1 +16222,1 +16236,1 +16248,1 +16251,2 +16256,1 +16257,1 +16259,1 +16280,1 +16290,1 +16304,1 +16307,1 +16345,1 +16347,1 +16350,1 +16356,1 +16359,1 +16371,1 +16375,1 +16380,1 +16394,1 +16395,1 +16399,1 +16417,1 +16429,1 +16434,1 +16441,1 +16444,1 +16451,1 +16452,1 +16453,1 +16461,1 +16462,2 +16463,1 +16468,1 +16473,1 +16475,1 +16485,1 +16486,1 +16492,2 +16496,1 +16502,1 +16512,1 +16515,1 +16532,1 +16533,1 +16534,2 +16540,1 +16542,1 +16549,1 +16554,1 +16558,1 +16559,2 +16567,1 +16579,1 +16590,1 +16592,1 +16604,1 +16606,1 +16611,1 +16622,1 +16631,1 +16632,1 +16633,2 +16643,3 +16650,1 +16652,1 +16675,1 +16698,1 +16724,1 +16728,1 +16739,1 +16748,1 +16757,1 +16790,1 +16826,1 +16830,1 +16831,1 +16859,2 +16890,1 +16912,2 +16923,1 +17022,1 +17026,1 +17030,1 +17033,1 +17037,1 +17039,1 +17088,1 +17133,1 +17170,1 +17171,1 +17174,1 +17196,1 +17211,1 +17253,1 +17297,1 +17326,1 +17360,1 +17448,1 +17469,1 +17476,1 +17552,1 +17613,1 +17633,1 +17641,1 +17677,1 +17763,1 +17802,1 +17804,1 +17815,1 +17878,1 +18129,1 +18208,1 +18226,2 +18322,1 +18437,1 +18690,1 +18752,1 +18834,1 +18864,1 +18908,1 +18933,1 +18938,1 +18951,1 +18952,1 +18962,1 +18986,1 +18987,1 +18990,1 +19038,1 +19045,1 +19059,1 +19063,1 +19064,1 +19117,1 +19159,1 +19165,1 +19166,1 +19193,1 +19201,1 +19216,1 +19241,1 +19252,1 +19287,1 +19295,1 +19313,1 +19341,1 +19342,1 +19381,1 +19382,1 +19391,1 +19406,1 +19766,1 +19785,1 +19796,1 +19802,1 +19807,1 +19830,1 +19880,1 +19901,1 +19950,1 +19962,1 +20218,1 +20493,1 +20500,1 +20532,1 +20647,2 +20659,1 +20711,1 +20786,1 +20802,1 +20873,1 +20911,1 +20987,1 +21068,1 +21534,1 +21639,1 +21680,1 +21793,1 +21848,1 +22060,1 +22119,1 +22374,1 +22391,1 +22397,1 +22497,1 +22664,1 +22928,1 +22983,1 +22984,1 +22993,1 +23087,1 +23090,1 +23125,1 +23203,1 +23296,1 +23374,1 +23444,2 +23497,1 +23514,1 +23516,1 +23847,1 +24046,1 +24319,1 +24534,1 +24671,1 +25005,1 +25255,1 +25311,1 +25404,1 +25584,1 +26324,1 +26667,1 +27323,1 +27802,1 +28284,1 +28313,1 +28315,1 +28316,1 +28330,1 +28362,1 +28372,1 +28375,1 +28404,1 +28446,1 +28485,1 +28520,1 +28665,1 +28986,1 +29051,1 +29725,1 +29742,1 +30356,1 +30526,1 +30531,1 +30576,1 +30722,1 +32314,1 +32356,1 +32374,1 +32465,1 +32500,1 +32504,1 +32562,1 +32584,2 +32593,1 +32826,1 +32832,1 +32842,1 +32854,2 +32912,1 +32934,1 +32938,1 +33252,1 +33838,1 +33840,1 +34034,1 +34086,1 +34149,1 +34167,1 +34171,1 +34268,1 +34270,1 +34280,1 +34315,1 +34360,1 +34547,1 +35063,1 +35242,1 +35291,1 +35342,1 +35594,1 +35736,1 +35774,1 +35819,1 +35911,1 +36164,1 +36232,1 +36501,1 +36723,1 +36728,1 +36925,1 +37122,1 +37332,1 +37892,1 +38343,1 +38592,1 +38687,1 +38842,1 +38951,1 +39008,1 +39263,1 +39266,1 +39294,1 +39419,1 +39635,1 +39725,1 +39982,1 +40501,1 +40516,1 +41459,1 +41549,1 +41921,1 +41944,1 +42032,1 +42695,1 +42933,1 +43012,1 +43321,1 +43461,1 +43805,1 +43996,1 +44004,1 +44138,1 +44148,1 +44183,1 +44197,1 +44214,1 +44217,1 +44366,1 +44395,1 +44406,1 +44631,1 +44729,1 +45003,1 +45070,1 +45072,1 +45144,2 +45177,1 +45180,1 +45508,1 +45716,1 +45749,1 +45817,1 +45850,1 +45932,1 +46110,1 +46764,1 +46771,1 +47610,1 +47631,1 +47674,1 +47727,1 +47996,1 +48046,1 +48116,1 +48177,1 +48288,1 +48306,1 +48371,1 +48389,1 +48513,1 +48532,1 +48725,1 +48849,1 +48885,1 +48942,1 +48966,1 +49027,1 +49159,1 +49309,1 +49330,1 +49368,1 +50066,1 +50113,1 +50176,1 +50208,1 +50283,1 +50341,1 +50395,1 +50404,1 +50421,1 +50427,1 +50459,1 +50489,1 +50611,1 +50643,1 +50689,1 +50726,1 +50775,1 +50781,1 +50788,1 +50826,1 +51330,1 +51349,1 +51371,1 +51470,1 +51511,1 +51636,1 +51723,1 +52048,1 +52184,1 +52228,1 +52268,1 +52273,1 +52328,1 +52448,1 +52504,1 +52680,1 +52733,1 +52826,1 +52850,1 +52913,1 +53051,1 +53120,1 +53315,1 +53324,1 +53349,1 +53400,1 +53658,1 +53843,1 +53990,1 +54038,1 +54137,1 +54157,1 +54176,1 +54184,1 +54256,1 +54278,1 +54325,1 +54462,1 +54463,1 +54551,1 +54669,1 +54701,1 +54727,1 +54737,1 +55176,1 +55214,1 +55284,1 +55682,1 +55757,1 +55803,1 +55819,1 +55836,1 +55851,1 +55878,1 +55895,1 +55904,1 +56151,1 +56211,1 +56298,1 +56330,1 +56420,1 +56450,1 +56646,1 +56656,1 +56802,1 +56840,1 +56911,1 +57070,1 +57079,1 +57093,1 +57239,1 +57287,1 +57451,1 +57461,1 +57590,1 +57617,1 +57734,1 +57737,1 +57796,1 +57825,1 +57860,1 +57863,1 +58144,1 +58228,1 +58255,1 +58257,1 +58348,1 +58478,1 +58497,1 +58612,1 +58672,1 +58766,1 +58816,1 +58939,1 +59029,1 +59126,1 +59157,1 +59473,1 +59590,1 +59607,1 +59615,1 +59626,1 +59701,1 +59703,1 +59791,1 +59821,1 +59848,1 +59929,1 +59948,1 +60118,1 +60232,1 +60731,1 +62275,1 +62351,1 +62395,1 +62473,1 +62659,1 +63407,1 +63862,1 +63969,1 +64696,1 +64741,1 +65466,1 +67932,1 +68152,1 +68452,1 +68487,1 +68758,1 +69548,1 +70158,1 +70733,1 +71296,1 +72860,1 +72963,1 +73066,1 +73273,1 +73441,1 +73540,1 +73629,1 +74059,1 +74711,1 +74716,1 +74746,1 +74889,1 +74974,1 +75003,1 +75187,1 +75236,1 +75267,1 +75429,1 +75470,1 +75532,1 +75731,1 +75762,1 +75894,1 +75958,1 +76043,1 +76232,1 +76235,1 +76431,1 +76513,1 +76549,1 +76775,1 +76915,1 +76957,1 +77218,1 +77635,1 +77693,1 +77697,1 +77764,1 +77869,1 +77880,1 +78214,1 +78663,1 +78731,1 +79501,1 +80010,1 +80066,1 +80133,1 +80242,1 +80697,1 +80715,1 +80975,1 +80992,1 +81013,1 +81480,1 +83157,1 +86850,1 +86856,1 +87225,1 +87642,1 +87648,1 +91991,1 +92171,1 +93550,1 +93563,1 +94673,1 +94837,1 +96347,1 +97071,1 +97955,1 +98833,1 +99257,1 +99271,1 +99533,1 +99769,1 +99772,1 +100942,1 +106626,1 +106668,1 +107239,1 +107290,1 +107319,1 +107606,1 +107910,1 +108028,1 +108819,1 +109033,1 +109801,1 +112913,1 +116613,1 +117339,1 +118554,1 +118878,1 +119193,1 +119266,1 +119409,1 +119413,1 +119414,1 +119650,1 +119651,1 +119793,1 +119934,1 +121239,1 +121402,1 +122053,1 +123379,1 +123488,1 +123592,1 +123600,1 +123850,1 +124114,1 +124148,1 +124246,1 +124834,1 +124858,1 +125520,1 +125630,1 +126462,1 +126602,1 +127805,1 +130779,1 +132006,1 +132067,1 +133066,1 +135893,1 +137413,1 +140911,1 +143833,1 +143867,1 +145773,1 +145850,1 +146013,1 +146113,1 +146293,1 +146531,1 +146607,1 +146853,1 +147107,1 +147155,1 +147359,1 +147459,1 +147968,1 +147994,1 +148255,1 +148309,1 +148319,1 +148337,1 +148455,1 +148475,1 +148697,1 +148959,1 +149002,1 +149033,1 +149071,1 +149225,2 +149304,1 +149365,1 +149437,1 +149693,1 +150141,1 +150165,1 +154300,1 +155364,1 +155594,1 +158408,1 +158734,1 +158975,1 +161034,1 +161797,1 +165036,1 +168045,1 +168363,1 +168816,1 +170028,1 +170207,1 +170553,1 +171414,1 +171459,1 +172785,1 +172796,1 +173078,1 +173630,1 +182705,1 +185634,1 +185760,1 +185827,1 +188043,1 +193293,1 +194647,1 +198595,1 +198745,1 +198760,1 +200663,1 +209140,1 +214603,1 +215794,1 +216002,1 +235466,1 +249072,1 +262974,1 +263904,1 +265264,1 +266211,1 +266751,1 +268727,1 +280683,1 +283684,1 +283799,1 +283999,1 +284183,1 +284186,1 +284546,1 +284747,1 +285228,1 +285237,1 +285612,1 +285921,1 +286016,1 +286094,1 +286772,1 +287587,1 +287759,1 +287811,1 +288123,1 +288345,1 +288469,1 +288737,1 +288992,1 +289056,1 +289915,1 +289938,1 +289970,1 +290026,1 +290027,1 +312747,1 +323836,1 +325373,1 +340919,1 +341146,1 +341209,1 +369692,1 +369695,1 +369863,1 +386365,1 +387772,1 +407410,1 +407640,1 +407687,1 +408404,1 +410642,1 +411297,1 +411592,1 +412852,1 +413702,1 +414766,1 +414919,1 +415544,1 +415629,1 +416681,1 +417220,1 +418035,1 +418150,1 +418277,1 +418434,1 +418942,1 +419106,1 +420011,1 +420647,1 +429120,1 +479781,1 +497525,1 +497864,1 +640915,1 +641545,1 +646483,1 +653062,1 +653278,1 +653851,1 +654084,1 +654331,1 +661103,1 +663689,1 +664049,1 +664720,1 +664934,1 +665274,1 +668041,1 +669331,1 +669348,1 +670671,1 +766503,1 +777891,1 +831151,1 +840192,1 +2808188,1 +2814067,1 +2814282,1 +2818067,1 +2818527,1 +2818732,1 +2838249,1 +2853898,1 +2856456,1 +2868145,1 +2876542,1 +2878470,1 +2879439,1 +2884044,1 diff --git a/doc/rfc/submitqueue/tango-eval/hive-go-diff-target-histogram-20260930.sql b/doc/rfc/submitqueue/tango-eval/hive-go-diff-target-histogram-20260930.sql new file mode 100644 index 000000000..236332454 --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/hive-go-diff-target-histogram-20260930.sql @@ -0,0 +1,27 @@ +-- Exact empirical distribution of the per-diff affected-target proxy for go-code single-diff requests. +-- Same population as hive-go-diff-targets-20260930.sql (September 1–29, 2026, latest event per diff ID), +-- aggregated to one row per distinct affected-target count so no individual diff is exported. +-- QueryBuilder report ETX3KWSXp, run xeBwJha1Z; Data Central execution 38adcea7-47e2-4521-957f-e61eb47c5e89. +-- Output: hive-go-diff-target-histogram-20260930.csv (4,953 rows; 58,876 diffs; sum 148,766,796, matching the summary query). +WITH single_diff_events AS ( + SELECT msg.targetschanged + msg.targetsadded + msg.targetsremoved AS affected_targets, + ROW_NUMBER() OVER ( + PARTITION BY msg.diffids[1] + ORDER BY ts DESC, datestr DESC + ) AS latest_event + FROM rawdata_user.kafka_hp_submitqueue_request_feature_event_nodedup + WHERE datestr BETWEEN '2026-09-01' AND '2026-09-29' + AND msg.queueid = 'go' + AND CARDINALITY(msg.diffids) = 1 + AND msg.stackheight = 1 + AND msg.diffids[1] IS NOT NULL + AND msg.targetschanged IS NOT NULL + AND msg.targetsadded IS NOT NULL + AND msg.targetsremoved IS NOT NULL + AND (hadoop_isdeleted = false OR hadoop_isdeleted IS NULL) +) +SELECT affected_targets, COUNT(*) AS diffs +FROM single_diff_events +WHERE latest_event = 1 +GROUP BY affected_targets +ORDER BY affected_targets diff --git a/doc/rfc/submitqueue/tango-eval/hive-go-diff-targets-20260930.json b/doc/rfc/submitqueue/tango-eval/hive-go-diff-targets-20260930.json new file mode 100644 index 000000000..6b5938fe5 --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/hive-go-diff-targets-20260930.json @@ -0,0 +1,56 @@ +{ + "source_table": "rawdata_user.kafka_hp_submitqueue_request_feature_event_nodedup", + "data_tier": "TIER_THREE", + "period_utc_datestr": [ + "2026-09-01", + "2026-09-29" + ], + "scope": "queueid=go, stackheight=1, cardinality(diffids)=1, latest event per diff ID by ts", + "affected_target_proxy": "targetschanged + targetsadded + targetsremoved", + "presto_report_id": "mcDHURW0X", + "presto_run_id": "PEOkkp0n", + "data_central_execution_uuid": "6a86dc71-00db-4c93-a38d-b07cb221e51d", + "raw_single_diff_event_rows": 62957, + "unique_diff_count": 58876, + "total_affected_targets": 148766796, + "avg_affected_targets_per_unique_diff": 2526.781642774645, + "avg_changed": 2524.6626639037977, + "avg_added": 1.6946973299816563, + "avg_removed": 0.4242815408655479, + "approx_p50_p90_p95_p99_p999": [ + 42, + 1047, + 3327, + 34815, + 524287 + ], + "zero_target_diffs": 5942, + "at_least_100000_target_diffs": 233, + "affected_target_sum_from_at_least_100000_diffs": 98355004, + "avg_below_100000": 859.6386951554321, + "broad_tail_cross_check": { + "sql": "hive-go-diff-tail-20260930.sql", + "presto_report_id": "F1fC14EDp", + "presto_run_id": "kMpN5D03F", + "data_central_execution_uuid": "216b3b36-a5b2-48c7-9463-20af6272b804", + "unique_diffs": 58876, + "total_affected_targets": 148766796, + "broad_diffs": 233, + "broad_affected_targets": 98355004, + "avg_below_100000": 859.6386951554321 + }, + "alternative_unique_diff_and_base_count": 60614, + "alternative_unique_diff_and_base_avg": 2656.7272082357213, + "null_event_timestamps": 0, + "negative_counts": 0, + "diffs_with_repeated_events": 3056, + "diffs_with_multiple_base_shas": 1648, + "diffs_with_varying_affected_counts": 1270, + "integer_N_rounded": 2527, + "benchmark_targets": { + "N": 2527, + "2N": 5054, + "5N": 12635 + }, + "caveat": "Feature-event counters are a proxy for affected targets, not actual Tango GetChangedTargets output. Heavy tail materially inflates the arithmetic mean." +} diff --git a/doc/rfc/submitqueue/tango-eval/hive-go-diff-targets-20260930.sql b/doc/rfc/submitqueue/tango-eval/hive-go-diff-targets-20260930.sql new file mode 100644 index 000000000..59fc2e730 --- /dev/null +++ b/doc/rfc/submitqueue/tango-eval/hive-go-diff-targets-20260930.sql @@ -0,0 +1,55 @@ +-- Go SubmitQueue request-feature events, September 1–29, 2026 (complete datestr partitions). +-- Hive table: rawdata_user.kafka_hp_submitqueue_request_feature_event_nodedup +-- QueryBuilder run: mcDHURW0X / PEOkkp0n +-- Data Central execution UUID: 6a86dc71-00db-4c93-a38d-b07cb221e51d +-- One latest event per immutable diff ID; exclude stacks so one observation is one code change. +-- Affected-target proxy = changed + added + removed from feature-event fields; +-- verify producer semantics before assuming the categories cannot overlap. This is not a Tango RPC result. +WITH single_diff_events AS ( + SELECT datestr, + msg.diffids[1] AS diff_id, + msg.basesha AS base_sha, + msg.targetschanged AS targets_changed, + msg.targetsadded AS targets_added, + msg.targetsremoved AS targets_removed, + msg.targetschanged + msg.targetsadded + msg.targetsremoved AS affected_targets, + ts, + ROW_NUMBER() OVER ( + PARTITION BY msg.diffids[1] + ORDER BY ts DESC, datestr DESC + ) AS latest_event + FROM rawdata_user.kafka_hp_submitqueue_request_feature_event_nodedup + WHERE datestr BETWEEN '2026-09-01' AND '2026-09-29' + AND msg.queueid = 'go' + AND CARDINALITY(msg.diffids) = 1 + AND msg.stackheight = 1 + AND msg.diffids[1] IS NOT NULL + AND msg.targetschanged IS NOT NULL + AND msg.targetsadded IS NOT NULL + AND msg.targetsremoved IS NOT NULL + AND (hadoop_isdeleted = false OR hadoop_isdeleted IS NULL) +), combined AS ( + SELECT 'raw_events' AS grain, targets_changed, targets_added, targets_removed, affected_targets + FROM single_diff_events + UNION ALL + SELECT 'unique_diff_latest' AS grain, targets_changed, targets_added, targets_removed, affected_targets + FROM single_diff_events + WHERE latest_event = 1 +) +SELECT grain, + COUNT(*) AS observations, + SUM(affected_targets) AS sum_affected_targets, + AVG(CAST(affected_targets AS DOUBLE)) AS avg_affected_targets, + AVG(CAST(targets_changed AS DOUBLE)) AS avg_changed, + AVG(CAST(targets_added AS DOUBLE)) AS avg_added, + AVG(CAST(targets_removed AS DOUBLE)) AS avg_removed, + APPROX_PERCENTILE(affected_targets, ARRAY[0.5, 0.9, 0.95, 0.99, 0.999]) AS p50_p90_p95_p99_p999, + COUNT_IF(affected_targets = 0) AS zero_target_observations, + COUNT_IF(affected_targets >= 1000) AS ge_1000, + COUNT_IF(affected_targets >= 10000) AS ge_10000, + COUNT_IF(affected_targets >= 100000) AS ge_100000, + MAX(affected_targets) AS max_affected_targets +FROM combined +GROUP BY grain +ORDER BY grain +LIMIT 10; diff --git a/tool/tangograph-eval/BUILD.bazel b/tool/tangograph-eval/BUILD.bazel new file mode 100644 index 000000000..3ba154d17 --- /dev/null +++ b/tool/tangograph-eval/BUILD.bazel @@ -0,0 +1,44 @@ +load("@rules_go//go:def.bzl", "go_binary", "go_library", "go_test") + +go_library( + name = "go_default_library", + srcs = [ + "basedrift.go", + "impact.go", + "loadbench.go", + "loadcodec.go", + "main.go", + "measure.go", + "parse.go", + "scale.go", + "workload.go", + ], + importpath = "github.com/uber/submitqueue/tool/tangograph-eval", + visibility = ["//visibility:private"], + deps = ["@org_golang_google_protobuf//encoding/protowire:go_default_library"], +) + +go_binary( + name = "tangograph-eval", + embed = [":go_default_library"], + visibility = ["//visibility:public"], +) + +go_test( + name = "go_default_test", + srcs = [ + "basedrift_test.go", + "impact_test.go", + "loadbench_test.go", + "measure_test.go", + "parse_test.go", + "scale_test.go", + "workload_test.go", + ], + embed = [":go_default_library"], + deps = [ + "@com_github_stretchr_testify//assert:go_default_library", + "@com_github_stretchr_testify//require:go_default_library", + "@org_golang_google_protobuf//encoding/protowire:go_default_library", + ], +) diff --git a/tool/tangograph-eval/basedrift.go b/tool/tangograph-eval/basedrift.go new file mode 100644 index 000000000..987782241 --- /dev/null +++ b/tool/tangograph-eval/basedrift.go @@ -0,0 +1,339 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import "slices" + +// The base-drift simulation checks, on small random DAGs, whether affected-target +// signatures computed once against the newest base at admission (and never recomputed +// as landed batches advance the base) can miss a conflict. Ground truth for a pair of +// changes that were allowed to run independently is "some target is affected by both +// in the first graph that contains both changes". + +// driftRule selects how a batch's comparison signature is formed. +type driftRule struct { + Name string `json:"name"` + // AddedDeps adds the targets that the change's new or rewired targets newly depend on. + AddedDeps bool `json:"added_deps"` + // Inherit unions the comparison signatures of the batch's dependencies at admission. + Inherit bool `json:"inherit_dependency_signatures"` + // StructuralOnly limits inheritance to dependencies that add edges or targets. + StructuralOnly bool `json:"inherit_structural_only"` +} + +type driftRuleResult struct { + Rule driftRule `json:"rule"` + // SameBasePairs counts independent (in-flight, in-flight) pairs admitted on one base. + SameBasePairs int `json:"same_base_independent_pairs"` + // SameBaseMissed counts those pairs whose changes truly share an affected target. + SameBaseMissed int `json:"same_base_missed_conflicts"` + // DriftPairs counts independent pairs whose signatures were computed on different bases. + DriftPairs int `json:"cross_base_independent_pairs"` + // DriftMissed counts those pairs whose changes truly share an affected target. + DriftMissed int `json:"cross_base_missed_conflicts"` + // FalseConflicts counts serialized pairs whose changes share no affected target. + FalseConflicts int `json:"false_conflicts"` + // MeanSignature is the mean comparison-signature size in targets. + MeanSignature float64 `json:"mean_comparison_signature_targets"` +} + +type driftReport struct { + Trials int `json:"trials"` + BaseNodes int `json:"base_nodes"` + Notes []string `json:"notes"` + RuleResult []driftRuleResult `json:"rules"` +} + +var driftRules = []driftRule{ + {Name: "affected"}, + {Name: "affected+added-deps", AddedDeps: true}, + {Name: "affected+added-deps+inherit-structural", AddedDeps: true, Inherit: true, StructuralOnly: true}, + {Name: "affected+added-deps+inherit-all", AddedDeps: true, Inherit: true}, +} + +type driftEdge struct{ from, to int32 } + +type driftChange struct { + touched []int32 + added []driftEdge + removed []driftEdge + // newNodes maps each new target (an ID unique to this change) to its dependencies. + newNodes map[int32][]int32 +} + +func (c driftChange) structural() bool { return len(c.added) > 0 || len(c.newNodes) > 0 } + +func (c driftChange) direct() []int32 { + out := slices.Clone(c.touched) + for _, e := range c.added { + out = append(out, e.from) + } + for _, e := range c.removed { + out = append(out, e.from) + } + for id := range c.newNodes { + out = append(out, id) + } + return out +} + +func (c driftChange) addedDeps() []int32 { + var out []int32 + for _, e := range c.added { + out = append(out, e.to) + } + for _, deps := range c.newNodes { + out = append(out, deps...) + } + return out +} + +// driftGraph maps each present target to its direct dependencies. +type driftGraph map[int32][]int32 + +func (g driftGraph) apply(c driftChange) driftGraph { + out := make(driftGraph, len(g)+len(c.newNodes)) + for id, deps := range g { + out[id] = slices.Clone(deps) + } + for id, deps := range c.newNodes { + out[id] = slices.Clone(deps) + } + for _, e := range c.removed { + out[e.from] = slices.DeleteFunc(out[e.from], func(d int32) bool { return d == e.to }) + } + for _, e := range c.added { + if !slices.Contains(out[e.from], e.to) { + out[e.from] = append(out[e.from], e.to) + } + } + return out +} + +func (g driftGraph) affected(seeds []int32) map[int32]bool { + reverse := make(map[int32][]int32, len(g)) + for id, deps := range g { + for _, d := range deps { + reverse[d] = append(reverse[d], id) + } + } + found := make(map[int32]bool) + var queue []int32 + for _, s := range seeds { + if _, present := g[s]; present && !found[s] { + found[s] = true + queue = append(queue, s) + } + } + for head := 0; head < len(queue); head++ { + for _, r := range reverse[queue[head]] { + if !found[r] { + found[r] = true + queue = append(queue, r) + } + } + } + return found +} + +func intersects(a, b map[int32]bool) bool { + if len(a) > len(b) { + a, b = b, a + } + for id := range a { + if b[id] { + return true + } + } + return false +} + +type driftRand struct{ state uint64 } + +func (r *driftRand) intn(n int) int { + r.state++ + return int(splitMix64(r.state) % uint64(n)) +} + +func (r *driftRand) chance(percent int) bool { return r.intn(100) < percent } + +func randomDriftBase(r *driftRand, nodes int) driftGraph { + g := make(driftGraph, nodes) + for i := range int32(nodes) { + g[i] = nil + for range r.intn(3) { + if i > 0 { + d := int32(r.intn(int(i))) + if !slices.Contains(g[i], d) { + g[i] = append(g[i], d) + } + } + } + } + return g +} + +// randomDriftChange edits targets of base; edges always point to lower IDs, so every +// combination of changes stays acyclic. New targets get IDs from firstNewID upward. +func randomDriftChange(r *driftRand, base driftGraph, nodes int, firstNewID int32) driftChange { + c := driftChange{newNodes: map[int32][]int32{}} + for range 1 + r.intn(2) { + c.touched = append(c.touched, int32(r.intn(nodes))) + } + if r.chance(50) { + from := int32(1 + r.intn(nodes-1)) + c.added = append(c.added, driftEdge{from: from, to: int32(r.intn(int(from)))}) + } + if r.chance(20) { + from := int32(r.intn(nodes)) + if deps := base[from]; len(deps) > 0 { + c.removed = append(c.removed, driftEdge{from: from, to: deps[r.intn(len(deps))]}) + } + } + if r.chance(30) { + var deps []int32 + for range 1 + r.intn(2) { + deps = append(deps, int32(r.intn(nodes))) + } + c.newNodes[firstNewID] = deps + } + return c +} + +type driftBatch struct { + change driftChange + signature map[int32]bool + dependsOn []int + comparison map[int32]bool +} + +func driftSignature(base driftGraph, c driftChange, rule driftRule) map[int32]bool { + sig := base.apply(c).affected(c.direct()) + if rule.AddedDeps { + for _, d := range c.addedDeps() { + sig[d] = true + } + } + return sig +} + +// admitDriftBatch compares the batch against every in-flight batch and records the +// comparison signature it will be checked with for the rest of its life. +func admitDriftBatch(base driftGraph, c driftChange, inFlight []driftBatch, rule driftRule) driftBatch { + b := driftBatch{change: c, signature: driftSignature(base, c, rule)} + b.comparison = b.signature + for i, other := range inFlight { + if intersects(b.signature, other.comparison) { + b.dependsOn = append(b.dependsOn, i) + } + } + if rule.Inherit { + b.comparison = make(map[int32]bool, len(b.signature)) + for id := range b.signature { + b.comparison[id] = true + } + for _, i := range b.dependsOn { + if rule.StructuralOnly && !inFlight[i].change.structural() { + continue + } + for id := range inFlight[i].comparison { + b.comparison[id] = true + } + } + } + return b +} + +func ordered(batches []driftBatch, later, earlier int) bool { + for _, d := range batches[later].dependsOn { + if d == earlier || ordered(batches, d, earlier) { + return true + } + } + return false +} + +func trulyConflict(g driftGraph, x, y driftChange) bool { + combined := g.apply(x).apply(y) + return intersects(combined.affected(x.direct()), combined.affected(y.direct())) +} + +// simulateBaseDrift admits 1–3 "landing" batches and batch A on base b0, lands every +// landing batch (A may depend on them; they never depend on A), then admits C on the +// advanced base and checks whether C was correctly ordered after A. +func simulateBaseDrift(trials, nodes int, seed uint64) driftReport { + report := driftReport{ + Trials: trials, + BaseNodes: nodes, + Notes: []string{ + "Toy random DAGs, not go-code: rates show whether a rule can miss conflicts, not how often production would.", + "Signatures follow Tango's GetChangedTargets: reverse closure, in the changed graph, of edited targets, rewired targets and new targets.", + "A pair is independent when neither batch transitively depends on the other; truth is a shared affected target in the first graph containing both changes.", + }, + } + for _, rule := range driftRules { + result := driftRuleResult{Rule: rule} + r := &driftRand{state: seed} + var signatureTargets, signatures int + for range trials { + base := randomDriftBase(r, nodes) + landing := 1 + r.intn(3) + nextNew := int32(nodes) + var changes []driftChange + for range landing + 2 { + changes = append(changes, randomDriftChange(r, base, nodes, nextNew)) + nextNew++ + } + var batches []driftBatch + for _, c := range changes[:landing+1] { + batches = append(batches, admitDriftBatch(base, c, batches, rule)) + } + a := landing + advanced := base + for _, c := range changes[:landing] { + advanced = advanced.apply(c) + } + c := admitDriftBatch(advanced, changes[landing+1], []driftBatch{batches[a]}, rule) + + for i := range landing { + if slices.Contains(batches[a].dependsOn, i) && !trulyConflict(advanced, changes[i], changes[a]) { + result.FalseConflicts++ + } + if ordered(batches, a, i) { + continue + } + result.SameBasePairs++ + if trulyConflict(advanced, changes[i], changes[a]) { + result.SameBaseMissed++ + } + } + if len(c.dependsOn) == 0 { + result.DriftPairs++ + if trulyConflict(advanced, changes[a], c.change) { + result.DriftMissed++ + } + } else if !trulyConflict(advanced, changes[a], c.change) { + result.FalseConflicts++ + } + for _, b := range append(batches, c) { + signatureTargets += len(b.comparison) + signatures++ + } + } + result.MeanSignature = float64(signatureTargets) / float64(signatures) + report.RuleResult = append(report.RuleResult, result) + } + return report +} diff --git a/tool/tangograph-eval/basedrift_test.go b/tool/tangograph-eval/basedrift_test.go new file mode 100644 index 000000000..559a4b5ca --- /dev/null +++ b/tool/tangograph-eval/basedrift_test.go @@ -0,0 +1,51 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package main + +import ( + "testing" + + "github.com/stretchr/testify/assert" +) + +func TestBaseDriftRules(t *testing.T) { + report := simulateBaseDrift(4000, 40, 1) + byName := map[string]driftRuleResult{} + for _, result := range report.RuleResult { + byName[result.Rule.Name] = result + } + t.Run("affected-target overlap misses conflicts on one base and across bases", func(t *testing.T) { + assert.Positive(t, byName["affected"].SameBaseMissed) + assert.Positive(t, byName["affected"].DriftMissed) + }) + t.Run("added dependencies alone still miss conflicts", func(t *testing.T) { + result := byName["affected+added-deps"] + assert.Positive(t, result.SameBaseMissed+result.DriftMissed) + }) + for _, name := range []string{"affected+added-deps+inherit-structural", "affected+added-deps+inherit-all"} { + t.Run(name+" misses nothing", func(t *testing.T) { + assert.Zero(t, byName[name].SameBaseMissed) + assert.Zero(t, byName[name].DriftMissed) + assert.Positive(t, byName[name].DriftPairs) + }) + } +} + +func TestDriftGraphApplyIsPure(t *testing.T) { + base := driftGraph{0: nil, 1: {0}} + changed := base.apply(driftChange{added: []driftEdge{{from: 1, to: 0}}, removed: []driftEdge{{from: 1, to: 0}}, newNodes: map[int32][]int32{2: {1}}}) + assert.Equal(t, driftGraph{0: nil, 1: {0}}, base) + assert.Equal(t, []int32{0}, changed[1]) + assert.Equal(t, map[int32]bool{0: true, 1: true, 2: true}, changed.affected([]int32{0})) +} diff --git a/tool/tangograph-eval/impact.go b/tool/tangograph-eval/impact.go new file mode 100644 index 000000000..4d3685b5e --- /dev/null +++ b/tool/tangograph-eval/impact.go @@ -0,0 +1,138 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "math/bits" + "slices" + "strings" +) + +type impactMeasurement struct { + CandidateSources uint64 `json:"eligible_main_repo_source_files"` + SourceSeeds uint64 `json:"sampled_source_file_seeds"` + Pairs uint64 `json:"source_seed_pairs"` + FullOverlapPairs uint64 `json:"full_reverse_closure_overlap_pairs"` + DirectOverlapPairs uint64 `json:"one_hop_overlap_pairs"` + MedianFullTargets uint64 `json:"median_full_closure_targets"` + P95FullTargets uint64 `json:"p95_full_closure_targets"` + MaxFullTargets uint64 `json:"max_full_closure_targets"` +} + +func measureSourceImpact(graph compactGraph, limit int, prefix string) impactMeasurement { + var sources []int32 + for _, target := range graph.targets { + if graph.ruleTypes[target.ruleType] == "source file" && + !target.external && + (prefix == "" || strings.HasPrefix(graph.names[target.id], prefix)) { + sources = append(sources, target.id) + } + } + if limit == 0 || len(sources) == 0 { + return impactMeasurement{CandidateSources: uint64(len(sources))} + } + n := min(limit, len(sources)) + reverseDeps := make([][]int32, len(graph.targets)+1) + for _, target := range graph.targets { + for _, dep := range target.directDependencies { + reverseDeps[dep] = append(reverseDeps[dep], target.id) + } + } + full := make([][]uint64, n) + direct := make([][]uint64, n) + sizes := make([]int, 0, n) + for i := range n { + source := sources[i*len(sources)/n] + full[i] = reverseClosure(reverseDeps, source, false) + direct[i] = reverseClosure(reverseDeps, source, true) + size := 0 + for _, word := range full[i] { + size += bits.OnesCount64(word) + } + sizes = append(sizes, size) + } + slices.Sort(sizes) + result := impactMeasurement{ + CandidateSources: uint64(len(sources)), + SourceSeeds: uint64(n), + MedianFullTargets: uint64(sizes[n/2]), + P95FullTargets: uint64(sizes[(n-1)*95/100]), + MaxFullTargets: uint64(sizes[n-1]), + } + for i := range n { + for j := i + 1; j < n; j++ { + result.Pairs++ + if intersectBitsets(full[i], full[j]) { + result.FullOverlapPairs++ + } + if intersectBitsets(direct[i], direct[j]) { + result.DirectOverlapPairs++ + } + } + } + return result +} + +func reverseClosure(reverseDeps [][]int32, start int32, oneHop bool) []uint64 { + found := make([]uint64, (len(reverseDeps)+63)/64) + set := func(id int32) bool { + word, mask := id/64, uint64(1)<<(id%64) + if found[word]&mask != 0 { + return false + } + found[word] |= mask + return true + } + set(start) + if oneHop { + for _, id := range reverseDeps[start] { + set(id) + } + return found + } + queue := []int32{start} + for head := 0; head < len(queue); head++ { + for _, id := range reverseDeps[queue[head]] { + if set(id) { + queue = append(queue, id) + } + } + } + return found +} + +func intersectBitsets(a, b []uint64) bool { + for i := range a { + if a[i]&b[i] != 0 { + return true + } + } + return false +} + +func intersectSortedFingerprints(a, b []uint64) bool { + i, j := 0, 0 + for i < len(a) && j < len(b) { + switch { + case a[i] < b[j]: + i++ + case a[i] > b[j]: + j++ + default: + return true + } + } + return false +} diff --git a/tool/tangograph-eval/impact_test.go b/tool/tangograph-eval/impact_test.go new file mode 100644 index 000000000..ee640d59f --- /dev/null +++ b/tool/tangograph-eval/impact_test.go @@ -0,0 +1,53 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "slices" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestSourceImpactMayMissSharedTransitiveTargetWhenCapped(t *testing.T) { + graph, err := compactTargets([]queryTarget{ + {name: "//app:a.go", ruleType: "source file"}, + {name: "//app:b.go", ruleType: "source file"}, + {name: "//app:a", ruleType: "go_library", deps: []string{"//app:a.go"}}, + {name: "//app:b", ruleType: "go_library", deps: []string{"//app:b.go"}}, + {name: "//app:integration_test", ruleType: "go_test", deps: []string{"//app:a", "//app:b"}}, + }) + require.NoError(t, err) + impact := measureSourceImpact(graph, 2, "//app:") + assert.Equal(t, uint64(2), impact.SourceSeeds) + assert.Equal(t, uint64(2), impact.CandidateSources) + assert.Equal(t, uint64(1), impact.Pairs) + assert.Equal(t, uint64(1), impact.FullOverlapPairs) + assert.Zero(t, impact.DirectOverlapPairs) + assert.Equal(t, uint64(3), impact.MaxFullTargets) +} + +func TestIntersectSortedFingerprints(t *testing.T) { + one := []uint64{fingerprint("//app:one"), fingerprint("//app:two")} + two := []uint64{fingerprint("//app:three"), fingerprint("//app:two")} + disjoint := []uint64{fingerprint("//app:four")} + slices.Sort(one) + slices.Sort(two) + slices.Sort(disjoint) + assert.True(t, intersectSortedFingerprints(one, two)) + assert.False(t, intersectSortedFingerprints(one, disjoint)) + assert.False(t, intersectSortedFingerprints(one, nil)) +} diff --git a/tool/tangograph-eval/loadbench.go b/tool/tangograph-eval/loadbench.go new file mode 100644 index 000000000..44fbcc398 --- /dev/null +++ b/tool/tangograph-eval/loadbench.go @@ -0,0 +1,449 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "cmp" + "crypto/sha256" + "encoding/binary" + "fmt" + "runtime" + "slices" + "strconv" + "strings" + "time" +) + +type loadBenchMode struct { + BlobBytes uint64 `json:"batch_blob_bytes"` + MaxBlobBytes uint64 `json:"max_single_batch_blob_bytes"` + DecodedHeapBytes uint64 `json:"decoded_go_heap_bytes"` + DeserializeMilliseconds float64 `json:"deserialize_milliseconds"` + CheckMicroseconds float64 `json:"check_microseconds"` + MatchingBatches int `json:"matching_batches"` +} + +type loadBenchTango struct { + RawBlobBytes uint64 `json:"raw_batch_blob_bytes"` + GzipBlobBytes uint64 `json:"gzip_batch_blob_bytes"` + DecodedHeapBytes uint64 `json:"decoded_go_heap_bytes"` + RawDeserializeMs float64 `json:"raw_deserialize_milliseconds"` + GzipDeserializeMs float64 `json:"gzip_deserialize_milliseconds"` + CheckMicroseconds float64 `json:"check_microseconds"` + MatchingBatches int `json:"matching_batches"` +} + +type loadBenchCase struct { + InFlight int `json:"in_flight_batches"` + TotalTargets uint64 `json:"total_batch_targets"` + MedianBatchTargets int `json:"median_batch_targets"` + MaxBatchTargets int `json:"max_batch_targets"` + CandidateTargets int `json:"candidate_targets"` + NameRegistryEntries int `json:"id_registry_entries"` + NameRegistryBytes uint64 `json:"id_registry_minimal_bytes"` + SortedIDs loadBenchMode `json:"id64"` + SortedNames loadBenchMode `json:"namekey"` + TangoSnapshot loadBenchTango `json:"tango_snapshot"` +} + +type loadBenchReport struct { + Input string `json:"input"` + InputBytes uint64 `json:"bazel_streamed_proto_input_bytes"` + GoCodeRevision string `json:"go_code_revision,omitempty"` + GoVersion string `json:"go_version"` + GraphTargets uint64 `json:"graph_targets"` + SourceSeeds int `json:"source_file_seeds"` + HistogramRequests uint64 `json:"histogram_requests"` + HistogramMean float64 `json:"histogram_mean_targets"` + Notes []string `json:"notes"` + Cases []loadBenchCase `json:"cases"` + Scale scaleReport `json:"scale_model"` +} + +var benchResultSink int + +func parseBenchmarkCounts(raw string) ([]int, error) { + var counts []int + for _, part := range strings.Split(raw, ",") { + count, err := strconv.Atoi(strings.TrimSpace(part)) + if err != nil || count <= 0 { + return nil, fmt.Errorf("invalid positive benchmark count %q", part) + } + counts = append(counts, count) + } + return counts, nil +} + +func benchmarkLoadAndCheck(graph queryGraph, histogram targetHistogram, batchesText string, candidateTargets int, store []blobStoreProfile, scaleTrials int) (loadBenchReport, error) { + batchCounts, err := parseBenchmarkCounts(batchesText) + if err != nil { + return loadBenchReport{}, err + } + report := loadBenchReport{ + GoVersion: runtime.Version(), + GraphTargets: graph.scanned, + HistogramRequests: histogram.requests(), + HistogramMean: histogram.mean(), + Notes: []string{ + "One new batch versus B in-flight batches; one batch is one request. Each batch's target count is drawn from the Hive per-request histogram (batch i always draws the same count, so smaller B is a prefix of larger B).", + "Affected sets are closure-shaped synthetic sets on the complete go-code graph: the reverse-dependency closure of a random source file, widened by climbing to a dependency until the drawn count is reached. They are not observed diffs.", + "ID64 stores sorted uint64 target IDs; NameKey stores sorted varint-prefixed UTF-8 labels; TangoSnapshot stores default-field GetTargetGraphResponse-shaped protobuf for the affected targets with their real direct dependencies in the response metadata, raw or per-message gzip/best-speed.", + "Go heap is measured after GC on all B decoded signatures plus the candidate, excluding the shared graph, workload indices, serialized buffers and ID64's dictionary held in storage.", + "Deserialize time is serial single-goroutine Go CPU for all B blobs; check time is local Go CPU for one candidate against all B decoded batches.", + }, + } + if candidateTargets <= 0 || candidateTargets > len(graph.targets) { + return loadBenchReport{}, fmt.Errorf("candidate targets %d must be between 1 and %d", candidateTargets, len(graph.targets)) + } + closures := newClosureGraph(graph) + graph.targets = nil + report.SourceSeeds = len(closures.sources) + runtime.GC() + + registeredIDs := make([]uint64, len(closures.names)) + registry := make(map[uint64]string, len(closures.names)) + for i, name := range closures.names { + registeredIDs[i] = registerTargetID(name, registry) + } + registry = nil + runtime.GC() + + generator := newAffectedSetGenerator(closures) + maxBatches := slices.Max(batchCounts) + sizes := sampledBatchTargets(histogram, maxBatches, len(closures.names)) + for _, batches := range batchCounts { + workload := newSampledWorkload(generator, sizes[:batches], candidateTargets) + sorted := slices.Sorted(slices.Values(sizes[:batches])) + entry := loadBenchCase{ + InFlight: batches, + TotalTargets: workload.totalTargets(), + MedianBatchTargets: sorted[len(sorted)/2], + MaxBatchTargets: sorted[len(sorted)-1], + CandidateTargets: len(workload.candidate), + } + entry.NameRegistryEntries, entry.NameRegistryBytes = minimalIDRegistryBytes(closures.names, workload) + entry.SortedIDs, err = benchmarkDecodedIDs(registeredIDs, workload) + if err != nil { + return loadBenchReport{}, err + } + runtime.GC() + entry.SortedNames, err = benchmarkDecodedNames(closures.names, workload) + if err != nil { + return loadBenchReport{}, err + } + runtime.GC() + entry.TangoSnapshot, err = benchmarkDecodedTango(closures, workload) + if err != nil { + return loadBenchReport{}, err + } + if entry.SortedIDs.MatchingBatches != entry.SortedNames.MatchingBatches || + entry.SortedIDs.MatchingBatches != entry.TangoSnapshot.MatchingBatches { + return loadBenchReport{}, fmt.Errorf("decoded modes disagree on matches for batches=%d", batches) + } + report.Cases = append(report.Cases, entry) + runtime.GC() + } + report.Scale = modelScale(report.Cases, histogram, store, scaleTrials) + runtime.KeepAlive(registeredIDs) + return report, nil +} + +func minimalIDRegistryBytes(names []string, workload benchWorkload) (int, uint64) { + seen := make([]bool, len(names)) + var entries int + var bytes uint64 + record := func(index uint32) { + if seen[index] { + return + } + seen[index] = true + entries++ + length := len(names[index]) + bytes += uint64(8 + unsignedVarintBytes(uint64(length)) + length) + } + for _, batch := range workload.batches { + for _, index := range batch { + record(index) + } + } + for _, index := range workload.candidate { + record(index) + } + return entries, bytes +} + +func benchmarkDecodedIDs(registeredIDs []uint64, workload benchWorkload) (loadBenchMode, error) { + baseline := liveHeap() + data := struct { + batches [][]uint64 + candidate []uint64 + }{ + batches: make([][]uint64, len(workload.batches)), + } + var result loadBenchMode + var decode time.Duration + for i, indices := range workload.batches { + blob := encodeIDSignature(indices, registeredIDs) + result.BlobBytes += uint64(len(blob)) + result.MaxBlobBytes = max(result.MaxBlobBytes, uint64(len(blob))) + start := time.Now() + decoded, err := decodeIDSignature(blob) + decode += time.Since(start) + if err != nil { + return loadBenchMode{}, err + } + data.batches[i] = decoded + } + result.DeserializeMilliseconds = milliseconds(decode) + data.candidate = make([]uint64, len(workload.candidate)) + for i, index := range workload.candidate { + data.candidate[i] = registeredIDs[index] + } + slices.Sort(data.candidate) + result.DecodedHeapBytes = heapAboveBaseline(baseline, data) + result.CheckMicroseconds, result.MatchingBatches = benchmarkLookup(func() int { + return countIntersectingSortedSignatures(data.batches, data.candidate) + }) + runtime.KeepAlive(data) + return result, nil +} + +func benchmarkDecodedNames(names []string, workload benchWorkload) (loadBenchMode, error) { + baseline := liveHeap() + data := struct { + batches [][]string + candidate []string + }{ + batches: make([][]string, len(workload.batches)), + } + var result loadBenchMode + var decode time.Duration + for i, indices := range workload.batches { + blob := encodeNameSignature(indices, names) + result.BlobBytes += uint64(len(blob)) + result.MaxBlobBytes = max(result.MaxBlobBytes, uint64(len(blob))) + start := time.Now() + decoded, err := decodeNameSignature(blob, len(indices)) + decode += time.Since(start) + if err != nil { + return loadBenchMode{}, err + } + data.batches[i] = decoded + } + result.DeserializeMilliseconds = milliseconds(decode) + data.candidate = make([]string, len(workload.candidate)) + for i, index := range workload.candidate { + data.candidate[i] = strings.Clone(names[index]) + } + slices.Sort(data.candidate) + result.DecodedHeapBytes = heapAboveBaseline(baseline, data) + result.CheckMicroseconds, result.MatchingBatches = benchmarkLookup(func() int { + return countIntersectingSortedSignatures(data.batches, data.candidate) + }) + runtime.KeepAlive(data) + return result, nil +} + +func benchmarkDecodedTango(graph closureGraph, workload benchWorkload) (loadBenchTango, error) { + baseline := liveHeap() + data := struct { + batches []decodedTangoBatch + candidate map[string]struct{} + }{ + batches: make([]decodedTangoBatch, len(workload.batches)), + candidate: make(map[string]struct{}, len(workload.candidate)), + } + for _, index := range workload.candidate { + data.candidate[graph.names[index]] = struct{}{} + } + var result loadBenchTango + var rawDecode, gzipDecode time.Duration + for i, indices := range workload.batches { + var rawFrames, gzipFrames [][]byte + err := streamProtoMessages(syntheticTangoResponse(graph, indices), defaultPayload, 4_250_000, func(frame []byte) error { + compressed, err := gzipTangoFrame(frame) + if err != nil { + return err + } + rawFrames = append(rawFrames, frame) + gzipFrames = append(gzipFrames, compressed) + result.RawBlobBytes += uint64(len(frame) + unsignedVarintBytes(uint64(len(frame)))) + result.GzipBlobBytes += uint64(len(compressed) + unsignedVarintBytes(uint64(len(compressed)))) + return nil + }) + if err != nil { + return loadBenchTango{}, err + } + raw := newDecodedTangoBatch(len(indices)) + start := time.Now() + for _, frame := range rawFrames { + if err := decodeTangoFrame(frame, &raw); err != nil { + return loadBenchTango{}, err + } + } + if err := validateDecodedTangoBatch(raw, len(indices)); err != nil { + return loadBenchTango{}, err + } + rawDecode += time.Since(start) + runtime.KeepAlive(raw) + raw = decodedTangoBatch{} + + decoded := newDecodedTangoBatch(len(indices)) + start = time.Now() + for _, compressed := range gzipFrames { + if err := decodeGzipTangoFrame(compressed, &decoded); err != nil { + return loadBenchTango{}, err + } + } + if err := validateDecodedTangoBatch(decoded, len(indices)); err != nil { + return loadBenchTango{}, err + } + gzipDecode += time.Since(start) + data.batches[i] = decoded + } + result.RawDeserializeMs = milliseconds(rawDecode) + result.GzipDeserializeMs = milliseconds(gzipDecode) + result.DecodedHeapBytes = heapAboveBaseline(baseline, data) + result.CheckMicroseconds, result.MatchingBatches = benchmarkLookup(func() int { + matches := 0 + for _, batch := range data.batches { + for _, target := range batch.targets { + if _, found := data.candidate[batch.names[target.id]]; found { + matches++ + break + } + } + } + return matches + }) + runtime.KeepAlive(data) + return result, nil +} + +// syntheticTangoResponse models a changed-targets response for one batch: one +// default-field optimized target per affected target, with its real direct +// dependencies, and response-local IDs for every named target. +func syntheticTangoResponse(graph closureGraph, indices []uint32) compactGraph { + targets := make([]optimizedTarget, 0, len(indices)) + names := make(map[int32]string, len(indices)*2) + ids := make(map[uint32]int32, len(indices)*2) + idFor := func(index uint32) int32 { + if id, ok := ids[index]; ok { + return id + } + id := int32(len(ids) + 1) + ids[index] = id + names[id] = graph.names[index] + return id + } + for _, index := range indices { + idFor(index) + } + for _, index := range indices { + target := optimizedTarget{id: ids[index], ruleType: 1} + for _, dep := range graph.deps[index] { + target.directDependencies = append(target.directDependencies, idFor(dep)) + } + targets = append(targets, target) + } + return compactGraph{targets: targets, names: names, ruleTypes: map[int32]string{1: "go_library"}} +} + +func milliseconds(d time.Duration) float64 { return float64(d.Nanoseconds()) / 1e6 } + +func unsignedVarintBytes(value uint64) int { + n := 1 + for value >= 0x80 { + value >>= 7 + n++ + } + return n +} + +// countIntersectingSortedSignatures counts batches sharing at least one element with +// the sorted candidate, stopping at the first shared element of each batch. +func countIntersectingSortedSignatures[T cmp.Ordered](batches [][]T, candidate []T) int { + matches := 0 + for _, batch := range batches { + existingIndex, candidateIndex := 0, 0 + for existingIndex < len(batch) && candidateIndex < len(candidate) { + if batch[existingIndex] == candidate[candidateIndex] { + matches++ + break + } + if batch[existingIndex] < candidate[candidateIndex] { + existingIndex++ + } else { + candidateIndex++ + } + } + } + return matches +} + +func splitMix64(x uint64) uint64 { + x += 0x9e3779b97f4a7c15 + x = (x ^ x>>30) * 0xbf58476d1ce4e5b9 + x = (x ^ x>>27) * 0x94d049bb133111eb + return x ^ x>>31 +} + +// registerTargetID derives a collision-checked 64-bit ID for a label against an +// in-memory stand-in for the durable ID-to-name dictionary. +func registerTargetID(label string, registry map[uint64]string) uint64 { + const domain, repository = "submitqueue-target-id/v1", "go-code" + for probe := uint32(0); ; probe++ { + payload := binary.AppendUvarint([]byte(domain), uint64(len(repository))) + payload = append(payload, repository...) + payload = binary.AppendUvarint(payload, uint64(len(label))) + payload = append(payload, label...) + payload = binary.BigEndian.AppendUint32(payload, probe) + digest := sha256.Sum256(payload) + id := binary.BigEndian.Uint64(digest[:8]) + if id == 0 { + continue + } + if existing, ok := registry[id]; ok { + if existing == label { + return id + } + continue + } + registry[id] = label + return id + } +} + +func liveHeap() uint64 { + runtime.GC() + var stats runtime.MemStats + runtime.ReadMemStats(&stats) + return stats.HeapAlloc +} + +// benchmarkLookup returns the mean microseconds per run and the matches of one run. +func benchmarkLookup(run func() int) (float64, int) { + runtime.GC() + start := time.Now() + iterations := 0 + totalMatches := 0 + for iterations < 2 || (time.Since(start) < 200*time.Millisecond && iterations < 1000) { + totalMatches += run() + iterations++ + } + duration := time.Since(start) + benchResultSink += totalMatches + return float64(duration.Nanoseconds()) / 1e3 / float64(iterations), totalMatches / iterations +} diff --git a/tool/tangograph-eval/loadbench_test.go b/tool/tangograph-eval/loadbench_test.go new file mode 100644 index 000000000..583a38600 --- /dev/null +++ b/tool/tangograph-eval/loadbench_test.go @@ -0,0 +1,206 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "fmt" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestSignatureCodecsRoundTrip(t *testing.T) { + t.Run("sorted numeric IDs", func(t *testing.T) { + encoded := encodeIDSignature([]uint32{2, 0, 1}, []uint64{30, 10, 20}) + require.Len(t, encoded, 24) + got, err := decodeIDSignature(encoded) + require.NoError(t, err) + assert.Equal(t, []uint64{10, 20, 30}, got) + _, err = decodeIDSignature(encoded[:23]) + require.Error(t, err) + }) + t.Run("sorted full names", func(t *testing.T) { + names := []string{"//app:one", "//app:two", "//other:three"} + encoded := encodeNameSignature([]uint32{2, 0, 1}, names) + got, err := decodeNameSignature(encoded, 3) + require.NoError(t, err) + assert.Equal(t, []string{"//app:one", "//app:two", "//other:three"}, got) + _, err = decodeNameSignature(encoded[:len(encoded)-1], 3) + require.Error(t, err) + _, err = decodeNameSignature(encoded, 2) + require.Error(t, err) + }) +} + +func TestTangoResponseFrameRoundTrip(t *testing.T) { + graph := compactGraph{ + targets: []optimizedTarget{{id: 1, directDependencies: []int32{2}, ruleType: 1}}, + names: map[int32]string{1: "//app:one", 2: "//app:dependency"}, + ruleTypes: map[int32]string{1: "go_library"}, + } + var frames [][]byte + require.NoError(t, streamProtoMessages(graph, defaultPayload, 128, func(frame []byte) error { + frames = append(frames, frame) + return nil + })) + require.NotEmpty(t, frames) + raw := newDecodedTangoBatch(1) + compressed := newDecodedTangoBatch(1) + for _, frame := range frames { + require.NoError(t, decodeTangoFrame(frame, &raw)) + gzipFrame, err := gzipTangoFrame(frame) + require.NoError(t, err) + require.NoError(t, decodeGzipTangoFrame(gzipFrame, &compressed)) + } + require.NoError(t, validateDecodedTangoBatch(raw, 1)) + require.NoError(t, validateDecodedTangoBatch(compressed, 1)) + assert.Equal(t, raw, compressed) + assert.Equal(t, int32(2), raw.targets[0].directDependencies[0]) + assert.Equal(t, "//app:one", raw.names[1]) + assert.Equal(t, "go_library", raw.ruleTypes[1]) + require.Error(t, decodeTangoFrame([]byte{0xff}, &raw)) + require.Error(t, decodeGzipTangoFrame([]byte{0xff}, &compressed)) +} + +func testChainGraph(n int) queryGraph { + graph := queryGraph{scanned: uint64(n)} + graph.targets = append(graph.targets, queryTarget{name: "//app:root.go", ruleType: "source file"}) + for i := 1; i < n; i++ { + graph.targets = append(graph.targets, queryTarget{ + name: fmt.Sprintf("//app:t%02d", i), + ruleType: "go_library", + deps: []string{graph.targets[i-1].name}, + }) + } + return graph +} + +func TestBenchmarkLoadAndCheck(t *testing.T) { + histogram := targetHistogram{values: []int{0, 3, 7}, cumulative: []uint64{1, 3, 4}} + result, err := benchmarkLoadAndCheck(testChainGraph(25), histogram, "2,5", 4, defaultBlobStoreProfiles, 4) + require.NoError(t, err) + require.Len(t, result.Cases, 2) + for _, entry := range result.Cases { + assert.Equal(t, entry.TotalTargets*8, entry.SortedIDs.BlobBytes) + assert.GreaterOrEqual(t, entry.SortedNames.BlobBytes, entry.SortedIDs.BlobBytes) + assert.Positive(t, entry.TangoSnapshot.RawBlobBytes) + assert.Positive(t, entry.NameRegistryBytes) + assert.Equal(t, entry.SortedIDs.MatchingBatches, entry.SortedNames.MatchingBatches) + assert.Equal(t, entry.SortedIDs.MatchingBatches, entry.TangoSnapshot.MatchingBatches) + assert.Equal(t, 4, entry.CandidateTargets) + } + assert.NotEmpty(t, result.Scale.Points) + assert.NotEmpty(t, result.Scale.Crossovers) +} + +func TestMinimalIDRegistryBytesDeduplicatesBatchAndCandidateLabels(t *testing.T) { + names := []string{"//a:one", "//b:two"} + workload := benchWorkload{ + batches: [][]uint32{{0, 1}, {0}}, + candidate: []uint32{0, 1}, + } + count, bytes := minimalIDRegistryBytes(names, workload) + assert.Equal(t, 2, count) + assert.Equal(t, uint64(2*(8+1+len("//a:one"))), bytes) +} + +func TestBenchmarkLoadAndCheckRejectsInvalidWorkload(t *testing.T) { + histogram := targetHistogram{values: []int{1}, cumulative: []uint64{1}} + for _, tt := range []struct { + name string + batches string + candidate int + }{ + {name: "invalid batches", batches: "0", candidate: 1}, + {name: "invalid candidate", batches: "1", candidate: 0}, + {name: "candidate larger than graph", batches: "1", candidate: 30}, + } { + t.Run(tt.name, func(t *testing.T) { + _, err := benchmarkLoadAndCheck(testChainGraph(25), histogram, tt.batches, tt.candidate, defaultBlobStoreProfiles, 4) + require.Error(t, err) + }) + } +} + +func TestParseBenchmarkCounts(t *testing.T) { + for _, tt := range []struct { + name string + input string + want []int + valid bool + }{ + {name: "two counts", input: "100, 500", want: []int{100, 500}, valid: true}, + {name: "zero", input: "100,0"}, + {name: "non-numeric", input: "one"}, + {name: "empty", input: ""}, + } { + t.Run(tt.name, func(t *testing.T) { + counts, err := parseBenchmarkCounts(tt.input) + if tt.valid { + require.NoError(t, err) + assert.Equal(t, tt.want, counts) + } else { + require.Error(t, err) + } + }) + } +} + +func TestRegisteredTargetIDMatchesName(t *testing.T) { + names := make(map[uint64]string) + one := registerTargetID("//app:one", names) + other := registerTargetID("//app:other", names) + assert.NotZero(t, one) + assert.NotEqual(t, one, other) + assert.Equal(t, one, registerTargetID("//app:one", names)) + assert.Equal(t, "//app:one", names[one]) + assert.Len(t, names, 2) +} + +func TestUnsignedVarintBytes(t *testing.T) { + for _, tt := range []struct { + value uint64 + want int + }{ + {value: 0, want: 1}, + {value: 127, want: 1}, + {value: 128, want: 2}, + {value: 16384, want: 3}, + } { + assert.Equal(t, tt.want, unsignedVarintBytes(tt.value)) + } +} + +func TestCountIntersectingSortedSignatures(t *testing.T) { + for _, tt := range []struct { + name string + batches [][]uint64 + candidate []uint64 + want int + }{ + {name: "overlap", batches: [][]uint64{{1, 3}, {2, 4}, {3, 5}}, candidate: []uint64{3, 7}, want: 2}, + {name: "disjoint", batches: [][]uint64{{1, 3}, {2, 4}}, candidate: []uint64{5, 7}}, + {name: "empty", batches: [][]uint64{{}, {1}}, candidate: []uint64{}}, + } { + t.Run(tt.name, func(t *testing.T) { + assert.Equal(t, tt.want, countIntersectingSortedSignatures(tt.batches, tt.candidate)) + }) + } + assert.Equal(t, 1, countIntersectingSortedSignatures( + [][]string{{"//app:a"}, {"//app:b"}}, + []string{"//app:b"}, + )) +} diff --git a/tool/tangograph-eval/loadcodec.go b/tool/tangograph-eval/loadcodec.go new file mode 100644 index 000000000..edacba4f6 --- /dev/null +++ b/tool/tangograph-eval/loadcodec.go @@ -0,0 +1,255 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "bytes" + "compress/gzip" + "encoding/binary" + "fmt" + "io" + "math" + "slices" + + "google.golang.org/protobuf/encoding/protowire" +) + +type decodedTangoBatch struct { + targets []optimizedTarget + names map[int32]string + ruleTypes map[int32]string +} + +func encodeIDSignature(indices []uint32, registeredIDs []uint64) []byte { + ids := make([]uint64, len(indices)) + for i, index := range indices { + ids[i] = registeredIDs[index] + } + slices.Sort(ids) + blob := make([]byte, len(ids)*8) + for i, id := range ids { + binary.LittleEndian.PutUint64(blob[i*8:], id) + } + return blob +} + +func decodeIDSignature(blob []byte) ([]uint64, error) { + if len(blob)%8 != 0 { + return nil, fmt.Errorf("ID signature length %d is not divisible by 8", len(blob)) + } + ids := make([]uint64, len(blob)/8) + for i := range ids { + ids[i] = binary.LittleEndian.Uint64(blob[i*8:]) + if i > 0 && ids[i] < ids[i-1] { + return nil, fmt.Errorf("ID signature is not sorted at index %d", i) + } + } + return ids, nil +} + +func encodeNameSignature(indices []uint32, names []string) []byte { + sorted := make([]string, len(indices)) + size := 0 + for i, index := range indices { + sorted[i] = names[index] + size += len(sorted[i]) + unsignedVarintBytes(uint64(len(sorted[i]))) + } + slices.Sort(sorted) + blob := make([]byte, 0, size) + for _, name := range sorted { + blob = binary.AppendUvarint(blob, uint64(len(name))) + blob = append(blob, name...) + } + return blob +} + +func decodeNameSignature(blob []byte, targetCount int) ([]string, error) { + names := make([]string, 0, targetCount) + for len(blob) > 0 { + size, prefix := binary.Uvarint(blob) + if prefix <= 0 || size > uint64(len(blob[prefix:])) { + return nil, fmt.Errorf("invalid length-prefixed target name") + } + name := string(blob[prefix : prefix+int(size)]) + if len(names) > 0 && name < names[len(names)-1] { + return nil, fmt.Errorf("target-name signature is not sorted") + } + names = append(names, name) + blob = blob[prefix+int(size):] + } + if len(names) != targetCount { + return nil, fmt.Errorf("got %d target names, expected %d", len(names), targetCount) + } + return names, nil +} + +func newDecodedTangoBatch(targetCount int) decodedTangoBatch { + return decodedTangoBatch{ + targets: make([]optimizedTarget, 0, targetCount), + names: make(map[int32]string, targetCount*3), + ruleTypes: make(map[int32]string, 1), + } +} + +func decodeTangoFrame(frame []byte, batch *decodedTangoBatch) error { + return walkFields(frame, func(field protowire.Number, _ protowire.Type, value []byte, _ uint64) error { + switch field { + case 1: + return walkFields(value, func(entry protowire.Number, _ protowire.Type, target []byte, _ uint64) error { + if entry != 1 { + return nil + } + decoded, err := decodeOptimizedTarget(target) + if err != nil { + return err + } + batch.targets = append(batch.targets, decoded) + return nil + }) + case 2: + return walkFields(value, func(mapping protowire.Number, _ protowire.Type, entry []byte, _ uint64) error { + if mapping != 1 && mapping != 2 { + return nil + } + id, name, err := decodeTangoMapping(entry) + if err != nil { + return err + } + if mapping == 1 { + batch.names[id] = name + } else { + batch.ruleTypes[id] = name + } + return nil + }) + } + return nil + }) +} + +func decodeOptimizedTarget(encoded []byte) (optimizedTarget, error) { + var target optimizedTarget + err := walkFields(encoded, func(field protowire.Number, _ protowire.Type, value []byte, number uint64) error { + switch field { + case 1: + if number == 0 || number > math.MaxInt32 { + return fmt.Errorf("invalid target ID %d", number) + } + target.id = int32(number) + case 3: + for len(value) > 0 { + dependency, n := protowire.ConsumeVarint(value) + if n < 0 { + return protowire.ParseError(n) + } + if dependency == 0 || dependency > math.MaxInt32 { + return fmt.Errorf("invalid dependency ID %d", dependency) + } + target.directDependencies = append(target.directDependencies, int32(dependency)) + value = value[n:] + } + case 5: + if number > math.MaxInt32 { + return fmt.Errorf("invalid rule type ID %d", number) + } + target.ruleType = int32(number) + case 6: + target.root = number != 0 + case 7: + target.external = number != 0 + } + return nil + }) + if err != nil { + return optimizedTarget{}, err + } + if target.id == 0 { + return optimizedTarget{}, fmt.Errorf("optimized target has no ID") + } + return target, nil +} + +func decodeTangoMapping(encoded []byte) (int32, string, error) { + var id int32 + var name string + err := walkFields(encoded, func(field protowire.Number, _ protowire.Type, value []byte, number uint64) error { + switch field { + case 1: + if number == 0 || number > math.MaxInt32 { + return fmt.Errorf("invalid metadata ID %d", number) + } + id = int32(number) + case 2: + name = string(value) + } + return nil + }) + if err != nil { + return 0, "", err + } + if id == 0 || name == "" { + return 0, "", fmt.Errorf("incomplete target metadata mapping") + } + return id, name, nil +} + +func validateDecodedTangoBatch(batch decodedTangoBatch, targetCount int) error { + if len(batch.targets) != targetCount { + return fmt.Errorf("got %d targets, expected %d", len(batch.targets), targetCount) + } + for _, target := range batch.targets { + if _, exists := batch.names[target.id]; !exists { + return fmt.Errorf("target %d has no response-local name mapping", target.id) + } + if _, exists := batch.ruleTypes[target.ruleType]; !exists { + return fmt.Errorf("target %d has no response-local rule type", target.id) + } + } + return nil +} + +func gzipTangoFrame(frame []byte) ([]byte, error) { + var compressed bytes.Buffer + writer, err := gzip.NewWriterLevel(&compressed, gzip.BestSpeed) + if err != nil { + return nil, err + } + if _, err := writer.Write(frame); err != nil { + return nil, err + } + if err := writer.Close(); err != nil { + return nil, err + } + return compressed.Bytes(), nil +} + +func decodeGzipTangoFrame(compressed []byte, batch *decodedTangoBatch) error { + reader, err := gzip.NewReader(bytes.NewReader(compressed)) + if err != nil { + return err + } + frame, readErr := io.ReadAll(io.LimitReader(reader, 4_250_001)) + closeErr := reader.Close() + if readErr != nil { + return readErr + } + if closeErr != nil { + return closeErr + } + if len(frame) > 4_250_000 { + return fmt.Errorf("decoded Tango frame exceeds limit") + } + return decodeTangoFrame(frame, batch) +} diff --git a/tool/tangograph-eval/main.go b/tool/tangograph-eval/main.go new file mode 100644 index 000000000..80b79a7c6 --- /dev/null +++ b/tool/tangograph-eval/main.go @@ -0,0 +1,188 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// tangograph-eval measures Tango-backed conflict-analysis designs on a Bazel +// streamed_proto query of a real repository, without running Tango or calculating +// target hashes: graph footprint (default), per-batch signature load and check costs +// against a request histogram (-benchmark-load), and whether signatures computed once +// against an advancing base can miss conflicts (-simulate-base-drift). +package main + +import ( + "encoding/json" + "flag" + "fmt" + "io" + "os" + "runtime" +) + +type report struct { + Input string `json:"input"` + InputBytes uint64 `json:"bazel_streamed_proto_input_bytes"` + ScopePrefix string `json:"scope_prefix,omitempty"` + Query string `json:"bazel_query,omitempty"` + GoCodeRevision string `json:"go_code_revision,omitempty"` + TangoRevision string `json:"tango_revision,omitempty"` + GoVersion string `json:"go_version"` + MaxMessageBytes int `json:"max_protobuf_message_bytes"` + ImpactSourcePrefix string `json:"impact_source_prefix,omitempty"` + Graph graphMeasurement `json:"sample"` + Projected *projectedMeasurement `json:"linear_projection,omitempty"` + ProjectionQualification string `json:"projection_qualification,omitempty"` +} + +func main() { + if err := run(os.Args[1:], os.Stdout, os.Stderr); err != nil { + fmt.Fprintln(os.Stderr, "tangograph-eval:", err) + os.Exit(1) + } +} + +func run(args []string, out, errOut io.Writer) error { + flags := flag.NewFlagSet("tangograph-eval", flag.ContinueOnError) + flags.SetOutput(errOut) + input := flags.String("input", "-", "Bazel --output=streamed_proto file, or - for stdin") + scope := flags.String("scope", "", "keep only Bazel labels with this prefix (for projection by subtree)") + query := flags.String("query", "", "Bazel query expression for the report provenance") + revision := flags.String("go-code-revision", "", "go-code git commit for the report provenance") + tangoRevision := flags.String("tango-revision", "", "Tango git commit whose wire schema is modeled") + sampledFiles := flags.Uint64("sample-build-files", 0, "tracked BUILD.bazel files in -scope") + totalFiles := flags.Uint64("total-build-files", 0, "tracked BUILD.bazel files in the full repo") + maxMessageBytes := flags.Int("max-message-bytes", 4_250_000, "limit for each modeled Tango protobuf message") + impactSeeds := flags.Int("impact-seeds", 64, "number of evenly distributed source-file nodes to test for reverse-closure overlap (0 to skip)") + impactPrefix := flags.String("impact-source-prefix", "", "select source-file change seeds under this label prefix (default: all main-repo source files)") + benchmarkLoad := flags.Bool("benchmark-load", false, "measure serialized bytes, decoded heap, deserialization, and one-candidate intersection separately") + benchmarkBatches := flags.String("benchmark-batches", "100,500,1000", "comma-separated in-flight batch counts") + targetHistogramPath := flags.String("target-histogram", "", "CSV of affected_targets,requests; each batch's target count is drawn from it") + candidateTargets := flags.Int("candidate-targets", 2527, "affected targets in the incoming candidate batch") + rescale := flags.String("rescale", "", "recompute the scale model of an existing -benchmark-load JSON report with -target-histogram and exit") + simulateDrift := flags.Int("simulate-base-drift", 0, "run this many random trials of the base-drift conflict simulation and exit") + if err := flags.Parse(args); err != nil { + return err + } + if *maxMessageBytes < 128 { + return fmt.Errorf("-max-message-bytes must be at least 128") + } + if *simulateDrift > 0 { + encoder := json.NewEncoder(out) + encoder.SetIndent("", " ") + return encoder.Encode(simulateBaseDrift(*simulateDrift, 40, 1)) + } + var histogram targetHistogram + if *benchmarkLoad || *rescale != "" { + if *targetHistogramPath == "" { + return fmt.Errorf("-benchmark-load and -rescale require -target-histogram") + } + f, err := os.Open(*targetHistogramPath) + if err != nil { + return err + } + histogram, err = readTargetHistogram(f) + f.Close() + if err != nil { + return err + } + } + if *impactSeeds < 0 || *impactSeeds > 256 { + return fmt.Errorf("-impact-seeds must be between 0 and 256") + } + if *impactPrefix == "" && *scope != "" { + *impactPrefix = *scope + } + if (*sampledFiles == 0) != (*totalFiles == 0) || *sampledFiles > *totalFiles { + return fmt.Errorf("both BUILD file counts must be positive, and the sample count cannot exceed the total") + } + if (*sampledFiles != 0 || *totalFiles != 0) && *scope == "" { + return fmt.Errorf("a linear projection requires -scope") + } + if *rescale != "" { + data, err := os.ReadFile(*rescale) + if err != nil { + return err + } + var measured loadBenchReport + if err := json.Unmarshal(data, &measured); err != nil { + return err + } + measured.Scale = modelScale(measured.Cases, histogram, defaultBlobStoreProfiles, 400) + encoder := json.NewEncoder(out) + encoder.SetIndent("", " ") + return encoder.Encode(measured) + } + source := io.Reader(os.Stdin) + if *input != "-" { + f, err := os.Open(*input) + if err != nil { + return err + } + defer f.Close() + source = f + } + counter := &countingReader{source: source} + runtime.GC() + var baseline runtime.MemStats + runtime.ReadMemStats(&baseline) + graph, err := readStreamedProto(counter, *scope) + if err != nil { + return err + } + if *benchmarkLoad { + measured, err := benchmarkLoadAndCheck(graph, histogram, *benchmarkBatches, *candidateTargets, defaultBlobStoreProfiles, 400) + if err != nil { + return err + } + measured.Input = *input + measured.InputBytes = counter.bytes + measured.GoCodeRevision = *revision + encoder := json.NewEncoder(out) + encoder.SetIndent("", " ") + return encoder.Encode(measured) + } + measured, err := measureGraph(graph, baseline.HeapAlloc, *maxMessageBytes, *impactSeeds, *impactPrefix) + if err != nil { + return err + } + result := report{ + Input: *input, + InputBytes: counter.bytes, + ScopePrefix: *scope, + Query: *query, + GoCodeRevision: *revision, + TangoRevision: *tangoRevision, + GoVersion: runtime.Version(), + MaxMessageBytes: *maxMessageBytes, + ImpactSourcePrefix: *impactPrefix, + Graph: measured, + } + if *sampledFiles != 0 { + projection := projectGraph(measured, *totalFiles, *sampledFiles) + result.Projected = &projection + result.ProjectionQualification = "Linear extrapolation of an observed subset, not a full Bazel or Tango run; excludes cross-scope dependencies and may misestimate external repositories, label lengths, tags, hash cycles, and BUILD files Bazel ignores." + } + encoder := json.NewEncoder(out) + encoder.SetIndent("", " ") + return encoder.Encode(result) +} + +type countingReader struct { + source io.Reader + bytes uint64 +} + +func (c *countingReader) Read(p []byte) (int, error) { + n, err := c.source.Read(p) + c.bytes += uint64(n) + return n, err +} diff --git a/tool/tangograph-eval/measure.go b/tool/tangograph-eval/measure.go new file mode 100644 index 000000000..be0c2a4f3 --- /dev/null +++ b/tool/tangograph-eval/measure.go @@ -0,0 +1,469 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "compress/gzip" + "crypto/sha1" + "encoding/hex" + "fmt" + "io" + "math" + "runtime" + "slices" + "sort" + "strings" + + "google.golang.org/protobuf/encoding/protowire" +) + +type optimizedTarget struct { + id int32 + hash string + directDependencies []int32 + ruleType int32 + tags []int32 + root bool + external bool + attributes map[int32]int32 +} + +type compactGraph struct { + targets []optimizedTarget + names map[int32]string + ruleTypes map[int32]string + tags map[int32]string + attrNames map[int32]string + attrValues map[int32]string + missingEdges uint64 + missingMain uint64 + missingExtern uint64 + presentEdges uint64 + uniquePackage uint64 + rules uint64 + sources uint64 + generated uint64 + packageGroups uint64 + external uint64 +} + +type dictionary struct { + ids map[string]int32 + names map[int32]string +} + +func newDictionary(capacity int) dictionary { + return dictionary{ + ids: make(map[string]int32, capacity), + names: make(map[int32]string, capacity), + } +} + +func (d dictionary) id(name string) int32 { + if n, ok := d.ids[name]; ok { + return n + } + n := int32(len(d.ids) + 1) + d.ids[name] = n + d.names[n] = name + return n +} + +func compactTargets(nodes []queryTarget) (compactGraph, error) { + if len(nodes) > math.MaxInt32 { + return compactGraph{}, fmt.Errorf("cannot assign int32 IDs to %d targets", len(nodes)) + } + slices.SortFunc(nodes, func(a, b queryTarget) int { return strings.Compare(a.name, b.name) }) + names := newDictionary(len(nodes)) + for _, node := range nodes { + if _, exists := names.ids[node.name]; exists { + return compactGraph{}, fmt.Errorf("duplicate Bazel target %q", node.name) + } + names.id(node.name) + } + ruleTypes := newDictionary(64) + tags := newDictionary(64) + attrNames := newDictionary(64) + attrValues := newDictionary(256) + graph := compactGraph{targets: make([]optimizedTarget, 0, len(nodes))} + referenced := make([]bool, len(nodes)+1) + packages := make(map[string]struct{}) + for _, node := range nodes { + t := optimizedTarget{ + id: names.ids[node.name], + ruleType: ruleTypes.id(node.ruleType), + external: strings.HasPrefix(node.name, "@") || strings.HasPrefix(node.name, "//external:"), + } + switch node.ruleType { + case "source file": + graph.sources++ + case "generated file": + graph.generated++ + case "package group": + graph.packageGroups++ + default: + graph.rules++ + } + if t.external { + graph.external++ + } + if colon := strings.IndexByte(node.name, ':'); colon >= 0 { + packages[node.name[:colon]] = struct{}{} + } + for _, dependency := range node.deps { + if id, exists := names.ids[dependency]; exists { + t.directDependencies = append(t.directDependencies, id) + referenced[id] = true + graph.presentEdges++ + } else { + graph.missingEdges++ + if strings.HasPrefix(dependency, "@") { + graph.missingExtern++ + } else { + graph.missingMain++ + } + } + } + for _, tag := range node.tags { + t.tags = append(t.tags, tags.id(tag)) + } + for _, attribute := range node.attributes { + if t.attributes == nil { + t.attributes = make(map[int32]int32, len(node.attributes)) + } + t.attributes[attrNames.id(attribute.name)] = attrValues.id(attribute.value) + } + // The upstream hash is content-derived; a label SHA-1 is used only + // to model its 40-byte hex-encoded, near-incompressible wire shape. + digest := sha1.Sum([]byte(node.name)) + t.hash = hex.EncodeToString(digest[:]) + graph.targets = append(graph.targets, t) + } + for i := range graph.targets { + t := &graph.targets[i] + t.root = !referenced[t.id] && + nodes[i].ruleType != "source file" && + nodes[i].ruleType != "package group" + } + graph.names = names.names + graph.ruleTypes = ruleTypes.names + graph.tags = tags.names + graph.attrNames = attrNames.names + graph.attrValues = attrValues.names + graph.uniquePackage = uint64(len(packages)) + return graph, nil +} + +type wireMeasurement struct { + Bytes uint64 `json:"protobuf_bytes"` + GzipBytes uint64 `json:"gzip_best_speed_bytes"` + Messages uint64 `json:"messages"` + MaxBytes uint64 `json:"largest_message_bytes"` +} + +type graphMeasurement struct { + ScannedTargets uint64 `json:"bazel_query_targets_scanned"` + Targets uint64 `json:"targets"` + RuleTargets uint64 `json:"rule_targets"` + SourceFiles uint64 `json:"source_files"` + GeneratedFiles uint64 `json:"generated_files"` + PackageGroups uint64 `json:"package_groups"` + ExternalTargets uint64 `json:"external_targets"` + UniquePackages uint64 `json:"unique_packages_in_query"` + IncludedEdges uint64 `json:"included_edges"` + ExcludedEdges uint64 `json:"edges_to_targets_outside_scope"` + ExcludedMainEdges uint64 `json:"excluded_main_repo_edges"` + ExcludedExternal uint64 `json:"excluded_external_repo_edges"` + HeapNamed uint64 `json:"go_heap_parsed_graph_bytes"` + HeapCompact uint64 `json:"go_heap_compact_graph_bytes"` + HeapFingerprints uint64 `json:"go_heap_sorted_fingerprints_bytes"` + FingerprintBytes uint64 `json:"sorted_fingerprints_raw_bytes"` + TangoDefault wireMeasurement `json:"tango_default"` + TangoAllFields wireMeasurement `json:"tango_all_fields_model"` + TangoNamesOnly wireMeasurement `json:"tango_names_only_model"` + Impact impactMeasurement `json:"synthetic_source_change_overlap"` +} + +type projectedMeasurement struct { + TotalBuildFiles uint64 `json:"total_tracked_build_files"` + SampledBuildFiles uint64 `json:"sampled_tracked_build_files"` + ScalingFactor float64 `json:"linear_scaling_factor"` + Targets uint64 `json:"estimated_targets"` + IncludedEdges uint64 `json:"estimated_in_sample_edges"` + ExcludedMainEdges uint64 `json:"estimated_cross_scope_main_repo_edges"` + ExcludedExternalEdges uint64 `json:"estimated_cross_scope_external_edges"` + DefaultProtoBytes uint64 `json:"estimated_default_proto_bytes"` + AllFieldsProtoBytes uint64 `json:"estimated_all_fields_proto_bytes"` + NamesOnlyProtoBytes uint64 `json:"estimated_names_only_proto_bytes"` + DefaultGzipBytes uint64 `json:"estimated_default_gzip_bytes"` + CompactGraphHeapBytes uint64 `json:"estimated_compact_graph_go_heap_bytes"` + FingerprintHeapBytes uint64 `json:"estimated_fingerprint_go_heap_bytes"` + FingerprintStorageBytes uint64 `json:"estimated_fingerprint_storage_bytes"` +} + +func projectGraph(measured graphMeasurement, total, sampled uint64) projectedMeasurement { + scale := float64(total) / float64(sampled) + scaled := func(n uint64) uint64 { return uint64(math.Round(float64(n) * scale)) } + return projectedMeasurement{ + TotalBuildFiles: total, + SampledBuildFiles: sampled, + ScalingFactor: scale, + Targets: scaled(measured.Targets), + IncludedEdges: scaled(measured.IncludedEdges), + ExcludedMainEdges: scaled(measured.ExcludedMainEdges), + ExcludedExternalEdges: scaled(measured.ExcludedExternal), + DefaultProtoBytes: scaled(measured.TangoDefault.Bytes), + AllFieldsProtoBytes: scaled(measured.TangoAllFields.Bytes), + NamesOnlyProtoBytes: scaled(measured.TangoNamesOnly.Bytes), + DefaultGzipBytes: scaled(measured.TangoDefault.GzipBytes), + CompactGraphHeapBytes: scaled(measured.HeapCompact), + FingerprintHeapBytes: scaled(measured.HeapFingerprints), + FingerprintStorageBytes: scaled(measured.FingerprintBytes), + } +} + +// measureGraph counts the modeled protobuf frames, including the response +// wrappers, and measures live Go heap at each stage with intervening GCs. +func measureGraph(graph queryGraph, baseline uint64, maxMessageBytes, impactSeeds int, impactPrefix string) (graphMeasurement, error) { + measured := graphMeasurement{ + ScannedTargets: graph.scanned, + Targets: uint64(len(graph.targets)), + HeapNamed: heapAboveBaseline(baseline, graph.targets), + } + compact, err := compactTargets(graph.targets) + if err != nil { + return graphMeasurement{}, err + } + measured.RuleTargets = compact.rules + measured.SourceFiles = compact.sources + measured.GeneratedFiles = compact.generated + measured.PackageGroups = compact.packageGroups + measured.ExternalTargets = compact.external + measured.UniquePackages = compact.uniquePackage + measured.IncludedEdges = compact.presentEdges + measured.ExcludedEdges = compact.missingEdges + measured.ExcludedMainEdges = compact.missingMain + measured.ExcludedExternal = compact.missingExtern + measured.Impact = measureSourceImpact(compact, impactSeeds, impactPrefix) + + graph.targets = nil + measured.HeapCompact = heapAboveBaseline(baseline, compact) + for _, mode := range []payloadMode{defaultPayload, allFieldsPayload, namesOnlyPayload} { + wire, err := measureProtoStream(compact, mode, maxMessageBytes) + if err != nil { + return graphMeasurement{}, err + } + switch mode { + case defaultPayload: + measured.TangoDefault = wire + case allFieldsPayload: + measured.TangoAllFields = wire + case namesOnlyPayload: + measured.TangoNamesOnly = wire + } + } + + fingerprints := make([]uint64, 0, len(compact.names)) + for _, label := range compact.names { + fingerprints = append(fingerprints, fingerprint(label)) + } + sort.Slice(fingerprints, func(i, j int) bool { return fingerprints[i] < fingerprints[j] }) + measured.FingerprintBytes = uint64(len(fingerprints) * 8) + compact = compactGraph{} + measured.HeapFingerprints = heapAboveBaseline(baseline, fingerprints) + return measured, nil +} + +func fingerprint(label string) uint64 { + const offset, prime = uint64(14695981039346656037), uint64(1099511628211) + hash := offset + for i := range len(label) { + hash = (hash ^ uint64(label[i])) * prime + } + return hash +} + +func heapAboveBaseline(baseline uint64, reachable any) uint64 { + runtime.GC() + var stats runtime.MemStats + runtime.ReadMemStats(&stats) + runtime.KeepAlive(reachable) + if stats.HeapAlloc <= baseline { + return 0 + } + return stats.HeapAlloc - baseline +} + +type payloadMode uint8 + +const ( + defaultPayload payloadMode = iota + allFieldsPayload + namesOnlyPayload +) + +type wireCounter struct{ bytes uint64 } + +func (w *wireCounter) Write(p []byte) (int, error) { + w.bytes += uint64(len(p)) + return len(p), nil +} + +func (w *wireMeasurement) addMessage(data []byte) error { + w.Bytes += uint64(len(data)) + w.Messages++ + w.MaxBytes = max(w.MaxBytes, uint64(len(data))) + var compressed wireCounter + writer, err := gzip.NewWriterLevel(&compressed, gzip.BestSpeed) + if err != nil { + return err + } + if _, err := writer.Write(data); err != nil { + return err + } + if err := writer.Close(); err != nil { + return err + } + w.GzipBytes += compressed.bytes + return nil +} + +func measureProtoStream(graph compactGraph, mode payloadMode, maxMessageBytes int) (wireMeasurement, error) { + var measured wireMeasurement + if err := streamProtoMessages(graph, mode, maxMessageBytes, measured.addMessage); err != nil { + return wireMeasurement{}, err + } + return measured, nil +} + +func streamProtoMessages(graph compactGraph, mode payloadMode, maxMessageBytes int, emit func([]byte) error) error { + var targets []byte + sendTargets := func() error { + response := appendBytesField(nil, 1, targets) + targets = nil + return emit(response) + } + for _, target := range graph.targets { + encoded := encodeOptimizedTarget(target, mode) + entry := appendBytesField(nil, 1, encoded) + if len(targets) != 0 && len(targets)+len(entry)+10 > maxMessageBytes { + if err := sendTargets(); err != nil { + return err + } + } + targets = append(targets, entry...) + } + if err := sendTargets(); err != nil { + return err + } + var meta []byte + sendMetadata := func() error { + response := appendBytesField(nil, 2, meta) + meta = nil + return emit(response) + } + addMapping := func(field protowire.Number, mapping map[int32]string) error { + for id := int32(1); id <= int32(len(mapping)); id++ { + entry := appendVarintField(nil, 1, uint64(id)) + entry = appendStringField(entry, 2, mapping[id]) + mapField := appendBytesField(nil, field, entry) + if len(meta) != 0 && len(meta)+len(mapField)+10 > maxMessageBytes { + if err := sendMetadata(); err != nil { + return err + } + } + meta = append(meta, mapField...) + } + return nil + } + if err := addMapping(1, graph.names); err != nil { + return err + } + if mode != namesOnlyPayload { + if err := addMapping(2, graph.ruleTypes); err != nil { + return err + } + } + if mode == allFieldsPayload { + for _, mapping := range []struct { + field protowire.Number + items map[int32]string + }{{3, graph.tags}, {4, graph.attrNames}, {5, graph.attrValues}} { + if err := addMapping(mapping.field, mapping.items); err != nil { + return err + } + } + } + if err := sendMetadata(); err != nil { + return err + } + return nil +} + +func encodeOptimizedTarget(target optimizedTarget, mode payloadMode) []byte { + b := appendVarintField(nil, 1, uint64(target.id)) + if mode == namesOnlyPayload { + return b + } + if mode == allFieldsPayload { + b = appendStringField(b, 2, target.hash) + } + if len(target.directDependencies) != 0 { + var packed []byte + for _, dep := range target.directDependencies { + packed = protowire.AppendVarint(packed, uint64(dep)) + } + b = appendBytesField(b, 3, packed) + } + if mode == allFieldsPayload && len(target.tags) != 0 { + var packed []byte + for _, tag := range target.tags { + packed = protowire.AppendVarint(packed, uint64(tag)) + } + b = appendBytesField(b, 4, packed) + } + b = appendVarintField(b, 5, uint64(target.ruleType)) + if target.root { + b = appendVarintField(b, 6, 1) + } + if target.external { + b = appendVarintField(b, 7, 1) + } + if mode == allFieldsPayload { + for name, value := range target.attributes { + entry := appendVarintField(nil, 1, uint64(name)) + entry = appendVarintField(entry, 2, uint64(value)) + b = appendBytesField(b, 8, entry) + } + } + return b +} + +func appendVarintField(b []byte, field protowire.Number, value uint64) []byte { + b = protowire.AppendTag(b, field, protowire.VarintType) + return protowire.AppendVarint(b, value) +} + +func appendStringField(b []byte, field protowire.Number, value string) []byte { + b = protowire.AppendTag(b, field, protowire.BytesType) + return protowire.AppendString(b, value) +} + +func appendBytesField(b []byte, field protowire.Number, value []byte) []byte { + b = protowire.AppendTag(b, field, protowire.BytesType) + return protowire.AppendBytes(b, value) +} + +var _ io.Writer = (*wireCounter)(nil) diff --git a/tool/tangograph-eval/measure_test.go b/tool/tangograph-eval/measure_test.go new file mode 100644 index 000000000..cbd72e722 --- /dev/null +++ b/tool/tangograph-eval/measure_test.go @@ -0,0 +1,127 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "os" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestEncodeOptimizedTarget(t *testing.T) { + target := optimizedTarget{ + id: 1, + hash: "abc", + directDependencies: []int32{2}, + ruleType: 1, + tags: []int32{4}, + root: true, + external: true, + attributes: map[int32]int32{3: 5}, + } + assert.Equal(t, []byte{0x08, 0x01}, encodeOptimizedTarget(target, namesOnlyPayload)) + assert.Equal(t, []byte{0x08, 0x01, 0x1a, 0x01, 0x02, 0x28, 0x01, 0x30, 0x01, 0x38, 0x01}, + encodeOptimizedTarget(target, defaultPayload)) + assert.Equal(t, []byte{ + 0x08, 0x01, 0x12, 0x03, 'a', 'b', 'c', 0x1a, 0x01, 0x02, + 0x22, 0x01, 0x04, 0x28, 0x01, 0x30, 0x01, 0x38, 0x01, + 0x42, 0x04, 0x08, 0x03, 0x10, 0x05, + }, encodeOptimizedTarget(target, allFieldsPayload)) +} + +func TestCompactTargetsAndWire(t *testing.T) { + graph, err := compactTargets([]queryTarget{ + {name: "//app:lib", ruleType: "go_library", deps: []string{"//app:file.go", "@@repo//:outside"}, + tags: []string{"manual"}, attributes: []stringAttribute{{name: "note", value: "hi"}}}, + {name: "//app:file.go", ruleType: "source file"}, + }) + require.NoError(t, err) + assert.Equal(t, uint64(1), graph.presentEdges) + assert.Equal(t, uint64(1), graph.missingEdges) + assert.Equal(t, uint64(1), graph.uniquePackage) + assert.True(t, graph.targets[1].root) + assert.Len(t, graph.tags, 1) + assert.Len(t, graph.attrValues, 1) + + for _, tt := range []struct { + name string + mode payloadMode + }{ + {"default", defaultPayload}, + {"all fields", allFieldsPayload}, + {"names only", namesOnlyPayload}, + } { + t.Run(tt.name, func(t *testing.T) { + measured, err := measureProtoStream(graph, tt.mode, 128) + require.NoError(t, err) + assert.Positive(t, measured.Bytes) + assert.Positive(t, measured.GzipBytes) + assert.Equal(t, uint64(2), measured.Messages) + assert.LessOrEqual(t, measured.MaxBytes, uint64(128)) + }) + } + allFields, err := measureProtoStream(graph, allFieldsPayload, 128) + require.NoError(t, err) + defaultFields, err := measureProtoStream(graph, defaultPayload, 128) + require.NoError(t, err) + namesOnly, err := measureProtoStream(graph, namesOnlyPayload, 128) + require.NoError(t, err) + assert.Greater(t, allFields.Bytes, defaultFields.Bytes) + assert.Greater(t, defaultFields.Bytes, namesOnly.Bytes) +} + +func TestProtoStreamChunksWithinBudget(t *testing.T) { + var targets []queryTarget + for i := range 100 { + targets = append(targets, queryTarget{name: fmt.Sprintf("//code/module:target%03d", i), ruleType: "go_library"}) + } + graph, err := compactTargets(targets) + require.NoError(t, err) + measurement, err := measureProtoStream(graph, allFieldsPayload, 128) + require.NoError(t, err) + assert.Greater(t, measurement.Messages, uint64(2)) + assert.LessOrEqual(t, measurement.MaxBytes, uint64(128)) +} + +func TestRunReportsScopeAndProjection(t *testing.T) { + in := streamedTargets( + bazelRule("//app:lib", "go_library", "//app:file.go", "@@ext//:dep"), + bazelFile(2, "//app:file.go"), + bazelRule("@@ext//:dep", "cc_library"), + ) + input := t.TempDir() + "/query.bin" + require.NoError(t, os.WriteFile(input, in, 0o600)) + var out, errOut bytes.Buffer + require.NoError(t, run([]string{ + "-input", input, + "-scope", "//app:", + "-sample-build-files", "1", + "-total-build-files", "3", + }, &out, &errOut)) + var result report + require.NoError(t, json.Unmarshal(out.Bytes(), &result)) + assert.Equal(t, uint64(len(in)), result.InputBytes) + assert.Equal(t, uint64(3), result.Graph.ScannedTargets) + assert.Equal(t, uint64(2), result.Graph.Targets) + assert.Equal(t, uint64(1), result.Graph.IncludedEdges) + assert.Equal(t, uint64(1), result.Graph.ExcludedEdges) + require.NotNil(t, result.Projected) + assert.Equal(t, uint64(6), result.Projected.Targets) +} diff --git a/tool/tangograph-eval/parse.go b/tool/tangograph-eval/parse.go new file mode 100644 index 000000000..ebab7b839 --- /dev/null +++ b/tool/tangograph-eval/parse.go @@ -0,0 +1,260 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "bufio" + "encoding/binary" + "fmt" + "io" + "strings" + + "google.golang.org/protobuf/encoding/protowire" +) + +// The Bazel query field numbers are from src/main/protobuf/build.proto. +// Parsing only the fields Tango's ResultToTargetGraph consumes avoids pulling +// Bazel's build-proto Go dependency into SubmitQueue. +type queryTarget struct { + name string + ruleType string + deps []string + tags []string + attributes []stringAttribute +} + +type stringAttribute struct { + name string + value string +} + +type queryGraph struct { + targets []queryTarget + scanned uint64 +} + +const maxTargetMessageBytes = 64 << 20 + +func readStreamedProto(r io.Reader, scope string) (queryGraph, error) { + reader := bufio.NewReaderSize(r, 1<<20) + var graph queryGraph + for { + if _, err := reader.Peek(1); err == io.EOF { + return graph, nil + } else if err != nil { + return queryGraph{}, fmt.Errorf("peek at next target: %w", err) + } + size, err := binary.ReadUvarint(reader) + if err != nil { + return queryGraph{}, fmt.Errorf("read target size: %w", err) + } + if size > maxTargetMessageBytes { + return queryGraph{}, fmt.Errorf("target message size %d exceeds %d", size, maxTargetMessageBytes) + } + payload := make([]byte, int(size)) + if _, err := io.ReadFull(reader, payload); err != nil { + return queryGraph{}, fmt.Errorf("read target %d: %w", graph.scanned+1, err) + } + target, err := parseTarget(payload) + if err != nil { + return queryGraph{}, fmt.Errorf("decode target %d: %w", graph.scanned+1, err) + } + graph.scanned++ + if scope == "" || strings.HasPrefix(target.name, scope) { + graph.targets = append(graph.targets, target) + } + } +} + +func parseTarget(data []byte) (queryTarget, error) { + var kind uint64 + var rule, source, generated, group []byte + err := walkFields(data, func(field protowire.Number, wireType protowire.Type, value []byte, number uint64) error { + switch field { + case 1: + kind = number + case 2: + rule = value + case 3: + source = value + case 4: + generated = value + case 5: + group = value + } + return nil + }) + if err != nil { + return queryTarget{}, err + } + switch kind { + case 1: + return parseRule(rule) + case 2: + return parseNamedTarget(source, "source file") + case 3: + return parseGeneratedFile(generated) + case 4: + return parseNamedTarget(group, "package group") + default: + return queryTarget{}, fmt.Errorf("unsupported Bazel target type %d", kind) + } +} + +func parseNamedTarget(data []byte, ruleType string) (queryTarget, error) { + var target queryTarget + target.ruleType = ruleType + err := walkFields(data, func(field protowire.Number, _ protowire.Type, value []byte, _ uint64) error { + if field == 1 { + target.name = string(value) + } + return nil + }) + if err != nil { + return queryTarget{}, err + } + if target.name == "" { + return queryTarget{}, fmt.Errorf("%s without a name", ruleType) + } + return target, nil +} + +func parseGeneratedFile(data []byte) (queryTarget, error) { + target, err := parseNamedTarget(data, "generated file") + if err != nil { + return queryTarget{}, err + } + err = walkFields(data, func(field protowire.Number, _ protowire.Type, value []byte, _ uint64) error { + if field == 2 { + target.deps = append(target.deps, string(value)) + } + return nil + }) + return target, err +} + +func parseRule(data []byte) (queryTarget, error) { + var target queryTarget + err := walkFields(data, func(field protowire.Number, _ protowire.Type, value []byte, _ uint64) error { + switch field { + case 1: + target.name = string(value) + case 2: + target.ruleType = string(value) + case 4: + attribute, tags, err := parseAttribute(value) + if err != nil { + return err + } + if attribute.name != "" { + target.attributes = append(target.attributes, attribute) + } + if tags != nil { + target.tags = tags + } + case 5: + dep := string(value) + if !isDoubledPackageLabel(dep) { + target.deps = append(target.deps, dep) + } + } + return nil + }) + if err != nil { + return queryTarget{}, err + } + if target.name == "" || target.ruleType == "" { + return queryTarget{}, fmt.Errorf("Bazel rule without a name or rule class") + } + if strings.HasPrefix(target.name, "//external:") { + target.ruleType = "external rule" + target.tags = nil + target.attributes = nil + } + return target, nil +} + +func isDoubledPackageLabel(label string) bool { + workspace := strings.Index(label, "//") + if workspace < 0 { + return false + } + parts := strings.SplitN(label[workspace+2:], ":", 2) + return len(parts) == 2 && parts[0] != "" && strings.HasPrefix(parts[1], parts[0]+"/") +} + +func parseAttribute(data []byte) (stringAttribute, []string, error) { + var name, stringValue string + var kind uint64 + var hasStringValue bool + err := walkFields(data, func(field protowire.Number, _ protowire.Type, value []byte, number uint64) error { + switch field { + case 1: + name = string(value) + case 2: + kind = number + case 5: + stringValue = string(value) + hasStringValue = true + } + return nil + }) + if err != nil { + return stringAttribute{}, nil, err + } + if kind == 2 && hasStringValue { + return stringAttribute{name: name, value: stringValue}, nil, nil + } + if kind == 5 && name == "tags" { + var tags []string + err := walkFields(data, func(field protowire.Number, _ protowire.Type, value []byte, _ uint64) error { + if field == 6 { + tags = append(tags, string(value)) + } + return nil + }) + return stringAttribute{}, tags, err + } + return stringAttribute{}, nil, nil +} + +func walkFields(data []byte, visit func(protowire.Number, protowire.Type, []byte, uint64) error) error { + for len(data) != 0 { + field, wireType, tagBytes := protowire.ConsumeTag(data) + if tagBytes < 0 { + return protowire.ParseError(tagBytes) + } + data = data[tagBytes:] + var value []byte + var number uint64 + var consumed int + switch wireType { + case protowire.BytesType: + value, consumed = protowire.ConsumeBytes(data) + case protowire.VarintType: + number, consumed = protowire.ConsumeVarint(data) + default: + consumed = protowire.ConsumeFieldValue(field, wireType, data) + } + if consumed < 0 { + return protowire.ParseError(consumed) + } + if err := visit(field, wireType, value, number); err != nil { + return err + } + data = data[consumed:] + } + return nil +} diff --git a/tool/tangograph-eval/parse_test.go b/tool/tangograph-eval/parse_test.go new file mode 100644 index 000000000..319e19680 --- /dev/null +++ b/tool/tangograph-eval/parse_test.go @@ -0,0 +1,125 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "bytes" + "encoding/binary" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/encoding/protowire" +) + +func streamedTargets(targets ...[]byte) []byte { + var out []byte + for _, target := range targets { + out = binary.AppendUvarint(out, uint64(len(target))) + out = append(out, target...) + } + return out +} + +func bazelRule(name, ruleType string, deps ...string) []byte { + rule := appendStringField(nil, 1, name) + rule = appendStringField(rule, 2, ruleType) + for _, dep := range deps { + rule = appendStringField(rule, 5, dep) + } + target := appendVarintField(nil, 1, 1) + return appendBytesField(target, 2, rule) +} + +func bazelFile(kind uint64, name string) []byte { + inner := appendStringField(nil, 1, name) + target := appendVarintField(nil, 1, kind) + return appendBytesField(target, protowire.Number(kind+1), inner) +} + +func TestReadStreamedProto(t *testing.T) { + attrs := appendStringField(nil, 1, "name") + attrs = appendVarintField(attrs, 2, 2) + attrs = appendStringField(attrs, 5, "lib") + tags := appendVarintField(nil, 2, 5) + tags = appendStringField(tags, 6, "manual") + tags = appendStringField(tags, 1, "tags") + tags = appendStringField(tags, 6, "large") + + rule := appendStringField(nil, 1, "//app:lib") + rule = appendStringField(rule, 2, "go_library") + rule = appendBytesField(rule, 4, attrs) + rule = appendBytesField(rule, 4, tags) + rule = appendStringField(rule, 5, "//app:file.go") + rule = appendStringField(rule, 5, "//app:app/resources") // Tango ignores these pseudo-labels. + in := streamedTargets( + appendBytesField(appendVarintField(nil, 1, 1), 2, rule), + bazelFile(2, "//app:file.go"), + bazelRule("@@external//:lib", "go_library"), + bazelRule("//external:repo", "http_archive"), + ) + for _, tt := range []struct { + name string + scope string + count int + }{ + {"full graph", "", 4}, + {"app subtree", "//app:", 2}, + {"no matches", "//nope:", 0}, + } { + t.Run(tt.name, func(t *testing.T) { + graph, err := readStreamedProto(bytes.NewReader(in), tt.scope) + require.NoError(t, err) + assert.Equal(t, uint64(4), graph.scanned) + require.Len(t, graph.targets, tt.count) + if tt.count == 2 { + assert.Equal(t, []string{"//app:file.go"}, graph.targets[0].deps) + assert.Equal(t, []string{"manual", "large"}, graph.targets[0].tags) + assert.Equal(t, []stringAttribute{{name: "name", value: "lib"}}, graph.targets[0].attributes) + } else if tt.scope == "" { + assert.Equal(t, "external rule", graph.targets[3].ruleType) + } + }) + } +} + +func TestReadStreamedProtoRejectsCorruption(t *testing.T) { + for _, tt := range []struct { + name string + data []byte + }{ + {"short target body", []byte{5, 0x08, 0x01}}, + {"short size varint", []byte{0x80}}, + {"oversized target", binary.AppendUvarint(nil, maxTargetMessageBytes+1)}, + {"malformed protobuf", streamedTargets([]byte{0xff})}, + {"unknown Bazel target", streamedTargets(appendVarintField(nil, 1, 5))}, + } { + t.Run(tt.name, func(t *testing.T) { + _, err := readStreamedProto(bytes.NewReader(tt.data), "") + require.Error(t, err) + }) + } +} + +func TestGeneratedFileReferencesGeneratingRule(t *testing.T) { + file := appendStringField(nil, 1, "//app:generated.go") + file = appendStringField(file, 2, "//app:generator") + target := appendBytesField(appendVarintField(nil, 1, 3), 4, file) + graph, err := readStreamedProto(bytes.NewReader(streamedTargets(target)), "") + require.NoError(t, err) + require.Len(t, graph.targets, 1) + assert.Equal(t, "generated file", graph.targets[0].ruleType) + assert.Equal(t, []string{"//app:generator"}, graph.targets[0].deps) +} diff --git a/tool/tangograph-eval/scale.go b/tool/tangograph-eval/scale.go new file mode 100644 index 000000000..987882b7c --- /dev/null +++ b/tool/tangograph-eval/scale.go @@ -0,0 +1,275 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "math" + "slices" +) + +// The scale model extends the measured cases to larger B. Every measured cost is fitted +// as perTarget·T, where T is the total affected targets across the B batches; T is then drawn by Monte Carlo from the request histogram, so tail requests +// enter at their observed frequency rather than as a mean. + +// blobStoreProfile is an assumed, not measured, per-controller fetch model. +type blobStoreProfile struct { + Name string `json:"name"` + FirstByteMs float64 `json:"first_byte_ms"` + Parallel int `json:"parallel_gets"` + MiBPerSecond float64 `json:"mib_per_second"` +} + +var defaultBlobStoreProfiles = []blobStoreProfile{ + {Name: "object-store", FirstByteMs: 150, Parallel: 32, MiBPerSecond: 100}, + {Name: "kv-point-read", FirstByteMs: 5, Parallel: 16, MiBPerSecond: 100}, +} + +type linearFit struct { + PerBatch float64 `json:"per_batch"` + PerTarget float64 `json:"per_target"` +} + +func (f linearFit) at(batches int, targets float64) float64 { + return f.PerBatch*float64(batches) + f.PerTarget*targets +} + +// fitPerTarget is the ratio estimator Σy/ΣT over the measured cases. A two-term fit +// with a per-batch coefficient is rejected: B and T rise together across cases, so the +// split is unidentifiable and attributed hundreds of KiB to batches whose median is 42 +// targets. Per-batch round trips are charged separately in loadSeconds. +func fitPerTarget(targets, y []float64) linearFit { + var sumTargets, sumY float64 + for i := range y { + sumTargets += targets[i] + sumY += y[i] + } + if sumTargets == 0 { + return linearFit{} + } + return linearFit{PerTarget: sumY / sumTargets} +} + +type designCost struct { + bytes, heap, decodeMs, checkUs linearFit +} + +type scaleDesign struct { + Name string `json:"name"` + BlobBytes linearFit `json:"blob_bytes"` + HeapBytes linearFit `json:"decoded_heap_bytes"` + DeserializeMs linearFit `json:"deserialize_milliseconds"` + CheckUs linearFit `json:"check_microseconds"` + cost designCost +} + +type scaleDesignPoint struct { + Name string `json:"name"` + BlobMiB [2]float64 `json:"blob_mib_p50_p99"` + HeapMiB [2]float64 `json:"heap_mib_p50_p99"` + LoadSeconds [][2]float64 `json:"load_seconds_p50_p99_by_profile"` + CheckMs [2]float64 `json:"check_ms_p50_p99"` +} + +type scalePoint struct { + InFlight int `json:"in_flight_batches"` + Targets [2]float64 `json:"total_targets_p50_p99"` + Designs []scaleDesignPoint `json:"designs"` +} + +type scaleBudget struct { + Metric string `json:"metric"` + Limit float64 `json:"limit"` +} + +type scaleCrossover struct { + Design string `json:"design"` + Metric string `json:"metric"` + Profile string `json:"profile,omitempty"` + Limit float64 `json:"limit"` + P50Batch int `json:"first_in_flight_exceeding_at_p50"` + P99Batch int `json:"first_in_flight_exceeding_at_p99"` +} + +type scaleReport struct { + Profiles []blobStoreProfile `json:"blob_store_profiles"` + Trials int `json:"monte_carlo_trials"` + Designs []scaleDesign `json:"fitted_designs"` + Points []scalePoint `json:"points"` + Crossovers []scaleCrossover `json:"crossovers"` +} + +var scaleDisplayBatches = []int{100, 200, 500, 1000, 2000, 5000, 10000, 20000, 50000} + +var scaleBudgets = []scaleBudget{ + {Metric: "load_seconds", Limit: 1}, + {Metric: "load_seconds", Limit: 10}, + {Metric: "load_seconds", Limit: 60}, + {Metric: "heap_mib", Limit: 1024}, + {Metric: "heap_mib", Limit: 4096}, + {Metric: "heap_mib", Limit: 16384}, +} + +func fittedScaleDesigns(cases []loadBenchCase) []scaleDesign { + targets := make([]float64, len(cases)) + for i, c := range cases { + targets[i] = float64(c.TotalTargets) + } + fit := func(value func(loadBenchCase) float64) linearFit { + y := make([]float64, len(cases)) + for i, c := range cases { + y[i] = value(c) + } + return fitPerTarget(targets, y) + } + id := designCost{ + bytes: fit(func(c loadBenchCase) float64 { return float64(c.SortedIDs.BlobBytes) }), + heap: fit(func(c loadBenchCase) float64 { return float64(c.SortedIDs.DecodedHeapBytes) }), + decodeMs: fit(func(c loadBenchCase) float64 { return c.SortedIDs.DeserializeMilliseconds }), + checkUs: fit(func(c loadBenchCase) float64 { return c.SortedIDs.CheckMicroseconds }), + } + names := designCost{ + bytes: fit(func(c loadBenchCase) float64 { return float64(c.SortedNames.BlobBytes) }), + heap: fit(func(c loadBenchCase) float64 { return float64(c.SortedNames.DecodedHeapBytes) }), + decodeMs: fit(func(c loadBenchCase) float64 { return c.SortedNames.DeserializeMilliseconds }), + checkUs: fit(func(c loadBenchCase) float64 { return c.SortedNames.CheckMicroseconds }), + } + tangoHeap := fit(func(c loadBenchCase) float64 { return float64(c.TangoSnapshot.DecodedHeapBytes) }) + tangoCheck := fit(func(c loadBenchCase) float64 { return c.TangoSnapshot.CheckMicroseconds }) + tangoRaw := designCost{ + bytes: fit(func(c loadBenchCase) float64 { return float64(c.TangoSnapshot.RawBlobBytes) }), + heap: tangoHeap, + decodeMs: fit(func(c loadBenchCase) float64 { return c.TangoSnapshot.RawDeserializeMs }), + checkUs: tangoCheck, + } + tangoGzip := designCost{ + bytes: fit(func(c loadBenchCase) float64 { return float64(c.TangoSnapshot.GzipBlobBytes) }), + heap: tangoHeap, + decodeMs: fit(func(c loadBenchCase) float64 { return c.TangoSnapshot.GzipDeserializeMs }), + checkUs: tangoCheck, + } + var designs []scaleDesign + for _, d := range []struct { + name string + cost designCost + }{ + {"TangoSnapshot raw", tangoRaw}, + {"TangoSnapshot gzip", tangoGzip}, + {"NameKey", names}, + {"ID64", id}, + } { + designs = append(designs, scaleDesign{ + Name: d.name, + BlobBytes: d.cost.bytes, + HeapBytes: d.cost.heap, + DeserializeMs: d.cost.decodeMs, + CheckUs: d.cost.checkUs, + cost: d.cost, + }) + } + return designs +} + +func loadSeconds(d designCost, profile blobStoreProfile, batches int, targets float64) float64 { + return math.Ceil(float64(batches)/float64(profile.Parallel))*profile.FirstByteMs/1000 + + d.bytes.at(batches, targets)/(1<<20)/profile.MiBPerSecond + d.decodeMs.at(batches, targets)/1000 +} + +// sampledTotalTargets returns the 50th and 99th percentile of T over trials draws of +// batches requests each. +func sampledTotalTargets(h targetHistogram, batches, trials int, limit int, seed uint64) [2]float64 { + totals := make([]float64, trials) + state := seed + for trial := range totals { + var total int + for range batches { + state++ + total += min(limit, h.sample(splitMix64(state))) + } + totals[trial] = float64(total) + } + slices.Sort(totals) + return [2]float64{totals[trials/2], totals[trials*99/100]} +} + +func modelScale(cases []loadBenchCase, h targetHistogram, profiles []blobStoreProfile, trials int) scaleReport { + const graphLimit = math.MaxInt + report := scaleReport{Profiles: profiles, Trials: trials, Designs: fittedScaleDesigns(cases)} + if len(cases) == 0 { + return report + } + for _, batches := range scaleDisplayBatches { + targets := sampledTotalTargets(h, batches, trials, graphLimit, uint64(batches)) + point := scalePoint{InFlight: batches, Targets: targets} + for _, d := range report.Designs { + dp := scaleDesignPoint{Name: d.Name} + for q := range 2 { + dp.BlobMiB[q] = d.cost.bytes.at(batches, targets[q]) / (1 << 20) + dp.HeapMiB[q] = d.cost.heap.at(batches, targets[q]) / (1 << 20) + dp.CheckMs[q] = d.cost.checkUs.at(batches, targets[q]) / 1000 + } + for _, profile := range profiles { + dp.LoadSeconds = append(dp.LoadSeconds, [2]float64{ + loadSeconds(d.cost, profile, batches, targets[0]), + loadSeconds(d.cost, profile, batches, targets[1]), + }) + } + point.Designs = append(point.Designs, dp) + } + report.Points = append(report.Points, point) + } + + var grid []int + for b := 10.0; b <= 100_000; b *= 1.05 { + if n := int(math.Round(b)); len(grid) == 0 || n != grid[len(grid)-1] { + grid = append(grid, n) + } + } + gridTargets := make([][2]float64, len(grid)) + for i, batches := range grid { + gridTargets[i] = sampledTotalTargets(h, batches, trials, graphLimit, 0xc0ffee+uint64(batches)) + } + first := func(exceeds func(batches int, targets float64) bool, q int) int { + for i, batches := range grid { + if exceeds(batches, gridTargets[i][q]) { + return batches + } + } + return 0 + } + for _, d := range report.Designs { + for _, budget := range scaleBudgets { + if budget.Metric == "heap_mib" { + exceeds := func(batches int, targets float64) bool { + return d.cost.heap.at(batches, targets)/(1<<20) > budget.Limit + } + report.Crossovers = append(report.Crossovers, scaleCrossover{ + Design: d.Name, Metric: budget.Metric, Limit: budget.Limit, + P50Batch: first(exceeds, 0), P99Batch: first(exceeds, 1), + }) + continue + } + for _, profile := range profiles { + exceeds := func(batches int, targets float64) bool { + return loadSeconds(d.cost, profile, batches, targets) > budget.Limit + } + report.Crossovers = append(report.Crossovers, scaleCrossover{ + Design: d.Name, Metric: budget.Metric, Profile: profile.Name, Limit: budget.Limit, + P50Batch: first(exceeds, 0), P99Batch: first(exceeds, 1), + }) + } + } + } + return report +} diff --git a/tool/tangograph-eval/scale_test.go b/tool/tangograph-eval/scale_test.go new file mode 100644 index 000000000..e476f71a4 --- /dev/null +++ b/tool/tangograph-eval/scale_test.go @@ -0,0 +1,33 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package main + +import ( + "testing" + + "github.com/stretchr/testify/assert" +) + +func TestFitPerTarget(t *testing.T) { + fit := fitPerTarget([]float64{100, 300}, []float64{200, 600}) + assert.Zero(t, fit.PerBatch) + assert.InDelta(t, 2, fit.PerTarget, 1e-9) + assert.Equal(t, linearFit{}, fitPerTarget([]float64{0}, []float64{5})) +} + +func TestLoadSeconds(t *testing.T) { + cost := designCost{bytes: linearFit{PerTarget: 1 << 20}, decodeMs: linearFit{PerTarget: 100}} + profile := blobStoreProfile{FirstByteMs: 100, Parallel: 10, MiBPerSecond: 1} + assert.InDelta(t, 3*0.1+5+0.5, loadSeconds(cost, profile, 21, 5), 1e-9) +} diff --git a/tool/tangograph-eval/workload.go b/tool/tangograph-eval/workload.go new file mode 100644 index 000000000..4b2f948ff --- /dev/null +++ b/tool/tangograph-eval/workload.go @@ -0,0 +1,236 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "encoding/csv" + "fmt" + "io" + "slices" + "strconv" + "strings" +) + +// targetHistogram is the empirical distribution of affected targets per request: +// values[i] targets were observed for counts[i] requests. +type targetHistogram struct { + values []int + cumulative []uint64 +} + +func readTargetHistogram(r io.Reader) (targetHistogram, error) { + rows, err := csv.NewReader(r).ReadAll() + if err != nil { + return targetHistogram{}, err + } + if len(rows) < 2 { + return targetHistogram{}, fmt.Errorf("histogram has no rows") + } + var h targetHistogram + var total uint64 + for i, row := range rows[1:] { + if len(row) != 2 { + return targetHistogram{}, fmt.Errorf("histogram row %d has %d columns, expected 2", i+1, len(row)) + } + value, err := strconv.Atoi(strings.TrimSpace(row[0])) + if err != nil || value < 0 { + return targetHistogram{}, fmt.Errorf("invalid target count %q in row %d", row[0], i+1) + } + count, err := strconv.ParseUint(strings.TrimSpace(row[1]), 10, 64) + if err != nil || count == 0 { + return targetHistogram{}, fmt.Errorf("invalid request count %q in row %d", row[1], i+1) + } + if len(h.values) > 0 && value <= h.values[len(h.values)-1] { + return targetHistogram{}, fmt.Errorf("histogram values must be strictly increasing at row %d", i+1) + } + total += count + h.values = append(h.values, value) + h.cumulative = append(h.cumulative, total) + } + return h, nil +} + +func (h targetHistogram) requests() uint64 { return h.cumulative[len(h.cumulative)-1] } + +func (h targetHistogram) mean() float64 { + var sum, previous uint64 + for i, value := range h.values { + sum += uint64(value) * (h.cumulative[i] - previous) + previous = h.cumulative[i] + } + return float64(sum) / float64(h.requests()) +} + +// sample maps a uniform 64-bit value to a target count, weighting each value by its +// observed request count. +func (h targetHistogram) sample(random uint64) int { + rank := random % h.requests() + i, _ := slices.BinarySearchFunc(h.cumulative, rank, func(c, r uint64) int { + if c <= r { + return -1 + } + return 1 + }) + return h.values[i] +} + +// closureGraph is the target graph reduced to what workload generation and the +// TangoSnapshot model need: names, real direct dependencies, and reverse edges. +type closureGraph struct { + names []string + deps [][]uint32 + reverse [][]uint32 + sources []uint32 +} + +func newClosureGraph(graph queryGraph) closureGraph { + index := make(map[string]uint32, len(graph.targets)) + g := closureGraph{ + names: make([]string, len(graph.targets)), + deps: make([][]uint32, len(graph.targets)), + } + for i, target := range graph.targets { + index[target.name] = uint32(i) + g.names[i] = target.name + } + reverseDegree := make([]uint32, len(graph.targets)) + for i, target := range graph.targets { + for _, dep := range target.deps { + if d, ok := index[dep]; ok && d != uint32(i) { + g.deps[i] = append(g.deps[i], d) + reverseDegree[d]++ + } + } + if target.ruleType == "source file" && strings.HasPrefix(target.name, "//") { + g.sources = append(g.sources, uint32(i)) + } + } + g.reverse = make([][]uint32, len(graph.targets)) + for i, degree := range reverseDegree { + g.reverse[i] = make([]uint32, 0, degree) + } + for i, deps := range g.deps { + for _, d := range deps { + g.reverse[d] = append(g.reverse[d], uint32(i)) + } + } + return g +} + +// affectedSetGenerator draws closure-shaped affected sets: the reverse-dependency +// closure of a random source file, widened by adding the closures of dependencies of +// targets already in the set until k targets are collected. Widening models a change +// to more widely used targets in one region rather than k unrelated targets. +type affectedSetGenerator struct { + graph closureGraph + mark []uint32 + gen uint32 +} + +func newAffectedSetGenerator(graph closureGraph) *affectedSetGenerator { + return &affectedSetGenerator{graph: graph, mark: make([]uint32, len(graph.names))} +} + +func (a *affectedSetGenerator) affectedSet(k int, seed uint64) []uint32 { + a.gen++ + out := make([]uint32, 0, k) + state := seed + random := func(n int) int { + state++ + return int(splitMix64(state) % uint64(n)) + } + visit := func(start uint32) { + if a.mark[start] == a.gen || len(out) == k { + return + } + a.mark[start] = a.gen + out = append(out, start) + for head := len(out) - 1; head < len(out) && len(out) < k; head++ { + for _, r := range a.graph.reverse[out[head]] { + if a.mark[r] != a.gen { + a.mark[r] = a.gen + out = append(out, r) + if len(out) == k { + return + } + } + } + } + } + misses := 0 + for len(out) < k { + var node uint32 + if misses < 64 && len(a.graph.sources) > 0 { + node = a.graph.sources[random(len(a.graph.sources))] + } else { + node = uint32(random(len(a.graph.names))) + } + if a.mark[node] == a.gen { + misses++ + continue + } + visit(node) + // Widen from inside the set: a dependency of a member has a reverse closure + // containing that member, so the set grows around one region of the graph. + for stalls := 0; len(out) < k && stalls < 64; { + member := out[random(len(out))] + deps := a.graph.deps[member] + if len(deps) == 0 { + stalls++ + continue + } + before := len(out) + visit(deps[random(len(deps))]) + if len(out) == before { + stalls++ + } else { + stalls = 0 + } + } + } + return out +} + +type benchWorkload struct { + batches [][]uint32 + candidate []uint32 +} + +func (w benchWorkload) totalTargets() uint64 { + var total uint64 + for _, batch := range w.batches { + total += uint64(len(batch)) + } + return total +} + +// sampledBatchTargets draws batch i's size from seed i, so a smaller B is a prefix of +// a larger B and every case shares the same leading batches. +func sampledBatchTargets(h targetHistogram, batches, limit int) []int { + out := make([]int, batches) + for i := range out { + out[i] = min(limit, h.sample(splitMix64(0x5eed0000+uint64(i)))) + } + return out +} + +func newSampledWorkload(gen *affectedSetGenerator, sizes []int, candidateTargets int) benchWorkload { + w := benchWorkload{batches: make([][]uint32, len(sizes))} + for i, size := range sizes { + w.batches[i] = gen.affectedSet(size, uint64(i+1)<<20) + } + w.candidate = gen.affectedSet(candidateTargets, 0xd123456789abcdef) + return w +} diff --git a/tool/tangograph-eval/workload_test.go b/tool/tangograph-eval/workload_test.go new file mode 100644 index 000000000..556a61dea --- /dev/null +++ b/tool/tangograph-eval/workload_test.go @@ -0,0 +1,77 @@ +// Copyright (c) 2026 Uber Technologies, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +package main + +import ( + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestReadTargetHistogram(t *testing.T) { + h, err := readTargetHistogram(strings.NewReader("affected_targets,diffs\n0,2\n10,1\n100,1\n")) + require.NoError(t, err) + assert.Equal(t, uint64(4), h.requests()) + assert.InDelta(t, 27.5, h.mean(), 1e-9) + counts := map[int]int{} + for i := range uint64(4000) { + counts[h.sample(splitMix64(i))]++ + } + assert.InDelta(t, 2000, counts[0], 200) + assert.InDelta(t, 1000, counts[10], 150) + assert.InDelta(t, 1000, counts[100], 150) + + for _, tt := range []struct { + name string + input string + }{ + {name: "no rows", input: "affected_targets,diffs\n"}, + {name: "negative value", input: "a,b\n-1,1\n"}, + {name: "zero count", input: "a,b\n1,0\n"}, + {name: "unsorted", input: "a,b\n5,1\n4,1\n"}, + } { + t.Run(tt.name, func(t *testing.T) { + _, err := readTargetHistogram(strings.NewReader(tt.input)) + require.Error(t, err) + }) + } +} + +func TestAffectedSetIsUniqueAndClosureShaped(t *testing.T) { + gen := newAffectedSetGenerator(newClosureGraph(testChainGraph(25))) + set := gen.affectedSet(5, 7) + require.Len(t, set, 5) + assert.Equal(t, set, gen.affectedSet(5, 7)) + seen := map[uint32]bool{} + for _, index := range set { + assert.False(t, seen[index]) + seen[index] = true + } + // In a chain every node's dependents are the higher-numbered nodes, so a closure + // grown from the only source file is the first k nodes. + assert.Equal(t, []uint32{0, 1, 2, 3, 4}, set) + assert.Len(t, gen.affectedSet(25, 9), 25) +} + +func TestSampledBatchTargetsIsPrefixStable(t *testing.T) { + h := targetHistogram{values: []int{1, 50, 900}, cumulative: []uint64{5, 8, 9}} + small := sampledBatchTargets(h, 10, 100) + large := sampledBatchTargets(h, 20, 100) + assert.Equal(t, small, large[:10]) + for _, k := range large { + assert.LessOrEqual(t, k, 100) + } +}