Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/pr-description/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Generate a pull request title and description for the current branch using the p
1. Determine the base branch:
- Use the argument if provided
- Otherwise, auto-detect: `git remote set-head origin --auto >/dev/null 2>&1 && git symbolic-ref refs/remotes/origin/HEAD 2>/dev/null | sed 's|refs/remotes/||'`
- Fall back to `v3.1-dev` if the command above fails
- Fall back to `v4.0-dev` if the command above fails

2. Gather context by running these git commands:
- `git log --oneline $(git merge-base HEAD <base>)..HEAD` — all commits on this branch
Expand Down
6 changes: 3 additions & 3 deletions NIGHTLY_STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

[![Nightly Tests](https://github.com/dashpay/platform/actions/workflows/tests.yml/badge.svg?event=schedule)](https://github.com/dashpay/platform/actions/workflows/tests.yml?query=event%3Aschedule)

> **Note:** This page is manually maintained. For live results, check the [latest nightly run](https://github.com/dashpay/platform/actions/workflows/tests.yml?query=event%3Aschedule+branch%3Av3.1-dev) directly.
> **Note:** This page is manually maintained. For live results, check the [latest nightly run](https://github.com/dashpay/platform/actions/workflows/tests.yml?query=event%3Aschedule+branch%3Av4.0-dev) directly.

Nightly tests run every day at **23:00 UTC** on the `v3.1-dev` branch via the [Tests workflow](https://github.com/dashpay/platform/actions/workflows/tests.yml?query=event%3Aschedule). They exercise the full CI pipeline including Docker image builds, E2E tests, and the platform test suite.
Nightly tests run every day at **23:00 UTC** on the `v4.0-dev` branch via the [Tests workflow](https://github.com/dashpay/platform/actions/workflows/tests.yml?query=event%3Aschedule). They exercise the full CI pipeline including Docker image builds, E2E tests, and the platform test suite.

## Nightly Jobs

Expand Down Expand Up @@ -52,7 +52,7 @@ The functional tests have been intermittently failing for months. This is a know
## Links

- [All nightly runs](https://github.com/dashpay/platform/actions/workflows/tests.yml?query=event%3Aschedule)
- [Latest nightly run](https://github.com/dashpay/platform/actions/workflows/tests.yml?query=event%3Aschedule+branch%3Av3.1-dev)
- [Latest nightly run](https://github.com/dashpay/platform/actions/workflows/tests.yml?query=event%3Aschedule+branch%3Av4.0-dev)
- [Long-running Rust nightly](https://github.com/dashpay/platform/actions/workflows/tests-rs-nightly-long-running.yml)
- [Security audits (Rust)](https://github.com/dashpay/platform/actions/workflows/security-audit-rust.yml)
- [Security audits (JS - npm)](https://github.com/dashpay/platform/actions/workflows/security-audit-js-npm.yml)
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@

<p align="center">
<a href="https://github.com/dashpay/platform/actions/workflows/tests.yml"><img alt="CI" src="https://github.com/dashpay/platform/actions/workflows/tests.yml/badge.svg"></a>
<a href="https://github.com/dashpay/platform/blob/v3.1-dev/NIGHTLY_STATUS.md"><img alt="Nightly Tests" src="https://img.shields.io/github/actions/workflow/status/dashpay/platform/tests.yml?event=schedule&label=Nightly%20Tests" title="Nightly test status"></a>
<a href="https://codecov.io/gh/dashpay/platform"><img alt="codecov" src="https://codecov.io/gh/dashpay/platform/branch/v3.1-dev/graph/badge.svg"></a>
<a href="https://github.com/dashpay/platform/blob/v4.0-dev/NIGHTLY_STATUS.md"><img alt="Nightly Tests" src="https://img.shields.io/github/actions/workflow/status/dashpay/platform/tests.yml?event=schedule&label=Nightly%20Tests" title="Nightly test status"></a>
<a href="https://codecov.io/gh/dashpay/platform"><img alt="codecov" src="https://codecov.io/gh/dashpay/platform/branch/v4.0-dev/graph/badge.svg"></a>
<a href="https://github.com/dashpay/platform/graphs/commit-activity"><img alt="commit activity" src="https://img.shields.io/github/commit-activity/m/dashpay/platform"></a>
<a href="https://github.com/dashpay/platform/commits"><img alt="last commit" src="https://img.shields.io/github/last-commit/dashpay/platform"></a>
<a href="https://discordapp.com/invite/PXbUxJB"><img alt="General Chat" src="https://img.shields.io/badge/discord-General_chat-738adb"></a>
Expand All @@ -24,10 +24,10 @@

| Crate | Lines | Coverage |
|-------|------:|----------|
| [rs-dpp](./packages/rs-dpp) | 129k | [![codecov](https://codecov.io/gh/dashpay/platform/branch/v3.1-dev/graph/badge.svg?component=dpp)](https://codecov.io/gh/dashpay/platform/component/dpp) |
| [rs-drive](./packages/rs-drive) | 171k | [![codecov](https://codecov.io/gh/dashpay/platform/branch/v3.1-dev/graph/badge.svg?component=drive)](https://codecov.io/gh/dashpay/platform/component/drive) |
| [rs-drive-abci](./packages/rs-drive-abci) | 125k | [![codecov](https://codecov.io/gh/dashpay/platform/branch/v3.1-dev/graph/badge.svg?component=drive-abci)](https://codecov.io/gh/dashpay/platform/component/drive-abci) |
| [rs-sdk](./packages/rs-sdk) | 23k | [![codecov](https://codecov.io/gh/dashpay/platform/branch/v3.1-dev/graph/badge.svg?component=sdk)](https://codecov.io/gh/dashpay/platform/component/sdk) |
| [rs-dpp](./packages/rs-dpp) | 129k | [![codecov](https://codecov.io/gh/dashpay/platform/branch/v4.0-dev/graph/badge.svg?component=dpp)](https://codecov.io/gh/dashpay/platform/component/dpp) |
| [rs-drive](./packages/rs-drive) | 171k | [![codecov](https://codecov.io/gh/dashpay/platform/branch/v4.0-dev/graph/badge.svg?component=drive)](https://codecov.io/gh/dashpay/platform/component/drive) |
| [rs-drive-abci](./packages/rs-drive-abci) | 125k | [![codecov](https://codecov.io/gh/dashpay/platform/branch/v4.0-dev/graph/badge.svg?component=drive-abci)](https://codecov.io/gh/dashpay/platform/component/drive-abci) |
| [rs-sdk](./packages/rs-sdk) | 23k | [![codecov](https://codecov.io/gh/dashpay/platform/branch/v4.0-dev/graph/badge.svg?component=sdk)](https://codecov.io/gh/dashpay/platform/component/sdk) |

</details>

Expand Down
12 changes: 6 additions & 6 deletions book/src/drive/average-index-examples.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Average Index Examples

This chapter walks through a representative contract and shows how **average queries** work on Drive. Every example uses the same `grade` document type on the **grades contract** at [`packages/rs-drive/tests/supporting_files/contract/grades/grades-contract.json`](https://github.com/dashpay/platform/blob/v3.1-dev/packages/rs-drive/tests/supporting_files/contract/grades/grades-contract.json).
This chapter walks through a representative contract and shows how **average queries** work on Drive. Every example uses the same `grade` document type on the **grades contract** at [`packages/rs-drive/tests/supporting_files/contract/grades/grades-contract.json`](https://github.com/dashpay/platform/blob/v4.0-dev/packages/rs-drive/tests/supporting_files/contract/grades/grades-contract.json).

The chapter assumes you've read [Document Count Trees](./document-count-trees.md) and [Document Sum Trees](./document-sum-trees.md) — averages are built directly on top of both, so understanding count + sum trees individually is the prerequisite. Here we take that machinery as given and look at the queries that need *both*.

> **Status:** the [`document_average_worst_case`](https://github.com/dashpay/platform/blob/v3.1-dev/packages/rs-drive/benches/document_average_worst_case.rs) bench lands the reproducible numbers below — same convention as the [Count](./count-index-examples.md) and [Sum](./sum-index-examples.md) chapters. All proof sizes are measured against a 31 620-grade fixture; verified `(count, sum)` values are the actual numbers the bench's matrix reports. The full surface — primary-key global average, point lookups, range aggregates, and both carrier variants — is wired through to grovedb PR #670's verifiers end-to-end.
> **Status:** the [`document_average_worst_case`](https://github.com/dashpay/platform/blob/v4.0-dev/packages/rs-drive/benches/document_average_worst_case.rs) bench lands the reproducible numbers below — same convention as the [Count](./count-index-examples.md) and [Sum](./sum-index-examples.md) chapters. All proof sizes are measured against a 31 620-grade fixture; verified `(count, sum)` values are the actual numbers the bench's matrix reports. The full surface — primary-key global average, point lookups, range aggregates, and both carrier variants — is wired through to grovedb PR #670's verifiers end-to-end.

## Why Averages Need a New Primitive

Expand Down Expand Up @@ -70,11 +70,11 @@ Five things to internalize before reading the queries:
2. **`byClass` / `byStudent` / `bySemester` are `countable: countable` + `summable: "score"`** (no range flags). Each per-key value-tree (one per class / student / semester) is a **`CountSumTree`** carrying both metrics at one merk lookup — point-lookup averages get the same shortcut count proofs and sum proofs do.
3. **`byClassSemester` and `byStudentSemester` set both range flags** (`rangeCountable: true` AND `rangeSummable: true`). The `semester` continuation under each (class | student) value-tree is a **`ProvableCountProvableSumTree`** (PCPS), the structurally-richest tree variant — every internal merk node carries both a per-node count *and* a per-node sum. This is what `AggregateCountAndSumOnRange` walks for "average for class X in semester range [a..b]" style queries.
4. **Every `summable` index here is also `countable`.** There's no `summable`-only index in this contract — averages need both axes, so a sum-only index would be unreachable from the average surface. (Pure-sum surfaces are covered by the [tip-jar contract](./sum-index-examples.md) in the previous chapter; the grades contract is deliberately the dual-axis counterpart.)
5. **`countableAllowingOffset` on the PCPS indexes** — `rangeCountable: true` requires the count tier to be `countable` or `countableAllowingOffset` (per the rule documented in [`Index::range_countable`](https://github.com/dashpay/platform/blob/v3.1-dev/packages/rs-dpp/src/data_contract/document_type/index/mod.rs)). The offset-allowing tier upgrades the property-name tree to a `ProvableCountTree` at minimum; combined with `summable: "score"` and `rangeSummable: true` the dispatcher resolves it to PCPS.
5. **`countableAllowingOffset` on the PCPS indexes** — `rangeCountable: true` requires the count tier to be `countable` or `countableAllowingOffset` (per the rule documented in [`Index::range_countable`](https://github.com/dashpay/platform/blob/v4.0-dev/packages/rs-dpp/src/data_contract/document_type/index/mod.rs)). The offset-allowing tier upgrades the property-name tree to a `ProvableCountTree` at minimum; combined with `summable: "score"` and `rangeSummable: true` the dispatcher resolves it to PCPS.

The bench populates **50 000 grades** under a deterministic, realistic-data-shaped schedule: 500 students × 10 classes × 10 semesters = 50 000 grade documents. The score model layers three deterministic axes:

- **Per-class baseline + spread** (see [`class_profile` in the bench](https://github.com/dashpay/platform/blob/v3.1-dev/packages/rs-drive/benches/document_average_worst_case.rs)) — hard classes get low means and wide spreads; easy classes cluster near 85–90 with narrow spreads. The 10 classes have semantic names matching the chapter's references:
- **Per-class baseline + spread** (see [`class_profile` in the bench](https://github.com/dashpay/platform/blob/v4.0-dev/packages/rs-drive/benches/document_average_worst_case.rs)) — hard classes get low means and wide spreads; easy classes cluster near 85–90 with narrow spreads. The 10 classes have semantic names matching the chapter's references:

| Class | Baseline mean | Spread | Profile |
|---|---|---|---|
Expand All @@ -94,7 +94,7 @@ The bench populates **50 000 grades** under a deterministic, realistic-data-shap

A skill score is *amplified by class spread* — `skill × spread / 8` — so a +10-skill student in PHYS101 (spread=12) gains +15 over baseline, while the same student in ARTS101 (spread=4) only gains +5. This produces the realistic spread real transcripts exhibit: a strong student stands out more in hard classes, struggling students fall further behind in hard classes, easy classes flatten the curve.

**Realistic enrollment.** Not every student takes every class — that's not how transcripts work. The bench walks all 500 × 10 × 10 = 50 000 possible `(student, class, semester)` triples but only emits a grade when [`is_enrolled`](https://github.com/dashpay/platform/blob/v3.1-dev/packages/rs-drive/benches/document_average_worst_case.rs) returns true, using a deterministic per-class popularity table:
**Realistic enrollment.** Not every student takes every class — that's not how transcripts work. The bench walks all 500 × 10 × 10 = 50 000 possible `(student, class, semester)` triples but only emits a grade when [`is_enrolled`](https://github.com/dashpay/platform/blob/v4.0-dev/packages/rs-drive/benches/document_average_worst_case.rs) returns true, using a deterministic per-class popularity table:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe just use permalinks (links referring commit ID) for easier maintenance?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not make commit-ID permalinks the blanket fix in this PR. These docs are being retargeted to the live v4.0-dev branch, so branch links make sense when the intent is “show the current source for this branch.”

For examples where the prose depends on an exact implementation snapshot, permalinks could be better, but that is a broader docs-link policy/audit than this branch-retarget PR. I would keep this PR focused on making the retargeted v4.0-dev links resolve correctly.


| Class | Popularity | Profile |
|---|---|---|
Expand Down Expand Up @@ -1699,7 +1699,7 @@ The split closely parallels the count and sum chapters — point lookups for Q1

## What's Next

The chapter is grounded in the [`document_average_worst_case`](https://github.com/dashpay/platform/blob/v3.1-dev/packages/rs-drive/benches/document_average_worst_case.rs) bench's measured numbers — Q1–Q7 verify cleanly end-to-end against the shared root hash `8b15f732…ffc7`.
The chapter is grounded in the [`document_average_worst_case`](https://github.com/dashpay/platform/blob/v4.0-dev/packages/rs-drive/benches/document_average_worst_case.rs) bench's measured numbers — Q1–Q7 verify cleanly end-to-end against the shared root hash `8b15f732…ffc7`.

A natural expansion follow-up (out of scope here): a worked example of "exact-precision" averages — for callers that need fractional averages (e.g. `avg = 50.7142857…` rather than `50.99`), the protocol-level approach is to return `(count, sum)` and let the client compute in its preferred numeric format (the chapter notes this in [Numerical Considerations](#numerical-considerations) above; a future expansion could walk through the fixed-point vs. floating-point trade-offs).

Expand Down
Loading
Loading