Fix dataset versioning: create immutable datasets per update and enable historical data retrieval

Ernesto Gutierrez
2026-03-31T14:43:46Z

### Background / Problem

Currently, RAFS reuses the same **dataset ID** when bulk data is updated. This causes a **dataset versioning bug**:

* The **Dataset Service only exposes the latest blob**, not historical versions.
* When a WPC ( SamplesAnalysis, SaturationFunctionSet etc.) record is versioned, its historical versions **still point to the same dataset ID**, making it impossible to retrieve the correct historical bulk data.
* As a result, **GET requests for WPC record only return (latest) bulk data**, breaking versioning guarantees and auditability.

This violates the expected immutability semantics of record versions and prevents reliable historical access.

## Proposed Solution

### 1. Immutable datasets per update (POST)

* **Do not reuse dataset IDs on update**
* Every bulk data update must:
* Create a **new dataset** and produce a **new dataset ID**
* Attach that dataset ID to the **new WPC record version**
* Older WPC versions must continue referencing their original dataset IDs

### 2. GET endpoint version support

* Extend RAFS GET endpoints to support **explicit WPC record version resolution**,
* When a historical version is requested:
* RAFS must resolve the correct WPC version
* Extract the dataset ID from that version
* Fetch and return the corresponding bulk data

## Acceptance Criteria

### Dataset creation & updates

* [ ] Every bulk data update creates a **new dataset ID**
* [ ] Dataset IDs are **never reused** across WPC record versions
* [ ] Each WPC record version references exactly one immutable dataset ID

---

### Historical correctness

* [ ] Given two versions of the same WPC record: Each version references a **different dataset ID**
* [ ] Fetching bulk data for an older WPC version returns the **correct historical content**, not the latest data

---

### GET endpoint behavior

* [ ] RAFS GET endpoints support fetching data by **explicit WPC record version**
* [ ] When a versioned record is requested:
* RAFS resolves the correct WPC version
* RAFS uses the dataset ID stored in that version
* [ ] If no version is provided assume latests version is requested

---

### Backward compatibility & safety

* [ ] Existing clients that request the latest record version continue to work unchanged
* [ ] No regression in current POST / GET flows for non-versioned access
* [ ] Clear error is returned if a requested record version does not exist