> ## Documentation Index
> Fetch the complete documentation index at: https://oxy.tech/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Anomaly Inbox Triage, Atomic Writes for Oxy Functions, UberEats and NetSuite Sources, and ClickHouse Observability
> Triage anomalies in bulk, write transactionally from Oxy Functions, and pull from two new Airway sources.
**August 18, 2026** · v0.5.127 – v0.5.129
### New Features
#### Insights Inbox: Feedback, Paging, and Bulk Triage
The Insights Inbox (**Semantic Layer → Anomalies**) held one page of anomalies and gave no sign whether an **Ack** or **Dismiss** had landed. All three gaps are closed:
* **Actions report what the server actually wrote** - Acking or dismissing now spins the button it was clicked on, disables both while it runs, and reports the outcome: zero rows warns rather than celebrates, and a partial apply says "3 of 5" instead of a blanket success. An action stays disabled until its rows come back, so a second click can't re-send it.
* **A row shows its status** - Anything past `new` carries its status under the severity badge. In the **All** filter a row doesn't move after an Ack, so a disappearing button used to be the only signal that anything happened.
* **Paging past the first hundred** - The list pages through the whole workspace rather than rendering whatever one request returned, with a total that counts the workspace instead of the payload. The count is shown as unknown rather than invented when it can't be computed, so the pager offers Prev/Next instead of page numbers rather than a wrong page count.
* **Multi-select and batch triage** - Rows carry checkboxes, and a bar above the table applies one status to the whole selection, naming the count it will actually move. A failed batch keeps the selection so it can be retried.
* **A long chain is triaged whole** - Batch actions name events, not the individual buckets a page happened to show, so a scan spanning more buckets than the page displays is fully triaged rather than leaving its tail behind under a clean success.
* **Dismissed anomalies aren't reversed by accident** - Live statuses (`new` and `acknowledged`) move together, so a fresh bucket chained onto an already-acknowledged event can't be stranded in **New** under a toast saying it was handled. Dismissed buckets are left alone unless you're acting from the **Dismissed** tab.
One behavior change worth knowing: in the **All** tab a wholly-dismissed event no longer offers **Ack**. Un-dismissing is done from the **Dismissed** tab, where those buckets are what you're looking at — which is what keeps an Ack in **All** from silently reversing a dismissal you can't see from there.
#### Atomic Multi-Statement Writes for Oxy Functions
`ctx.warehouse.insert`, `exec`, and `upsert` are each a single statement, so the canonical transactional write — insert an order, then its line items, then decrement inventory — could half-apply. A new `ctx.tx()` gives an Oxy Function a real transaction:
* **The runtime owns the bracket** - `ctx.tx(database, fn)` commits when your function resolves and rolls back when it throws, rethrowing your error either way. A function that dies mid-flight leaves nothing behind — the database rolls the transaction back on disconnect, so there is no cleanup path to trust.
* **Bound parameters, not string concatenation** - `query` and `exec` inside a transaction take `$1`, `$2` placeholders with bound arguments. On a surface that accepts end-user input this is the security half, not an ergonomic one: a hostile payload round-trips as data instead of executing.
* **Results feed the next statement** - An `INSERT … RETURNING id` can be read and used by the statements that follow, which is what a batch API couldn't express.
* **Gated by the same allowlist** - A transaction resolves through `ctx.warehouse`'s fail-closed `destinations` allowlist, checked before any credential is built.
Available against any configured Postgres database, including one already pasted into workspace secrets. Other warehouse backends refuse a transaction by name rather than emulating one — an emulated transaction would report success on a half-applied write, which is the exact failure this exists to prevent.
```typescript theme={null}
await ctx.tx("orders_db", async (tx) => {
const { rows } = await tx.query(
"INSERT INTO orders (customer_id) VALUES ($1) RETURNING id",
[customerId],
);
await tx.exec("INSERT INTO order_lines (order_id, sku) VALUES ($1, $2)", [
rows[0].id,
sku,
]);
});
```
One limit to know: a `numeric` column can't be read back directly. The path fails naming the column and the cast that fixes it (`amount::text`) rather than returning a `f64` that loses cents; writing into a `numeric` works via `$1::text::numeric`.
#### UberEats and NetSuite Airway Sources
Two new first-class **Airway** sources:
* **UberEats payment details** - Pulls payment detail reports, scoped to named stores. Store scoping keys on store name rather than store ID, since UberEats ships reports with the ID blank for an entire store. A reporting period must name both year and month — half a period is refused rather than half-guessed into a month nobody named.
* **NetSuite** - Six resources over NetSuite's SuiteQL API, authenticated with a private key held in the secret manager. Note that the credential is a PEM, so whatever stores it must preserve newlines end to end; a re-wrapped key fails at connection time naming the credential rather than as an opaque signature error.
* **Column types survive a fully-null file** - Declared column types now carry through the pipeline instead of being re-inferred from data. A column that is entirely null in the first file loaded no longer drops out of the landed table, which made a table's shape depend on which file happened to load first.
#### Upload UberEats Reports from the Pipeline Page
UberEats reports had to reach the landing zone out of band before a pipeline could read them. A **Reports** tab now sits on the pipeline page and does the drop from the browser:
* **Shown only where it works** - The tab appears for an UberEats pipeline and nowhere else, and it stays hidden on a workspace that hasn't been compiled yet rather than offering an upload that would fail.
* **Batch upload** - Drop several report files at once, each with its own progress and result, so one rejected file doesn't take the rest of the batch with it.
* **Re-dropping the same file is safe** - A file is identified by its contents, so uploading the same report twice merges onto the same rows instead of landing a duplicate copy. A corrected file is genuinely different and lands as its own report.
* **Landing a file is not running the pipeline** - Uploading puts reports in the landing zone and stops there. **Run after uploading** is an explicit opt-in that fires only when something actually landed, and a run that then fails is reported as a warning — the reports are in the zone either way, so the two failures stay apart.
* **Reports can't collide across pipelines** - Every workspace and every pipeline writes under its own prefix, so two pipelines posting the same report for the same period no longer overwrite each other.
The landing zone bucket is a deployment setting rather than something a pipeline chooses. A pipeline whose `base_path` disagrees with it is refused with both values named, instead of quietly writing reports where nothing reads them.
#### ClickHouse Is Now the Only Observability Backend
Observability capture consolidates onto ClickHouse; the DuckDB, Postgres, and Airhouse backends are removed. Telemetry is a serving workload — continuous ingest, fixed query shapes, TTL retention, concurrent panel reads — and it needs to fail independently of the data planes it observes. A traces query previously poisoned the shared analytical instance and took span capture down with it for days.
* **One schema, one dialect, one failure unit** - A bad query costs that query, not the capture pipeline.
* **A removed backend fails loudly** - `OXY_OBSERVABILITY_BACKEND` set to `duckdb`, `postgres`, or `airhouse` now reports a migration error naming what to set instead, rather than silently defaulting to something that no longer works.
**Action required before upgrading.** A deployment still setting
`OXY_OBSERVABILITY_BACKEND` to a removed backend boots with observability
**disabled**. Provision ClickHouse and set `OXY_OBSERVABILITY_BACKEND=clickhouse`
plus the `OXY_CLICKHOUSE_*` settings before or together with this rollout.
Existing data in the old stores is not migrated.
***
### Platform Improvements
#### Airway
* **NetSuite items load completely** - The `items` resource only fetched rows whose last-modified date fell inside the run's window, so an item that had settled and stopped changing was never pulled at all — on one account 43 of 671 listed items had simply never landed, with nothing in the run to indicate a gap. `items` is now loaded as a full snapshot, so the next run closes the gap on its own with no backfill or schema reset.
* **Faster filtered NetSuite queries** - `transaction_lines` is now partitioned by location and sorted by item, and `transactions` sorted by transaction date, so a query filtered to one location stops reading every other location's data. Existing tables keep their current layout until compaction rewrites them, so the speedup accrues over the following runs rather than switching on at once.
* **Child tables no longer accumulate a copy per load** - A merged parent had no way to say "and drop this parent's old children", so its nested child rows appended without bound while the parent table stayed correct — one table was carrying roughly 3.4 copies of every row. Worse, it compounded: each repair run over the parents added another full copy to the children. Children are now replaced by parent, so a shrinking line count drops the stale rows too, and deletion keys on the parent's own identity rather than every propagated column, so one order's children can't take a sibling's with them. Most affected tables self-repair on their next run; a table whose rows predate the propagated parent key needs a one-off drop and a parent re-run.
#### Semantic Layer
* **Metric Tree loads again in the cloud** - Every Metric Tree call on app.oxygen-hq.com failed with "Failed to load semantic layer", in every workspace, even though the same workspace's semantic explorer, compile, and SQL queries all worked. The Metric Tree now reads the compiled semantic layer instead of a copy of the workspace on disk, so it serves from any instance rather than only the one holding a checkout. Decomposition, drill-down, distribution, and opportunity sizing are all restored.
* **A workspace that hasn't compiled yet says so** - Opening the Metric Tree before the workspace has finished compiling now reports a retryable "not ready yet" and kicks off the compile, instead of a flat server error that looked like a broken model.
#### Automations
* **Long delegated steps are no longer failed at 30 minutes** - An automation step that hands work to a sub-run — most often an Airway pipeline load — was failed once it had waited 30 minutes. Measured against 90 days of runs that genuinely completed, roughly one in eleven took longer than that, with a p99 of 1h25m. The ceiling is now 4 hours, comfortably above the observed maximum and deliberately below the pipeline lease timeout, so a genuinely stuck run surfaces as "delegation timed out" naming the step that hung rather than as a run that simply looks slow for six hours. The trade: a hung sub-automation or loop fan-out, which normally finishes in seconds, now takes 4 hours to hit the backstop instead of 30 minutes — those remain visible in the run view long before either ceiling.