Opening a custom app from the home launcher used to run it embedded inside HQ, which meant the app no longer owned the address bar. It now opens in its own tab, from every entry point:
Deep links work again - An app’s screen lives in its own URL, so a filtered view — a vendor, a month, a report — can be copied out of the address bar and sent to someone. Previously the address bar showed the launcher no matter what you were looking at.
Back and forward behave - App navigation is the app’s own history rather than a branch of HQ’s, so Back goes one screen back instead of landing somewhere unpredictable a hop or two away.
Reload stays in the app - Refreshing returns to the screen you were on rather than bouncing back to the launcher — the difference is biggest for people who live in a single app all day.
One tab per app - Clicking the same app again brings its existing tab forward instead of stacking duplicates, and tabs are kept separate per organization so working in two tenants never switches the tab out from under you.
Ordinary link behavior - App cards and rail tiles are plain links now, so cmd-click, middle-click, “open in new tab” and “copy link address” all do what the browser normally does. Both announce that they open in a new tab for screen readers.
The Oxy Starter example app grows from one page to four — an overview, a filterable list, a detail screen, and identity — navigated the way a real custom app navigates. It is the working reference for how an app should own its URL, and it doubles as the fixture for checking that Back, Forward, reload, and a pasted deep link all land where you expect.Worth knowing when reading your app’s Activity tab: navigation that only changes the query string records no separate pageview by design, since query strings are never captured. Two different filters on the same screen are one row.
A pre-aggregated rollup is a local Parquet summary the semantic model answers from instead of the warehouse. Custom apps were the one surface that could never reach one — the bundle-facing semantic query endpoint always took the warehouse path, no matter what the workspace’s semantic model declared. It does not anymore, and three silent gaps in the rollup read path closed with it:
A bundle’s query can be answered by a rollup - A store-by-month aggregate over a 37-million-row line-item table went from 4.28s to 0.14s. Row caps and column naming are identical on both paths, so the same question can’t return a different shape depending on which tier answered it.
Coarser grains reach the rollup too - A monthly question over a rollup stored by day quietly fell back to the warehouse every time, because the stored time column couldn’t be rolled up in place. 1.26s to 0.33s on a real dashboard query.
Date-range filters are no longer dropped - A range filter over a rollup was being discarded rather than declining the rollup, so a 90-day question was answered with the view’s entire history. Details below.
The rest of the semantic surfaces caught up - The rollup path now also serves anomaly scan and explain, the metric-tree operations a bundle calls, ctx.semantic inside an Oxy Function, Slack, Data App tasks, and background and queued work — all of which were resolving to the warehouse regardless of what was cached.
Anomaly monitoring insists on a fresh rollup - Every display surface serves a stale rollup and carries the Pre-aggregated badge while the rebuild catches up. A monitor scan is different in kind: its number becomes an assertion written to the Insights Inbox that can page Slack, so a scan or an explain declines a stale rollup and pays for the warehouse scan instead.
The badge still can’t lie - A rollup that won’t read falls through to the warehouse rather than failing the query, and the badge follows whichever tier actually answered.
Re-run any saved result that filtered a pre-aggregated view by date range. The range
predicate was silently erased rather than sent, so the answer covered the view’s whole
history instead of the window asked for — one case measured returned 321 trading days per
store for a 90-day question. This predates the pre-aggregation work shipped on August 24
and affected every surface that already had rollups turned on. Numbers with no date-range
filter, and every warehouse-path answer, were never affected.
Opening a custom app in the admin console shows a live preview beside its manifest, builds and request log. None of that view lived in the address bar, and the preview’s own navigation was being written into the admin console’s history:
Every part of the view is in the URL - Device, channel, which dossier section is open, which function is selected, and where the preview itself has navigated. An operator can now send a teammate the exact view they are looking at.
Back returns to the app list - Clicking around inside a previewed dashboard used to fill the admin console’s Back stack invisibly, so Back stepped backwards inside the preview while the admin UI stayed put. Back now undoes what the operator did, not what the app did.
The preview has its own back, forward and reload - A control row above the canvas, with a read-out of where the preview currently is, so the two histories stay legible as two: window chrome walks the console, this row walks the app inside it.
The popped-out dossier is a real route - It shares the same URL binding rather than falling back to its own local state.
Preferences stay preferences - Dock mode, dossier pinning and section collapse are personal layout, not location, so they stay on the machine instead of in the link.
Seller Central reports — ledger, FBA inventory, FBA health, shipments — no longer have to be exported by hand and uploaded to a bucket for a filesystem pipeline to read. Amazon Selling Partner is an Airway source kind, and it is an option in the New Pipeline wizard:
Set it up from the UI - Choose Amazon Selling Partner, paste the client id and the refresh token Seller Central issued for your app, and name the secrets. Values go to the secret manager; only the names are written into the .airway.yml.
Marketplace is a picker, not a free-text field - The list of reachable marketplaces is served by the backend, so the wizard can never offer one a run would reject. An unreachable marketplace id previously failed at run time with a 403, which reads like an expired credential and sends you to re-check the token you just set.
A start date is required on purpose - The connector only pulls forward, so that single date is the entire backfill policy: set it too recent and earlier history is simply absent, too far back and the first run spends its report-job budget catching up. Neither shows up in the output, so the wizard asks rather than quietly picking one.
Third-Party Connections for Custom Apps, Starting With Google Drive
A custom app can now hold a live credential for an outside service. Settings → Connections is a new workspace-admin section for connecting one:
Google Drive is the first provider - Connected with Google’s drive.file scope, which lets an app write documents and read back what it wrote. Indexing files an app did not create needs a broader, Google-verified scope and is deliberately not enabled by this.
Adding a provider is configuration, not a new integration - The existing QuickBooks flow was generalized rather than copied, so every provider shares one authorize/callback path. Connected QuickBooks accounts keep working unchanged — their callback URL is registered with Intuit and was left exactly as it was.
Larger documents fetch cleanly - ctx.fetch had a fixed 10 MiB response ceiling, which a real 9.8 MiB lease document sat just under. A function can now raise its own limit with fetch.maxResponseBytes in the manifest, bounded by a server-side ceiling, so one app’s large download can’t raise the ceiling for every other app on the instance.
Oxy Functions Can Receive Webhooks and Send Binary Data
Oxy Functions could not verify an incoming webhook or upload a file, so integrations that should run unattended had to run in a browser with someone at the keyboard. Four additions close that:
The whole request, not just its body - A handler now receives req.method and req.headers alongside req.body.
Headers are an allowlist - A short fixed list plus any x-* header, which is what lets a new provider’s signature header arrive without a platform change. Headers that carry a viewer’s Oxy session — cookie, authorization, x-api-key, x-oxy-* — are never passed through.
Signature verification is built in - ctx.crypto.hmac, ctx.crypto.verifyHmac and ctx.crypto.timingSafeEqual compare in constant time. A malformed signature returns false, so a forged request becomes a clean rejection rather than a 500; an unsupported algorithm throws, because that is an author mistake worth surfacing.
Binary request bodies - ctx.fetch accepts bodyEncoding: "base64". A request body was previously sent as text, so any byte above 0x7F was re-encoded as UTF-8 and the far end received a corrupt file with nothing to indicate it.
A custom app can now be installed the way a native app is — added to a phone’s home screen or a desktop dock, and opened without browser chrome:
Every app gets a web app manifest - Served by the platform, generated per app and per surface, so an app installs correctly whether it is reached on a path or on its own subdomain. Nothing has to be added to a bundle, and nothing about an app’s build needs to change.
An installed app keeps its identity - Renaming an app or moving its path updates the app already on someone’s home screen, instead of installing a second copy beside the first and leaving the old one pointing at nothing.
A label that fits the home screen - There is room for about twelve characters under an icon, so a long app name falls back to the app’s short slug rather than being truncated mid-word.
A rename reaches installed devices the same day - The manifest is cached for five minutes rather than indefinitely.
Per-app theme colors and screenshots, which feed the richer install prompt some browsers show, are not exposed yet. Safari offers no install prompt at all, so iOS is still Share → Add to Home Screen.
Oxy assumed everyone had a mailbox: an email address was required and unique on a user, and it was the only thing a session could use to name a person. Hourly staff working from a tablet on a wall do not have one. They can now sign in with a name and a PIN:
A roster and a PIN instead of an email - A worker picks their name from their location’s roster and enters a PIN. An email address is now optional, and email becomes one credential among several rather than the only one.
Frontline standing is not organization membership - Deliberately separate, so enrolling a shift worker never grants the workspace reach an org Member has. Databases, Secrets and warehouse settings stay out of reach by construction rather than by a policy someone could loosen later.
App access is always explicit - A frontline worker reaches only the apps they have been granted, even for an app shared organization-wide. Opening one app to the whole organization can never hand every worker on a roster every app in the tenant.
Two layers of brute-force protection - Each credential locks itself out after repeated failures, and a per-organization rate limit catches someone trying one common PIN against forty different names — a pattern a per-credential lockout never sees, because each attempt is against a different account. Every failure returns the same message, so the login screen is not a roster oracle.
Signing in survives a deploy - The roster and login endpoints read only the database, so a restart never locks a store out of its own checklists.
ctx.user.email can be null - An Oxy Function handling a frontline user receives null rather than an empty string, so a handler that emails the caller has an obvious check to write instead of a message rejected at send time.
Chat is part of the platform now rather than something to integrate:
Channels, messages and read state - Named channels scoped to an organization, with paged history, per-member read tracking and unread counts. There is deliberately no channel spanning two organizations — a conversation with no single owner has nobody who could answer a deletion request about it.
Create and join - Anyone with standing in the organization can start a channel and seed its members, or join a named channel later. Every seeded member is checked individually, so an id belonging to another tenant is refused rather than quietly given read access to the conversation.
Live delivery on every instance - Messages arrive over a live stream regardless of which instance accepted the message and which one is serving the reader, and the stream keeps working through a deploy.
Unread counts can’t drift - Counted per request rather than kept in a column every writer has to maintain, because a wrong unread badge is the most-noticed bug a chat product can ship.
Direct messages, channel administration, per-channel notification routing and message search are not in this release.
Anything that needs to tell someone something — an announcement, a piece of overdue work — now has somewhere to record that it did:
A durable inbox - Notifications are stored, with unread counts, mark-read and mark-all-read. A notification that exists only as a push is gone the moment a send fails or a device is offline, which is exactly when someone starts asking whether a store was warned.
Reads are self-scoped - You see your own notifications and nobody else’s, and someone else’s reads as not found rather than forbidden, so a refusal never confirms that it exists.
A link works from whichever surface it is opened on - Stored relative, so the same notification opens correctly from an organization subdomain, a custom-app subdomain, or the admin host. A notification outlives the request that created it, and its sender cannot know where it will be read.
Web Push where it is configured - A deployment holding a VAPID keypair sends real Web Push to registered browsers, covering Android Chrome and every desktop browser. The public key a browser needs in order to subscribe is served by the platform, so a client no longer has to be built with it baked in. Half-configured is treated as unconfigured, because a sender that reports itself as working while failing every send is worse than none.
The push carries no message content - It is a wake signal; the client re-reads the inbox, which is the record. A missed wake is repaired by the next one or by a navigation, and the push service is never handed content it has no business seeing. Wake signals expire after four hours, since a nudge landing a day later is about a shift that has ended.
A device token belongs to a device, not a person - Re-registering a token replaces the earlier registration, so a phone handed to a colleague — or an app reinstalled under a different login — never leaves one person receiving another’s notifications.
Apple and Google push (APNs and FCM) are not implemented yet.
Tasks, site visits, location launches, training and compliance all reduce to the same fact — somebody owes somebody a piece of work, at a place, by a time — and every one of them needs the same two views: assigned to me, and supervised by me. That is now one shared foundation apps build on instead of five near-identical ones:
Locations have a lifecycle and a timezone - Open, launching, pre-launch. A launching store is where the work is densest, which an open/closed flag cannot express. A per-location timezone means “due by close” is closing time at that store rather than 23:00 UTC two zones away.
Organization-defined roles - Tenants name their own roles — Shift Lead, Area Manager — instead of squeezing them into Owner / Admin / Member. These are labels for routing work and deliberately cannot become permissions, so a tenant role named “Admin” grants nothing.
Work assignable to a person or to a role - “The closing checklist” belongs to whoever is Shift Lead at that store tonight, and re-assigning it every time the rota changes is how it stops getting done. “Re-test the reading you logged wrong” belongs to a person. Assigned-to-me answers both, and a location-scoped role matches only at the locations you hold it at, so a Shift Lead at one store never sees another store’s checklist.
Work remembers why it exists - Each item records what created it — a failed answer, a visit finding, a launch template, a training path, an expiring document.
Reachable without organization membership - Assigned work is visible to whoever it was assigned to, including a frontline worker who holds no membership by design, and it keeps working through a deploy.
Recurrence, work-item comments and bulk assignment are not included.
oxy api is replaced by oxyc, published as @oxy-hq/cli — npm i -g @oxy-hq/cli, or npx @oxy-hq/cli:
Route discovery comes from the deployment - oxyc routes asks the server what it actually serves. The old command answered from a table compiled into the binary, which described what that build could mount; several mounts are mode-dependent, so it could confidently list a path that 404s on the deployment in front of you.
Endpoint shapes without leaving the terminal - oxyc schema /api/orgs prints the request and response shape for one endpoint. The endpoints a script or an agent actually has to build a body for — SQL query, project query, semantic query, and the lookups that turn a customer into ids — are documented now instead of answering with a shrug.
Shaped like gh api - -f / -F / --input / -q / --paginate behave the way the GitHub CLI’s do, and exit codes are a documented contract (oxyc exit-codes), so a script can branch on auth versus not-found versus request error rather than parsing stderr.
One login for both clients - oxyc login and the Rust oxy login read and write the same credentials file, so authenticating with either authenticates the other.
One behavior worth knowing: an expired token makes the current-user endpoint answer 200 with an empty body rather than 401, which oxyc whoami reports as no longer resolving to a user.
Database errors name their cause - Every Postgres failure through ctx.tx and ctx.oltp reached function authors as the literal string db error, so a missing table, a permission denial, and a syntax error were indistinguishable. Errors now carry the SQLSTATE code, the server’s message, and its DETAIL — [42P01] relation "store_notes" does not exist instead of db error — which also lets a handler branch on what actually went wrong rather than treating every failure the same.
Guidance messages are visible again - The errors that name their own fix, like the row cap and the amount::text cast needed to read a numeric column, were arriving behind an opaque leading cause and a tripled prefix. The prefix now has one owner, so the useful part of the message is the part you read first.
A function can write an airhouse-managed database - Every function connected with the read-only default, so ctx.warehouse.query worked and any insert or update came back Reader role cannot execute Update statements. A write now runs with the invoking user’s workspace role — Owner writes as Admin, Admin as Writer, Member and Viewer stay read-only, which is the same answer they would get querying the warehouse directly. Worth designing around: a scheduled or background run stays read-only on airhouse, so a function whose job is to write on a timer belongs on ctx.oltp or an Airway pipeline, where a scheduled run writes exactly as a request does.
A rejection now arrives as a rejection - A handler answering 400, 403 or 409 was framed as an ordinary success, so every client had to infer refusal from the shape of the body and a catch written for a status was dead code. The status now travels with the result, including on an idempotent retry, and the SDK raises a FunctionStatusError carrying the status, the parsed body and the logs captured before it. Non-2xx results are also no longer cached — a refusal is a statement about one caller and one request, not a result worth replaying to everyone for the life of a cache entry. This changes SDK behavior: an app that reads { error: ... } out of a resolved value should move that handling into a catch.
Offline precaching and asset preloading actually work - The asset list stored with a published build repeated the app’s mount path in every entry, so every precache entry and every preload hint named a URL that does not exist. Pages always rendered — the served HTML carries correct paths — which is why this went unnoticed: offline caching had been silently doing nothing, and every navigation opened connections for assets that 404. Builds published before this release keep the old list until they are republished; until then they behave exactly as they have been.
An app holding a transactional database can be deleted - Deleting a custom app refused while it still owned a ctx.oltp schema, and pointed at a command that only ever acts on a local database. Releasing a single app’s database access is now its own admin operation, deliberately on a different path from the request that destroys a tenant’s entire database.
A dev server on 127.0.0.1 is no longer rejected - Custom-app queries from a local dev server reached at http://127.0.0.1:5173 failed with a 403 origin not allowed while the same server at localhost worked, and the app’s own error panel blamed the workspace instead. Loopback development origins are now recognized on any port, with the safety property preserved by requiring both ends of the request to be loopback.
A workspace that an instance could not read was reporting an empty answer rather than saying it could not look — so several surfaces rendered a healthy workspace as a bare one. Absence is now distinguished from emptiness everywhere it was being flattened:
Workspace counts are honest - The workspace list showed “0 agents · 0 automations · 0 apps” for workspaces full of them. Counts an instance can’t compute now read as unknown instead of zero.
Anomaly monitors are found - Scans reported “No monitors configured” and finished successfully having scanned nothing, for workspaces that had monitors configured. A monitor set that can’t be read is now an error rather than a silent opt-out.
The Pre-aggregations panel lists rollups - The panel vanished entirely for views with built rollups, because it was built from worker output instead of the declarations. It now lists every declared rollup with its cache state, so “Not cached” is a state you can actually see, and the aggregation badge renders its type instead of undefined.
The Tables row is back in the Context Graph - A category the instance couldn’t enumerate dropped out of the panel rather than reporting that it was unavailable.
Deleting a workspace’s files does what it says - A delete requesting file removal could report success having removed nothing.
Semantic queries reach the right view - Looker-backed queries could silently resolve to the wrong view, and a request naming a specific branch could be answered from main with no indication of the substitution. Both are now refused rather than answered wrongly.
A secret that can’t be decrypted says so - An undecryptable secret reported “no such secret”, which sent people looking for a missing row instead of a key problem.
“Not compiled yet” is retryable - Opening a workspace mid-compile reports a retryable not-ready state and starts the compile, rather than surfacing an error that reads like a broken model.
Intermittent 30-second hangs and 500s across the product — including sign-in, which every page passes through — turned out to be database connections held by background pollers rather than anything the affected pages did:
Unrunnable work is retired instead of retried forever - A queued run whose workspace no longer exists can never be resolved, but it was being skipped rather than closed out, so the next poll re-selected the same row 300ms later, permanently. Two such rows were enough to account for most of what the database was being asked to do. Runs like these are now dead-lettered along with their queued tasks, and the same treatment covers a workspace with no path on disk.
Per-run workers exit when their run does - A worker polling the durable queue had no exit condition and kept polling for the lifetime of the process, long after the run it belonged to had finished.
Connection pressure backs off rather than crowding requests out - A worker that can’t get a connection now waits, growing from 1 to 30 seconds, instead of immediately re-entering the queue ahead of user traffic.
A full database says so - The server now reports its connection budget at startup and probes connection health every 30 seconds, so “too many clients” surfaces within seconds instead of being invisible from inside the application.
The Developer Portal and published apps no longer share one cached model - A single instance reads two semantic sources: the working copy you are editing, and the compiled revision that serves published and custom apps. Both were cached under the workspace alone, so whichever was read first decided what the other saw for the next minute — an IDE panel could show main instead of the branch you were editing, or a custom app could answer from an uncommitted working copy. Both caches are now keyed on the source they were actually read from.
Semantic models are built once, not per request - The Metric Tree, World Model, custom-app queries and automation steps each rebuilt the semantic engine on every request — on exactly the workspaces with the largest layers. They now share one cache, and reloading a layer drops the models built from it.
A ClickHouse query no longer scans the whole schema first - Every query built a fresh connector that eagerly fetched the entire warehouse catalog and then threw it away — 457KB over the wire on a 5,105-column warehouse, plus a quadratic pass over every id column, before the query’s own SQL ran. The catalog is now fetched only by the schema browser that actually reads it. SELECT 1 through the SQL API went from 1.95s to 0.72s, and every query on that warehouse got the same fixed cost back.
A heavy pipeline no longer competes with the web tier - Worker pods were only tidying the queue; the work itself still ran on a node also serving the UI, so a memory-hungry pipeline could take the interface down with it. Workers now drive scheduled and stranded runs themselves, and schedules, monitor scans and health checks fire from them too.
A pipeline started from the UI runs on a worker too - The last case: a run submitted interactively executed on the node that accepted the request — the one also serving the interface, and the least able to afford a memory-heavy load. Those runs are now queued for the worker fleet like every other kind, which makes them crash-recoverable as a side effect: a run whose worker dies is picked up and resumed elsewhere instead of sitting at “running” forever.
Cancelling a run no longer corrupts it - Cancel could write “failed — cancelled by user” onto a run that was still executing, and close its live event stream on a run that had not finished, leaving the browser waiting forever. This predates the fleet change and affects a scheduled pipeline cancelled from the UI today.
A run’s event stream keeps up across instances - With a run executing on a worker, the instance streaming its events is not the one producing them, so a stream could go quiet while the run carried on. It now re-checks on a backoff and reads the run’s own state, so a finished run always ends its stream — and a run parked waiting on a person no longer holds a query open twice a second per open browser.
Large multi-file loads stop exhausting memory - A filesystem source read every file into a single batch, which is the shape that ran pods out of memory. It now streams one batch per file.
Every Airway Postgres connection is encrypted - Some connections were still being opened without TLS.
Live order events reach the people watching - Order webhooks are received by one instance and the LIVE EVENTS panel is served by another, and events were only ever delivered within a single process — so the panel showed nothing at all for orders. Camera events were unaffected, which is why the panel looked half-alive rather than dead. Events are now durable rows every instance follows, so a publish on any instance reaches every viewer exactly once.
A viewer who joins mid-shift sees history - The panel backfills the most recent events before the live stream takes over, instead of opening empty and staying that way until the next order. Rates like orders per minute are countable for the same reason: a reconnect no longer loses everything that came before it.
Trace retention actually applies - The retention rule written to ClickHouse on every boot was rejected outright for a type mismatch, and the failure was only printed rather than logged, so nothing could alert on it. No observability table has ever carried a retention policy, and traces grew without bound — which eventually produces partitions too large to merge, the state in which even a row count over the table fails. Retention now applies, and a failure to apply it is a logged error. It removes nothing inside the 90-day window.
A struggling trace store no longer floods the logs - When the span buffer filled, one warning was written per dropped span: 11,000 lines in half an hour, about half of all logging, at exactly the moment the store was already under pressure and competing for the same resources. Drops are now summarized once per flush window, with the count as a queryable field rather than a number baked into a message.
Server-side paging is visible to clients - Six admin, audit and assume-session endpoints accepted page or limit parameters and answered with a bare array carrying no total and no sign that more existed, so a client reading the first page had no way to tell it was one page. They now advertise the next page in a standard Link header, and the next link carries the filters forward so a continuation is the same result set rather than an unfiltered page two. Response bodies are unchanged, so nothing reading them is affected.
Assume-session history is bounded - It selected every session ever recorded across every tenant and then looked up an organization per row. It now pages, with a stable order so a page boundary neither repeats nor drops a row.
--paginate says when it cannot page - The endpoints that are still silent about paging warn on stderr instead of quietly returning the first page as though it were everything.
oxyc skills install no longer leaves skills that stop working days later - Run through npx, it reported every skill installed and linked them into a cache the package manager reclaims on its own schedule. The links went dead silently: the skills directory still looked full, but nothing loaded them. The install is now refused up front, naming npm i -g @oxy-hq/cli as the fix, and OXYC_SKILLS_DIR pointed somewhere durable is accepted like any other source. Anyone who installed skills through npx should reinstall from a global install — existing links may already be dangling.
The standalone oxyc binary is exercised before it ships - The binaries behind the curl install unpack their skills, templates and schemas at first run, a path that never executed anywhere in testing. Each release now runs the compiled binary and checks that it actually lists what it bundled, so a build whose assets never unpacked is caught before publish rather than by the person installing it.
“Semantic layer” is now “semantic model” everywhere - Product copy, documentation, onboarding and the agent’s own prompts still said “semantic layer” in around 500 places while the rest of the product said semantic model. All of it reads semantic model now. File names, routes and configuration keys are deliberately unchanged, so nothing in a workspace needs editing.
How a function’s database write is authorized is documented - Which database plane consults the invoking user and which does not, the workspace-role mapping for airhouse, and the four cases that stay read-only. The symptom this explains — queries fine, writes permission-denied — is the one an app author actually hits.
