> ## Documentation Index
> Fetch the complete documentation index at: https://oxy.tech/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Example: production setup on AWS

> A reference Airhouse deployment on Amazon EKS with RDS for PostgreSQL, S3, EKS Pod Identity, and the Helm chart, with Terraform and values examples.

<Warning>
  **Access required.** Airhouse is proprietary and not open source. The container image and Helm chart are available only to customers with an agreement with Oxygen. [Contact Oxygen](https://oxygen-hq.com/contact) to get access, or see [Getting access](../access).
</Warning>

This page walks through a reference production deployment on AWS, similar to the setup Oxygen runs. It's an example to adapt, not a turnkey module. Every name, size, and region below is a placeholder. Replace `<region>`, `<account-id>`, and names like `airhouse-data-<env>` with your own. For the general installation steps and all deployment options, see [Installation](../installation).

## Architecture

```mermaid theme={null}
flowchart LR
    subgraph vpc["Your VPC"]
      subgraph eks["Amazon EKS"]
        apps["Your apps / BI tools<br/>(allowed namespaces)"]
        hap["HAProxy<br/>:5445"]
        dp["Data plane pods<br/>(HPA)"]
        cp["Control plane<br/>Admin API :8080"]
        eso["External Secrets<br/>Operator"]
      end
      rds[("RDS for PostgreSQL<br/>registry + catalogs")]
    end
    s3[("S3 bucket<br/>SSE-KMS")]
    sm["Secrets Manager"]
    iam["IAM role<br/>(EKS Pod Identity)"]

    apps --> hap --> dp
    dp --> rds
    cp --> rds
    dp --> s3
    cp --> s3
    eso --> sm
    dp -. credentials .-> iam
    cp -. credentials .-> iam
```

| Piece | Choice in this example |
| - | - |
| **Compute** | Amazon EKS. Nodes come from a general-purpose pool of Graviton (`arm64`) instances, since the Airhouse image is multi-arch. |
| **Catalog and registry** | One RDS for PostgreSQL instance, in private subnets, reachable only from the EKS nodes. It holds the Airhouse registry database (`airhouse_cp`) and one catalog database per Airhouse database. Use separate instances if you want to scale or isolate them independently. |
| **Data** | One S3 bucket with SSE-KMS, public access blocked, and a lifecycle rule. Each Airhouse database writes under its own prefix. |
| **AWS credentials** | EKS Pod Identity. An IAM role is associated with the control-plane and data-plane service accounts, so pods get short-lived credentials and there are no static S3 keys. |
| **Database auth** | RDS IAM authentication for the control plane's registry and catalog-admin connections. Per-database catalog roles use passwords that Airhouse generates when it creates each database. |
| **Secrets** | AWS Secrets Manager, synced into Kubernetes by the External Secrets Operator. |
| **Client access** | Port `5445` goes through the chart's HAProxy Service (`ClusterIP`, not exposed outside the cluster). The chart doesn't restrict which pods can reach HAProxy, so this example adds its own NetworkPolicy for it. The chart's NetworkPolicies lock down the control and data planes and pin egress. |
| **Admin API** | Private. Port `8080` isn't exposed through an ingress. Only the read-only dashboard is published, behind the ingress controller's authentication. |
| **Deployment** | Argo CD installs the Helm chart from the OCI registry with a values file kept in Git. |

## Terraform: AWS resources

The snippets assume you already have a VPC and an EKS cluster, for example from the `terraform-aws-modules/vpc` and `terraform-aws-modules/eks` modules, referenced here as `module.vpc` and `module.eks`.

### KMS key and S3 bucket

```hcl s3.tf theme={null}
resource "aws_kms_key" "airhouse" {
  description         = "Airhouse data and secrets"
  enable_key_rotation = true
}

resource "aws_s3_bucket" "airhouse" {
  bucket = "airhouse-data-<env>"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "airhouse" {
  bucket = aws_s3_bucket.airhouse.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.airhouse.arn
    }
    bucket_key_enabled = true
  }
}

resource "aws_s3_bucket_public_access_block" "airhouse" {
  bucket                  = aws_s3_bucket.airhouse.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_s3_bucket_lifecycle_configuration" "airhouse" {
  bucket = aws_s3_bucket.airhouse.id

  rule {
    id     = "abort-incomplete-multipart-uploads"
    status = "Enabled"
    filter {}

    abort_incomplete_multipart_upload {
      days_after_initiation = 7
    }
  }
}
```

<Note>
  Airhouse's maintenance jobs delete Parquet files that are no longer referenced. If you enable S3 versioning for extra protection, add a lifecycle rule that expires noncurrent versions. Otherwise deleted files keep accruing storage cost.
</Note>

### RDS for PostgreSQL

```hcl rds.tf theme={null}
resource "aws_db_subnet_group" "airhouse" {
  name       = "airhouse-<env>"
  subnet_ids = module.vpc.private_subnets
}

resource "aws_security_group" "airhouse_rds" {
  name   = "airhouse-<env>-postgres"
  vpc_id = module.vpc.vpc_id

  ingress {
    description     = "Postgres from EKS nodes"
    from_port       = 5432
    to_port         = 5432
    protocol        = "tcp"
    security_groups = [
      module.eks.cluster_primary_security_group_id,
      module.eks.node_security_group_id,
    ]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# Query statistics and slow-query logging, useful for tuning catalog load.
resource "aws_db_parameter_group" "airhouse" {
  name   = "airhouse-<env>-postgres17"
  family = "postgres17"

  parameter {
    name         = "shared_preload_libraries"
    value        = "pg_stat_statements,auto_explain"
    apply_method = "pending-reboot"
  }
  parameter {
    name  = "pg_stat_statements.track"
    value = "all"
  }
  parameter {
    name  = "auto_explain.log_min_duration"
    value = "10000"
  }
  parameter {
    name  = "log_min_duration_statement"
    value = "5000"
  }
}

resource "aws_db_instance" "airhouse" {
  identifier     = "airhouse-<env>"
  engine         = "postgres"
  engine_version = "17"
  instance_class = "db.t4g.medium" # starting point; see Sizing below

  allocated_storage     = 20
  max_allocated_storage = 100
  storage_type          = "gp3"
  storage_encrypted     = true
  kms_key_id            = aws_kms_key.airhouse.arn

  db_name                       = "postgres" # the catalog-admin connection target
  username                      = "airhouse_master"
  manage_master_user_password   = true       # master password lives in Secrets Manager
  master_user_secret_kms_key_id = aws_kms_key.airhouse.arn

  iam_database_authentication_enabled = true

  multi_az               = true
  publicly_accessible    = false
  db_subnet_group_name   = aws_db_subnet_group.airhouse.name
  vpc_security_group_ids = [aws_security_group.airhouse_rds.id]
  parameter_group_name   = aws_db_parameter_group.airhouse.name

  backup_retention_period   = 14
  copy_tags_to_snapshot     = true
  deletion_protection       = true
  skip_final_snapshot       = false
  final_snapshot_identifier = "airhouse-<env>-final"

  performance_insights_enabled    = true
  enabled_cloudwatch_logs_exports = ["postgresql", "upgrade"]
  auto_minor_version_upgrade      = true
}
```

After the instance is up, create the two Postgres roles Airhouse uses. Run this once as the master user, for example from a one-off Kubernetes Job or a bastion host. RDS IAM authentication applies to roles that are granted `rds_iam`.

```sql bootstrap.sql theme={null}
CREATE DATABASE airhouse_cp;

-- Control plane: owns the registry database.
CREATE USER airhouse_cp;
GRANT rds_iam TO airhouse_cp;
ALTER DATABASE airhouse_cp OWNER TO airhouse_cp;

-- Catalog admin: creates one database and role per Airhouse database.
CREATE USER airhouse_catalog_admin CREATEDB CREATEROLE;
GRANT rds_iam TO airhouse_catalog_admin;

\c airhouse_cp
GRANT ALL ON SCHEMA public TO airhouse_cp;
```

### IAM role for EKS Pod Identity

One role serves both planes. It grants read and write on the bucket, use of the KMS key, and IAM database login for the two control-plane users.

```hcl iam.tf theme={null}
data "aws_caller_identity" "current" {}

resource "aws_iam_role" "airhouse" {
  name = "airhouse-<env>"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "pods.eks.amazonaws.com" }
      Action    = ["sts:AssumeRole", "sts:TagSession"]
      Condition = {
        StringEquals = { "aws:SourceAccount" = data.aws_caller_identity.current.account_id }
        ArnEquals    = { "aws:SourceArn" = module.eks.cluster_arn }
      }
    }]
  })
}

resource "aws_iam_policy" "airhouse" {
  name = "airhouse-<env>"
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "BucketReadWrite"
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:PutObject",
          "s3:DeleteObject",
          "s3:ListBucket",
          "s3:GetBucketLocation",
          "s3:ListBucketMultipartUploads",
          "s3:ListMultipartUploadParts",
          "s3:AbortMultipartUpload",
        ]
        Resource = [aws_s3_bucket.airhouse.arn, "${aws_s3_bucket.airhouse.arn}/*"]
      },
      {
        Sid      = "BucketKms"
        Effect   = "Allow"
        Action   = ["kms:Decrypt", "kms:GenerateDataKey*"]
        Resource = [aws_kms_key.airhouse.arn]
      },
      {
        Sid    = "RdsIamLogin"
        Effect = "Allow"
        Action = ["rds-db:connect"]
        Resource = [
          "arn:aws:rds-db:${var.region}:${data.aws_caller_identity.current.account_id}:dbuser:${aws_db_instance.airhouse.resource_id}/airhouse_cp",
          "arn:aws:rds-db:${var.region}:${data.aws_caller_identity.current.account_id}:dbuser:${aws_db_instance.airhouse.resource_id}/airhouse_catalog_admin",
        ]
      },
    ]
  })
}

resource "aws_iam_role_policy_attachment" "airhouse" {
  role       = aws_iam_role.airhouse.name
  policy_arn = aws_iam_policy.airhouse.arn
}

# The service account names must match serviceAccount.*.name in the Helm values.
resource "aws_eks_pod_identity_association" "airhouse_cp" {
  cluster_name    = module.eks.cluster_name
  namespace       = "airhouse"
  service_account = "airhouse-cp"
  role_arn        = aws_iam_role.airhouse.arn
}

resource "aws_eks_pod_identity_association" "airhouse_dp" {
  cluster_name    = module.eks.cluster_name
  namespace       = "airhouse"
  service_account = "airhouse-dp"
  role_arn        = aws_iam_role.airhouse.arn
}
```

The cluster needs the EKS Pod Identity Agent add-on. If you use IRSA instead, annotate the service accounts with the role ARN through `serviceAccount.controlPlane.annotations` and `serviceAccount.dataPlane.annotations`, and use an IRSA trust policy.

### Admin API token in Secrets Manager

```hcl secrets.tf theme={null}
resource "random_password" "airhouse_admin_token" {
  length  = 48
  special = false
}

resource "aws_secretsmanager_secret" "airhouse_admin_token" {
  name       = "airhouse/<env>/admin-token"
  kms_key_id = aws_kms_key.airhouse.arn
}

resource "aws_secretsmanager_secret_version" "airhouse_admin_token" {
  secret_id     = aws_secretsmanager_secret.airhouse_admin_token.id
  secret_string = jsonencode({ token = random_password.airhouse_admin_token.result })
}
```

Your backend reads the same secret when it calls the Admin API to create databases, users, or service accounts.

## Kubernetes: secrets and Helm values

### Sync the admin token with External Secrets

```yaml admin-token-external-secret.yaml theme={null}
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: airhouse-admin
  namespace: airhouse
spec:
  refreshInterval: 24h
  secretStoreRef:
    name: <your-secrets-manager-store>
    kind: ClusterSecretStore
  target:
    name: airhouse-admin
    creationPolicy: Owner
  data:
    - secretKey: token
      remoteRef:
        key: airhouse/<env>/admin-token
        property: token
```

If you use password authentication for Postgres instead of IAM, sync the `airhouse_cp` and `airhouse_catalog_admin` passwords the same way, and reference them with `controlPlaneDatabase.passwordSecretRef` and `catalogDatabase.adminPasswordSecretRef`.

### Helm values

```yaml values-aws.yaml theme={null}
mode: split

image:
  repository: ghcr.io/oxy-hq/airhouse
  tag: "<version>"                  # always pin the release Oxygen recommends
imagePullSecrets:
  - name: ghcr-credentials           # only if Oxygen gave you registry credentials

serviceAccount:
  create: true
  controlPlane:
    name: airhouse-cp                # must match the Pod Identity associations
  dataPlane:
    name: airhouse-dp

controlPlaneDatabase:
  host: <rds-endpoint>               # aws_db_instance.airhouse.address
  port: 5432
  database: airhouse_cp
  username: airhouse_cp
  iamAuth:
    enabled: true                    # SigV4 token instead of a password
  sslMode: require

catalogDatabase:
  adminHost: <rds-endpoint>
  adminPort: 5432
  adminDatabase: postgres
  adminUsername: airhouse_catalog_admin
  iamAuth:
    enabled: true
  sslMode: require
  tenantSslMode: require             # per-database catalog connections

s3:
  endpoint: s3.<region>.amazonaws.com   # required; must match the region
  region: <region>
  bucket: airhouse-data-<env>
  useSsl: true
  pathStyle: false                   # AWS S3 uses virtual-host addressing
  useCredentialChain: true           # credentials from EKS Pod Identity

adminToken:
  secretRef:
    name: airhouse-admin
    key: token

controlPlane:
  replicaCount: 2
  resources:
    requests: { cpu: 250m, memory: 512Mi }
    limits: { cpu: 500m, memory: 2Gi }
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: ScheduleAnyway
      labelSelector:
        matchLabels:
          app.kubernetes.io/component: control-plane
  ui:                                # read-only dashboard only; never /admin/v1 or /api/query
    enabled: true
    className: nginx
    host: airhouse.example.com
    annotations: {}                  # add your ingress controller's auth annotations here

dataPlane:
  resources:
    requests: { cpu: "2", memory: 6Gi }  # memory request == limit
    limits: { cpu: "4", memory: 6Gi }
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 6
  governance:                      # per data-plane pod; scale out with the HPA for more concurrency
    maxConcurrentQueries: 4          # 4 x 1024 MB + ~1 GB base = ~5 GB, under the 6 GiB limit
    memoryPerSessionMB: 1024         # DuckDB memory per session; spills to disk beyond it
    maxTenantInstances: 4            # warm per-database DuckDB instances kept in memory (LRU)
    podMemoryBudgetMB: 4608          # 4 instances x 1024 MB = 4096 MB fits; below the limit for untracked buffers
    threadsPerSession: 1             # 4 queries x 1 thread = 4 threads = CPU limit
    perUserMaxConcurrent: 2          # no single user takes every slot
    perTenantMaxConcurrent: 2        # no single database takes every slot
    maxConnections: 32               # open sessions per pod, including HAProxy health checks
    queryTimeoutSecs: 300            # chart default is 120
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: ScheduleAnyway
      labelSelector:
        matchLabels:
          app.kubernetes.io/component: data-plane

haproxy:
  enabled: true
  service:
    type: ClusterIP                  # in-cluster clients only

networkPolicy:
  enabled: true
  # Namespaces allowed to reach port 8080 on BOTH planes: the control-plane Admin
  # API (/admin/v1) and SQL over HTTP (/api/query), which bypasses HAProxy.
  # List only trusted namespaces. This does NOT govern port 5445 (see below).
  allowedNamespaces:
    - <backend-namespace>            # the service that calls the Admin API
    - <ingress-controller-namespace> # needed for controlPlane.ui (the ingress proxies to the CP)
    - <monitoring-namespace>         # needed for the ServiceMonitors to scrape /metrics on both planes
  # Egress is open by default. Pin Postgres to your RDS subnets.
  postgresEgressCidrs:
    - <rds-subnet-cidr-a>
    - <rds-subnet-cidr-b>
  # Pinning 443 (S3, STS, Secrets Manager) also works, but S3 uses public ranges
  # unless you have VPC interface endpoints, so leave it empty unless you do.
  httpsEgressCidrs: []
  # EKS Pod Identity serves credentials over HTTP/80 on a link-local address that
  # the 443 rule doesn't cover. Required for RDS IAM auth and the S3 credential chain.
  allowPodIdentityEndpoint: true

metrics:
  serviceMonitor:
    enabled: true
  grafanaDashboard:
    enabled: true
    folder: Airhouse
  prometheusRule:
    enabled: true

logging:
  format: json
  level: info
  logStatementText: false            # don't write SQL text to logs
```

### Install with Argo CD

```yaml airhouse-application.yaml theme={null}
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: airhouse
  namespace: argocd
spec:
  project: default
  sources:
    - repoURL: https://github.com/<your-org>/<your-gitops-repo>
      targetRevision: main
      ref: values
    - repoURL: ghcr.io/oxy-hq/helm-charts
      chart: airhouse
      targetRevision: "<chart-version>"
      helm:
        valueFiles:
          - $values/airhouse/values-aws.yaml
  destination:
    server: https://kubernetes.default.svc
    namespace: airhouse
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true
```

Without Argo CD, run `helm install airhouse oci://ghcr.io/oxy-hq/helm-charts/airhouse --version <chart-version> -n airhouse -f values-aws.yaml`.

## Client access and TLS

* **Clients inside the cluster** connect to the HAProxy Service on port `5445`, for example `host=airhouse-haproxy.airhouse.svc port=5445` for a release named `airhouse` in the `airhouse` namespace.

* **Restricting port 5445:** the chart ships NetworkPolicies for the control plane and data plane only. HAProxy has none, so any pod in the cluster can reach port `5445` (clients still need valid credentials). `networkPolicy.allowedNamespaces` doesn't change this: it only opens port `8080` on the two planes. To limit which namespaces can connect, add a policy like this one. It's not part of the chart. Its pod selector matches the labels the chart puts on HAProxy pods:

  ```yaml haproxy-networkpolicy.yaml theme={null}
  # Your own manifest, applied alongside the chart.
  apiVersion: networking.k8s.io/v1
  kind: NetworkPolicy
  metadata:
    name: airhouse-haproxy-clients
    namespace: airhouse
  spec:
    podSelector:
      matchLabels:
        app.kubernetes.io/name: airhouse
        app.kubernetes.io/instance: airhouse      # your Helm release name
        app.kubernetes.io/component: haproxy
    policyTypes:
      - Ingress
    ingress:
      - from:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: <your-app-namespace>
        ports:
          - port: 5445
            protocol: TCP
      - from:                                     # kubelet probes, and HAProxy's own stats/health port
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: kube-system
        ports:
          - port: 8404
            protocol: TCP
  ```

  Add a rule for port `5446` if you enable `haproxy.analyticsBackend`. If you expose HAProxy through a load balancer, allow the load balancer's source ranges too.

* **Clients outside the cluster:** set `haproxy.service.type: LoadBalancer` and add AWS Load Balancer Controller annotations for a Network Load Balancer. Prefer an `internal` scheme, reachable over VPN or peering, to an internet-facing one:

  ```yaml theme={null}
  haproxy:
    service:
      type: LoadBalancer
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: external
        service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
        service.beta.kubernetes.io/aws-load-balancer-scheme: internal
  ```

* **TLS for clients:** HAProxy passes TCP through, so TLS terminates on the data plane. Enable it with a `kubernetes.io/tls` Secret and point the server at the mounted files:

  ```yaml theme={null}
  tls:
    enabled: true
    secretName: airhouse-tls
  dataPlane:
    extraEnv:
      - name: AIRHOUSE_TLS_CERT
        value: /etc/airhouse/tls/tls.crt
      - name: AIRHOUSE_TLS_KEY
        value: /etc/airhouse/tls/tls.key
  ```

  Enable TLS for any client traffic that leaves the cluster. Connections from Airhouse to RDS use TLS (`sslMode: require`), and connections to S3 use HTTPS (`useSsl: true`).

* **Admin API:** keep it private. Reach it from your backend inside the cluster (control-plane Service, port `8080`), or with `kubectl port-forward` for manual administration. Don't expose it through an ingress.

## Operations

### Metrics and alerts

With `metrics.serviceMonitor.enabled`, Prometheus or a compatible scraper (for example VictoriaMetrics' agent) collects `/metrics` from both planes. The scraper's namespace must be in `networkPolicy.allowedNamespaces`, or the chart's NetworkPolicies block the scrape and none of the alerts below can fire. `metrics.prometheusRule.enabled` installs the chart's alert rules, including:

* `AirhouseQueryAvailabilityBurning` and `AirhouseQueryAvailabilitySlowBurn`: wire-protocol query error budget.
* `AirhouseQueryLatencyP99High`
* `AirhouseAdminAvailabilityBurning` and `AirhouseMintErrorRateHigh`: Admin API and credential minting.
* `AirhouseCircuitBreakerOpen`: a pod is shedding all queries after repeated backend failures.
* `AirhouseDataPlaneRestart` and `AirhouseControlPlaneLivenessRestart`
* `AirhouseQueryQueueSaturated`, `AirhouseWorkerPoolExhausted`, and `AirhouseQueriesSpillingToDisk`: capacity pressure.
* `AirhouseCPDBPoolSaturated` and `AirhouseSweeperLag`

In addition, the reference setup pages on these availability conditions, using kube-state-metrics. The Deployment names below assume a Helm release named `airhouse`:

```yaml airhouse-availability-rules.yaml theme={null}
groups:
  - name: airhouse-availability
    rules:
      - alert: AirhouseHAProxyNoReadyReplicas
        expr: kube_deployment_status_replicas_ready{namespace="airhouse", deployment="airhouse-haproxy"} == 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Airhouse HAProxy has no ready pods; clients can't connect"
      - alert: AirhouseDataPlaneNoReadyReplicas
        expr: kube_deployment_status_replicas_ready{namespace="airhouse", deployment="airhouse-dp"} == 0
        for: 3m
        labels:
          severity: critical
        annotations:
          summary: "Airhouse data plane has no ready pods"
```

A data-plane pod that fails readiness but stays alive doesn't restart on its own. `kubectl rollout restart` of the data-plane Deployment clears it.

For RDS, CloudWatch alarms on `DBLoad` (above 50% and 100% of vCPUs), `CPUUtilization` (above 80%), `DatabaseConnections`, `ReadLatency` and `WriteLatency` (above 20 ms), `FreeableMemory`, and `FreeStorageSpace` are a good baseline.

### Logs and traces

* Logs are JSON (`logging.format: json`). Collect them with your cluster's log agent, such as an OpenTelemetry Collector or Fluent Bit. Keep `logging.logStatementText: false` if SQL text may contain sensitive values.
* Traces: set `tracing.enabled: true` and `tracing.endpoint` to an OTLP/HTTP collector, for example `http://<otel-collector>.<namespace>.svc:4318`.

### Backups

* **Catalogs and registry:** RDS automated backups (14 days here) and snapshots cover every Airhouse database's catalog and the registry. Keep `deletion_protection` on, and test restores periodically.
* **Data:** Parquet files in S3 are immutable once written. S3 versioning, or replication to a second bucket, protects against accidental deletes. Restore the catalog and data to the same point in time, because a catalog references specific data files.
* **Time travel window:** snapshot expiry (below) sets how far back [time travel](../guides/tables-and-storage#snapshots-and-time-travel) can reach.

### Maintenance jobs

DuckLake needs periodic maintenance: compacting small files, expiring old snapshots, and deleting files that are no longer referenced. By default the control plane runs this in process every 6 hours and keeps 30 days of snapshots (`controlPlane.vacuum`). The reference setup moves it into the chart's vacuum CronJob, so a large merge can't compete with the control plane for memory. The CronJob is the one job that expires snapshots and deletes files:

```yaml theme={null}
controlPlane:
  vacuum:
    enabled: false                  # replaced by the vacuum CronJob below

vacuum:
  cronjob:
    enabled: true
    schedule: "0 */6 * * *"         # concurrencyPolicy Forbid: cycles never overlap
    retentionDays: 7                # snapshots older than this are expired: the time-travel window
```

Choose `retentionDays` to match how far back you need time travel and point-in-time restores.

The chart also ships a native compaction drain (`compactionDrain`) for working through a large backlog of small files. It's a manual Job: keep it suspended, and trigger it on demand instead of scheduling it. Leave its expiry, cleanup, and reap switches off so it doesn't duplicate the vacuum CronJob's work:

```yaml theme={null}
compactionDrain:
  enabled: true                     # renders a suspended CronJob to trigger from
  suspend: true                     # never runs on a schedule
  dryRun: true                      # measure only
  expireEnabled: false              # expiry and cleanup stay with the vacuum CronJob
  cleanupEnabled: false
  reapEnabled: false
```

```bash theme={null}
kubectl -n airhouse create job --from=cronjob/airhouse-cp-compact-drain airhouse-drain-$(date +%s)
```

A dry run measures and reports the backlog without changing data. Setting `dryRun: false` makes the drain rewrite Parquet files, so make that a separate, deliberate change: read the dry-run report, pause the vacuum CronJob (`vacuum.cronjob.suspend: true`) so the two don't work on the same catalogs at once, run one drain, then resume the CronJob.

### Sizing starting point

| Component | Starting point |
| - | - |
| Node pool | Graviton general-purpose and memory-optimized families (`m6g`/`m7g`/`m8g`, `r6g`/`r7g`/`r8g`), larger than `medium`, spot and on-demand. Put the data plane on on-demand capacity if you can't tolerate spot interruptions. |
| Data plane | Requests 2 CPU / 6 GiB, limits 4 CPU / 6 GiB, HPA from 2 to 6 replicas. Memory is usually the constraint. |
| Control plane | 2 replicas, requests 250m / 512 MiB, limit 2 GiB |
| HAProxy | Chart default (2 replicas) |
| RDS | `db.t4g.medium`, Multi-AZ, 20 GiB gp3 with autoscaling. Catalog rows are small, so move up for many databases or high write rates rather than for data volume. |
| Limits (per data-plane pod) | 4 concurrent queries at 1 GiB each, 2 per user and per database, 32 connections, 4 warm database instances, and a 300-second timeout. Total concurrency grows with the replica count. See [Query limits](../reference/configuration#query-limits). |

To resize, keep `maxConcurrentQueries × memoryPerSessionMB + ~1 GB` and `maxTenantInstances × memoryPerSessionMB` (which must stay under `podMemoryBudgetMB`) below `resources.limits.memory`, and keep `maxConcurrentQueries × threadsPerSession` near the CPU limit. For example, a 10 GiB pod fits 8 queries at 1 GiB. Watch `AirhouseQueriesSpillingToDisk` and pod memory. Add replicas for more concurrency, and raise pod memory together with `memoryPerSessionMB` for larger queries.

## Next steps

* [Create your first database and user](../installation#create-your-first-database-and-user)
* [Security](../concepts/security)
* [Architecture](../concepts/architecture)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.