celld-operator

WorkerApp reference

The complete celld-operator.io/v1alpha1 WorkerApp API: every spec field with its default and validation, and the full status surface. One WorkerApp is one celld fleet serving one application, so the resource, the fleet, and the app are 1:1:1.

Validation happens twice: the CRD's schema and CEL rules reject bad objects at admission, and the operator re-validates every object (including ones admitted by older CRDs) against its allowlists before creating anything. A failure sets SpecValid: False and phase Degraded. The resource name must be a DNS label of at most 48 characters.

Spec

Top-level fields

FieldTypeDefaultDescription
appVersionstring, requirednoneThe expected deployment: the Version ID celld deploy printed. celld adopts the bucket pointer on its own; status verifies that every pod serves this. The sentinel auto follows the bucket's deploy/current.json and requires an s3:// bucket and deployTrackingSecretRef. See Deploying updates.
deploymentPolicyRestart | InPlaceRestartWhether an appVersion change also runs a gated restart (Restart) or relies on celld's in-place adoption alone (InPlace).
deployTrackingSecretRefstringunsetA Secret with a read-only S3 credential (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, optional AWS_SESSION_TOKEN) for reading deploy/current.json. Required for auto; in pinned mode it enables the VersionMismatch drift check.
hostnames[]string (set)unsetHostnames routed to this app. One route object carrying every hostname is reconciled in the app's namespace; its kind depends on the operator's ingress mode. Omit for internal-only apps.
replicasint32 ≥ 13Fleet size when autoscaling is disabled; the initial size otherwise.
durabilitybucket | fleetbucketHow celld proves a write before acknowledging it (CELLD_DURABILITY). bucket: the write is in the bucket. fleet: follower nodes fsync it and the upload follows; requires storage, replicas ≥ 3, and autoscaling.minReplicas ≥ 3 when autoscaling. Changing it rolls the fleet.
websocketsboolfalseSelects the WebSocket profile: long edge timeouts where the ingress mode can express them, and a 30-minute scale-down window. No session affinity.
trustForwardedHeadersboolfalseLets X-Forwarded-Host/-Proto set request.url (CELLD_TRUST_FORWARDED_HEADERS=1). Only when every hop in front of the fleet replaces both headers.
clusterDomainDNS namecluster.localThe cluster's DNS suffix, used in peer advertise addresses and VirtualService hosts.

celld

FieldTypeDefaultDescription
celld.imagestring, requirednoneThe celld image, e.g. ghcr.io/denoland/celld:v0.6.2. The tag must carry a numeric major.minor.patch version (also when digest-pinned), and the image must match the operator's allowedImagePrefixes.
celld.updateStrategyRolling | RecreateRollingRolling is partition-stepped and restoring-gated, and is refused across a non-rolling boundary. Recreate scales to zero first, an availability event by design. Downgrades from v0.3+ to older runtimes are refused either way.

bucket

Bucket identity (name, endpoint, storageAccount) is immutable; migrate to a new WorkerApp to change it.

FieldTypeDefaultDescription
bucket.namestring, requirednoneBucket and prefix: s3://, gs://, or az:// (the container), e.g. s3://platform-cells/apps/chat. At most 1024 characters. The store must satisfy the storage contract.
bucket.endpointstringunsetS3-compatible endpoint when not AWS S3 (e.g. an R2 account endpoint). s3:// only; must be HTTPS and on the operator's allowedBucketEndpoints.
bucket.regionstringunsetStorage region when it cannot be inferred (auto for R2).
bucket.storageAccountstring, 3–24 lowercase alphanumericsunsetThe Azure storage account (AZURE_STORAGE_ACCOUNT_NAME). Required for az://.
bucket.credentialsFrom.iamRolestringunsetAn IAM role ARN, rendered as the EKS IRSA annotation on the fleet ServiceAccount; must be on allowedIAMRoles. The literal auto (operator-provisioned role) is not implemented and reports BucketCredentialsReady: False. Leave credentialsFrom empty for EKS Pod Identity.
bucket.credentialsFrom.azureClientIDstringunsetA Microsoft Entra workload identity client ID for az:// (AKS workload identity); must be on allowedAzureClientIDs.
bucket.credentialsFrom.secretRefstringunsetA Secret injected as environment: AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY for s3://, AZURE_STORAGE_ACCOUNT_KEY for az://. Rotation rolls the fleet.

At most one of iamRole, azureClientID, and secretRef may be set.

resources

FieldTypeDefaultDescription
resources.memoryGiint32, 1–655368Memory request and limit per pod, in GiB. The operator sets CELLD_MAX_RSS_MB to 80% of it.
resources.maxResidentCellsint32 ≥ 11000Hard per-node resident-cell admission limit (CELLD_MAX_RESIDENT_CELLS). Upstream sizing: about 1,000 cells per 8 GiB.
resources.cpuMillisint32 ≥ 1500CPU request per pod, in millicores. No CPU limit is set.
resources.ephemeralStorageGiint32 ≥ 110Ephemeral-storage limit per pod, in GiB.

storage and scheduling

FieldTypeDefaultDescription
storage.sizeGiint32, 1–65536unsetProvisions a retained PersistentVolumeClaim per ordinal for /var/lib/celld. Without storage, local state is an emptyDir. Claims survive scale-down and WorkerApp deletion. The whole storage block is immutable.
storage.storageClassNamestringcluster defaultThe claim's StorageClass. Node-local storage does not survive permanent node loss.
scheduling.nodeSelectormapunsetPod node selector, e.g. to pin fleets to a dedicated node pool.
scheduling.tolerations[]TolerationunsetPod tolerations.
scheduling.spreadAcrossZonesboolfalseWith storage, also require one pod per zone (pods with storage always require distinct hosts). Provision enough zones for maxReplicas.

service and vars

FieldTypeDefaultDescription
service.typeClusterIP | LoadBalancer | NodePortClusterIPType of the serving Service fronting the Worker listener. ClusterIP suits ingress backends and in-cluster consumers (<app>-celld.<ns>.svc:8080).
service.annotationsmapunsetMerged onto the serving Service (internal-LB flags, health-check tuning); removed again when deleted from the spec.
vars.secretRefstringunsetcelld before v0.5 only. A Secret whose vars.env key (NAME=value lines) is mounted and passed via CELLD_VARS_FILE. Rejected for v0.5+ images: use the deployment's Wrangler vars.

autoscaling

FieldTypeDefaultDescription
autoscaling.enabledboolfalseMaterializes a KEDA ScaledObject over the operator's /state-derived metrics; paused whenever the fleet is not Ready. Requires KEDA v2.12.0+.
autoscaling.minReplicasint32 ≥ 12Scale floor; keep ≥ 2 for HA. Must not exceed maxReplicas.
autoscaling.maxReplicasint32 ≥ 110The tenant's cost ceiling.
autoscaling.targets.residentCellUtilizationint32, 1–10070Target fleet-average percentage of occupied cells vs maxResidentCells. Any shedding pod triggers scale-up regardless.
autoscaling.targets.p95LatencyMsint32 ≥ 1unsetOptional latency target from Istio destination telemetry in the same Prometheus. Unset disables the latency signal.

telemetry

FieldTypeDefaultDescription
telemetry.enabledbooltrueEnables celld tracing. false sets CELLD_OTEL=0.
telemetry.sinkbucket | otlpbucketDestination. Unset means bucket, unless otlpEndpoint is set, which implies otlp.
telemetry.otlpEndpointstringhttp://localhost:4318 for otlpCollector base URL. On celld v0.5+ it becomes the value of CELLD_OTEL.
telemetry.retentionstring30dCELLD_OTEL_RETENTION for the bucket sink, or none to defer to bucket lifecycle rules. Ignored by the otlp sink.

Status

Phase

PhaseMeaning
PendingCreating or converging: pods not all ready, the fleet state incomplete, restoring above zero, or some pod not yet serving the expected deployment.
ReadyEvery pod is ready and updated, the fleet reports restoring=0, and every pod serves the expected deployment with nothing draining or swapping.
RollingOutThe gated rollout is stepping through ordinals, or waiting for KEDA to pause.
RecreatingA Recreate change is in progress: scale to zero, wait for every old pod to disappear, start the new fleet.
DegradedA refusal: the spec failed validation (SpecValid: False), or a celld change is not rolling-safe, is an unsupported downgrade, or names an unparseable image. The message names it. Rollouts have no timeout, so a stuck gate stays in RollingOut.

Fields

FieldDescription
status.expectedAppVersionThe resolved deployment target: spec.appVersion, or the tracked pointer in auto mode.
status.rolledOutAppVersionSet only when every pod serves the expected version with nothing draining or swapping; empty otherwise.
status.deployments[]Fresh per-pod observations from /state: name, version, generation, draining, swapping. Missing pods are not inferred.
status.rollout.partitionThe StatefulSet partition owned by the rollout controller; 0 when no rollout is in progress.
status.rollout.waitingOnThe gate the operator is waiting on, e.g. chat-celld-2: not ready or waiting for rollout settle interval. Empty when not waiting.
status.fleet.readyReady pods, from the StatefulSet.
status.fleet.restoringFleet-wide sum of cold activations holding or awaiting a permit. Rollouts step only at 0.

Conditions

ConditionMeaning when not True
SpecValidInvalidConfiguration: schema, CEL, or allowlist validation failed; nothing is created or changed until it is fixed.
Available / Progressing / DegradedPhase reflection; Progressing and Degraded carry waitingOn as their message.
FleetStateReadyUnavailable: the operator could not get a complete /state sweep (a pod missing, unready, or unreachable).
BucketCredentialsReadyProvisioningNotImplemented: iamRole: auto is not implemented; annotate the fleet ServiceAccount yourself.
DeployTrackingReadyPresent only with tracking. BucketUnreachable (holding the last known version), UnsupportedStore (auto on gs:///az://), or VersionMismatch (pinned mode: the bucket pointer and spec.appVersion disagree).
IngressReadyPresent only with hostnames. Unknown for Ingress and VirtualService; False for missing CRDs, route errors, or dropped retries. See Networking.
MeshPolicyReadyIstioUnavailable: NetworkPolicy alone guards :8081.
AutoscalingReadyPresent only with autoscaling. KEDAUnavailable: spec.autoscaling has no effect.

Printer columns

$ kubectl get workerapps -n tenant-acme
NAME   PHASE   APP                READY   RESTORING   AGE
chat   Ready   a3f9c1d2e4b57081   3       0           2m

Complete example

A production-shaped fleet with fleet durability, persistent storage, bucket tracking, and autoscaling. The IAM role and endpoint must be on the operator's allowlists.

apiVersion: celld-operator.io/v1alpha1
kind: WorkerApp
metadata:
  name: chat
  namespace: tenant-acme
spec:
  hostnames: ["chat.acme.example.com"]
  appVersion: auto
  deployTrackingSecretRef: chat-deploy-reader
  deploymentPolicy: InPlace
  celld:
    image: ghcr.io/denoland/celld:v0.6.2
    updateStrategy: Rolling
  replicas: 3
  bucket:
    name: s3://platform-cells/apps/chat
    endpoint: https://ACCOUNT.r2.cloudflarestorage.com
    region: auto
    credentialsFrom:
      iamRole: arn:aws:iam::123456789012:role/celld-chat
  resources:
    memoryGi: 8
    maxResidentCells: 1000
    cpuMillis: 2000
  durability: fleet
  storage:
    sizeGi: 100
    storageClassName: your-durable-class  # pods now require distinct hosts
  service:
    type: ClusterIP
  websockets: true
  autoscaling:
    enabled: true
    minReplicas: 3
    maxReplicas: 10
    targets:
      residentCellUtilization: 70
      p95LatencyMs: 250
  telemetry:
    enabled: true
    retention: 30d
Edit this page on GitHubUpdated 2026-10-18