Architecture

Three controllers, three CRDs, one Server-Side Apply engine. This page covers the system components, reconciliation flow, and key design decisions.

System Overview

Datasource support

MySQL: fully supported (v1.0+). PostgreSQL: planned for v1.2.

Components at a Glance

Component	Purpose	Example
LynqHub	Reads database, creates LynqNode CRs	MySQL every 30s → 6 LynqNode CRs
LynqForm	Resource blueprint per row	Deployment + Service per active node
LynqNode	Instance for one row × one form	acme-corp-web-app → 5 K8s resources

Naming: LynqNode CRs follow {uid}-{form-name}. A hub with 3 active rows (acme, beta, corp) and 2 forms (web-app, worker) creates 6 LynqNodes: acme-web-app, acme-worker, beta-web-app, beta-worker, corp-web-app, corp-worker.

Reconciliation Flow

Three-Controller Design

LynqHub Controller

Syncs the database on spec.source.syncInterval (default: 30s):

1. Queries external datasource; filters rows where activate is truthy
2. Calculates desired LynqNode set: referencingForms × activeRows
3. Creates missing LynqNode CRs, updates existing ones, deletes excess
4. Emits events: LynqNodeDeleting, LynqNodeDeleted, LynqNodeDeletionFailed
5. Updates status.{referencingTemplates, desired, ready, failed}

LynqForm Controller

Validates form-hub relationships:

• Verifies spec.hubId references an existing LynqHub
• Ensures resource IDs are unique within the form
• Validates Go template syntax
• Detects dependency cycles in dependIds

LynqNode Controller

The core reconciler. Runs on LynqNode create/update, child-resource changes (Owns() watches), and 30s periodic requeue:

1. Finalizer handling — run cleanup before deletion, then remove finalizer (lynqnode.operator.lynq.sh/finalizer)
2. Template evaluation — render all resource specs with node data
3. Orphan detection — compare status.appliedResources with current desired set; handle each orphan per its DeletionPolicy
4. Dependency resolution — build DAG from dependIds; fail fast on cycles
5. Force-reapply gating — if time.Since(status.lastFullReconcileAt) >= ForceReapplyInterval (default 10 min), set forceReapply=true so this cycle bypasses the per-resource skip check. Nil lastFullReconcileAt is treated as "stamp now, defer first force by one full interval" — protects against re-apply storms on controller restart.
6. Resource application — apply in topological order; skip if lynq.sh/applied-hash on the live resource matches the desired-spec hash and forceReapply is false; skip if a dependency failed (respects skipOnDependencyFailure)
7. Readiness gate — wait for Ready condition when waitForReady: true (default); timeout measured from lynq.sh/apply-start-time annotation (stamped at apply, preserved across reconciles when spec is unchanged) at timeoutSeconds (default: 300)
8. Status update — write readyResources, failedResources, desiredResources, appliedResources; after a successful force-reapply, advance lastFullReconcileAt to now

Key Design Patterns

Server-Side Apply (SSA)

All resources use SSA with fieldManager: lynq-operator. This means:

• Operator owns only the fields it sets; other controllers (HPA, ESO, Kyverno mutate webhooks, etc.) can own other fields and Lynq preserves them
• Conflict detection when another manager owns a field Lynq wants to change
• ConflictPolicy: Force overrides with force=true flag (SSA only — does not apply to patchStrategy: merge or replace)
• Drift correction operates on two channels (see below)

Drift Correction

Two channels work together:

1. Watch-driven (immediate). Owns() watches on child resources fire on metadata.generation or non-lynq.sh/* annotation changes. If the external edit also altered or stripped lynq.sh/applied-hash, the next reconcile detects the hash mismatch and re-applies. If the edit preserved applied-hash, the skip path short-circuits and the next channel handles it.
2. Periodic force-reapply (~10 min, gated by lastFullReconcileAt). Every ForceReapplyInterval, the controller bypasses the per-resource skip check and re-applies every child resource unconditionally — the backstop for external mutations that preserved applied-hash on a child resource.

lynq.sh/applied-hash and lynq.sh/apply-start-time are stamped atomically as part of the same SSA / Update payload — there is no follow-up MergePatch. This is the contract that makes the annotation-only skip path race-free.

Resource Tracking

Two mechanisms, chosen automatically based on namespace and deletion policy:

Scenario	Mechanism	Why
Same-namespace, DeletionPolicy: Delete	OwnerReference	Kubernetes GC handles cleanup automatically
Cross-namespace, Namespace resources, DeletionPolicy: Retain	Labels lynq.sh/node + lynq.sh/node-namespace	OwnerReferences can't cross namespaces; Retain requires manual lifecycle

Dependency Management

deployments:
  - id: app
    # ...
services:
  - id: svc
    dependIds: ["app"]   # applies only after app is ready
    waitForReady: true

Lynq builds a DAG, detects cycles (fails fast), performs topological sort, and applies in order. Blocked dependencies (not-yet-ready) wait silently; failed dependencies trigger skipOnDependencyFailure logic.

Orphan Resource Management

When a resource is removed from a LynqForm template, Lynq detects it by comparing status.appliedResources (previous state) against the current desired set. The stored DeletionPolicy annotation determines what happens next:

• DeletionPolicy: Delete — resource is removed from the cluster
• DeletionPolicy: Retain — resource receives orphan markers and stays:
  • Label: lynq.sh/orphaned: "true"
  • Annotation: lynq.sh/orphaned-at: "<RFC3339>"
  • Annotation: lynq.sh/orphaned-reason: "RemovedFromTemplate"

Re-adding a resource to the template removes all orphan markers and re-adopts it.

# Find all orphaned resources cluster-wide
kubectl get all -A -l lynq.sh/orphaned=true

Performance

Controller Concurrency

--hub-concurrency=3    # default
--form-concurrency=5   # default
--node-concurrency=10  # default

Watch Predicates

Lynq watches 12 resource types via Owns() (same-namespace) and Watches() (cross-namespace, label-based). The shared predicate fires on generation changes, non-lynq.sh/* annotation changes, and on a curated set of status fields needed for real-time readiness propagation (Deployment / StatefulSet / DaemonSet replica & condition counts, Job success/failure counts, Ingress load-balancer status, HPA current/desired replicas). Other status-only updates and internal lynq.sh/* annotation rewrites are filtered.

Requeue Strategy

The 30-second periodic requeue (RequeueAfter: 30s) ensures child resource status changes (e.g., a Deployment becoming Ready) are reflected in the LynqNode status quickly, without relying solely on watch events.

See Also

• Introduction — what Lynq is and when to use it
• API Reference — full CRD schemas for LynqHub, LynqForm, LynqNode
• Policies — creation, deletion, and conflict policy reference
• Dependencies — dependency graph, ordering, and failure handling