Architecture

This document provides a detailed overview of the Lynq's architecture, including system components, reconciliation flow, and key design decisions.

System Overview

Database Support

• MySQL: Fully supported (v1.0+)
• PostgreSQL: Planned for v1.2

Architecture at a Glance

Quick reference for the three main components:

Component	Purpose	Example
LynqHub	Connects to database, syncs node rows	MySQL every 30s → Creates LynqNode CRs
LynqForm	Defines resource blueprint	Deployment + Service per node
LynqNode	Instance of a single node	acme-corp-web-app → 5 K8s resources

Workflow: Database row → LynqHub syncs → Creates LynqNode CR → LynqNode controller applies LynqForm → Kubernetes resources created.

---

Reconciliation Flow

Three-Controller Design

The operator uses a three-controller architecture to separate concerns and optimize reconciliation:

1. LynqHub Controller

Purpose: Syncs database (e.g., 1m interval) → Creates/Updates/Deletes LynqNode CRs

Responsibilities:

• Periodically queries external datasource at spec.source.syncInterval
• Filters active rows where activate field is truthy
• Calculates desired LynqNode set: referencingTemplates × activeRows
• Creates missing LynqNode CRs (naming: {uid}-{template-name})
• Updates existing LynqNode CRs with fresh data
• Deletes LynqNode CRs for inactive/removed rows (garbage collection)
• Updates Hub status with counts

Key Status Fields:

status:
  referencingTemplates: 2 # Number of templates using this hub
  desired: 6 # referencingTemplates × activeRows
  ready: 5 # Ready LynqNodes across all templates
  failed: 1 # Failed LynqNodes across all templates

2. LynqForm Controller

Purpose: Validates template-registry linkage and invariants

Responsibilities:

• Validates that spec.hubId references an existing LynqHub
• Ensures template syntax is valid (Go text/template)
• Validates resource IDs are unique within template
• Detects dependency cycles in dependIds
• Updates template status

3. LynqNode Controller

Purpose: Renders templates → Resolves dependencies → Applies resources via SSA

Responsibilities:

• Builds template variables from LynqNode spec
• Resolves resource dependencies (DAG + topological sort)
• Renders all templates (names, namespaces, specs)
• Applies resources using Server-Side Apply
• Waits for resource readiness (if waitForReady=true)
• Updates LynqNode status with resource counts and conditions
• Handles conflicts according to ConflictPolicy
• Manages finalizers for proper cleanup

CRD Architecture

LynqHub

Defines external datasource configuration and sync behavior:

apiVersion: operator.lynq.sh/v1
kind: LynqHub
metadata:
  name: my-saas-hub
spec:
  source:
    type: mysql
    mysql:
      host: mysql.default.svc.cluster.local
      port: 3306
      database: nodes
      table: node_data
      username: node_reader
      passwordRef:
        name: mysql-secret
        key: password
    syncInterval: 30s
  valueMappings:
    uid: node_id # Required
    hostOrUrl: domain # Required
    activate: is_active # Required
  extraValueMappings:
    planId: subscription_plan
    deployImage: container_image

Multi-Form Support: One hub can be referenced by multiple LynqForms, creating separate LynqNode CRs for each form-row combination.

LynqForm

Blueprint for resources to create per node:

apiVersion: operator.lynq.sh/v1
kind: LynqForm
metadata:
  name: web-app
spec:
  hubId: my-saas-hub
  deployments:
    - id: app-deployment
      nameTemplate: "{{ .uid }}-app"
      spec:
        apiVersion: apps/v1
        kind: Deployment
        spec:
          replicas: 2
          # ... deployment spec

Supported Resource Types:

• serviceAccounts
• deployments, statefulSets, daemonSets
• services
• configMaps, secrets
• persistentVolumeClaims
• jobs, cronJobs
• ingresses
• namespaces
• manifests (raw resources)

LynqNode

Instance representing a single node:

apiVersion: operator.lynq.sh/v1
kind: LynqNode
metadata:
  name: acme-web-app
spec:
  uid: acme
  templateRef: web-app
  hubId: my-saas-hub
  # ... resolved resource arrays
status:
  desiredResources: 10
  readyResources: 10
  failedResources: 0
  appliedResources:
    - "Deployment/default/acme-app@app-deployment"
    - "Service/default/acme-svc@app-service"
  conditions:
    - type: Ready
      status: "True"
      lastTransitionTime: "2024-01-15T10:30:00Z"

Key Design Patterns

Server-Side Apply (SSA)

All resources are applied using Kubernetes Server-Side Apply with fieldManager: lynq. This provides:

• Conflict-free updates: Multiple controllers can manage different fields
• Declarative management: Operator owns only fields it sets
• Drift detection: Automatic detection of manual changes
• Force mode: Optional force ownership with ConflictPolicy: Force

Resource Tracking

Two mechanisms based on namespace and deletion policy:

1. OwnerReference-based (automatic GC):

   • Same-namespace resources with DeletionPolicy=Delete
   • Kubernetes garbage collector handles cleanup

2. Label-based (manual lifecycle):

   • Cross-namespace resources
   • Namespace resources
   • Resources with DeletionPolicy=Retain
   • Labels: lynq.sh/node, lynq.sh/node-namespace

Dependency Management

Resources are applied in order based on dependIds:

deployments:
  - id: app-deployment
    # ...

services:
  - id: app-service
    dependIds: ["app-deployment"]
    waitForReady: true
    # ...

The operator:

1. Builds a Directed Acyclic Graph (DAG)
2. Detects cycles (fails fast if found)
3. Performs topological sort
4. Applies resources in dependency order

Drift Detection & Auto-Correction

The operator continuously monitors managed resources through:

• Event-driven watches: Immediate reconciliation on resource changes
• Watch predicates: Only trigger on meaningful changes (Generation/Annotation)
• Fast requeue: 30-second periodic requeue for status reflection
• Auto-correction: Reverts manual changes to maintain desired state

Garbage Collection

Automatic cleanup when:

• Database row is deleted
• Row's activate field becomes false
• Template no longer references the hub
• LynqNode CR is deleted (with finalizer-based cleanup)

Resources respect DeletionPolicy:

• Delete: Removed from cluster
• Retain: Orphaned with labels for manual cleanup

Orphan Resource Management

When resources are removed from templates:

• Detected via comparison of status.appliedResources
• Resource key format: kind/namespace/name@id
• DeletionPolicy preserved in annotation: lynq.sh/deletion-policy
• Orphaned resources marked with:
  • Label: lynq.sh/orphaned: "true"
  • Annotations: orphaned-at, orphaned-reason
• Re-adoption: Orphan markers removed when resource re-added to template

Performance Considerations

Controller Concurrency

Configurable worker pools for each controller:

• --hub-concurrency=N (default: 3)
• --form-concurrency=N (default: 5)
• --node-concurrency=N (default: 10)

Reconciliation Optimization

• Fast status reflection: 30-second requeue interval
• Smart watch predicates: Filter status-only updates
• Event-driven architecture: Immediate reaction to changes
• Resource caching: Frequently accessed resources cached

Scalability

The operator is designed to scale horizontally:

• Leader election for single-writer pattern
• Optional sharding by namespace or node ID
• Resource-type worker pools for parallel processing
• SSA batching for bulk applies

See Also

• API Reference - Complete CRD specification
• Policies - Lifecycle management policies
• Dependencies - Dependency graph system
• Monitoring - Observability and metrics