Architecture
Big picture
Karmada is a standalone control plane: a dedicated karmada-apiserver and etcd hold both plain Kubernetes resource templates and Karmada's CRDs. Member clusters register as Cluster objects. A set of controllers and a scheduler turn one template plus a PropagationPolicy into per-cluster Work objects, which are then applied to the member clusters either by the control plane (Push) or by an agent inside the member (Pull). Each component is a cobra application under cmd/, where func main calls an app.NewXxxCommand.
Components
karmada-controller-manager
The core process. It runs the resource detector and the controller set that build ResourceBinding objects and Work objects. Entry point cmd/controller-manager/controller-manager.go:30.
karmada-scheduler
Assigns clusters to each ResourceBinding. It runs a filter/score/select/assign pipeline and writes the result back into the binding's spec.Clusters. Lives under cmd/scheduler and pkg/scheduler.
karmada-agent
For Pull mode. It runs inside a member cluster, connects out to the control plane, and applies Work locally. Lives under cmd/agent.
Supporting components
karmada-aggregated-apiserver(cmd/aggregated-apiserver): aggregated APIs such ascluster/proxy.karmada-search(cmd/karmada-search): cross-cluster resource search and cache.karmada-deschedulerandkarmada-scheduler-estimator(cmd/descheduler,cmd/scheduler-estimator): rescheduling and real available-capacity estimation in member clusters.karmada-metrics-adapter(cmd/metrics-adapter): aggregates metrics across clusters for federated HPA.karmada-webhook(cmd/webhook): admission.karmadactl/kubectl-karmada(cmd/karmadactl,cmd/kubectl-karmada): the CLI;initbootstraps a control plane (pkg/karmadactl/cmdinit/cmdinit.go:121).operator/: manages Karmada instances declaratively through aKarmadaCRD.
How a request flows
Propagating a single Deployment to member clusters:
- The detector picks up the template change in
pkg/detector/detector.go:231(Reconcile), matches aPropagationPolicy, and applies it inApplyPolicy(detector.go:441).BuildResourceBinding(detector.go:822) creates aResourceBindingwhosespec.Resourcereferences the template and whosespec.Placementcarries the policy's placement rules. - The scheduler reacts in
pkg/scheduler/scheduler.go:359(scheduleNext), routes throughdoScheduleBinding(scheduler.go:395) toscheduleResourceBinding(scheduler.go:571), and for affinity-based placement toscheduleResourceBindingWithClusterAffinity(scheduler.go:590), which callss.Algorithm.Schedule(...)(scheduler.go:600). - The scheduling algorithm in
pkg/scheduler/core/generic_scheduler.go:71(Schedule) runs filter plugins (findClustersThatFit,generic_scheduler.go:119), score plugins (prioritizeClusters,generic_scheduler.go:166),selectClusters(generic_scheduler.go:196, viaSelectClustersincore/common.go:34), andassignReplicas(generic_scheduler.go:201, viaAssignReplicasincore/common.go:51). The scheduler patches the result into the binding'sspec.Clusters(patchScheduleResultForResourceBinding,scheduler.go:610). - The binding controller in
pkg/controllers/binding/binding_controller.go:70(Reconcile) callssyncBinding(binding_controller.go:110) and thenensureWork(pkg/controllers/binding/common.go:53). For each target cluster it deep-copies the workload, overwrites replicas with the scheduled value, applies override policies (common.go:109), and creates aWorkin the per-cluster execution namespacekarmada-es-<cluster>. - The execution controller in
pkg/controllers/execution/execution_controller.go:82(Reconcile) callssyncWork(execution_controller.go:151) andsyncToClusters(execution_controller.go:266), which applies each manifest to the member API viaObjectWatcher.Create/Update(execution_controller.go:324,:332) and marks theWorkApplied on success (execution_controller.go:302).
Status controllers under pkg/controllers/status run the reverse path, aggregating each member's real state from Work back up into the ResourceBinding and the template.
Key design decisions
- Push vs Pull: Push has the control plane call member APIs directly; Pull has
karmada-agentconnect outward, which works for clusters behind NAT or firewalls (karmada.io). - Templates stay untouched, scheduling owns replicas:
ensureWorkalways overwrites the workload's replica count with the scheduling result rather than trusting the template, so a hand-edited template cannot bypass the scheduler's quota and queue accounting (pkg/controllers/binding/common.go:80-96). - Execution isolation per cluster: each member cluster's
Worklives in its own namespacekarmada-es-<cluster>(pkg/util/names/names.go:80,92), keeping delivery state separated by destination.
Extension points
- PropagationPolicy / ClusterPropagationPolicy and OverridePolicy CRDs to control placement and per-cluster customization (
pkg/apis/policy/v1alpha1/propagation_types.go:52). - Resource Interpreter Framework: an interface (
pkg/resourceinterpreter/interpreter.go:50) plus a Lua-script implementation (ResourceInterpreterCustomizationCRD) that teaches Karmada how to read replicas, revise them, and aggregate status for arbitrary CRDs. - Scheduler framework plugins: filter and score plugins with scores bounded to 0..100 (
pkg/scheduler/framework/interface.go:39-42). - Admission webhooks (
cmd/webhook).