Kubernetes objects

1. Cluster state as a collection of objects
2. Manifest files and objects schemas
3. Object metadata
4. Objects management
5. Objects querying

• Objects (or resources) are persistent entities that store the desired and current state of a cluster, for instance :

  1. Which pods should be running on which nodes vs pods that are actually running.
  2. Resources that should be made available to pods vs resources actually available.
  3. Cluster policies that should be enforced (restarting, rollback, security, etc ...).

• An object's spec property stores its desired state while the status property stores its current state.

• The status property is constantly updated by K8s as it tries to have the cluster state match spec using the controller pattern.

• As a result, the entire cluster state can be represented using a collection of objects.

• Objects can be written to the control plane through kube-apiserver to update the desired cluster state.

• Updates can be described in .yaml manifest files for consumption by command-line tools like kubectl :

  required manifest field	usage
  apiVersion	K8s API group and version
  kind	Object resource type
  metadata	Additional data for unique identification in the cluster : name, labels etc
  spec	The declaration of the desired state for the object

Notes :

• The spec for the different kind of objects have a specific schema that kube-apiserver will recognize.
• Objects can be nested : spec for some objects can contain spec for other objects, depending on kind.
• For instance, a Deployment spec must contain a PodTemplate spec as well so pods can be started as part of the Deployment.

• The metadata fields may be populated manually by the user in the manifest or automatically by K8s :

  field	managed by	description
  name	user	Name of the object, see below for naming conventions
  namespace	user	Identifies the object as part of a specific group of resources
  labels	user	List of key / value pairs for use in conjunction with selectors
  annotations	user	List of key / value pairs not used for identification or grouping purposes
  generation	K8s	Number, increments each time the object's spec is updated
  ownerReferences	K8s	List of objects referencing the current object
  uid	K8s	UUID, uniquely identifies the object in the cluster
  creationTimestamp	K8s	Date / time of object creation in the desired cluster state
  deletionTimestamp	K8s	Date / time of object deletion from the desired cluster state

• The current object will automatically be garbage collected once ownerReferences becomes empty.

• At this point, it makes sense to envision objects as nodes in a graph, the graph being the cluster state.

• namespace metadata is used to restrict the scope (visibility and reachability) of an object in the cluster.
• Objects that represent cluster resources (as opposed to workload resources) cannot be namespaced.
• Examples of cluster resources include ClusterRole, Node, PersistentVolume, PriorityClass, etc.
• The command kubectl api-resources --namespaced=false will list all such kind of objects.
• Nevertheless, a best practice is to always create a dedicated namespace when deploying a workload to a cluster.
• However, workloads may also require cluster resources (for instance PersistentVolume objects) for proper operation.
• The recommended approach is to provision independant, dedicated namespaced cluster resources for each workload :
  1. This granular approach is compatible with the vast majority of use cases even if the cluster runs multiple workloads.
  2. It provides consistency as to the use of namespaces with objects.
  3. It is painless : the K8s API will ignore the namespace metadata when updating the cluster state.

• "Fully qualified" unique objects have to be encoded as REST paths to be exposed by kube-apiserver.
• As a result, names of objects that exist in the cluster state have to adhere to the following :
  1. name must be identical across all API versions.
  2. name must be unique for the tuple API group / resource type / namespace.
• DNS subdomain names, RFC 1123 and RFC 1035 label names are often used to name K8s objects.

• The desired cluster state can be updated with kubectl.

• Desired state update requests for object(s) can be expressed using current spec and next spec.

• The diff between current and next spec can be written to the desired cluster state in different ways :

  command	management type	behavior
  kubectl create <obj> --params	Imperative command	No manifest, actual diff unclear, do not use
  kubectl create -f manifest.yaml	Imperative config	Creates objects defined in next spec
  kubectl delete -f manifest.yaml	Imperative config	Deletes objects defined in next spec
  kubectl replace -f manifest.yaml	Imperative config	Overwrites current spec with next spec
  kubectl apply -f manifest.yaml	Declarative config	Merges the diff into the current spec, recommended

• Best practices :

  1. The workflow for state updates should always be manifest.yaml -> spec -> status (current cluster state).
  2. This approach allows representation of the desired cluster state as a set of files.
  3. As a result kubectl apply should always be preferred (it is also the only command that can process directories).
  4. Important : no field should exist in the current spec if it doesn't exist in the manifest for the next spec.
  5. The above eliminates the need for kubectl replace or kubectl patch in favor of kubectl apply --force-conflicts true.

• labels is a set of key / value pairs used to identify and characterize an object as part of the cluster.
• A label key comprises an optional prefix and a name, for instance platform.gg/domain.
• Prefixes, when present, are used to provide interoperability with external systems.
• kubernetes.io/ and k8s.io/ prefixes are reserved for K8s core components.

See also : recommended labels for interoperability.

• A selector is a core grouping primitive used to identify a set of objects.
• Selectors can be passed as arguments to kubectl or used in manifest files to reference other objects.
• Multiple selectors can be comma-separated : the resulting query will perform a logical && on all the specified selectors.
• Selectors can be equality-based or set-based and returned objects will satisfy all constraints :

# combined equality based selectors
# supported operators are =, == and !=
kubectl get pods -l "environment=production,tier=frontend"

# combined set based selectors
# supported operators are in, notin, <key> and !<key> (whether the key exists or not)
kubectl get pods -l "environment in (production),tier in (frontend)"

• Equivalent example in a manifest file (matchLabels and matchExpressions results will be combined) :

selector:
  # YAML map
  matchLabels:
    environment: production
  # YAML list
  matchExpressions:
    - { key: tier, operator: In, values: [frontend] }

• In manifest files, operators for set-based selectors are In, NotIn, Exists and DoesNotExist.
• Using a set-based selector to match a label against multiple possible values requires using matchExpressions.

• Field selectors are an independant mechanism that allows filtering objects by field value.
• However, they are seldomly used in object manifests and exist mainly for observability purposes.