06 - Policy Management¶

Domain 6: Policy Management (10%)

Curriculum topics, in order: Policy Reports, PolicyExceptions, Kyverno Metrics.

Policy reports¶

Kyverno records what its rules found as Kubernetes resources. The reports page says reports are generated for validate, mutate, generate and verifyImages rules, and that they reflect the current cluster state with no history: "Policy reports show policy results for current resources in the cluster only." Entries disappear when the resource is deleted.

Both use the open report format from the Kubernetes Policy working group, API version wgpolicyk8s.io/v1alpha2. That means other tools can read them too. Each result has fields such as policy, rule, result, severity, source and a timestamp, and each report has a summary with counts.

Where results come from¶

Two intermediate report types feed the final reports, merged by the reports controller:

• Admission reports (AdmissionReport, ClusterAdmissionReport) - results from resources matched during admission.
• Background scan reports (BackgroundScanReport, ClusterBackgroundScanReport) - results from scanning existing resources. Scans run hourly by default, check validate and verifyImages rules, never block, and are disabled per policy with background: false.

Result values¶

kubectl get policyreport -A          # namespaced summaries
kubectl get clusterpolicyreport      # cluster-wide report
kubectl get polr <name> -o yaml      # one report in full

Commands as described on the reports page. Not run.

A working routine: roll a new policy out in Audit, run kubectl get policyreport -A, find the fail rows, remediate them, and only then change the action to Enforce.

📖 Policy reports - report types, results, background scans

PolicyExceptions¶

Sometimes one workload has a legitimate reason to break a rule, and rewriting the policy would weaken it for everyone. A PolicyException carves out a narrow, auditable exception.

Legacy form (kyverno.io/v2, namespaced):

apiVersion: kyverno.io/v2
kind: PolicyException
metadata:
  name: delta-exception
  namespace: delta
spec:
  exceptions:
    - policyName: disallow-host-namespaces
      ruleNames:
        - host-namespaces
        - autogen-host-namespaces
  match:
    any:
      - resources:
          kinds:
            - Pod
          names:
            - important-tool*

• policyName names the policy; for a namespaced Policy use <namespace>/<policy-name>.
• ruleNames lists the rules to bypass; wildcards work. Autogenerated rules must be listed too (autogen-host-namespaces above).
• match selects the excepted resources with any or all; an optional conditions block refines it.
• The exception's Namespace need not match the Namespace of the excepted resources.

Exceptions are off by default. Turn them on with two container flags: enablePolicyException=true and exceptionNamespace=<namespace> (* for all). Since 1.13 they are no longer enabled in all Namespaces by default (CVE-2024-48921), so restrict them to a dedicated Namespace that only platform admins can write to. Otherwise anyone who can create an exception can switch a policy off for their own workload.

In reports, a matched exception turns a fail into skip.

The CEL replacement¶

The exceptions page says the legacy kyverno.io/v2 PolicyException "is deprecated as of Kyverno v1.19 and will be removed in v1.20." The replacement is a PolicyException in policies.kyverno.io/v1 (introduced in 1.14), for ValidatingPolicy, ImageValidatingPolicy and GeneratingPolicy. It uses spec.policyRefs to name the policy and spec.matchConditions with CEL; if a condition is true, the referenced rule is skipped. Skipped rules appear in reports as result: skip, with the rule name exception and the exception name in properties.exceptions.

📖 PolicyExceptions - legacy and CEL forms, flags, reports

Kyverno metrics¶

Kyverno exposes Prometheus metrics, so a platform team can watch policy behavior and cost.

• Endpoint: port 8000, path /metrics. With Helm each controller can have a metrics Service; the admission controller's is kyverno-svc-metrics. The default type is ClusterIP, so only in-cluster scrapers reach it.
• Prometheus: scrape the Service, or enable a ServiceMonitor through Helm (serviceMonitor.enabled). The monitoring page names that value and says it needs prometheus-operator installed (kube-prometheus-stack includes it); it gives no values example.
• OpenTelemetry: run a Collector that receives OTLP over gRPC on port 8000 and exposes a Prometheus endpoint on 9090; start Kyverno with metricsPort and otelConfig=grpc.

The metrics page is inconsistent about _total suffixes (some sections omit them, and its alert runbook uses kyverno_policy_changes_total). Check the names your version exposes with a query before you write alerts.

Controlling metric volume¶

The kyverno-metrics ConfigMap (set through the Helm metricsConfig values) has namespaces.include and namespaces.exclude (exclude wins), bucketBoundaries for histograms, and metricsExposure per metric: enabled: false turns a metric off, disabledLabelDimensions drops labels. Excluding Namespaces and dropping labels can substantially reduce memory use.

📖 Kyverno monitoring - endpoint, metrics, ServiceMonitor, ConfigMap

Key terms¶

• PolicyReport - a namespaced resource that lists policy results for resources in one Namespace.
• ClusterPolicyReport - a cluster-scoped report for cluster-level resources.
• wgpolicyk8s.io - the API group of the open policy report format from the Kubernetes Policy working group.
• Admission report - an intermediate report of results from resources seen at admission.
• Background scan report - an intermediate report of results from scanning existing resources.
• skip (result) - a report result meaning preconditions were unmet or an exception applied.
• PolicyException - a resource that exempts specific resources from specific policy rules.
• exceptionNamespace - the flag naming the only Namespace where exceptions are honored.
• kyverno-svc-metrics - the Service exposing the admission controller's metrics.
• ServiceMonitor - a Prometheus Operator resource that tells Prometheus what to scrape.