This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Troubleshooting

Troubleshoot Kf resources or installation.

1: Temporarily stop reconciliation
2: Troubleshoot Apps
3: Troubleshoot Builds
4: Troubleshoot ClusterServiceBrokers
5: Troubleshoot Routes
6: Troubleshoot ServiceBrokers
7: Troubleshoot ServiceInstanceBindings
8: Troubleshoot ServiceInstances
9: Troubleshoot SourcePackages
10: Troubleshoot Spaces
11: Troubleshoot Tasks

1 - Temporarily stop reconciliation

Stop resource reconciliation to change values for debugging.

Sometimes it may be necessary to stop the Kf controller or Kf operator from changing settings on a Kubernetes resource if you need to change values on it for debugging.

Operator managed resources

Resources managed by the Kf operator, usually found in the kf namespace, can have reconciliation disabled using the operator.knative.dev/mode annotation:

Value	Behavior
`EnsureExists`	The resource will be created if it doesn’t exist, but values won’t be overwritten.
`Reconcile` or blank	The resource will be created if it doesn’t exist, and values will be overwritten.

To change values on operator managed resources for testing:

Make a local copy of the resource so you can revert it later.
Disable reconciliation by setting the annotation to EnsureExists.
Example disabling reconciliation on the Kf controller
```
kubectl annotate --overwrite -n kf deployment controller operator.knative.dev/mode=EnsureExists
```
Update the resource as needed for testing.
When done, restore the annotation to the original value and optionally restore the changed values.

Kf managed resources

Kf managed resources are child resources created in response to configuration set on Spaces, Apps, Builds, etc. These usually have the label app.kubernetes.io/managed-by: kf.

You can disable reconciliation by removing the metadata.ownerReference that references the parent Kf resource. Save a copy of the field so you can add it back when you’re done.

Make a local copy of the resource so you can revert it later.

Disable reconciliation by removing the metadata.ownerReference field.

Example editing a Namespaced owned by a Kf Space:

$ kubectl edit namespace test

Empty the metadata.ownerReference field:

apiVersion: v1
kind: Namespace
metadata:
    creationTimestamp: "2023-04-19T20:34:50Z"
    name: test
    ownerReferences: []

Update the resource as needed for testing.
When done, restore the resource to the original value.

2 - Troubleshoot Apps

Use these steps to troubleshoot various issues that can occur when using Kf Apps.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get apps.kf.dev -n SPACE_NAME APP_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME app/APP_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get apps.kf.dev -n SPACE_NAME APP_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME app/APP_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get apps.kf.dev -n SPACE_NAME APP_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME app/APP_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

3 - Troubleshoot Builds

Use these steps to troubleshoot various issues that can occur when using Kf Builds.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get builds.kf.dev -n SPACE_NAME BUILD_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME build/BUILD_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get builds.kf.dev -n SPACE_NAME BUILD_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME build/BUILD_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get builds.kf.dev -n SPACE_NAME BUILD_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME build/BUILD_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

4 - Troubleshoot ClusterServiceBrokers

Use these steps to troubleshoot various issues that can occur when using Kf ClusterServiceBrokers.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get clusterservicebrokers.kf.dev CLUSTERSERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor clusterservicebroker/CLUSTERSERVICEBROKER_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get clusterservicebrokers.kf.dev CLUSTERSERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor clusterservicebroker/CLUSTERSERVICEBROKER_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get clusterservicebrokers.kf.dev CLUSTERSERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor clusterservicebroker/CLUSTERSERVICEBROKER_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

5 - Troubleshoot Routes

Use these steps to troubleshoot various issues that can occur when using Kf Routes.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get routes.kf.dev -n SPACE_NAME ROUTE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME route/ROUTE_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get routes.kf.dev -n SPACE_NAME ROUTE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME route/ROUTE_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get routes.kf.dev -n SPACE_NAME ROUTE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME route/ROUTE_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

6 - Troubleshoot ServiceBrokers

Use these steps to troubleshoot various issues that can occur when using Kf ServiceBrokers.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get servicebrokers.kf.dev -n SPACE_NAME SERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME servicebroker/SERVICEBROKER_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get servicebrokers.kf.dev -n SPACE_NAME SERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME servicebroker/SERVICEBROKER_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get servicebrokers.kf.dev -n SPACE_NAME SERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME servicebroker/SERVICEBROKER_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

7 - Troubleshoot ServiceInstanceBindings

Use these steps to troubleshoot various issues that can occur when using Kf ServiceInstanceBindings.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstancebindings.kf.dev -n SPACE_NAME SERVICEINSTANCEBINDING_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstancebinding/SERVICEINSTANCEBINDING_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstancebindings.kf.dev -n SPACE_NAME SERVICEINSTANCEBINDING_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstancebinding/SERVICEINSTANCEBINDING_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstancebindings.kf.dev -n SPACE_NAME SERVICEINSTANCEBINDING_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstancebinding/SERVICEINSTANCEBINDING_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

8 - Troubleshoot ServiceInstances

Use these steps to troubleshoot various issues that can occur when using Kf ServiceInstances.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstances.kf.dev -n SPACE_NAME SERVICEINSTANCE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstance/SERVICEINSTANCE_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstances.kf.dev -n SPACE_NAME SERVICEINSTANCE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstance/SERVICEINSTANCE_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstances.kf.dev -n SPACE_NAME SERVICEINSTANCE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstance/SERVICEINSTANCE_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Backing resource reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstances.kf.dev -n SPACE_NAME SERVICEINSTANCE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstance/SERVICEINSTANCE_NAME

Possible Cause Solution

Backing resource DeprovisionFailed error.

Possible Cause	Solution
Backing resource DeprovisionFailed error.	This error usually occurs when backing resources (MySQL database hosted at an external OSB server) fails to be deprovisioned. Kf can not safely determine if the dependent resource is deprovisioned. To recover from the error, it is recommended that user reads the detail error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe servipropogatesceinstance my-service -n my-space`. Once the error message is confirmed, have an administrator check the backing resource and clean it up manually. Once the backing resource is determined to be safely released, the impacted Kf resource can be reconciled successfully by manually removing the `Finalizer` from the object spec, use the `kubectl edit serviceinstance my-service -n my-space` command.

This error usually occurs when backing resources (MySQL database hosted at an external OSB server) fails to be deprovisioned. Kf can not safely determine if the dependent resource is deprovisioned.

To recover from the error, it is recommended that user reads the detail error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe servipropogatesceinstance my-service -n my-space.

Once the error message is confirmed, have an administrator check the backing resource and clean it up manually. Once the backing resource is determined to be safely released, the impacted Kf resource can be reconciled successfully by manually removing the Finalizer from the object spec, use the kubectl edit serviceinstance my-service -n my-space command.

9 - Troubleshoot SourcePackages

Use these steps to troubleshoot various issues that can occur when using Kf SourcePackages.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get sourcepackages.kf.dev -n SPACE_NAME SOURCEPACKAGE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME sourcepackage/SOURCEPACKAGE_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get sourcepackages.kf.dev -n SPACE_NAME SOURCEPACKAGE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME sourcepackage/SOURCEPACKAGE_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get sourcepackages.kf.dev -n SPACE_NAME SOURCEPACKAGE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME sourcepackage/SOURCEPACKAGE_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

10 - Troubleshoot Spaces

Use these steps to troubleshoot various issues that can occur when using Kf Spaces.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get spaces.kf.dev SPACE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor space/SPACE_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get spaces.kf.dev SPACE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor space/SPACE_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get spaces.kf.dev SPACE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor space/SPACE_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

11 - Troubleshoot Tasks

Use these steps to troubleshoot various issues that can occur when using Kf Tasks.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get tasks.kf.dev -n SPACE_NAME TASK_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME task/TASK_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get tasks.kf.dev -n SPACE_NAME TASK_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME task/TASK_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get tasks.kf.dev -n SPACE_NAME TASK_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME task/TASK_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.