Kf offers developers the Cloud Foundry experience while empowering operators to adopt declarative Kubernetes practices. It makes migrating Cloud Foundry workloads to Kubernetes straightforward, and most importantly, avoids major changes to developer workflows.

1 - Getting Started

Get started with Kf as an app runtime.

1.1 - Getting Started Overview

With Kf, you can migrate your Cloud Foundry apps to Kubernetes without changing your developer workflows.

Use the following sections to get started deploying Cloud Foundry apps to Kf.

Get started with a quickstart

Use the quickstart to install Kf and deploy a simple test Cloud Foundry app. This introduces you to the basic steps you’d perform for most app deployments.

Get an introduction to key concepts

For an introduction to the value of Kf, as well as high-level overviews of the technology, see the following documents:

To learn more about how Kf works, see Kf dependencies and architecture.
For a side-by-side comparison of Cloud Foundry and Kf components, see Compare Cloud Foundry and Kf services.
To learn how to use service brokers with Kf, see Service brokers.
For more information about Kf security design, see the Security overview.

1.2 - Quickstart

In this quickstart, you will deploy a sample Cloud Foundry app on an existing Kf cluster.

Push an application

Prerequisites

The following are required to complete this section:

The kf CLI installed and in your path.

You have connected to the Kf Kubernetes cluster:

gcloud container clusters get-credentials CLUSTER_NAME \
    --project=CLUSTER_PROJECT_ID \
    --zone=CLUSTER_LOCATION

The git CLI installed and in your path.

Prepare space

Create new space:
```
kf create-space test-space
```
Target the space:
```
kf target -s test-space
```

Push the Cloud Foundry test app

Clone the test-app repo.

git clone https://github.com/cloudfoundry-samples/test-app go-test-app

cd go-test-app

Push the app.
Note: It will take a few minutes for the build to complete.
```
kf push test-app
```
Get the application’s URL.
```
kf apps
```
Note: The app will have a random route by default.
Open the URL in your browser where you should see the app running.
Successful app push on Kf

Clean up

These steps should return the cluster to the starting state.

Delete the application.
```
kf delete test-app
```
Delete the Space.
```
kf delete-space test-space
```

1.3 - Compare Cloud Foundry and Kf services

This document provides a side-by-side comparison of the various services available on Cloud Foundry (CF) and those that Kf integrates with on Google Cloud.

Service category	Service	CF	Kf
Platform	Infrastructure Orchestrator	BOSH	Kubernetes
	PaaS	CF Application Runtime (CFAR)	Kf
Data management	Service Broker	Service Broker Tile	Kubernetes Deployed Service Brokers
	MySQL	MySQL Tile	Kf Cloud Service Broker
	MongoDB	MongoDB Tile	Kf Cloud Service Broker
	RabbitMQ	RabbitMQ Tile	Kf Cloud Service Broker
	Redis	Redis Tile	Kf Cloud Service Broker
	Eureka	Spring Cloud Services Tile	Service Discovery
	Spring Cloud Config	Spring Cloud Services Tile	Spring Cloud Config
Operations tooling	Continuous Integration (CI)	Concourse Tile	Concourse Helm Chart
Logging	Google Cloud	Google Cloud Firehose Nozzle	Google Cloud Logging Kubernetes Agent
	Elastic	Elastic Firehose Nozzle	Elastic Stack Agent
	Splunk	Splunk Firehose Nozzle	Splunk Connect
	Metrics	CF App Metrics	Google Cloud Monitoring Kubernetes AGent

2 - Develop Applications on Kf

Learn to use Kf to deploy and maintain applications.

2.1 - Build and deploy applications

Learn how to build and deploy applications in Kf.

2.1.1 - Deploy an application

Guide to deploying applications with Kf.

When pushing an app (via kf push) to Kf, there are three lifecycles that Kf uses to take your source code and allow it to handle traffic:

Source code upload
Build
Run

Source code upload

The first thing that happens when you kf push is the Kf CLI (kf) packages up your directory (either current or --path/-p) into a container and publishes it to the container registry configured for the Space. This is called the source container. The Kf CLI then creates an App type in Kubernetes that contains both the source image and configuration from the App manifest and push flags.

Ignore files during push

In many cases, you will not want to upload certain files during kf push (i.e., “ignore” them). This is where a .kfignore (or .cfignore) file can be used. Similar to a .gitignore file, this file instructs the Kf CLI which files to not include in the source code container.

To create a .kfignore file, create a text file named .kfignore in the base directory of your app (similar to where you would store the manifest file). Then populate it with a newline delimited list of files and directories you don’t want published. For example:

bin
.idea

This will tell the Kf CLI to not include anything in the bin or .idea directories.

Kf supports gitignore style syntax.

Build

The Build lifecycle is handled by a Tekton TaskRun. Depending on the flags that you provide while pushing, it will choose a specific Tekton Task. Kf currently has the following Tekton Tasks:

buildpackv2
buildpackv3
kaniko

Kf tracks each TaskRun as a Build. If a Build succeeds, the resulting container image is then deployed via the Run lifecycle (described below).

More information can be found at Build runtime.

Run

The Run lifecycle is responsible for taking a container image and creating a Kubernetes Deployment.

It also creates:

More information can be found at Build runtime.

Push timeouts

Kf supports setting an environment variable to instruct the CLI to time out while pushing apps. If set, the variables KF_STARTUP_TIMEOUT or CF_STARTUP_TIMEOUT are parsed as a golang style duration (for example 15m, 1h). If a value is not set, the push timeout defaults to 15 minutes.

2.1.2 - Get started with buildpacks

How to use built-in buildpacks for various languages.

Kf supports a variety of buildpacks. This document covers some starter examples for using them.

Before you begin

You should have Kf running on a cluster.
You should have run kf target -s <space-name> to target your space.

Java (v2) buildpack

Use spring initializr to create a Java 8 maven project with a spring web dependency and JAR packaging. Download it, extract it, and once extracted you can generate a JAR.

./mvnw package

Push the JAR to Kf with the Java v2 buildpack.

kf push java-v2 --path target/helloworld-0.0.1-SNAPSHOT.jar

Java (v3) buildpack

Use spring initializr to create a Java 8 maven project with a spring web dependency and JAR packaging. Download it, extract it, and once extracted, push to Kf with the cloud native buildpack.

kf push java-v3 --stack org.cloudfoundry.stacks.cflinuxfs3

Python (v2) buildpack

Create a new directory with files as shown in the following structure.

tree
.
├── Procfile
├── requirements.txt
└── server.py

cat Procfile
web: python server.py

cat requirements.txt
Flask

cat server.py
from flask import Flask
import os

app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

if __name__ == "__main__":
  port = int(os.getenv("PORT", 8080))
  app.run(host='0.0.0.0', port=port)

Push the Python flask app using v2 buildpacks.

kf push python --buildpack python\_buildpack

Python (v3) buildpack

(same as above)

Push the Python flask app using cloud native buildpacks.

kf push pythonv3 --stack org.cloudfoundry.stacks.cflinuxfs3

Staticfile (v2) buildpack

Create a new directory that holds your source code.

Add an index.html file with this content.

<!DOCTYPE html>

<html lang="en">

<head><title>Hello, world!</title></head>

<body><h1>Hello, world!</h1></body>

</html>

Push the static content with the staticfile buildpack.

kf push staticsite --buildpack staticfile\_buildpack

2.1.3 - Reduce deployment risk with blue-green deployments

How to deploy highly available applications.

This page shows you how to deploy a new version of your application and migrate traffic over from an old to a new version.

Push the initial App

Use the Kf CLI to push the initial version of your App with any routes:

$ kf push app-v1 --route my-app.my-space.example.com

Push the updated App

Use the Kf CLI to push a new version of your App without any routes:

$ kf push app-v2 --no-route

Add routes to the updated App

Use the Kf CLI to bind all existing routes to the updated App with a weight of 0 to ensure that they don’t get any requests:

$ kf map-route app-v2 my-space.example.com --hostname my-app --weight 0

Shift traffic

Start shifting traffic from the old App to the updated App by updating the weights on the routes:

$ kf map-route app-v1 my-space.example.com --hostname my-app --weight 80
$ kf map-route app-v2 my-space.example.com --hostname my-app --weight 20

If the deployment is going well, you can shift more traffic by updating the weights again:

$ kf map-route app-v1 my-space.example.com --hostname my-app --weight 50
$ kf map-route app-v2 my-space.example.com --hostname my-app --weight 50

Complete traffic shifting

After you’re satisfied that the new service hasn’t introduced regressions, complete the rollout by shifting all traffic to the new instance:

$ kf map-route app-v1 my-space.example.com --hostname my-app --weight 0
$ kf map-route app-v2 my-space.example.com --hostname my-app --weight 100

Turn down the original App

After you’re satisfied that quick rollbacks aren’t needed, remove the original route and stop the App:

$ kf unmap-route app-v1 myspace.example.com --hostname my-app
$ kf stop app-v1

Or delete the App and all associated route mappings:

$ kf delete app-v1

2.1.4 - Compare V2 and V3 Buildpacks

Learn how to decide wihch buildpacks to use.

A buildpack converts source code into an executable, and is used to deliver a simple, reliable, and repeatable way to create containers. Kf supports both V2 and V3 buildpacks, and it is important to understand the differences between them.

V2 buildpacks

Most Cloud Foundry applications already use V2 buildpacks. When using V2 buildpacks with Kf, the lifecycle binaries and the buildpacks are downloaded and configured from their git URLs. Kf then uses the lifecycle CLI to execute each buildpack against the source code.

Pros

Ready out of the box without pipeline or code changes.

Cons

Legacy buildpack supersceded by V3.
Weaker performance and reliability. The Kf build pipeline requires more IO for V2 buildpacks.
Fewer community resources.
Kf only supports OSS git repos.

V3 buildpacks

V3 buildpacks are a Cloud Native Computing Foundation (CNCF) project with a well defined spec, a CLI (pack) and a growing community that is innovating around different languages and frameworks. Google Cloud also has its own set of OSS buildpacks.

V3 buildpacks have two overarching OCI containers:

Builder image
Run image

Builder image

The builder image is used while your source code is being built into a runnable container. The image has the necessary detect scripts and other utilities to compile source code.

Run image

The run image is the base image that a container is built on. This means that it is the base image that will run when the App executes.

Layers

V3 buildpacks use layers to compose the final container. Each buildpack included in a build is given the opportunity to manipulate the file system and environment variables of the App. This layering approach allows for buildpacks to be thinner and more generic.

V3 buildpacks are built on OCI containers. This requires that the V3 builder image be stored in a container registry that the Kf build pipeline has access to. The build pipeline uses the builder image to apply the underlying scripts to build the source code into a runnable container.

Pros

Google supported builder and run image.
Works with various CI/CD runtimes like Cloud Build.
Growing community and buildpack registry.

Cons

May require code/process updates. For example, the Java buildpack requires source code while the V2 buildpack requires a jar file.
V3 buildpacks are newer and might require additional validation (is using community developed buildpacks).

Kf Stacks

View Stacks

When pushing an App, the build pipeline determines the buildpack based on the selected Stack (specified via the --stack flag or the manifest).

To see which Stacks are available in a Space first ensure a Space is targeted:

kf target -s myspace

The kf stacks subcommand can then be used to list the Stacks:

kf stacks

The output shows both V2 and V3 Stacks:

Getting stacks in Space: myspace
Version  Name                                Build Image                                                                                          Run Image
V2       cflinuxfs3                          cloudfoundry/cflinuxfs3@sha256:5219e9e30000e43e5da17906581127b38fa6417f297f522e332a801e737928f5      cloudfoundry/cflinuxfs3@sha256:5219e9e30000e43e5da17906581127b38fa6417f297f522e332a801e737928f5
V3       kf-v2-to-v3-shim                    gcr.io/kf-releases/v2-to-v3:v2.7.0                                                                   gcr.io/buildpacks/gcp/run:v1                                                                       This is a stack added by the integration tests to assert that v2->v3 shim works
V3       google                              gcr.io/buildpacks/builder:v1                                                                         gcr.io/buildpacks/gcp/run:v1                                                                       Google buildpacks (https://github.com/GoogleCloudPlatform/buildpacks)
V3       org.cloudfoundry.stacks.cflinuxfs3  cloudfoundry/cnb:cflinuxfs3@sha256:f96b6e3528185368dd6af1d9657527437cefdaa5fa135338462f68f9c9db3022  cloudfoundry/run:full-cnb@sha256:dbe17be507b1cc6ffae1e9edf02806fe0e28ffbbb89a6c7ef41f37b69156c3c2  A large Cloud Foundry stack based on Ubuntu 18.04

V2 to V3 Buildpack Migration

Kf provides a V3 stack to build applications that were built with standard V2 buildpacks, using a stack named kf-v2-to-v3-shim. The kf-v2-to-v3-shim stack is created following the standard V3 buildpacks API. A Google maintained builder image is created with each Kf release, following the standard buildpack process. The builder image aggregates a list of V3 buildpacks created by the same process used with the kf wrap-v2-buildpack command. The V3 buildpack images are created using the standard V2 buildpack images. It’s important to note that the V3 buildpacks do not contain the binary of the referenced V2 buildpacks. Instead, the V2 buildpack images are referenced, and the bits are downloaded at App build time (by running kf push).

At App build time, the V2 buildpack is downloaded from the corresponding git repository. When V3 detection runs, it delegates to the downloaded V2 detection script. For the first buildpack group that passes detection, it proceeds to the build step, which delegates the build execution to the downloaded V2 builder script.

The following V2 buildpacks are supported in the kf-v2-to-v3-shim stack:

Buildpack	Git Repository
java_buildpack	https://github.com/cloudfoundry/java-buildpack
dotnet_core_buildpack	https://github.com/cloudfoundry/dotnet-core-buildpack
nodejs_buildpack	https://github.com/cloudfoundry/nodejs-buildpack
go_buildpack	https://github.com/cloudfoundry/go-buildpack
python_buildpack	https://github.com/cloudfoundry/python-buildpack
binary_buildpack	https://github.com/cloudfoundry/binary-buildpack
nginx_buildpack	https://github.com/cloudfoundry/nginx-buildpack

Option 1: Migrate Apps built with standard V2 buildpacks

To build Apps with the kf-v2-to-v3-shim stack, use the following command:

kf push myapp --stack kf-v2-to-v3-shim

The kf-v2-to-v3-shim stack will automatically detect the runtime with the wrapped V2 buildpacks. The resulting App image is created using the V3 standard and build pipeline, but the builder of the equivalent V2 buildpack.

Option 2: Migrate Apps built with custom V2 buildpacks

Kf has a buildpack migration tool that can take a V2 buildpack and wrap it with a V3 buildpack. The wrapped buildpack can then be used anywhere V3 buildpacks are available.

kf wrap-v2-buildpack gcr.io/your-project/v2-go-buildpack https://github.com/cloudfoundry/go-buildpack --publish

This will create a buildpack image named gcr.io/your-project/v2-go-buildpack. It can then be used to create a builder by following the create a builder docs.

This subcommand uses the following CLIs transparently:

go
git
pack
unzip

2.1.5 - App Manifest

Reference guide for the application manifest.yml format.

App manifests provide a way for developers to record their App’s execution environment in a declarative way. They allow Apps to be deployed consistently and reproducibly.

Format

Manifests are YAML files in the root directory of the App. They must be named manifest.yml or manifest.yaml.

Kf App manifests are allowed to have a single top-level element: applications. The applications element can contain one or more application entries.

Application fields

The following fields are valid for objects under applications:

Field	Type	Description
`name`	`string`	The name of the application. The app name should be lower-case alphanumeric characters and dashes. It must not start with a dash.
`path`	`string`	The path to the source of the app. Defaults to the manifest’s directory.
`buildpacks`	`string[]`	A list of buildpacks to apply to the app.
`stack`	`string`	Base image to use for to use for apps created with a buildpack.
`docker`	`object`	A docker object. See the Docker Fields section for more information.
`env`	`map`	Key/value pairs to use as the environment variables for the app and build.
`services`	`string[]`	A list of service instance names to automatically bind to the app.
`disk_quota`	`quantity`	The amount of disk the application should get. Defaults to 1GiB.
`memory`	`quantity`	The amount of RAM to provide the app. Defaults to 1GiB.
`cpu` †	`quantity`	The amount of CPU to provide the application. Defaults to 1/10th of a CPU.
`cpu-limit` †	`quantity`	The maximum amount of CPU to provide the application. Defaults to unlimited.
`instances`	`int`	The number of instances of the app to run. Defaults to 1.
`routes`	`object`	A list of routes the app should listen on. See the Route Fields section for more.
`no-route`	`boolean`	If set to true, the application will not be routable.
`random-route`	`boolean`	If set to true, the app will be given a random route.
`timeout`	`int`	The number of seconds to wait for the app to become healthy.
`health-check-type`	`string`	The type of health-check to use `port`, `process`, `none`, or `http`. Default: `port`
`health-check-http-endpoint`	`string`	The endpoint to target as part of the health-check. Only valid if `health-check-type` is `http`.
`health-check-invocation-timeout`	`int`	Timeout in seconds for an individual health check probe to complete. Default: `1`.
`command`	`string`	The command that starts the app. If supplied, this will be passed to the container entrypoint.
`entrypoint` †	`string`	Overrides the app container’s entrypoint.
`args` †	`string[]`	Overrides the arguments the app container.
`ports` †	`object`	A list of ports to expose on the container. If supplied, the first entry in this list is used as the default port.
`startupProbe` †	`probe`	Sets the app container’s startup probe.
`livenessProbe` †	`probe`	Sets the app container’s liveness probe.
`readinessProbe` †	`probe`	Sets the app container’s readiness probe.
`metadata`	`object`	Additional tags for applications and their underlying resources.

† Unique to Kf

Docker fields

The following fields are valid for application.docker objects:

Field	Type	Description
`image`	`string`	The docker image to use.

Route fields

The following fields are valid for application.routes objects:

Field	Type	Description
`route`	`string`	A route to the app including hostname, domain, and path.
`appPort`	`int`	(Optional) A custom port on the App the route will send traffic to.

Note: If you specify an appPort, that port MUST also be declared in the ports field.

Port fields

The following fields are valid for application.ports objects:

Field	Type	Description
`port`	`int`	The port to expose on the App’s container.
`protocol`	`string`	The protocol of the port to expose. Must be `tcp`, `http` or `http2`. Default: `tcp`

Note: The protocol field is a hint about the traffic that goes over the port. The hint is used by the service mesh for better tracing and metrics.

Warning: Kf doesn’t currently support TCP port-based routing. You must use a Kubernetes LoadBalancer if you want to expose a TCP port to the Internet. Ports are available on the cluster internal App address <app-name>.<space>.

Metadata fields

The following fields are valid for application.metadata objects:

Field	Type	Description
`labels`	`string -> string map`	Labels to add to the app and underlying application Pods.
`annotations`	`string -> string map`	Annotations to add to the app and underlying application Pods.

Probe fields

Probes allow a subset of functionality from Kubernetes probes.

A probe must contain one action and other settings.

Field	Type	Description
`failureThreshold`	`int`	Minimum consecutive failures for the probe to be considered failed.
`initialDelaySeconds`	`int`	Number of seconds to wait after container initialization to start the probe.
`periodSeconds`	`int`	How often (in seconds) to perform the probe.
`successThreshold`	`int`	Minimum consecutive successes for the probe to be considered successful.
`timeoutSeconds`	`int`	Number of seconds after a single invocation of the probe times out.
`tcpSocket`	`TCPSocketAction` object	Action specifying a request to a TCP port.
`httpGet`	`HTTPGetAction` object	Action specifying a request to a TCP port.

TCPSocketAction fields

Describes an action based on TCP requests.

Field	Type	Description
`host`	`string`	Host to connect to, defaults to the App’s IP.

HTTPGetAction fields

Describes an action based on HTTP get requests.

Field	Type	Description
`host`	`string`	Host to connect to, defaults to the App’s IP.
`path`	`string`	Path to access on the HTTP server.
`scheme`	`string`	Scheme to use when connecting to the host. Default: `http`
`httpHeaders`	array of `{"name": <string>, "value": <string>}` objects	Additional headers to send.

Examples

Minimal App

This is a bare-bones manifest that will build an App by auto-detecting the buildpack based on the uploaded source, and deploy one instance of it.

---
applications:
- name: my-minimal-application

Simple App

This is a full manifest for a more traditional Java App.

---
applications:
- name: account-manager
  # only upload src/ on push
  path: src
  # use the Java buildpack
  buildpacks:
  - java
  env:
    # manually configure the buildpack's Java version
    BP_JAVA_VERSION: 8
    ENVIRONMENT: PRODUCTION
  # use less disk and memory than default
  disk_quota: 512M
  memory: 512M
  # Give the app a minimum of 2/10ths of a CPU
  cpu: 200m
  instances: 3
  # make the app listen on three routes
  routes:
  - route: accounts.mycompany.com
  - route: accounts.datacenter.mycompany.internal
  - route: mycompany.com/accounts
  # set up a longer timeout and custom endpoint to validate
  # when the app comes up
  timeout: 300
  health-check-type: http
  health-check-http-endpoint: /healthz
  # attach two services by name
  services:
  - customer-database
  - web-cache

Docker App

Kf can deploy Docker containers as well as manifest deployed App. These Docker Apps MUST listen on the PORT environment variable.

---
applications:
- name: white-label-app
  # use a pre-built docker image (must listen on $PORT)
  docker:
    image: gcr.io/my-company/white-label-app:123
  env:
    # add additional environment variables
    ENVIRONMENT: PRODUCTION
  disk_quota: 1G
  memory: 1G
  # Give the app a minimum of 2 full CPUs
  cpu: 2000m
  instances: 1
  routes:
  - route: white-label-app.mycompany.com

App with multiple ports

This App has multiple ports to expose an admin console, website, and SMTP server.

---
applications:
- name: b2b-server
  ports:
  - port: 8080
    protocol: http
  - port: 9090
    protocol: http
  - port: 2525
    protocol: tcp
  routes:
  - route: b2b-admin.mycompany.com
    appPort: 9090
  - route: b2b.mycompany.com
    # gets the default (first) port

Health check types

Kf supports three different health check types:

port (default)
http
process (or none)

port and http set a Kubernetes readiness and liveness probe that ensures the application is ready before sending traffic to it.

The port health check will ensure the port found at $PORT is being listened to. Under the hood Kf uses a TCP probe.

The http health check will use the configured value in health-check-http-endpoint to check the application’s health. Under the hood Kf uses an HTTP probe.

A process health check only checks to see if the process running on the container is alive. It does NOT set a Kubernetes readiness or liveness probe.

Known differences

The following are known differences between Kf manifests and CF manifests:

Kf does not support deprecated CF manifest fields. This includes all fields at the root-level of the manifest (other than applications) and routing fields.
Kf is missing support for the following v2 manifest fields:
- docker.username
Kf does not support auto-detecting ports for Docker containers.

2.1.6 - App runtime

Reference guide for the application runtime container environment.

The app runtime is the environment apps are executed in.

	Buildpack Apps	Container Image Apps
System libraries	Provided by the Stack	Provided in the container
Network access	Full access through Envoy sidecar	Full access through Envoy sidecar
File system	Ephemeral storage	Ephemeral storage
Language runtime	Supplied by the Stack or Buildpack	Built into the container
User	Specified by the Stack	Specified on the container
Isolation mechanism	Kubernetes Pod	Kubernetes Pod
DNS	Provided by Kubernetes	Provided by Kubernetes

Environment variables

Environment variables are injected into the app at runtime by Kubernetes. Variables are added based on the following order, where later values override earlier ones with the same name:

Space (set by administrators)
App (set by developers)
System (set by Kf)

Kf provides the following system environment variables:

Variable	Purpose
`CF_INSTANCE_ADDR`	The cluster-visible IP:PORT of the App instance.
`CF_INSTANCE_GUID`	The UUID of the App instance.
`INSTANCE_GUID`	Alias of `CF_INSTANCE_GUID`.
`CF_INSTANCE_INDEX`	The index number of the App instance, this will ALWAYS be 0.
`INSTANCE_INDEX`	Alias of `CF_INSTANCE_INDEX`.
`CF_INSTANCE_IP`	The cluster-visible IP of the App instance.
`CF_INSTANCE_INTERNAL_IP`	Alias of `CF_INSTANCE_IP`
`VCAP_APP_HOST`	Alias of `CF_INSTANCE_IP`
`CF_INSTANCE_PORT`	The cluster-visible port of the App instance. In Kf this is the same as `PORT`.
`DATABASE_URL`	The first URI found in a `VCAP_SERVICES` credential.
`LANG`	Required by Buildpacks to ensure consistent script load order.
`MEMORY_LIMIT`	The maximum amount of memory in MB the App can consume.
`PORT`	The port the App should listen on for requests.
`VCAP_APP_PORT`	Alias of `PORT`.
`VCAP_APPLICATION`	A JSON structure containing App metadata.
`VCAP_SERVICES`	A JSON structure specifying bound services.

Service credentials from bound services get injected into Apps via the VCAP_SERVICES environment variable. The variable is a valid JSON object with the following structure.

VCAPServices

A JSON object where the keys are Service labels and the values are an array of VCAPService. The array represents every bound service with that label. User provided services are placed under the label user-provided.

Example

{
  "mysql": [...],
  "postgresql": [...],
  "user-provided": [...]
}

VCAPService

This type represents a single bound service instance.

Example

{
  "binding_name": string,
  "instance_name": string,
  "name": string,
  "label": string,
  "tags": string[],
  "plan": string,
  "credentials": object
}

Fields

Field	Type	Description
`binding_name`	`string`	The name assigned to the service binding by the user.
`instance_name`	`string`	The name assigned to the service instance by the user.
`name`	`string`	The `binding_name` if it exists; otherwise the `instance_name`.
`label`	`string`	The name of the service offering.
`tags`	`string[]`	An array of strings an app can use to identify a service instance.
`plan`	`string[]`	The service plan selected when the service instance was created.
`credentials`	`object`	The service-specific credentials needed to access the service instance.

Note: Use the tags field to discover services in your App rather than filtering by plan/label so you can swap implementations and providers per environment.

Note: The credentials field is often an object with string keys and values. However, the values are allowed to be any type.

VCAP_APPLICATION

TheVCAP_APPLICATION environment variable is a JSON object containing metadata about the App.

Example

{
  "application_id": "12345",
  "application_name": "my-app",
  "application_uris": ["my-app.example.com"],
  "limits": {
    "disk": 1024,
    "mem": 256
  },
  "name": "my-app",
  "process_id": "12345",
  "process_type": "web",
  "space_name": "my-ns",
  "uris": ["my-app.example.com"]
}

Fields

Field	Type	Description
`application_id`	`string`	The GUID identifying the App.
`application_name`	`string`	The name assigned to the App when it was pushed.
`application_uris`	`string[]`	The URIs assigned to the App.
`limits`	`object`	The limits to disk space, and memory permitted to the App. Memory and disk space limits are supplied when the App is deployed, either on the command line or in the App manifest. Disk and memory limits are represented as integers, with an assumed unit of MB.
`name`	`string`	Identical to `application_name`.
`process_id`	`string`	The UID identifying the process. Only present in running App containers.
`process_type`	`string`	The type of process. Only present in running App containers.
`space_name`	`string`	The human-readable name of the Space where the App is deployed.
`uris`	`string[]`	Identical to `application_uris`.

Missing Fields

Some fields in VCAP_APPLICATION that are in Cloud Foundry are currently not supported in Kf.

Besides CF-specific and deprecated fields (cf_api, host, users) the fields that are not supported in Kf are:

application_version (identical to version)
organization_id
organization_name
space_id
start (identical to started_at)
started_at_timestamp (identical to state_timestamp)

2.1.7 - Build runtime

Reference guide for the application build container environment.

The Build runtime is the environment Apps are built in.

	Buildpack Builds	Docker Builds
System libraries	Provided by the Stack	User supplied
Network access	Full access through Envoy sidecar	Full access through Envoy sidecar
File system	No storage	No storage
Language runtime	Provided by the Stack	User supplied
User	Specified by the Stack	User supplied
Isolation mechanism	Kubernetes Pod	Kubernetes Pod
DNS	Provided by Kubernetes	Provided by Kubernetes

Environment variables

Environment variables are injected into the Build at runtime. Variables are added based on the following order, where later values override earlier ones with the same name:

Space (set by administrators)
App (set by developers)
System (set by Kf)

Kf provides the following system environment variables to Builds:

Variable	Purpose
`CF_INSTANCE_ADDR`	The cluster-visible IP:PORT of the Build.
`INSTANCE_GUID`	Alias of `CF_INSTANCE_GUID`.
`CF_INSTANCE_IP`	The cluster-visible IP of the Build.
`CF_INSTANCE_INTERNAL_IP`	Alias of `CF_INSTANCE_IP`
`VCAP_APP_HOST`	Alias of `CF_INSTANCE_IP`
`CF_INSTANCE_PORT`	The cluster-visible port of the Build.
`LANG`	Required by Buildpacks to ensure consistent script load order.
`MEMORY_LIMIT`	The maximum amount of memory in MB the Build can consume.
`VCAP_APPLICATION`	A JSON structure containing App metadata.
`VCAP_SERVICES`	A JSON structure specifying bound services.

2.2 - Backing services

Discover, create, and use backing services.

2.2.1 - Backing Services Overview

Learn about how Kf works with backing services.

Backing services are any processes that the App contacts over the network during its operation. In traditional operating systems, these services could have been accessed over the network, a UNIX socket, or could even be a sub-process. Examples include the following:

Databases — for example: MySQL, PostgreSQL
File storage — for example: NFS, FTP
Logging services — for example: syslog endpoints
Traditional HTTP APIs — for example: Google Maps, WikiData, Parcel Tracking APIs

Connecting to backing services over the network rather than installing them all into the same machine allows developers to focus on their App, independent security upgrades for different components, and flexibility to swap implementations.

Backing services in Kf

Kf supports two major types of backing services:

Managed services: These services are created by a service broker and are tied to the Kf cluster.
User-provided services: These services are created outside of Kf, but get can be bound to apps in the same way as managed services.

2.2.2 - Use managed services

Learn to use the marketplace to find a service, create it, and bind it to an app.

Find a service

Use the kf marketplace command to find a service you want to use in your App. Running the command without arguments will show all the service classes available. A service class represents a specific type of service e.g. a MySQL database or a Postfix SMTP relay.

$ kf marketplace
5 services can be used in Space "test", use the --service flag to list the plans for a service

Broker      Name        Space      Status  Description
minibroker  mariadb                Active  Helm Chart for mariadb
minibroker  mongodb                Active  Helm Chart for mongodb
minibroker  mysql                  Active  Helm Chart for mysql
minibroker  postgresql             Active  Helm Chart for postgresql
minibroker  redis                  Active  Helm Chart for redis

Service classes can have multiple plans available. A service plan generally corresponds to a version or pricing tier of the software. You can view the plans for a specific service by supplying the service name with the marketplace command:

$ kf marketplace --service mysql
Name    Free  Status  Description
5-7-14  true  Active  Fast, reliable, scalable, and easy to use open-source relational database system.
5-7-27  true  Active  Fast, reliable, scalable, and easy to use open-source relational database system.
5-7-28  true  Active  Fast, reliable, scalable, and easy to use open-source relational database system.

Provision a service

Once you have identified a service class and plan to provision, you can create an instance of the service using kf create-service:

$ kf create-service mysql 5-7-28 my-db
Creating service instance "my-db" in Space "test"
Waiting for service instance to become ready...
Success

Services are provisioned into a single Space. You can see the services in the current Space by running kf services:

$ kf services
Listing services in Space: "test"
Name   ClassName  PlanName  Age   Ready  Reason
my-db  mysql      5-7-28    111s  True   <nil>

You can delete a service using kf delete-service:

$ kf delete-service my-db

Bind a service

Once a service has been created, you can bind it to an App, which will inject credentials into the App so the service can be used. You can create the binding using kf bind-service:

$ kf bind-service my-app my-db
Creating service instance binding "binding-my-app-my-db" in Space "test"
Waiting for service instance binding to become ready...
Success

You can list all bindings in a Space using the kf bindings command:

$ kf bindings
Listing bindings in Space: "test"
Name                  App     Service  Age  Ready
binding-my-app-my-db  my-app  my-db    82s  True

Once a service is bound, restart the App using kf restart and the credentials will be in the VCAP_SERVICES environment variable.

You can delete a service binding with the kf unbind-service command:

$ kf unbind-service my-app my-db

2.2.3 - Configure user-provided services

Learn how to inject existing services into your app.

Users can leverage services that aren’t available in the marketplace by creating user-provided service instances. Once created, user-provided service instances behave like managed service instances created through kf create-service. Creating, listing, updating, deleting, binding, and unbinding user-provided services are all supported in Kf.

Create a user-provided service instance

Tip: The alias for kf create-user-provided-service is kf cups.

The name given to a user-provided service must be unique across all service instances in a Space, including services created through a service broker.

Deliver service credentials to an app

A user-provided service instance can be used to deliver credentials to an App. For example, a database admin can have a set of credentials for an existing database managed outside of Kf, and these credentials include the URL, port, username, and password used to connect to the database.

The admin can create a user-provided service with the credentials and the developer can bind the service instance to the App. This allows the credentials to be shared without ever leaving the platform. Binding a service instance to an App has the same effect regardless of whether the service is a user-provided service or a marketplace service.

The App is configured with the credentials provided by the user, and the App runtime environment variable VCAP_SERVICES is populated with information about all of the bound services to that App.

A user-provided service can be created with credentials and/or a list of tags.

kf cups my-db -p '{"username":"admin", "password":"test123", "some-int-val": 1, "some-bool": true}' -t "comma, separated, tags"

This will create the user-provided service instance my-db with the provided credentials and tags. The credentials passed in to the -p flag must be valid JSON (either inline or loaded from a file path).

To deliver the credentials to one or more Apps, the user can run kf bind-service.

Suppose we have an App with one bound service, the user-provided service my-db defined above. The VCAP_SERVICES environment variable for that App will have the following contents:

{
  "user-provided": [
    {
      "name": "my-db",
      "instance_name": "my-db",
      "label": "user-provided",
      "tags": [
        "comma",
        "separated",
        "tags"
      ],
      "credentials": {
        "username": "admin",
        "password": "test123",
        "some-int-val": 1,
        "some-bool": true
      }
    }
  ]
}

Update a user-provided service instance

Tip: The alias for kf update-user-provided-service is kf uups.

A user-provided service can be updated with the uups command. New credentials and/or tags passed in completely overwrite existing ones. For example, if the user created the user-provided service my-db above, called kf bind-service to bind the service to an App, then ran the command.

kf uups my-db -p '{"username":"admin", "password":"new-pw", "new-key": "new-val"}'

The updated credentials will only be reflected on the App after the user unbinds and rebinds the service to the App. No restart or restage of the App is required. The updated VCAP_SERVICES environment variable will have the following contents:

{
  "user-provided": [
    {
      "name": "my-db",
      "instance_name": "my-db",
      "label": "user-provided",
      "tags": [
        "comma",
        "separated",
        "tags"
      ],
      "credentials": {
        "username": "admin",
        "password": "new-pw",
        "new-key": "new-val"
      }
    }
  ]
}

The new credentials overwrite the old credentials, and the tags are unchanged because they were not specified in the update command.

2.2.4 - User Provided Service Templates

Learn how to set up user provided services for MySQL, Redis, and RabbitMQ.

Before you begin

Ensure your service is running and accessible on the same network running your Kf cluster.
Ensure you have targeted the Space where you want to create the service.

Create the user-provided instance

The following examples use the most common parameters used by applications to autoconfigure services. Most libraries use tags to find the right bound service and a URI to connect.

MySQL

MySQL libraries usually expect the tag mysql and the following parameters:

uri: Example mysql://username:password@host:port/dbname. The MySQL documentation can help with creating a URI string. The port is usually 3306.
username: The connection username, required by some libraries even if included in uri.
password: The connection password, required by some libraries even if included in uri.

kf cups service-instance-name \
  -p '{"username":"my-username", "password":"my-password", "uri":"mysql://my-username:my-password@mysql-host:3306/my-db"}' \
  -t "mysql"

RabbitMQ

RabbitMQ libraries usually expect the tag rabbitmq and the following parameters:

uri: Example amqp://username:password@host:port/vhost?query. The RabbitMQ documentation can help with creating a URI string. The port is usually 5672.

Example:

kf cups service-instance-name \
  -p '{"uri":"amqp://username:password@host:5672"}' \
  -t "rabbitmq"

Redis

Redis libraries usually expect the tag redis and the following parameters:

uri: Example redis://:password@host:port/uery. The IANA Redis URI documentation can help with creating a URI string. The port is usually 6379.

Example for Redis with no AUTH configured:

kf cups service-instance-name \
  -p '{"uri":"redis://redis-host:6379"}' \
  -t "redis"

Example for Redis with AUTH configured:

kf cups service-instance-name \
  -p '{"uri":"redis://:password@redis-host:6379"}' \
  -t "redis"

Bind your App

Once the user-provided service has been created, you can bind your App to the user provided service by name, just like a managed service:

kf bind-service application-name service-instance-name

What’s next

Learn about how the credentials are injected into your app.

2.2.5 - Configure NFS volumes

Learn how to use an NFS volume as a mounted drive.

Kf supports mounting NFS volumes using the kf marketplace.

Prerequisites

Your administrator must have completed the NFS platform setup guide.

Create an NFS service instance

Run kf marketplace to see available services. The built-in NFS service appears on the list if NFS is enabled on the platform.

Broker           Name  Namespace  Description
nfsvolumebroker  nfs              mount nfs shares

Mount an external filesystem

Create a service instance

To mount to an existing NFS service:

kf create-service nfs existing SERVICE-INSTANCE-NAME -c '{"share":"SERVER/SHARE", "capacity":"CAPACITY"}'

Replace variables with your values.

SERVICE-INSTANCE-NAME is the name you want for this NFS volume service instance.
SERVER/SHARE is the NFS address of your server and share.
CAPACITY uses the Kubernetes quantity format.

Confirm that the NFS volume service appears in your list of services. You can expect output similar to this example:

$ kf services
...
Listing services in Space: demo-space
Name                Type      ClassName         PlanName  Age    Ready  Reason
filestore-nfs       volume    nfs               existing  6s     True   <nil>
...

Bind your service instance to an App

To bind an NFS service instance to an App, run:

kf bind-service YOUR-APP-NAME SERVICE-NAME -c '{"uid":"2000","gid":"2000","mount":"MOUNT-PATH","readonly":true}'

Replace variables with your values.

YOUR-APP-NAME is the name of the App for which you want to use the volume service.
SERVICE-NAME is the name of the volume service instance you created in the previous step.
uid:UID and gid:GID specify the directory permissions of the mounting share.
Note: For V2 buildpack Apps, the value for uid and gid is always 2000. Otherwise, the user specified by uid and gid should be the uid and gid of the running App process.
MOUNT-PATH is the path the volume should be mounted to within your App.
(Optional) "readonly":true is an optional JSON string that creates a read-only mount. By default, Volume Services mounts a read-write file system.
Note: Your App automatically restarts when the NFS binding changes.

You can list all bindings in a Space using the kf bindings command. You will see output similar to this example:

$ kf bindings
...
Listing bindings in Space: demo-space
Name                                     App           Service             Age  Ready
binding-spring-music-filestore-nfs       spring-music  filestore-nfs       71s  True
...

Access the volume service from your App

To access the volume service from your App, you must know which file path to use in your code. You can view the file path in the details of the service binding, which are visible in the environment variables for your App.

View environment variables for your App:

kf vcap-services YOUR-APP-NAME

Replace YOUR-APP-NAME with the name of your App.

The following is example output of the kf vcap-services command:

$ kf vcap-services YOUR-APP-NAME
{
  "nfs": [
    {
      "instance_name": "nfs-instance",
      "name": "nfs-instance",
      "label": "nfs",
      "tags": [],
      "plan": "existing",
      "credentials": {
        "capacity": "1Gi",
        "gid": 2000,
        "mount": "/test/mount",
        "share": "10.91.208.210/test",
        "uid": 2000
      },
      "volume_mounts": [
        {
          "container_dir": "/test/mount",
          "device_type": "shared",
          "mode": "rw"
        }
      ]
    }
  ]
}

Use the properties under volume_mounts for any information required by your App.

Property	Description
`container_dir`	String containing the path to the mounted volume that you bound to your App.
`device_type`	The NFS volume release. This currently only supports shared devices. A shared device represents a distributed file system that can mount on all App instances simultaneously.
`mode`	String that informs what type of access your App has to NFS, either `ro` (read-only), or `rw` (read and write).

2.3 - Configure and Use Service Accounts

Configure and Use Kubernetes Service Accounts from your Apps.

By default, all applications in Kf are assigned a unique Kubernetes Service Account (KSA) named sa-<APP_NAME>. Kf uses this KSA as the “user” it runs application instances and tasks under.

Each App KSA receives a copy of the container registry credentials used by the Space’s build KSA so Kf apps can pull container images that were created during kf push.

Using the service account

Kuberenetes Pods (the building blocks of Apps and Tasks) automatically receive a JWT for the KSA mounted in the container:

$ ls /var/run/secrets/kubernetes.io/serviceaccount/
ca.crt
namespace
token

ca.crt The Kubernetes control plane’s certificate.
namespace The Kubernetes namespace of the workload.
token A Base64 encoded JWT for the Kf App’s Service Account.

Below is an example of what the JWT looks like, note that:

It expires and needs to be periodically refreshed from disk.
It’s audience is only valid within the Kubernetes cluster.

{
    "aud": [
        "<KUBERNETES_CLUSTER_URI>"
    ],
    "exp": 3600,
    "iat": 0,
    "iss": "<KUBERNETES_CLUSTER_URI>",
    "kubernetes.io": {
        "namespace": "<SPACE_NAME>",
        "pod": {
            "name": "<APP_NAME>-<RANDOM_SUFFIX>",
            "uid": "<APP_GUID>"
        },
        "serviceaccount": {
            "name": "sa-<APP_NAME>",
            "uid": "<SERVICE_ACCOUNT_GUID>"
        },
        "warnafter": 3500
    },
    "nbf": 0,
    "sub": "system:serviceaccount:<SPACE_NAME>:sa-<APP_NAME>"
}

You can use this credential to connect to the Kubernetes control plane listed in the issuer (iss) field.

Customizing the service account

You want to use a different service account than the default one Kf provides, for example to:

Allow blue/green apps to have the same identity.
Use Kf with a federated identity system.
Provide custom image pull credentials for a specific app.

You can enable this by adding the apps.kf.dev/service-account-name annotation to your app manifest. The value should be the name of the KSA you want the application and tasks to use.

Example:

applications:
- name: my-app
  metadata:
    annotations:
      "apps.kf.dev/service-account-name": "override-sa-name"

Limitations:

Only KSAs within the same Kubernetes namespace–corresponding to a Kf Space–are allowed.
The KSA must exist and be readable by Kf, otherwise the app will not deploy.
The KSA or the cluster must have permission to pull the application’s container images.

Additional resources

If you use GKE, learn how to inject apps with a Google Service Account credential.
Learn how to use federated identity in GCP to allow authenticating KSAs to GCP infrastructure.

2.4 - Scaling

Learn to scale apps up or down.

2.4.1 - Scaling overview

Learn about how Kf scales apps.

Kf leverages the Kubernetes Horizontal Pod Autoscaler (HPA) to automatically scale the number of Pods in a App. When autoscaling is enabled for an App, an HPA object is created and bound to the App object. It then dynamically calculates the target scale and sets it for the App.

Kf Apps are also compatible with HPA policies created outside of Kf.

How Kf scaling works

The number of Pods that are deployed for a Kf App is controlled by its underlying Deployment object’s replicas field. The target number of Deployment replicas is set through the App’s replicas field.

Scaling can be done manually with the kf scale command. This command is disabled when autoscaling is enabled to avoid conflicting targets.

How the Kubernetes Horizontal Pod Autoscaler works

The Horizontal Pod Autoscaler (HPA) is implemented as a Kubernetes API resource (the HPA object) and a control loop (the HPA controller) which periodically calculates the number of desired replicas based on current resource utilization. The HPA controller then passes the number to the target object that implements the Scale subresource. The actual scaling is delegated to the underlying object and its controller. You can find more information in the Kubernetes documentation.

How the Autoscaler determines when to scale

Periodically, the HPA controller queries the resource utilization against the metrics specified in each HorizontalPodAutoscaler definition. The controller obtains the metrics from the resource metrics API for each Pod. Then the controller calculates the utilization value as a percentage of the equivalent resource request. The desired number of replicas is then calculated based on the ratio of current percentage and desired percentage. You can read more about the autoscaling algorithm in the Kubernetes documentation.

Metrics

Kf uses HPA v1 which only supports CPU as the target metric.

How the Kubernetes Horizontal Autoscaler works with Kf

When autoscaling is enabled for a Kf App, the Kf controller will create an HPA object based on the scaling limits and rules specified on the App. Then the HPA controller fetches the specs from the HPA object and scales the App accordingly.

The HPA object will be deleted if Autoscaling is disabled or if the corresponding App is deleted.

2.4.2 - Manage Autoscaling

Learn to use autoscaling for your app.

Kf supports two primary autoscaling modes:

Built-in autosacling similar to Cloud Foundry.
Advanced autoscaling through the Kubernetes Horizontal Pod Autoscaler (HPA).

Built-in autoscaling

Kf Apps can be automatically scaled based on CPU usage. You can configure autoscaling limits for your Apps and the target CPU usage for each App instance. Kf automatically scales your Apps up and down in response to demand.

By default, autoscaling is disabled. Follow the steps below to enable autoscaling.

View Apps

You can view the autoscaling status for an App using the kf apps command. If autoscaling is enabled for an App, Instances includes the autoscaling status.

$ kf apps

Name   Instances              Memory  Disk  CPU
app1   4 (autoscaled 4 to 5)  256Mi   1Gi   100m
app2   1                      256Mi   1Gi   100m

Autoscaling is enabled for app1 with min-instances set to 4 and max-instances set to 5. Autoscaling is disabled for app2.

Update autoscaling limits

You can update the instance limits using the kf update-autoscaling-limits command.

kf update-autoscaling-limits app-name min-instances max-instances

Create autoscaling rule

You can create autoscaling rules using the kf create-autoscaling-rule command.

kf create-autoscaling-rule app-name CPU min-threshold max-threshold

Delete autoscaling rules

You can delete all autoscaling rules with the kf delete-autoscaling-rule command. Kf only supports one autoscaling rule.

kf delete-autoscaling-rules app-name

Enable and disable autoscaling

Autoscaling can be enabled by using enable-autoscaling and disabled by using disable-autoscaling. When it is disabled, the configurations, including limits and rules, are preserved.

kf enable-autoscaling app-name

kf disable-autoscaling app-name

Advanced autoscaling

Kf Apps support the Kubernetes Horizontal Pod Autoscaler interface and will therefore work with HPAs created using kubectl.

Kubernetes HPA policies are less restrictive than Kf’s built-in support for autoscaling.

They include support for:

Scaling on memory, CPU, or disk usage.
Scaling based on custom metrics, such as traffic load or queue length.
Scaling on multiple metrics.
The ability to tune reactivity to smooth out rapid scaling.

Using custom HPAs with apps

You can follow the Kubernetes HPA walkthrough to learn how to set up autoscalers.

When you create the HPA, make sure to set the scaleTargetRef to be your application:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-scaler
  namespace: SPACE_NAME
spec:
  scaleTargetRef:
    apiVersion: kf.dev/v1alpha1
    kind: App
    name: APP_NAME
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 60

Caveats

You shouldn’t use Kf autoscaling with an HPA.
When you use an HPA, kf apps will show the current number of instances, it won’t show that the App is being autoscaled.

2.4.3 - Managing Resources for Apps

Learn to set resources on apps.

When you create an app, you can optionally specify how much of each resource an instance of the application will receive when it runs.

Kf simplifies the Kubernetes model of resources and provides defaults that should work for most I/O bound applications out of the box.

Resource types

Kf supports three types of resources, memory, CPU, and ephemeral disk.

Memory specifies the amount of RAM an application receives when running. If it exceeds this amount then the container is restarted.
Ephemeral disk specifies how much an application can write to a local disk. If an application exceeds this amount then it may not be able to write more.
CPU specifies the number of CPUs an application receives when running.

Manifest

Resources are specified using four values in the manifest:

memory sets the guaranteed minimum an app will receive and the maximum it’s permitted to use.
disk_quota sets the guaranteed minimum an app will receive and the maximum it’s permitted to use.
cpu sets the guaranteed minimum an app will receive.
cpu-limit sets the maximum CPU an app can use.

Example:

applications:
- name: "example"
  disk_quota: 512M
  memory: 512M
  cpu: 200m
  cpu-limit: 2000m

Defaults

Memory and ephemeral storage are both set to 1Gi if not specified.

CPU defaults to one of the following

1/10th of a CPU if the platform operator hasn’t overridden it.
A CPU value proportionally scaled by the amount of memory requested.
A minimum CPU value set by the platform operator.

Resource units

Memory and disk

Cloud Foundry used the units T, G, M, and K to represent powers of two. Kubernetes uses the units Ei, Pi, Gi, Mi, and Ki for the same.

Kf allows you to specify memory and disk in either units.

CPU

Kf and Kubernetes use the unit m for CPU, representing milli-CPU cores (thousandths of a core).

Sidecar overhead

When Kf schedules your app’s container as a Kubernetes Pod, it may bundle additional containers to your app to provide additional functionality. It’s likely your application will also have an Istio sidecar which is responsible for networking.

These containers will supply their own resource requests and limits and are overhead associated with running your application.

Best practices

All applications should set memory and disk quotas.
CPU intensive applications should set a CPU request and limit to guarantee they’ll have the resources they need without starving other apps.
I/O bound applications shouldn’t set a CPU limit so they can burst during startup.

Additional reading

Learn how Kubernetes defines resources and schedules apps based on resource use

2.5 - Service discovery

Connect to other Apps in the Kf cluster.

This document is an overview of Kubernetes DNS-based service discovery and how it can be used with Kf.

When to use Kubernetes service discovery with Kf

Kubernetes service discovery can be used by applications that need to locate backing services in a consistent way regardless of where the application is deployed. For example, a team might want to use a common URI in their configuration that always points at the local SMTP gateway to decouple code from the environment it ran in.

Service discovery helps application teams by:

Reducing the amount of per-environment configuration.
Decoupling client and server applications.
Allowing applications to be portable to new environments.

You can use Kubernetes service discovery when:

Applications use their container’s DNS configurations to resolve hosts.
Applications are deployed with their backing services in the same Kubernetes cluster or namespace.
Backing services have an associated Kubernetes service. Kf creates these for each app.
Kubernetes NetworkPolicies allow traffic between an application and the Kubernetes service it needs to communicate with. Kf creates these policies in each Kf space.

You should not use Kubernetes service discovery if:

Applications need to failover between multiple clusters.
You override the DNS resolver used by your application.
Applications need specific types of load balancing.

How Kubernetes service discovery works

Kubernetes service discovery works by modifying the DNS configuration of containers running on a Kubernetes node. When an application looks up an unqualified domain name, the local DNS resolver will first attempt to resolve the name in the local cluster.

Domains without multiple parts will be resolved against the names of Kubernetes services in the container’s namespace. Each Kf app creates a Kubernetes service with the same name. If two Kf apps ping and pong were deployed in the same Kf space, then ping could use the URL http://pong to send traffic to the other service.

Domains with a single dot will be resolved against the Kubernetes services in the Kubernetes namespace with the same name as the label after the dot. For example, if there was a PostgreSQL database with a customers service in the database namespace, an application in another namespace could resolve it using postgres://customers.database.

How to use service discovery with Kf

Kubernetes DNS based service discovery can be used in any Kf app. Each Kf app creates a Kubernetes service of the same name, and each Kf space creates a Kubernetes namespace with the same name.

Refer to a Kf app in the current space using protocol://app-name.
Refer to a Kf app in a different space using protocol://app-name.space-name.
Refer to a Kf app in the current space listening on a custom port using protocol://app-name:port.
Refer to a Kf app in a different space listening a custom port using protocol://app-name.space-name:port.

Note: For Kubernetes services managed by Kf, TCP port 80 always maps to the same port in the app’s PORT environment variable. This is even true if the Kf app doesn’t serve HTTP traffic.

Best practices

Applications that are going to be the target of DNS based service discovery should have frequent health checks to ensure they are rapidly added and removed from the pool of hosts that accept connections.

Applications using DNS based service discovery should not cache the IP addresses of the resolved services because they are not guaranteed to be stable.

If environment specific services exist outside of the cluster, they can be resolved using Kubernetes DNS if you set up ExternalName Kubernetes services. These Kubernetes services provide the same resolution capabilities, but return a CNAME record to redirect requests to an external authority.

Comparison to Eureka

Eureka is an open source client-side load-balancer created by Netflix. It is commonly used as part of the Spring Cloud Services service broker. Eureka was built to be a regional load balancer and service discovery mechanism for services running in an environment that caused frequent disruptions to workloads leading to unstable IP addresses.

Eureka is designed as a client/server model. Clients register themselves with the server indicating which names they want to be associated with and periodically send the server heartbeats. The server allows all connected clients to resolve names.

In general, you should use Kubernetes DNS rather than Eureka in Kubernetes for the following reasons:

DNS works with all programming languages and applications without the need for libraries.
Your application’s existing health check will be reused reducing combinations of errors.
Kubernetes manages the DNS server, allowing you to rely on fewer dependencies.
Kubernetes DNS respects the same policy and RBAC constraints as the rest of Kubernetes.

There are a few times when deploying a Eureka server would be advantageous:

You need service discovery across Kubernetes and VM based applications.
You need client based load-balancing.
You need independent health checks.

What’s next

Read more about service discovery in GKE.
Learn about Service Directory, a managed offering similar to Eureka.

2.6 - Debugging workloads

Understand how to debug workloads using Kubernetes.

Kf uses Kubernetes under the hood to schedule application workloads onto a cluster. Ultimately, every workload running on a Kubernetes cluster is scheduled as a Pod, but the Pods may have different properties based on the higher level abstractions that schedule them.

Once you understand how to debug Pods, you can debug any running workload on the cluster including the components that make up Kf and Kubernetes.

General debugging

Kubernetes divides most resources into Namespaces. Each Kf Space creates a Namespace with the same name. If you’re debugging a Kf resource, you’ll want to remember to set the -n or --namespace CLI flag on each kubectl command you run.

Finding a Pod

You can list all the Pods in a Namespace using the command:

kubectl get pods -n NAMESPACE

This will list all Pods in the Namespace. You’ll often want to filter these using a label selector like the following:

kubectl get pods -n NAMESPACE -l "app.kubernetes.io/component=app-server,app.kubernetes.io/name=echo"

You can get the definition of a Pod using a command like the following:

kubectl get pods -n NAMESPACE POD_NAME -o yaml

Note: kubectl supports a variety of output types similar to Kf using the -o flag. yaml, json, and jsonpath are common ones.

Understand Kubernetes objects

Most Kubernetes objects follow the same general structure. Kubernetes also has extensive help documentation for each type of object (including Kf’s).

If you need to quickly look up what a field is for, use the kubectl explain command with the object type and the path to the field you want documentation on.

$ kubectl explain pod.metadata.labels
KIND:     Pod
VERSION:  v1

FIELD:    labels <map[string]string>

DESCRIPTION:
     Map of string keys and values that can be used to organize and categorize
     (scope and select) objects. May match selectors of replication controllers
     and services. More info: http://kubernetes.io/docs/user-guide/labels

When you run kubectl get RESOURCE_TYPE RESOURCE_NAME -oyaml you’ll see the stored version of the object. An annotated example of an Pod running the instance of an App is below:

apiVersion: v1
kind: Pod
metadata:
    # Annotations hold security information and configuration for the 
    # object.
    annotations:
        kubectl.kubernetes.io/default-container: user-container
        kubectl.kubernetes.io/default-logs-container: user-container
        sidecar.istio.io/inject: "true"
        traffic.sidecar.istio.io/includeOutboundIPRanges: '*'
    # Labels hold information useful for filtering resources.
    labels:
        # Kf sets many of these labels to help find Pods.
        app.kubernetes.io/component: app-server
        app.kubernetes.io/managed-by: kf
        app.kubernetes.io/name: echo
        kf.dev/networkpolicy: app
    name: echo-6b759c978b-zwrt8
    namespace: development
    # Contains the object(s) that "own" this resource, this usually
    # means the ones that were responsible for creating it.
    ownerReferences:
    - apiVersion: apps/v1
        blockOwnerDeletion: true
        controller: true
        kind: ReplicaSet
        name: echo-6b759c978b
        uid: 7f0ee42d-f1e8-4c4f-b0c8-f0c5d7f27a0a
    # The ID of the resource, if deleted and re-created the ID will
    # change.
    uid: 0d49d5d7-afa4-4904-9f69-f98ce1923745
spec:
    # Contains the desired state of the object, written by a human or
    # one of the metadata.ownerReferences.
    # Omitted for brevity.
status:
    # Contains the state of the object as written by the controller.
    # Omitted for brevity.

When you see an object with an metadata.ownerReferences set, you can run kubectl get again to find that object’s information all the way back up until you find the root object responsible for creating it. In this case the chain would look like Pod -> ReplicaSet -> Deployment -> App.

Getting logs

You can get logs for a specific Pod using kubectl logs:

kubectl logs -c user-container -n NAMESPACE POD_NAME

You can find a list of containers on the Pod in the .spec.containers.name field:

kubectl get pods -n NAMESPACE -o jsonpath='{.spec.containers[*].name}' POD_NAME

Note: Kf calls the container running the App code user-container. On newer versions of Kf, you won’t have to supply -c due to the kubectl.kubernetes.io/default-container annotation on the Pod that kubectl knows how to read.

Port forwarding

You can port-forward to a specific port using kubectl port-forward.

# Bind remote port 8080 to local port 8080.
kubectl port-forward -n NAMESPACE POD_NAME 8080

# Bind remote port 8080 to local port 9000.
kubectl port-forward -n NAMESPACE POD_NAME 9000:8080

Open a shell

You can open a shell to a container using kubectl exec.

kubectl exec --stdin --tty -n NAMESPACE -c user-container POD_NAME -- /bin/bash

Kf Specifics

The following sections show specific information about Kf types that schedule Pods.

App Pods

Label selector

All Apps:

app.kubernetes.io/component=app-server,app.kubernetes.io/managed-by=kf

Specific App (replace APP_NAME):

app.kubernetes.io/component=app-server,app.kubernetes.io/managed-by=kf,app.kubernetes.io/name=APP_NAME

Expected containers

user-container Container running the application code.
istio-proxy Connects the application code to the virtual network.

Ownership hierarchy

Each resource will have a metadata.ownerReferences to the resource below it:

Kubernetes Pod Runs a single instance of the application code.
Kubernetes ReplicaSet Schedules all the instances for one version of an App.
Kubernetes Deployment Manages rollouts and scaling of multiple versions of the App.
Kf App Orchestrates routing, rollouts, service bindings, builds, etc. for an App.

Build Pods

Label selector

All Builds:

app.kubernetes.io/component=build,app.kubernetes.io/managed-by=kf

Builds for a specific App (replace APP_NAME):

app.kubernetes.io/component=build,app.kubernetes.io/managed-by=kf,app.kubernetes.io/name=APP_NAME

Specific Build for an App (replace APP_NAME and TASKRUN_NAME):

app.kubernetes.io/component=build,app.kubernetes.io/managed-by=kf,app.kubernetes.io/name=APP_NAME,tekton.dev/taskRun=TASKRUN_NAME

Expected containers

step-.* Container(s) that execute different steps of the build process. Specific steps depend on the type of build.
istio-proxy (Optional) Connects the application code to the virtual network.

Ownership hierarchy

Each resource will have a metadata.ownerReferences to the resource below it:

Kubernetes Pod Runs the steps of the build.
Tekton TaskRun Schedules the Pod, ensures it runs to completion once, and cleans it up once done.
Kf Build Creates a TaskRun with the proper steps to build an App from source.
Kf App Creates Builds with app-specific information like environment variables.

Task Pods

Label selector

All Tasks:

app.kubernetes.io/component=task,app.kubernetes.io/managed-by=kf

Tasks for a specific App (replace APP_NAME):

app.kubernetes.io/component=task,app.kubernetes.io/managed-by=kf,app.kubernetes.io/name=APP_NAME

Specific Task for an App (replace APP_NAME and TASKRUN_NAME):

app.kubernetes.io/component=task,app.kubernetes.io/managed-by=kf,app.kubernetes.io/name=APP_NAME,tekton.dev/taskRun=TASKRUN_NAME

Expected containers

step-user-container Container running the application code.
istio-proxy Connects the application code to the virtual network.

Ownership hierarchy

Kubernetes Pod Runs an instance of the application code.
Tekton TaskRun Schedules the Pod, ensures it runs to completion once, and cleans it up once done.
Kf Task Creates a TaskRun with the proper step to run a one-off Task.
Kf App Creates Builds with app-specific information like environment variables.

Next steps

Explore the Kubernetes guide to debugging applications.
Understand how to write label selectors.

2.7 - Configure routes and domains

Make your app reachable over the network.

This page describes how routes and domains work in Kf, and how developers and administrators configure routes and domains for an App deployed on Kf cluster.

You must create domain and routes to give external access to your application.

Internal routing

Kf apps can communicate internally with other apps in the cluster directly using a service mesh without leaving the cluster network. By default, all traffic on the service mesh is encrypted using mutual TLS.

All apps deployed in the Kf cluster come with an internal endpoint configured by default. You can use the address app-name.space-name.svc.cluster.local for internal communication between apps. To use this internal address no extra steps are required. Mutual TLS is enabled by default for internal routes. Note that this internal address is only accessible from the pods running the apps and not accessible from outside the cluster.

App load balancing

Traffic is routed by Istio to healthy instances of an App using a round-robin policy. Currently, this policy can’t be changed.

Route capabilities

Routes tell the cluster’s ingress gateway where to deliver traffic and what to do if no Apps are available on the given address. By default, if no App is available on a Route and the Route receives a request it returns an HTTP 503 status code.

Routes are comprised of three parts: host, domain, and path. For example, in the URI payroll.mydatacenter.example.com/login:

The host is payroll
The domain is mydatacenter.example.com
The path is /login

Routes must contain a domain, but the host and path is optional. Multiple Routes can share the same host and domain if they specify different paths. Multiple Apps can share the same Route and traffic will be split between them. This is useful if you need to support legacy blue/green deployments. If multiple Apps are bound to different paths, the priority is longest to shortest path.

Warning: Kf doesn’t currently support TCP port-based routing. You must use a Kubernetes LoadBalancer if you want to expose a TCP port to the Internet. Ports are available on the cluster internal App address <app-name>.<space>.

Manage routes

The following sections describe how to use the kf CLI to manage Routes.

List routes

Developers can list Routes for the current Space using the kf routes command.

$ kf routes
Getting Routes in Space: my-space
Found 2 Routes in Space my-space

HOST    DOMAIN       PATH    APPS
echo    example.com  /       echo
*       example.com  /login  uaa

Create a route

Developers can create Routes using the kf create-route command.

# Create a Route in the targeted Space to match traffic for myapp.example.com/*
$ kf create-route example.com --hostname myapp

# Create a Route in the Space myspace to match traffic for myapp.example.com/*
$ kf create-route -n myspace example.com --hostname myapp

# Create a Route in the targeted Space to match traffic for myapp.example.com/mypath*
$ kf create-route example.com --hostname myapp --path /mypath

# You can also supply the Space name as the first parameter if you have
# scripts that rely on the old cf style API.
$ kf create-route myspace example.com --hostname myapp # myapp.example.com

After a Route is created, if no Apps are bound to it then an HTTP 503 status code is returned for any matching requests.

Map a route to your app

Developers can make their App accessible on a Route using the kf map-route command.

$ kf map-route MYAPP mycluster.example.com --hostname myapp --path mypath

Note: map-route creates the Route if it doesn’t exist yet.

Unmap a route

Developers can remove their App from being accessible on a Route using the kf unmap-route command.

$ kf unmap-route MYAPP mycluster.example.com --hostname myapp --path mypath

Delete a route

Developers can delete a Route using the kf delete-route command.

$ kf delete-route mycluster.example.com --hostname myapp --path mypath

Deleting a Route will stop traffic from being routed to all Apps listening on the Route.

Manage routes declaratively in your app manifest

Routes can be managed declaratively in your app manifest file. They will be created if they do not yet exist.

---
applications:
- name: my-app
  # ...
  routes:
  - route: example.com
  - route: www.example.com/path

You can read more about the supported route properties in the manifest documentation.

Routing CRDs

There are four types that are relevant to routing:

VirtualService
Route
Service
App

Each App has a Service, which is an abstract name given to all running instances of your App. The name of the Service is the same as the App. A Route represents a single external URL. Routes constantly watch for changes to Apps, when an App requests to be added to a Route, the Route updates its list of Apps and then the VirtualService. A VirtualService represents a single domain and merges a list of all Routes in a Space that belong to that domain.

Istio reads the configuration on VirtualServices to determine how to route traffic.

2.8 - Tasks

Learn how to run one-off jobs using Kf.

2.8.1 - Tasks Overview

Understand how tasks work in Kf.

About Tasks

Unlike Apps which run indefinitely and restart if the process terminates, Tasks run a process until it completes and then stop. Tasks are run in their own containers and are based on the configuration and binary of an existing App.

Tasks are not accessible from routes, and should be used for one-off or scheduled recurring work necessary for the health of an application.

Use cases for Tasks

Migrating a database
Running a batch job (scheduled/unscheduled)
Sending an email
Transforming data (ETL)
Processing data (upload/backup/download)

How Tasks work

Tasks are executed asynchronously and run independently from the parent App or other Tasks running on the same App. An App created for running Tasks does not have routes created or assigned, and the Run lifecycle is skipped. The Source code upload and Build lifecycles still proceed and result in a container image used for running Tasks after pushing the App (see App lifecycles at Deploying an Application).

The lifecycle of a Task is as follows:

You push an App for running tasks with the kf push APP_NAME --task command.
You run a Task on the App with the kf run-task APP_NAME command. Task inherits the environment variables, service bindings, resource allocation, start-up command, and security groups bound to the App.
Kf creates a Tekton PipelineRun with values from the App and parameters from the run-task command.
The Tekton PipelineRun creates a Kubernetes Pod which launches a container based on the configurations on the App and Task.
Task execution stops (Task exits or is terminated manually), the underlying Pod is either stopped or terminated. Pods of stopped Tasks are preserved and thus Task logs are accessible via the kf logs APP_NAME --task command.
If you terminate a Task before it stops, the Tekton PipelineRun is cancelled (see Cancelling a PipelineRun), the underlying Pod together with the logs are deleted. The logs of termianted Tasks are delivered to the cluster level logging streams if configured (e.g. Stackdriver, Fluentd).
If the number of Tasks run on an App is greater than 500, the oldest Tasks are automatically deleted.

Tasks retention policy

Tasks are created as custom resources in the Kubernetes cluster, therefore, it is important not to exhaust the space of the underlying etcd database. By default, Kf only keeps the latest 500 Tasks per each App. Once the number of Tasks reach 500, the oldest Tasks (together with the underlying Pods and logs) will be automatically deleted.

Task logging and execution history

Any data or messages the Task outputs to STDOUT or STDERR is available by using the kf logs APP_NAME --task command. Cluster level logging mechanism (such as Stackdriver, Fluentd) will deliver the Task logs to the configured logging destination.

Scheduling Tasks

As described above, Tasks can be run asynchronously by using the kf run-task APP_NAME command. Alternatively, you can schedule Tasks for execution by first creating a Job using the kf create-job command, and then scheduling it with the kf schedule-job JOB_NAME command. You can schedule that Job to automatically run Tasks on a specified unix-cron schedule.

How Tasks are scheduled

Create and schedule a Job to run the Task. A Job describes the Task to execute and automatically manages Task creation.

Tasks are created on the schedule even if previous executions of the Task are still running. If any executions are missed for any reason, only the most recently missed execution are executed when the system recovers.

Deleting a Job deletes all associated Tasks. If any associated Tasks were still in progress, they are forcefully deleted without running to completion.

Tasks created by a scheduled Job are still subject to the Task retention policy.

Differences from PCF Scheduler

PCF Scheduler allows multiple schedules for a single Job while Kf only supports a single schedule per Job. You can replicate the PCF Scheduler behavior by creating multiple Jobs, one for each schedule.

2.8.2 - Run Tasks

Learn how to use tasks to run one-off jobs.

You can execute short-lived workflows by running them as Tasks in Kf. Tasks are run under Apps, meaning that each Task must have an associated App. Each Task execution uses the build artifacts from the parent App. Because Tasks are short-lived, the App is not deployed as a long-running application, and no routes should be created for the App or the Task.

Push an App for running Tasks

Clone the test-app repo repo:

git clone https://github.com/cloudfoundry-samples/test-app test-app

cd test-app

Push the App.
Push the App with the kf push APP_NAME --task command. The --task flag indicates that the App is meant to be used for running Tasks, and thus no routes are created on the App, and it is not deployed as a long-running application:
```
kf push test-app --task
```

Confirm that no App instances or routes were created by listing the App:

kf apps

Notice that the App is not started and has no URLs:

Listing Apps in Space: test-space
Name                     Instances  Memory  Disk  CPU   URLs
test-app                 stopped    1Gi     1Gi   100m  <nil>

Run a Task on the App

When you run a Task on the App, you can optionally specify a start command by using the --command flag. If no start command is specified, it uses the start command specified on the App. If the App doesn’t have a start command specified, it looks up the CMD configuration of the container image. A start command must exist in order to run the Task successfully.

kf run-task test-app --command "printenv"

You see something similar to this, confirming that the Task was submitted:

Task test-app-gd8dv is submitted successfully for execution.

The Task name is automatically generated, prefixed with the App name, and suffixed with an arbitrary string. The Task name is a unique identifier for Tasks within the same cluster.

Specify Task resource limits

Resource limits (such as CPU cores/Memory limit/Disk quota) can be specified in the App (during kf push) or during the kf run-task command. The limits specified in the kf run-task command take prededence over the limits specified on the App.

To specify resource limits in an App, you can use the --cpu-cores, --memory-limit, and --disk-quota flags in the kf push command:

kf push test-app --command "printenv" --cpu-cores=0.5 --memory-limit=2G --disk-quota=5G --task

To override these limits in the App, you can use the --cpu-cores, --memory-limit, and --disk-quota flags in the kf run-task command:

kf run-task test-app --command "printenv" --cpu-cores=0.5 --memory-limit=2G --disk-quota=5G

Specify a custom display name for a Task

You can optionally use the --name flag to specify a custom display name for a Task for easier identification/grouping:

$ kf run-task test-app --command "printenv" --name foo
Task test-app-6swct is submitted successfully for execution.

$ kf tasks test-app
Listing Tasks in Space: test space
Name              ID  DisplayName        Age    Duration  Succeeded  Reason
test-app-6swct    3   foo                1m     21s       True       <nil>

Manage Tasks

View all Tasks of an App with the kf tasks APP_NAME command:

$ kf tasks test-app
Listing Tasks in Space: test space
Name              ID  DisplayName        Age    Duration  Succeeded  Reason
test-app-gd8dv    1   test-app-gd8dv     1m     21s       True       <nil>

Cancel a Task

Cancel an active Task by using the kf terminate-task command:

Cancel a Task by Task name:

$ kf terminate-task test-app-6w6mz
Task "test-app-6w6mz" is successfully submitted for termination

Or cancel a Task by APP_NAME + Task ID:

$ kf terminate-task test-app 2
Task "test-app-6w6mz" is successfully submitted for termination

Cancelled Tasks have PipelineRunCancelled status.

$ kf tasks test-app
Listing Tasks in Space: test space
Name              ID  DisplayName        Age    Duration  Succeeded  Reason
test-app-gd8dv    1   test-app-gd8dv     1m     21s       True       <nil>
test-app-6w6mz    2   test-app-6w6mz     38s    11s       False      PipelineRunCancelled

View Task logs

View logs of a Task by using the kf logs APP_NAME --task command:

$ kf logs test-app --task

2.8.3 - Schedule Tasks

Learn how to schedule tasks to run periodic jobs.

You can execute short-lived workflows by running them as Tasks. Running Tasks describes how to run Tasks under Apps.

You can also schedule Tasks to run at recurring intervals specified using the unix-cron format. With scheduled Tasks, you first push an App running the Task as you do with an unscheduled Task, and then create a Job to schedule the Task.

You can define a schedule so that your Task runs multiple times a day or on specific days and months.

Push an App for running scheduled Tasks

Clone the test-app repo:

git clone https://github.com/cloudfoundry-samples/test-app test-app

cd test-app

Push the App.
Push the App with the kf push APP_NAME --task command. The --task flag indicates that the App is meant to be used for running Tasks, and thus no routes will be created on the App and it will not be deployed as a long-running application.
```
kf push test-app --task
```

Confirm that no App instances or routes were created by listing the App:

kf apps

Notice that the App is not started and has no URLs:

Listing Apps in Space: test-space
Name                     Instances  Memory  Disk  CPU   URLs
test-app                 stopped    1Gi     1Gi   100m  <nil>

Create a Job

To run a Task on a schedule, you must first create a Job that describes the Task:

kf create-job test-app test-job "printenv"

The Job starts suspended or unscheduled, and does not create Tasks until it is manually executed by kf run-job or scheduled by kf schedule-task.

Run a Job manually

Jobs can be run ad hoc similar to running Tasks by kf run-task. This option can be useful for testing the Job before scheduling or running as needed in addition to the schedule.

kf run-job test-job

This command runs the Task defined by the Job a single time immediately.

Schedule a Job

To schedule the Job for execution, you must provide a unix-cron schedule in the kf schedule-job command:

kf schedule-job test-job "* * * * *"

This command triggers the Job to automatically create Tasks on the specified schedule. In this example a Task runs every minute.

You can update a Job’s schedule by running kf schedule-task with a new schedule. Jobs in Kf can only have a single cron schedule. This differs from the PCF Scheduler, which allows multiple schedules for a single Job. If you require multiple cron schedules, then you can achieve that with multiple Jobs.

Manage Jobs and schedules

View all Jobs, both scheduled and unscheduled, in the current Space by using the kf jobs command:

$ kf jobs
Listing Jobs in Space: test space
Name               Schedule    Suspend  LastSchedule  Age  Ready  Reason
test-job           * * * * *   <nil>    16s           2m   True   <nil>
unscheduled-job    0 0 30 2 *  true     16s           2m   True   <nil>

Note: The unscheduled-job has a default schedule set (0 0 30 2 *). This schedule is not active because the Job is suspended and is only present as a placeholder schedule for unscheduled Jobs.

Additionally, you can view only Jobs that are actively scheduled with the kf job-schedules command.

$ kf job-schedules
Listing job schedules in Space: test space
Name           Schedule   Suspend  LastSchedule  Age  Ready  Reason
test-job       * * * * *  <nil>    16s           2m   True   <nil>

Notice how the unscheduled-job is not listed in the kf job-schedules output.

Cancel a Job’s schedule

You can stop a scheduled Job with the kf delete-job-schedule command:

kf delete-job-schedule test-job

This command suspends the Job and stops it from creating Tasks on the previous schedule. The Job is not deleted and can be scheduled again by kf schedule-job to continue execution.

Delete a Job

The entire Job can be deleted with the kf delete-job command:

kf delete-job test-job

This command deletes the Job and all Tasks that were created by the Job, both scheduled and manual executions. If any Tasks are still running, this command forcefully deletes them.

If you want to ensure that running Tasks are not interrupted, then first delete the Jobs schedule with kf delete-job-schedule, wait for all Tasks to complete, and then delete the job by calling kf delete-job.

3 - Operate and Maintain Kf

Learn to run and maintain Kf as a platform.

3.1 - Service brokers

Learn to install and operate backing service marketplaces for Kf.

3.1.1 - Service Brokers Overview

Kf supports binding and provisioning apps to Open Service Broker (OSB) services.

Any compatible service broker can be installed using the create-service-broker command, but only the Kf Cloud Service Broker is fully supported.

Special services such as syslog drains, volume services, route services, service keys, and shared services aren’t currently supported.

3.1.2 - About Kf Cloud Service Broker

The Kf Cloud Service Broker is a Service Broker bundle that includes the open source Cloud Service Broker and Google Cloud Brokerpak. It is made available as a public Docker image and ready to deploy as a Kubernetes service in Kf clusters. Once the Kf Cloud Service Broker service is deployed in a cluster, developers can provision Google Cloud backing services through the Kf Cloud Service Broker service, and bind the backing services to Kf Apps.

Requirements

Kf Cloud Service Broker requires a MySQL instance and a service account for accessing the MySQL instance and Google Cloud backing services to be provisioned. Connection from the Kf Cloud Service Broker to the MySQL instance goes through the Cloud SQL Auth Proxy.
Requests to access Google Cloud services (for example: Cloud SQL for MySQL or Cloud Memorystore for Redis) are authenticated via Workload Identity.

Override Brokerpak defaults

Brokerpaks are essentially a Terraform plan and related dependencies in a tar file. You can inspect the Terraform plans to see what the defaults are, and then you can tell Kf Cloud Service Broker to override them when creating new services.

For example, the Terraform plan for MySQL includes a variable called authorized_network. If not overridden, the default VPC will be used. If you’d like to override the default, you can pass that during service creation. Here are some examples:

Override the compute region config.

 kf create-service csb-google-postgres small spring-music-postgres-db -c '{"config":"YOUR_COMPUTE_REGION"}'

Override the authorized_network and compute region config.

 kf create-service csb-google-postgres small spring-music-postgres-db -c '{"config":"YOUR_COMPUTE_REGION","authorized_network":"YOUR_CUSTOM_VPC_NAME"}'

You can learn more by reading the MySQL Plans and Configs documentation.

Architecture

The following Kf Cloud Service Broker architecture shows how instances are created.

The Kf Cloud Service Broker (CSB) is installed in its own namespace.
On installation, a MySQL instance must be provided to persist business logic used by Kf Cloud Service Broker. Requests are sent securely from the Kf Cloud Service Broker pod to the MySQL instance via the MySQL Auth Proxy.
On service provisioning, a Kf Service custom resource is created. The reconciler of the Kf Service provisions Google Cloud backing services using the Open Service Broker API.
When a request to provision/deprovision backing resources is received, Kf Cloud Service Broker sends resource creation/deletion requests to the corresponding Google Cloud service, and these requests are authenticated with Workload Identity. It also persists the business logics (e.g. mapping of Kf services to backing services, service bindings) to the MySQL instance.
On backing service creation success, the backing service is bound to an App via VCAP_SERVICES.

What’s next?

{% endblock %}

3.1.3 - Deploy Kf Cloud Service Broker

This page shows you how to deploy Kf Cloud Service Broker and use it to provision or deprovision backing resources. Read about the concepts and architecture to learn more about the Kf Cloud Service Broker.

Create environment variables

Linux

export PROJECT_ID=YOUR_PROJECT_ID
export CLUSTER_PROJECT_ID=YOUR_PROJECT_ID
export CSB_IMAGE_DESTINATION=YOUR_DOCKER_REPO_CSB_PATH
export CLUSTER_NAME=kf-cluster
export INSTANCE_NAME=cloud-service-broker
export COMPUTE_REGION=us-central1

Windows PowerShell

Set-Variable -Name PROJECT_ID -Value YOUR_PROJECT_ID
Set-Variable -Name CLUSTER_PROJECT_ID -Value YOUR_PROJECT_ID
Set-Variable -Name CSB_IMAGE_DESTINATION=YOUR_DOCKER_REPO_CSB_PATH
Set-Variable -Name CLUSTER_NAME -Value kf-cluster
Set-Variable -Name INSTANCE_NAME -Value cloud-service-broker
Set-Variable -Name COMPUTE_REGION -Value us-central1

Build the broker

First you’ll want to download and build the broker and push it to your container registry:

git clone --single-branch --branch main https://github.com/google/kf.git kf
cd kf/samples/cloud-service-broker
docker build --tag ${CSB_IMAGE_DESTINATION} .
docker push ${CSB_IMAGE_DESTINATION}

Set up the Kf Cloud Service Broker database

Create a MySQL instance.
Note: Read Creating and managing MySQL users for Google Cloud SQL and set a secure password for the default root user.
```
gcloud sql instances create ${INSTANCE_NAME} --cpu=2 --memory=7680MB --require-ssl --region=${COMPUTE_REGION}
```
Create a database named servicebroker in the MySQL instance.
Note: Document the database name since it is used in later steps.
```
gcloud sql databases create servicebroker -i ${INSTANCE_NAME}
```
Create a username and password to be used by the broker.
Note: Document these values since they will be used in later steps.
```
gcloud sql users create csbuser -i ${INSTANCE_NAME} --password=csbpassword
```

Set up a Google Service Account for the broker

Create a Google Service Account.

 gcloud iam service-accounts create csb-${CLUSTER_NAME}-sa \
     --project=${CLUSTER_PROJECT_ID} \
     --description="GSA for CSB at ${CLUSTER_NAME}" \
     --display-name="csb-${CLUSTER_NAME}"

Grant roles/cloudsql.client permissions to the Service Account. This is required to connect the service broker pod to the CloudSQL for MySQL instance through the CloudSQL Proxy.

 gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
     --member="serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
     --role="roles/cloudsql.client"

Grant additional Google Cloud permissions to the Service Account.

Note: In the example below, we grant IAM roles required to provision an instance of CloudSQL and Cloud Memorystore. You must grant this service account the appropriate roles to provision instances of other Google Cloud managed services listed in kf marketplace.

 gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
     --member="serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
     --role="roles/compute.networkUser"

 gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
     --member="serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
     --role="roles/cloudsql.admin"

 gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
     --member="serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
     --role="roles/redis.admin"

Verify the permissions.
Warning: Replace the CSB_SERVICE_ACCOUNT_NAME variable in the YAML below with the full service account resolved from csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com.
```
 gcloud projects get-iam-policy ${CLUSTER_PROJECT_ID} \
     --filter='bindings.members:serviceAccount:"CSB_SERVICE_ACCOUNT_NAME"' \
     --flatten="bindings[].members"
```

Set up Workload Identity for the broker

Bind the Google Service Account with the Kubernetes Service Account.

 gcloud iam service-accounts add-iam-policy-binding "csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
     --project=${CLUSTER_PROJECT_ID} \
     --role="roles/iam.workloadIdentityUser" \
     --member="serviceAccount:${CLUSTER_PROJECT_ID}.svc.id.goog[kf-csb/csb-user]"

Verify the binding.

 gcloud iam service-accounts get-iam-policy "csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
     --project=${CLUSTER_PROJECT_ID}

Set up a Kubernetes Secret to share configuration with the broker

Create a config.yml file.

Note: Replace the default user/password if desired. Ensure you have set the CLUSTER_PROJECT_ID in the Create environment variables step.

cat << EOF >> ./config.yml
gcp:
  credentials: ""
  project: ${CLUSTER_PROJECT_ID}

db:
  host: 127.0.0.1
  password: csbpassword
  user: csbuser
  tls: false
api:
  user: servicebroker
  password: password
EOF

Create the kf-csb namespace.
```
kubectl create ns kf-csb
```

Create the Kubernetes Secret.

kubectl create secret generic csb-secret --from-file=config.yml -n kf-csb

Install the Kf Cloud Service Broker

Copy the kf-csb-template.yaml into kf-csb.yaml for working:
```
cp kf-csb-template.yaml /tmp/kf-csb.yaml
```

Edit /tmp/kf-csb.yaml and replace placeholders with final values. In the example below, sed is used.

sed -i "s|<GSA_NAME>|csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com|g" /tmp/kf-csb.yaml

sed -i "s|<INSTANCE_CONNECTION_NAME>|${CLUSTER_PROJECT_ID}:${COMPUTE_REGION}:${INSTANCE_NAME}|g" /tmp/kf-csb.yaml

sed -i "s|<DB_PORT>|3306|g" /tmp/kf-csb.yaml

sed -i "s|<CSB_IMAGE_DESTINATION>|${CSB_IMAGE_DESTINATION}|g" /tmp/kf-csb.yaml

Apply yaml for Kf Cloud Service Broker.
```
kubectl apply -f /tmp/kf-csb.yaml
```
Verify the Cloud Service Broker installation status.
```
kubectl get pods -n kf-csb
```

Create a service broker

kf create-service-broker cloud-service-broker servicebroker password http://csb-controller.kf-csb/

Validate installation

Check for available services in the marketplace.

kf marketplace

If everything is installed and configured correctly, you should see the following:

$ kf marketplace

Broker                Name                          Namespace  Description
cloud-service-broker  csb-google-bigquery                      A fast, economical and fully managed data warehouse for large-scale data analytics.
cloud-service-broker  csb-google-dataproc                      Dataproc is a fully-managed service for running Apache Spark and Apache Hadoop clusters in a simpler, more cost-efficient way.
cloud-service-broker  csb-google-mysql                         Mysql is a fully managed service for the Google Cloud Platform.
cloud-service-broker  csb-google-postgres                      PostgreSQL is a fully managed service for the Google Cloud Platform.
cloud-service-broker  csb-google-redis                         Cloud Memorystore for Redis is a fully managed Redis service for the Google Cloud Platform.
cloud-service-broker  csb-google-spanner                       Fully managed, scalable, relational database service for regional and global application data.
cloud-service-broker  csb-google-stackdriver-trace             Distributed tracing service
cloud-service-broker  csb-google-storage-bucket                Google Cloud Storage that uses the Terraform back-end and grants service accounts IAM permissions directly on the bucket.

Clean up

Delete cloud-service-broker.

kf delete-service-broker cloud-service-broker

Delete CSB components.
```
kubectl delete ns kf-csb
```

Delete the broker’s database instance.

 gcloud sql instances delete ${INSTANCE_NAME} --project=${CLUSTER_PROJECT_ID}

Remove the IAM policy bindings.

 gcloud projects remove-iam-policy-binding ${CLUSTER_PROJECT_ID} \
 --member='serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com' \
 --role=roles/cloudsql.client

 gcloud projects remove-iam-policy-binding ${CLUSTER_PROJECT_ID} \
 --member='serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com' \
 --role=roles/compute.networkUser

 gcloud projects remove-iam-policy-binding ${CLUSTER_PROJECT_ID} \
 --member='serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com' \
 --role=roles/redis.admin

Remove the GSA.

 gcloud iam service-accounts delete csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com \
   --project=${CLUSTER_PROJECT_ID}

3.1.4 - Set up NFS platform

Kf supports Kubernetes native NFS, and exposes them with a dedicated nfsvolumebroker service broker for developers to consume. This broker has an nfs service offering which has a service plan named existing.

Use kf marketplace to see the service offering:

$ kf marketplace
...
Broker           Name  Namespace  Description
nfsvolumebroker  nfs              mout nfs shares
...

Use kf marketplace -s nfs to see the service plan:

$ kf marketplace -s nfs
...
Broker           Name      Free  Description
nfsvolumebroker  existing  true  mount existing nfs server
...

Requirements

You need an NFS volume that can be accessed by your Kubernetes cluster. For example Cloud Filestore which Google’s managed NFS solution that provides access to clusters in the same gcloud project.

Prepare NFS

If you have an existing NFS service, you can use that. If you want a Google managed NFS service, create a Filestore instance and Kf will automaticlaly configure the cluster to use it.

Warning: You only need to create the NFS instance. Kf will create relevant Kubernetes objects, including PersistentVolume and PersistentVolumeClaims. Do not manually mount the volume.

What’s next?

Configure NFS volumes

3.2 - Customizing Kf

Learn about how to configure cluster-wide settings.

3.2.1 - Customizing Overview

Learn about how Kf uses configuration.

Kf supports customizing some cluster wide configurations by manipulating the kfsystem custom rsource.

3.2.2 - Cluster Configuration

Learn to configure your Kf cluster’s settings.

Kf uses a Kubernetes configmap named config-defaults in the kf namespace to store cluster wide configuration settings. This document explains its structure and fields.

Note: This configuration is usually copied from the kfsystem by the Kf operator. The values can be set in the operator under the spec.kf.config path.

Structure of the config-defaults configmap

The configmap contains three types of key/value pairs in the .data field:

Comment keys prefixed by _ contain examples, notes, and warnings.
String keys contain plain text values.
Object keys contain a JSON or YAML value that has been encoded as a string.

Example:

_note: "This is some note"
stringKey: "This is a string key that's not encoded as JSON or YAML."
objectKey: |
  - "These keys contain nested YAML or JSON."
  - true
  - 123.45

Example section

The example section under the _example key contains explanations for other fields and examples. Changes to this section have no effect.

Space container registry

The spaceContainerRegistry property is a plain text value that specifies the default container registry each space uses to store built images.

Example:

spaceContainerRegistry: gcr.io/my-project

Space cluster domains

The spaceClusterDomains property is a string encoded YAML array of domain objects.

Each space in the cluster adds all items in the array to its list of domains that developers can bind their apps to.

Fields

Fields
`domain`	`string` The domain name to make available. May contain one of the following substitutions: `$(SPACE_NAME)` - Replaced in each space with the name of the space. `$(CLUSTER_INGRESS_IP)` - The IP address of the cluster ingress gateway.
`gatewayName`	`string` (Optional) Overrides the Istio gateway routes will be bound to. Defaults to `kf/external-gateway`, but any other gateway in the `kf` namespace may be used.

domain

string

The domain name to make available. May contain one of the following substitutions:

$(SPACE_NAME) - Replaced in each space with the name of the space.
$(CLUSTER_INGRESS_IP) - The IP address of the cluster ingress gateway.

gatewayName

string

(Optional)

Overrides the Istio gateway routes will be bound to. Defaults to kf/external-gateway, but any other gateway in the kf namespace may be used.

Note: All domains should contain the $(SPACE_NAME) substitution to prevent Apps in different namespaces from listening to the same hostname which causes undefined behavior.

Example:

spaceClusterDomains: |
  # Support canonical and vanity domains
  - domain: $(SPACE_NAME).prod.example.com
  - domain: $(SPACE_NAME).kf.us-east1.prod.example.com

  # Using a dynamic DNS resolver
  - domain: $(SPACE_NAME).$(CLUSTER_INGRESS_IP).nip.io

  # Creating an internal domain only visible within the cluster
  - domain: $(SPACE_NAME)-apps.internal
    gatewayName: kf/internal-gateway

Buildpacks V2 lifecycle builder

The buildpacksV2LifecycleBuilder property contains the version of the Cloud Foundry builder binary used execute buildpack v2 builds.

The value is a Git reference. To use a specific version, append an @ symbol followed by a Git SHA to the end.

Example:

buildpacksV2LifecycleBuilder: "code.cloudfoundry.org/buildpackapplifecycle/builder@GIT_SHA"

Buildpacks V2 lifecycle launcher

The buildpacksV2LifecycleLauncher property contains the version of the Cloud Foundry launcher binary built into every buildpack V2 application.

The value is a Git reference. To use a specific version, append an @ symbol followed by a Git SHA to the end.

Example:

buildpacksV2LifecycleLauncher: "code.cloudfoundry.org/buildpackapplifecycle/launcher@GIT_SHA"

Buildpacks V2 list

The spaceBuildpacksV2 property is a string encoded YAML array that holds an ordered list of default buildpacks that are used to build applications compatible with the V2 buildpacks process.

Fields

Fields
`name`	`string` A short name developers can use to reference the buildpack by in their application manifests.
`url`	`string` The URL used to fetch the buildpack.
`disabled`	`boolean` Used to prevent this buildpack from executing.

name

string

A short name developers can use to reference the buildpack by in their application manifests.

url

string

The URL used to fetch the buildpack.

disabled

boolean

Used to prevent this buildpack from executing.

Stacks V2 list

The spaceBuildpacksV2 property is a string encoded YAML array that holds an ordered list of stacks that can be used with Cloud Foundry compatible builds.

Fields

name

string

A short name developers can use to reference the stack by in their application manifests.

image

string

URL of the container image to use as the stack. For more information, see https://kubernetes.io/docs/concepts/containers/images.

Stacks V3 list

The spaceStacksV3 property is a string encoded YAML array that holds an ordered list of stacks that can be used with Cloud Native Buildpack builds.

Fields
`name`	`string` A short name developers can use to reference the stack by in their application manifests.
`description`	`string` A short description of the stack shown when running `kf stacks`.
`buildImage`	`string` URL of the container image to use as the builder. For more information, see https://kubernetes.io/docs/concepts/containers/images.
`runImage`	`string` URL of the container image to use as the base for all apps built with . For more information, see https://kubernetes.io/docs/concepts/containers/images.
`nodeSelector`	`map (key: string, value: string)` (Optional) A NodeSelector used to indicate which nodes applications built with this stack can run on.

Example:

spaceStacksV3: |
  - name: heroku-18
    description: The official Heroku stack based on Ubuntu 18.04
    buildImage: heroku/pack:18-build
    runImage: heroku/pack:18
    nodeSelector:
       kubernetes.io/os: windows

Default to V3 Stack

The spaceDefaultToV3Stack property contains a quoted value true or false indicating whether spaces should use V3 stacks if a user doesn’t specify one.

Feature flags

The featureFlags property contains a string encoded YAML map of feature flags that can enable and disable features of Kf.

Flag names that aren’t supported by Kf will be ignored.

Flag Name	Default	Purpose
`disable_custom_builds`	`false`	Disable developer access to arbitrary Tekton build pipelines.
`enable_dockerfile_builds`	`true`	Allow developers to build source code from dockerfiles.
`enable_custom_buildpacks`	`true`	Allow developers to specify external buildpacks in their applications.
`enable_custom_stacks`	`true`	Allow developers to specify custom stacks in their applications.

Example:

featureFlags: |
  disable_custom_builds: false
  enable_dockerfile_builds: true
  enable_some_feature: true

ProgressDeadlineSeconds

ProgressDeadlineSeconds contains a configurable quoted integer indicating the maximum allowed time between state transition and reaching a stable state before provisioning or deprovisioning when pushing an application. The default value is 600 seconds.

TerminationGracePeriodSeconds

The TerminationGracePeriodSeconds contains a configurable quoted integer indicating the time between when the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. The default value is 30 seconds.

3.2.3 - Customizing Kf Features

Learn to configure your Kf cluster’s settings.

Build Retention

You can control how many Kf Builds are kept before being garbage collected.

kubectl patch \
kfsystem kfsystem \
--type='json' \
-p="[{'op': 'replace', 'path': '/spec/kf/config/buildRetentionCount', 'value': 1}]"

Task Retention

You can control how many Kf Tasks are kept before being garbage collected.

kubectl patch \
kfsystem kfsystem \
--type='json' \
-p="[{'op': 'replace', 'path': '/spec/kf/config/taskRetentionCount', 'value': 1}]"

Enable or Disable the Istio Sidecar

If you do not require the Istio sidecar for the Build pods, then they can be disabled by setting the value to true. Enable by setting the value to false.

kubectl patch \
kfsystem kfsystem \
--type='json' \
-p="[{'op': 'replace', 'path': '/spec/kf/config/buildDisableIstioSidecar', 'value': true}]"

Enable or Disable Routing Retries

Allows enabling/disbaling retries in the VirtualServices that route traffic to Apps. Kf leaves this value unset by default and it’s inherited from Istio.

Istio’s default retry mechanism attempts to make up for instability inherent in service meshes, however allowing retries requires the contents of the payload to be buffered within Envoy. This may fail for large payloads and the buffering will need to be disabled at the expense of some stability.

Values for routeDisableRetries:

false Inherit Istio’s retry settings. (Default)
true Set retries to 0.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/routeDisableRetries','value':true}]"

Enable or Disable Routing Hosts Ignoring Any Port Numbers

Allows enabling/disabling routing hosts ignoring any specified port number. By default hosts are matched using the exact value specified in the route Host (e.g a request with a Host header value of example.com:443 does not match with preconfigured route Host example.com). By enabling, ports are ignored and only hosts are used (e.g example.com:443 matches example.com)

Note: Feature will only work in clusters with istio 1.15+. In older versions it will function as though it where disabled.

Values for routeHostIgnoringPort:

false Will match the Host header in request to the exact configured route Host. (Default)
true Will use regexp to match to the configured route Host ignoring any port specified in the Host header of the request.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/routeHostIgnoringPort','value':true}]"

Build Pod Resource Limits

The default pod resource size can be increased from the default to accommodate very large builds. The units for the value are in Mi or Gi.

Note: This is only applicable for built-in Tasks (which is normal for a kf push build). For V2 buildpack builds, this will be set on two steps and one for V3 buildpacks or Dockerfiles. This means that for a V2 build the required Pod size will be double the limit. For example, if the memory limit is 1Gi, then the pod will require 2Gi.

kubectl patch \
kfsystem kfsystem \
--type='json' \
-p="[{'op': 'replace', 'path': '/spec/kf/config/buildPodResources', 'value': {'limits': {'memory': '234Mi'}}}]"

Read Kubernetes container resource docs for more information about container resource management.

Robust Build Snapshots

Kf uses Kaniko to build the final application containers in the v2 buildpack lifecycle. To produce image layers, Kaniko needs to take “snapsohts” of the image which requires iterating over all files in the image and checking if they’ve changed.

The fast mode checks for file attribute changes (like timestamps and size) and the slow mode checks full file hashes to determine if a file changed.

Kf apps aren’t expected to overwrite operating system files in their build, so fast mode should be used to reduce disk pressure.

Note: This value should be set to false in Kaniko versions older than v1.15.0.

Values for buildKanikoRobustSnapshot:

false Will use a fast snapshot mode for v2 builds. (Default)
true Will use a robust snapshot mode for v2 builds to catch uncommon cases.

kubectl patch \
kfsystem kfsystem \
--type='json' \
-p="[{'op': 'replace', 'path': '/spec/kf/config/buildKanikoRobustSnapshot', 'value': true}]"

Self Signed Certificates for Service Brokers

If you want to use self signed certificates for TLS (https instead of http) for the service broker URL, the Kf controller requires the CA certificate. To configure Kf for this scenario, create an immutable Kubernetes secret in the kf namespace and update the kfsystem.spec.kf.config.secrets.controllerCACerts.name object to point to it.

Create a secret to store the self-signed certificate.
Note: Customize the secret name if desired, or leave the default name of cacerts. Replace /path/to/cert/certs.pem with the path to the self-signed certificate.
```
 kubectl create secret generic cacerts -nkf --from-file /path/to/cert/certs.pem
 
```

Make the secret immutable.

 kubectl patch -nkf secret cacerts \
     --type='json' \
     -p="[{'op':'add','path':'/immutable','value':true}]"

Update kfsystem to point to the secret.

kubectl patch \
  kfsystem kfsystem \
  --type='json' \
  -p="[{'op':'add','path':'/spec/kf/config/secrets','value':{'controllerCACerts':{'name':'<var>cacerts</var>'}}}]"

Set CPU minimums and ratios

Application default CPU ratios and minimums can be set in the operator.

Values are set in CPU units. Units are typically expressed in millicpus (m), or thousandths of a CPU.

The spec.kf.config.appCPUMin property specifies a minimum amount of CPU per application, even if the developer has specified less.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/appCPUMin','value':'<var>200m</var>'}]"

The spec.kf.config.appCPUPerGBOfRAM property specifies a default amount of CPU to give each app per GB or RAM requested.

You can choose different approaches based on the desired outcome:

Choose the ratio of CPU to RAM for the cluster’s nodes if you want to maximize utilization.
Choose a ratio of 1 CPU to 4 GB of RAM which typically works well for I/0 or memory bound web applications.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/appCPUPerGBOfRAM','value':'<var>250m</var>'}]"

Set buildpacks using git tags

Buildpacks can support pinning by using git tags instead of automatically sourcing the latest buildpack from a git repository.

Note: Buildpacks can be added using the kfsystem operator and kubectl patch.

Add a new buildpack as follows and use a git tag to specify which version of the buildpack the app should use. Otherwise the buildpack will default to the latest version.

For example, to pin Golang buildpack version 1.9.49 do:

kubectl patch \
kfsystem kfsystem \
--type='json' \
 -p='[{"op":"add","path":"data/spec/kf/config/spaceBuildpacksV2","value":[{"name":"go_buildpack_v1.9.49","url":"https://github.com/cloudfoundry/go-buildpack.git#v1.9.49"}]}]'

This command will add the following to the config-defaults configmaps resource:

data:
  SpaceBuildpacksV2: |
    - name: go_buildpack_v1.9.49
      url: https://github.com/cloudfoundry/go-buildpack.git#v1.9.49

The kubectl patch command will replace all the existing buildpacks in the config-defaults configmaps. If the user would like the existing buildpacks to remain, these too need to be included in the command.

To get the list of existing buildpacks in the configmaps run the following command:

kubectl describe configmaps config-defaults -n kf

Set ProgressDeadlineSeconds

ProgressDeadlineSeconds can be set in the kfsystem operator.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/progressDeadlineSeconds','value':100}]"

Set TerminationGracePeriodSeconds

TerminationGracePeriodSeconds can be set in the kfsystem operator.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/terminationGracePeriodSeconds','value':200}]"

Enable/Disable App Start Command Lookup

Allows enabling/disbaling start command lookup in the App reconciler. This behavior requires the reconciler to fetch container configuration for every app from the container registry and enables displaying the start command on kf push and in kf app.

Enabling this behavior on a large cluster may make the reconcilation times for Apps slow.

Values for appDisableStartCommandLookup:

false Enable start command lookup. (Default)
true Disable start command lookup.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/appDisableStartCommandLookup','value':true}]"

Enable/Disable Kubernetes service Account (KSA) overrides

Allows enabling/disbaling the ability to override the Kubernetes Service Account for Apps via annotation.

Values for appEnableServiceAccountOverride:

false Don’t allow overriding service account. (Default)
true Allow developers to override KSAs for their Apps.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/appEnableServiceAccountOverride','value':true}]"

Set default Kf Task timeout

Kf uses Tekton TaskRuns as its mechanism to run Kf Tasks. Tekton may impose a default timeout on TaskRuns that differs depending on the version of Tekton you have installed.

You can override this setting either on the Tekton configmap – which will set the value for both Kf Tasks and Builds or on the Kf operator to apply the value only to Tasks.

The following values are supported:

null - Inherit the value from Tekton (Default).
Value <= 0 - Tasks get an infinite timeout.
Value >= 0 - Tasks get the given timeout.

Consider setting a long, but non-infinite timeout to prevent improperly programmed tasks from consuming resources.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/taskDefaultTimeoutMinutes','value':-1}]"

Enable/Disbale NFS in Kf Tasks

You can enable/disable the ability for Tasks run from Apps to mount NFS volumes.

Mounting NFS volumes requires FUSE which grants Task Pods with NFS additional system privileges on the node and may be a security concern.

kubectl patch \
    kfsystem kfsystem \
    --type='json' \
    -p="[{'op':'add','path':'/spec/kf/config/taskDisableVolumeMounts','value':true}]"

3.3 - Testing Kf Components

Understand the Kf test suite and what it covers.

This page describes the tests that Kf runs and notable gaps you may want to cover when using Kf as your platform.

Components

Unit tests

Kf uses Go’s unit testing functionality to perform extensive validation of the business logic in the CLI, the control plane components, and “golden” files which validate the structure of I/O that Kf expects to be deterministic (e.g. certain CLI output).

Unit tests can be executed using the ./hack/unit-test.sh shell script in the Kf repository. This script skips exercising the end to end tests.

Unit tests are naturally limited because they have to mock interactions with external services e.g. the Kubernetes control plane and container registries.

End to end tests

Kf uses specially marked Go tests for acceptance and end to end testing.

These can be executed using the ./hack/integration-test.sh and ./hack/acceptance-test.sh shell scripts. These scripts will test against the currently targeted Kf cluster.

These tests check for behavior between Kf and other components, but are still limited because they tend to test one specific thing at a time and are built for speed.

Load tests

Kf can run load tests against the platform using the ./ci/cloudbuild/test.yaml Cloud Build template with the _STRESS_TEST_BASELOAD_FACTOR environment variable set.

This will deploy the Diego stresds test workloads to Kf to check how the platform behaves with failing Apps.

Gaps

Kf runs unit tests and end to end tests for each release. You may want to augment with additional qualificatoin tests on your own cluster to check for:

Compatibility with your service brokers.
Compatibility with all buildpacks you normally use.
Compatibility with represenative workloads.
Compatibility with intricate combinations of features (e.g. service bindings, docker containers, buildpacks, routing).
Compatibility with non-Kf Kubernetes components.

3.4 - Kf dependencies and architecture

Kf requires Kubernetes and several other OSS projects to run. Some of the dependencies are satisfied with Google-managed services—for example, GKE provides Kubernetes.

Dependencies

Diagram showing how Kf components interact.

Get CRD details

Kf supports the kubectl subcommand explain. It allows you to list the fields in Kf CRDs to understand how to create Kf objects via automation instead of manually via the CLI. This command is designed to be used with ConfigSync to automate creation and management of resources like Spaces across many clusters. You can use this against any of the component kinds below.

In this example, we examine the kind called space in the spaces CRD:

kubectl explain space.spec

The output looks similar to this:

$ kubectl explain space.spec
KIND:     Space
VERSION:  kf.dev/v1alpha1

RESOURCE: spec <Object>

DESCRIPTION:
     SpaceSpec contains the specification for a space.

FIELDS:
   buildConfig  <Object>
     BuildConfig contains config for the build pipelines.

   networkConfig        <Object>
     NetworkConfig contains settings for the space's networking environment.

   runtimeConfig        <Object>
     RuntimeConfig contains settings for the app runtime environment.

Kf components

Kf installs several of its own Kubernetes custom resources and controllers. The custom resources effectively serve as the Kf API and are used by the kf CLI to interact with the system. The controllers use Kf’s CRDs to orchestrate the other components in the system.

You can view the CRDs installed and used by Kf by running the following command:

kubectl api-resources --api-group=kf.dev

The output of that command is as follows:

NAME                      SHORTNAMES   APIGROUP   NAMESPACED   KIND
apps                                   kf.dev     true         App
builds                                 kf.dev     true         Build
clusterservicebrokers                  kf.dev     false        ClusterServiceBroker
routes                                 kf.dev     true         Route
servicebrokers                         kf.dev     true         ServiceBroker
serviceinstancebindings                kf.dev     true         ServiceInstanceBinding
serviceinstances                       kf.dev     true         ServiceInstance
spaces                                 kf.dev     false        Space

Apps

Apps represent a twelve-factor application deployed to Kubernetes. They encompass source code, configuration, and the current state of the application. Apps are responsible for reconciling:

Kf Builds
Kf Routes
Kubernetes Deployments
Kubernetes Services
Kubernetes ServiceAccounts
Kubernetes Secrets

You can list Apps using Kf or kubectl:

kf apps

kubectl get apps -n space-name

Builds

Builds combine the source code and build configuration for Apps. They provision Tekton TaskRuns with the correct steps to actuate a Buildpack V2, Buildpack V3, or Dockerfile build.

You can list Builds using Kf or kubectl:

kf builds

kubectl get builds -n space-name

ClusterServiceBrokers

ClusterServiceBrokers hold the connection information necessary to extend Kf with a service broker. They are responsible for fetching the catalog of services the broker provides and displaying them in the output of kf marketplace.

You can list ClusterServiceBrokers using kubectl:

kubectl get clusterservicebrokers

Routes

Routes are a high level structure that contain HTTP routing rules. They are responsible for reconciling Istio VirtualServices.

You can list Routes using Kf or kubectl:

kf routes

kubectl get routes -n space-name

ServiceBrokers

ServiceBrokers hold the connection information necessary to extend Kf with a service broker. They are responsible for fetching the catalog of services the broker provides and displaying them in the output of kf marketplace.

You can list ServiceBrokers using kubectl:

kubectl get servicebrokers -n space-name

ServiceInstanceBinding

ServiceInstanceBindings hold the parameters to create a binding on a service broker and the credentials the broker returns for the binding. They are responsible for calling the bind API on the broker to bind the service.

You can list ServiceInstanceBindings using Kf or kubectl:

kf bindings

kubectl get serviceinstancebindings -n space-name

ServiceInstance

ServiceInstances hold the parameters to create a service on a service broker. They are responsible for calling the provision API on the broker to create the service.

You can list ServiceInstances using Kf or kubectl:

kf services

kubectl get serviceinstances -n space-name

Spaces

Spaces hold configuration information similar to Cloud Foundry organizations and spaces. They are responsible for:

Creating the Kubernetes Namespace that other Kf resources are provisioned into.
Creating Kubernetes NetworkPolicies to enforce network connection policies.
Holding configuration and policy for Builds, Apps, and Routes.

You can list Spaces using Kf or kubectl:

kf spaces

kubectl get spaces

Kf RBAC / Permissions

The following sections list permissions for Kf and its components to have correct access at the cluster level. These permissions are required and enabled by default in Kf; do not attempt to disable them.

Components	Namespace	Service Account
`controller`	kf	controller
`subresource-apiserver`	kf	controller
`webhook`	kf	controller
`appdevexperience-operator`	appdevexperience	appdevexperience-operator

Note that the appdevexperience-operator service account has the same set of permissions as controller. The operator is what deploys all Kf components, including custom resource definitions and controllers.

RBAC for Kf service accounts

The following apiGroup definitions detail which access control permissions components in {{product_name}} have on which API groups and resources for both the controller and appdevexperience-operator service accounts.

- apiGroups:
  - "authentication.k8s.io"
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - "authorization.k8s.io"
  resources:
  - subjectaccessreviews
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - persistentvolumeclaims
  - persistentvolumes
  - endpoints
  - events
  - configmaps
  - secrets
  verbs: *
- apiGroups:
  - ""
  resources:
  - services
  - services/status
  verbs:
  - create
  - delete
  - get
  - list
  - watch
- apiGroups:
  - "apps"
  resources:
  - deployments
  - daemonsets
  - replicasets
  - statefulsets
  verbs: *
- apiGroups:
  - "apps"
  resources:
  - deployments/finalizers
  verbs:
  - get
  - list
  - create
  - update
  - delete
  - patch
  - watch
- apiGroups:
  - "rbac.authorization.k8s.io"
  resources:
  - clusterroles
  - roles
  - clusterrolebindings
  - rolebindings
  verbs:
  - create
  - delete
  - update
  - patch
  - escalate
  - get
  - list
  - deletecollection
  - bind
- apiGroups:
  - "apiregistration.k8s.io"
  resources:
  - apiservices
  verbs:
  - update
  - patch
  - create
  - delete
  - get
  - list
- apiGroups:
  - "pubsub.cloud.google.com"
  resources:
  - topics 
  - topics/status
  verbs: *
- apiGroups:
  - ""
  resources:
  - namespaces
  - namespaces/finalizers
  - serviceaccounts
  verbs: 
  - get
  - list
  - create
  - update
  - watch
  - delete
  - patch
  - watch
- apiGroups:
  - "autoscaling"
  resources:
  - horizontalpodautoscalers
  verbs: 
  - create
  - delete
  - get
  - list
  - update
  - patch
  - watch
- apiGroups:
  - "coordination.k8s.io"
  resources:
  - leases
  verbs: *
- apiGroups:
  - "batch"
  resources:
  - jobs
  - cronjobs
  verbs: 
  - get
  - list
  - create
  - update
  - patch
  - delete
  - deletecollection
  - watch
- apiGroups:
  - "messaging.cloud.google.com"
  resources:
  - channels
  verbs: 
  - delete
- apiGroups:
  - "pubsub.cloud.google.com"
  resources:
  - pullsubscriptions
  verbs: 
  - delete
  - get
  - list
  - watch
  - create
  - update
  - patch
- apiGroups:
  - "pubsub.cloud.google.com"
  resources:
  - [pullsubscriptions/status
  verbs: 
  - get
  - update
  - patch
- apiGroups:
  - "events.cloud.google.com"
  resources: *
  verbs: *
- apiGroups:
  - "keda.k8s.io"
  resources: *
  verbs: *
- apiGroups:
  - "admissionregistration.k8s.io"
  resources:
  - mutatingwebhookconfigurations
  - validatingwebhookconfigurations
  verbs:
  - get
  - list
  - create
  - update
  - patch
  - delete
  - watch
- apiGroups:
  - "extensions"
  resources:
  - ingresses
  - ingresses/status
  verbs: *
- apiGroups:
  - ""
  resources: 
  - endpoints/restricted
  verbs:
  - create
- apiGroups:
  - "certificates.k8s.io"
  resources: 
  - certificatesigningrequests
  - certificatesigningrequests/approval
  - certificatesigningrequests/status
  verbs: 
  - update
  - create
  - get
  - delete
- apiGroups:
  - "apiextensions.k8s.io"
  resources:
  - customresourcedefinitions
  verbs:   
  - get
  - list
  - create
  - update
  - patch
  - delete
  - watch
- apiGroups:
  - "networking.k8s.io"
  resources: 
  - networkpolicies
  verbs: 
  - get
  - list
  - create
  - update
  - patch
  - delete
  - deletecollection
  - watch
- apiGroups:
  - ""
  resources: 
  - nodes
  verbs: 
  - get
  - list
  - watch
  - update
  - patch
- apiGroups:
  - ""
  resources: 
  - nodes/status
  verbs: 
  - patch

The following table lists how the RBAC permissions are used in Kf, where:

view includes the verbs: get, list, watch
modify includes the verbs: create, update, delete, patch

Permissions	Reasons
Can view all `secrets`	Kf reconcilers need to read secrets for functionalities such as space creation and service instance binding.
Can modify `pods`	Kf reconcilers need to modify pods for functionalities such as building/pushing Apps and Tasks.
Can modify `secrets`	Kf reconcilers need to modify secrets for functionalities such as building/pushing Apps and Tasks and service instance binding.
Can modify `configmaps`	Kf reconcilers need to modify configmaps for functionalities such as building/pushing Apps and Tasks.
Can modify `endpoints`	Kf reconcilers need to modify endpoints for functionalities such as building/pushing Apps and route binding.
Can modify `services`	Kf reconcilers need to modify pods for functionalities such as building/pushing Apps and route binding.
Can modify `events`	Kf controller creates and emits events for the resources managed by Kf.
Can modify `serviceaccounts`	Kf needs to modify service accounts for App deployments.
Can modify `endpoints/restricted`	Kf needs to modify endpoints for App deployments.
Can modify `deployments`	Kf needs to modify deployments for functionalities such as pushing Apps.
Can modify `mutatingwebhookconfiguration`	Mutatingwebhookconfiguration is needed by {{mesh_name}}, a Kf dependency, for admission webhooks.
Can modify `customresourcedefinitions customresourcedefinitions/status`	Kf manages resources through Custom Resources such as Apps, Spaces and Builds.
Can modify `horizontalpodautoscalers`	Kf supports autoscaling based on Horizontal Pod Autoscalers.
Can modify `namespace/finalizer`	Kf needs to set owner reference of webhooks.

Third-party libraries

Third-party library source code and licenses can be found in the /third_party directory of any Kf container image.

You can also run kf third-party-licenses to view the third-party licenses for the version of the Kf CLI that you downloaded.

3.5 - Kf reference architecture diagrams

Learn about common ways of deploying and managing Kf clusters.

Kf can benefit from the rich ecosystem of Anthos and GKE, including automation, managed backing services, and development tools.

GKE reference architecture

Kf clusters are managed just like any other GKE cluster, and access resources in the same way.

ConfigSync

Kf can work with ConfigSync to automate Space creation across any number of clusters. Create new namespaces on non-Kf clusters, and Spaces on Kf clusters. You can also manage Spaces by removing the Kf CLI from Space management, if desired. Use kubectl explain to get Kf CRD specs to fully manage your product configuration via GitOps.

3.6 - Logging and Monitoring

3.6.1 - Create and user monitoring dashboards.

You can use Google Cloud Monitoring dashboards to create custom dashboards and charts. Kf comes with a default template which can be used to create dashboards to monitor the performance of your applications.

Application performance dashboard

Run the following commands to deploy a dashboard in your monitoring workspace in Cloud monitoring dashboards to monitor performance of your apps. This dashboard has application performance metrics like requests/sec, round trip latency, HTTP error codes, and more.

git clone https://github.com/google/kf
cd ./kf/dashboards
./create-dashboard.py my-dashboard my-cluster my-space

System resources and performance dashboard

You can view all the system resources and performance metrics such as list of nodes, pods, containers and much more using a built-in dashboard. Click the link below to access the system dashboard.

System dashboard

More details about this dashboard can be found here.

Create SLO and alerts

You can create SLOs and Alerts on available metrics to monitor performance and availability of both system and applications. For example, you can use the metrics istio.io/service/server/response_latencies to setup an alert on the application roundtrip latency.

Configure dashboard access control

Follow these instructions to provide dashboard access to developers and other members on the team. The role roles/monitoring.dashboardViewer provides read-only access to dashboards.

3.6.2 - Logging and monitoring

Kf can use GKE’s Google Cloud integrations to send a log of events to your Cloud Monitoring and Cloud Logging project for observability. For more information, see Overview of GKE operations.

Kf deploys two server side components:

Controller
Webhook

To view the logs for these components, use the following Cloud Logging query:

resource.type="k8s_container"
resource.labels.project_id=<PROJECT ID>
resource.labels.location=<GCP ZONE>
resource.labels.cluster_name=<CLUSTER NAME>
resource.labels.namespace_name="kf"
labels.k8s-pod/app=<controller OR webhook>

3.6.3 - Logging and monitoring overview

By default, Kf includes native integration with Cloud Monitoring and Cloud Logging. When you create a cluster, both Monitoring and Cloud Logging are enabled by default. This integration lets you monitor your running clusters and help analyze your system and application performance using advanced profiling and tracing capabilities.

Application level performance metrics is provided by Istio sidecar injection which is injected alongside applications deployed via Kf. You can also create SLO and Alerts using this default integration to monitor performance and availability of both system and applications.

Ensure the following are setup on your cluster:

Cloud Monitoring and Cloud Logging are enabled on the Kf cluster by default unless you disabled them explicitly, so no extra step is required.
Istio sidecar injection is enabled. Sidecar proxy will inject application level performance metrics.

3.6.4 - View logs

Kf provides you with several types of logs. This document describes these logs and how to access them.

Application logs

All logs written to standard output stdout and standard error stderr, are uploaded to Cloud Logging and stored under the log name user-container.

Open Cloud Logging and run the following query:

resource.type="k8s_container"
log_name="projects/YOUR_PROJECT_ID/logs/user-container"
resource.labels.project_id=YOUR_PROJECT_ID
resource.labels.location=GCP_COMPUTE_ZONE (e.g. us-central1-a)
resource.labels.cluster_name=YOUR_CLUSTER_NAME
resource.labels.namespace_name=YOUR_KF_SPACE_NAME
resource.labels.pod_name:YOUR_KF_APP_NAME

You should see all your application logs written on standard stdout and standard error stderr.

Access logs for your applications

Kf provides access logs using Istio sidecar injection. Access logs are stored under the log name server-accesslog-stackdriver.

Open Cloud Logging and run the following query:

resource.type="k8s_container"
log_name="projects/YOUR_PROJECT_ID/logs/server-accesslog-stackdriver"
resource.labels.project_id=YOUR_PROJECT_ID
resource.labels.location=GCP_COMPUTE_ZONE (e.g. us-central1-a)
resource.labels.cluster_name=YOUR_CLUSTER_NAME
resource.labels.namespace_name=YOUR_KF_SPACE_NAME
resource.labels.pod_name:YOUR_KF_APP_NAME

You should see access logs for your application. Sample access log:

{
  "insertId": "166tsrsg273q5mf",
  "httpRequest": {
    "requestMethod": "GET",
    "requestUrl": "http://test-app-38n6dgwh9kx7h-c72edc13nkcm.***. ***.nip.io/",
    "requestSize": "738",
    "status": 200,
    "responseSize": "3353",
    "remoteIp": "10.128.0.54:0",
    "serverIp": "10.48.0.18:8080",
    "latency": "0.000723777s",
    "protocol": "http"
  },
  "resource": {
    "type": "k8s_container",
    "labels": {
      "container_name": "user-container",
      "project_id": ***,
      "namespace_name": ***,
      "pod_name": "test-app-85888b9796-bqg7b",
      "location": "us-central1-a",
      "cluster_name": ***
    }
  },
  "timestamp": "2020-11-19T20:09:21.721815Z",
  "severity": "INFO",
  "labels": {
    "source_canonical_service": "istio-ingressgateway",
    "source_principal": "spiffe://***.svc.id.goog/ns/istio-system/sa/istio-ingressgateway-service-account",
    "request_id": "0e3bac08-ab68-408f-9b14-0aec671845bf",
    "source_app": "istio-ingressgateway",
    "response_flag": "-",
    "route_name": "default",
    "upstream_cluster": "inbound|80|http-user-port|test-app.***.svc.cluster.local",
    "destination_name": "test-app-85888b9796-bqg7b",
    "destination_canonical_revision": "latest",
    "destination_principal": "spiffe://***.svc.id.goog/ns/***/sa/sa-test-app",
    "connection_id": "82261",
    "destination_workload": "test-app",
    "destination_namespace": ***,
    "destination_canonical_service": "test-app",
    "upstream_host": "127.0.0.1:8080",
    "log_sampled": "false",
    "mesh_uid": "proj-228179605852",
    "source_namespace": "istio-system",
    "requested_server_name": "outbound_.80_._.test-app.***.svc.cluster.local",
    "source_canonical_revision": "asm-173-6",
    "x-envoy-original-dst-host": "",
    "destination_service_host": "test-app.***.svc.cluster.local",
    "source_name": "istio-ingressgateway-5469f77856-4n2pw",
    "source_workload": "istio-ingressgateway",
    "x-envoy-original-path": "",
    "service_authentication_policy": "MUTUAL_TLS",
    "protocol": "http"
  },
  "logName": "projects/*/logs/server-accesslog-stackdriver",
  "receiveTimestamp": "2020-11-19T20:09:24.627065813Z"
}

Audit logs

Audit Logs provides a chronological record of calls that have been made to the Kubernetes API Server. Kubernetes audit log entries are useful for investigating suspicious API requests, for collecting statistics, or for creating monitoring alerts for unwanted API calls.

Open Cloud Logging and run the following query:

resource.type="k8s_container"
log_name="projects/YOUR_PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"
resource.labels.project_id=YOUR_PROJECT_ID
resource.labels.location=GCP_COMPUTE_ZONE (e.g. us-central1-a)
resource.labels.cluster_name=YOUR_CLUSTER_NAME
protoPayload.request.metadata.name=YOUR_APP_NAME
protoPayload.methodName:"deployments."

You should see a trace of calls being made to the Kubernetes API server.

Configure logging access control

Follow these instructions to provide logs access to developers and other members on the team. The role roles/logging.viewer provides read-only access to logs.

Use Logs Router

You can also use Logs Router to route the logs to supported destinations.

3.7 - Networking

Learn about common cluster networking setups.

3.7.1 - Set up a custom domain

Learn to set up a DNS domain apps can use on your cluster.

All Kf Apps that serve HTTP traffic to users or applications outside of the cluster must be associated with a domain name.

Kf has three locations where domains can be configured. Ordered by precedence, they are:

Apps
Spaces
The config-defaults ConfigMap in the kf Namespace

Edit the `config-defaults` ConfigMap

The config-defaults ConfigMap holds cluster-wide settings for Kf and can be edited by cluster administrators. The values in the ConfigMap are read by the Spaces controller and modify their configuration. Domain values are reflected in the Space’s status.networkConfig.domains field.

To modify Kf cluster’s domain, edit the kfsystem, the operator will then popluate the change to config-defaults configmap under kf namespace:

kubectl edit kfsystem

Add or update the entry for the spaceClusterDomains key under spec.kf.config like the following:

spaceClusterDomains: my-domain.com

To validate the configuration was updated correctly, check the domain value in a Space:

kf space SPACE_NAME -o "jsonpath={.status.networkConfig.domains[]['domain']}"

The output will look similar to:

Getting Space some-space
some-space.my-domain.com

Each Space prefixes the cluster domains with its own name. This prevents conflicts between Apps.

Warning: Updating the spaceClusterDomains in config-defaults will immediately be reflected on all Spaces and Apps that haven’t overridden the domain.

Assign Space domains

Spaces are the authoritative location for domain configuration. You can assign domains and sub-domains to each Space for developers to use. The field for configuring domains is spec.networkConfig.domains.

Use kf space to view the domains assigned to a Space:

kf space SPACE_NAME

In the output, the Spec field contains specific configuration for the Space and the Status field reflects configuration for the Space with cluster-wide defaults appended to the end:

...
Spec:
  Network Config:
    Domains:
      Domain: my-space.mycompany.com
...
Status:
  Network Config:
    Domains:
      Domain: my-space.mycompany.com
      Domain: my-space.prod.us-east1.kf.mycompany.com

Add or remove domains using the CLI

The kf CLI supports mutations on Space domains. Each command outputs a diff between the old and new configurations.

Add a new domain with kf configure-space append-domain:

kf configure-space append-domain SPACE_NAME myspace.mycompany.com

Add or make an existing domain the default with kf configure-space set-default-domain:

kf configure-space set-default-domain SPACE_NAME myspace.mycompany.com

And finally, remove a domain:

kf configure-space remove-domain SPACE_NAME myspace.mycompany.com

Note: You can override a cluster domains by inserting a new domain on the spec with the same domain. This ensures the space won’t be updated if the cluster domain changes.

Use Apps to specify domains

Apps can specify domains as part of their configuration. Routes are mapped to Apps during kf push using the following logic:

let current_routes  = The set of routes already on the app
let manifest_routes = The set of routes defined by the manifest
let flag_routes     = The set of routes supplied by the --route flag(s)
let no_route        = Whether the manifest has no-route:true or --no-route is set
let random_route    = Whether the manifest has random-route:true or --random-route is set

let new_routes = Union(current_routes, manifest_routes, flag_routes)

if new_routes.IsEmpty() then
  if random_route then
    new_routes.Add(CreateRandomRoute())
  else
    new_routes.Add(CreateDefaultRoute())
  end
end

if no_route then
  new_routes.RemoveAll()
end

return new_routes

If an App doesn’t specify a Route, or requests a random Route, the first domain on the Space is used. If the first domain on a Space changes, all Apps in the Space using the default domain are updated to reflect it.

Customize domain templates

Kf supports variable substitution in domains. Substitution allows a single cluster-wide domain to be customized per-Space and to react to changes to the ingress IP. Substitution is performed on variables with the syntax $(VARIABLE_NAME) that occur in a domain.

Variable	Description
`CLUSTER_INGRESS_IP`	The IPV4 address of the cluster ingress.
`SPACE_NAME`	The name of the Space.

Note: Use the SPACE_NAME variable in domains to allow developers in different Spaces to avoid DNS conflicts if they push Apps with the same name.

Examples

The following examples demonstrate how domain variables can be used to support a variety of different organizational structures and cluster patterns.

Using a wildcard DNS service like nip.io:

$(SPACE_NAME).$(CLUSTER_INGRESS_IP).nip.io

Domain for an organization with centrally managed DNS:
```
$(SPACE_NAME).cluster-name.example.com
```
Domain for teams who manage their own DNS:
```
cluster-name.$(SPACE_NAME).example.com
```
Domain for a cluster with warm failover and external circuit breaker:
```
$(SPACE_NAME)-failover.cluster-name.example.com
```

Differences between Kf and CF

Kf Spaces prefix the cluster-wide domain with the Space name.
Kf does not check for domain conflicts on user-specified routes.

3.7.2 - Set up HTTPS ingress

Learn to set up a TLS certificate to enable HTTPS.

You can secure the ingress gateway with HTTPS by using simple TLS, and enable HTTPS connections to specific webpages. In addition, you can redirect HTTP connections to HTTPS.

HTTPS creates a secure channel over an insecure network, protecting against man-in-the-middle attacks and encrypting traffic between the client and server. To prepare a web server to accept HTTPS connections, an administrator must create a public key certificate for the server. This certificate must be signed by a trusted certificate authority for a web browser to accept it without warning.

Edit the gateway named external-gateway in the kf namespace using the built-in Kubernetes editor:

kubectl edit gateway -n kf external-gateway

Assuming you have a certificate and key for your service, create a Kubernetes secret for the ingress gateway. Make sure the secret name does not begin with istio or prometheus. For this example, the secret is named myapp-https-credential.
Under servers:
Add a section for port 443.
Under tls:, set the credentialName to the name of the secret you just created.
Under hosts:, add the host name of the service you want to secure with HTTPS. This can be set to an entire domain using a wildcard (e.g. *.example.com) or scoped to just one hostname (e.g. myapp.example.com).
There should already be a section under servers: for port 80 HTTP. Keep this section in the Gateway definition if you would like all traffic to come in as HTTP.
To redirect HTTP to HTTPS, add the value httpsRedirect: true under tls in the HTTP server section. See the Istio Gateway documentation for reference. Note that adding this in the section where hosts is set to * means that all traffic is redirected to HTTPS. If you only want to redirect HTTP to HTTPS for a single app/domain, add a separate HTTP section specifying the redirect.

Shown below is an example of a Gateway spec that sets up HTTPS for myapp.example.com and redirects HTTP to HTTPS for that host:

spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - myapp.example.com
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      credentialName: myapp-https-credential
      mode: SIMPLE
  - hosts:
    - myapp.example.com
    port:
      name: http-my-app
      number: 80
      protocol: HTTP
    tls:
      httpsRedirect: true
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP

3.7.3 - Set up network policies

Learn to set up a Kubernetes NetworkPolicies to configure traffic.

Kf integrates tightly with Kubernetes and Istio to provide robust network policy enforcement.

By default, Kf workloads are run in the Kubernetes cluster and resolve addresses using Kubernetes DNS. This DNS resolver will first attempt to resolve addresses within the cluster, and only if none are found will attempt external resolution.

Each Kf App gets run with an Envoy sidecar injected by Istio or the Anthos Service Mesh (ASM). This sidecar proxies all network traffic in and out of the Kubernetes Pod.

Each Kubernetes Pod is executed on a Node, a physical or virtual machine responsible for managing the container images that make up a Pod. Nodes exist on a physical or virtual network.

Together, these form a hierarchy of systems you can apply network policies. These are listed below from least to most granular.

Network level policies

Workload protection starts with the network your GKE cluster is installed on.

If you’re running Kf on a GKE cluster on GCP, Kf recommends:

Placing your GKE cluster on a Virtual Private Cloud (VPC) network.
- With Private Google Access enabled.
Using Cloud NAT to control egress.

Node level policies

You can set up policies for containers running on the Node using Kubernetes NetworkPolicies. These are the closest mapping to Cloud Foundry network policies that exist in Kubernetes.

NetworkPolicies are backed by a Kubernetes add-on. If you set up your own GKE cluster, you will need to enable NetworkPolicy enforcement.

Kf labels Apps with kf.dev/networkpolicy=app and builds with kf.dev/networkpolicy=build. This allows you to target NetworkPolicies directly at Pods running Apps or Builds.

Each Kf Space creates two NetworkPolicies to start with, one targeting Apps and one targeting Builds. You can change the configuration on the Space’s spec.networkConfig.(app|build)NetworkPolicy.(in|e)gress fields. These fields can be set to one of the following values:

Enum Value	Description
`PermitAll`	Allows all traffic.
`DenyAll`	Denies all traffic.

By default Kf uses a permissive network policy. This allows the following functionality that Kf uses:

North/South routing to the cluster ingress gateway
Egress to the Internet e.g. to fetch Buildpacks
East/West routing between Apps
Access to the Kubernetes DNS server
Access to container registries
Direct access to the VPC network
Access to Google services like Cloud Logging
Access to the Workload Identity server for automatic rotating credentials

Note: Setting a default DenyAll policy will break existing and new Apps and Builds unless you create additional NetworkPolicies to add back the connections listed above.

Service mesh policies

If you need fine-grained networking control, authentication, authorization, and observability you can apply policies using Anthos Service Mesh.

A service mesh is an infrastructure layer that enables managed, observable and secure communication across your services, letting you create robust enterprise applications made up of many microservices on your chosen infrastructure.

You can see the list of supported features here.

3.8 - Security

Learn about security considerations for your Kf cluster.

3.8.1 - Security Overview

Unerstand Kf’s security posture.

Kf aims to provide a similar developer experience to Cloud Foundry, replicating the build, push, and deploy lifecycle. It does this by building a developer UX layer on top of widely-known, broadly used and adopted technologies like Kubernetes, Istio, and container registries rather than by implementing all the pieces from the ground up.

When making security decisions, Kf aims to provide complete solutions that are native to their respective components and can be augmented with other mechanisms. Breaking that down:

Complete solutions means that Kf tries not to provide partial solutions that can lead to a false sense of security.
Native means that the solutions should be a part of the component rather than a Kf construct to prevent breaking changes.
Can be augmented means the approach Kf takes should work seamlessly with other Kubernetes and Google Cloud tooling for defense in depth.

Important considerations

In addition to the Current limitations described below, it is important that you read through and understand the items outlined in this section.

Workload Identity

By default, Kf uses Workload Identity to provide secure delivery and rotation of the Service Account credentials used by Kf to interact with your Google Cloud Project. Workload Identity achieves this by mapping a Kubernetes Service Account (KSA) to a Google Service Account (GSA). The Kf controller runs in the kf namespace and uses a KSA named controller mapped to your GSA to do the following things:

Write metrics to Stackdriver
When a new Kf space is created (kf create-space), the Kf controller creates a new KSA named kf-builder in the new space and maps it to the same GSA.
The kf-builder KSA is used by Tekton to push and pull container images to Google Container Registry (gcr.io)

This diagram illustrates those interactions:

NFS

In order to mimic Cloud Foundry’s UID/GID mapping, containers in Kf that mount NFS volumes need the ability to run as root and the ability to access the FUSE device of the kernel running the Node.

Current limitations

A developer pushing an app with Kf can also create Pods (with kubectl) that can use the kf-builder KSA with the permissions of its associated GSA.
Kf uses the same Pod to fetch, build, and store images. Assume that any credentials that you provide can be known by the authors and publishers of the buildpacks you use.
Kf doesn’t support quotas to protect against noisy neighbors. Use Kubernetes resource quotas.

Other resources

Google Cloud

General

Recommended protections

Advanced protections

3.8.2 - Role-based access control

Learn about how to share a Kf cluster using roles.

Kf provides a set of Kubernetes roles that allow multiple teams to share a Kf cluster. This page describes the roles and best practices to follow when using them.

When to use Kf roles

Kf roles allow multiple teams to share a Kubernetes cluster with Kf installed. The roles provide access to individual Kf Spaces.

Use Kf roles to share access to a cluster if the following are true:

The cluster is used by trusted teams.
Workloads on the cluster share the same assumptions about the level of security provided by the environment.
The cluster exists in a Google Cloud project that is tightly controlled.

Kf roles will not:

Protect your cluster from untrusted developers or workloads. See the GKE shared responsibility model for more information.
Provide isolation for your workloads. See the guide to harden your cluster for more information.
Prevent additional Kubernetes roles from being defined that interact with Kf.
Prevent access from administrators who have access to the Google Cloud project or cluster.

Kf roles

The following sections describe the Kubernetes RBAC Roles provided by Kf and how they interact with GKE IAM.

Predefined roles

Kf provides several predefined Kubernetes roles to help you provide access to different subjects that perform different roles. Each predefined role can be bound to a subject within a Kubernetes Namespace managed by a Kf Space.

When a subject is bound to a role within a Kubernetes Namespace, their access is limited to objects that exist in the Namespace that match the grants listed in the role. In Kf, some resources are defined at the cluster scope. Kf watches for changes to subjects in the Namespace and grants additional, limited, roles at the cluster scope.

Role	Title	Description	Scope
`space-auditor`	Space Auditor	Allows read-only access to a Space.	Space
`space-developer`	Space Developer	Allows application developers to deploy and manage applications within a Space.	Space
`space-manager`	Space Manager	Allows administration and the ability to manage auditors, developers, and managers in a Space.	Space
`SPACE_NAME-manager`	Dynamic Space Manager	Provides write access to a single Space object, automatically granted to all subjects with the `space-manager` role within the named Space.	Cluster
`kf-cluster-reader`	Cluster Reader	Allows read-only access to cluster-scoped Kf objects, automatically granted to all `space-auditor`, `space-developer`, and `space-manager`.	Cluster

Information about the policy rules that make up each predefined role can be found in the Kf roles reference documentation.

Google Cloud IAM roles

Kf roles provide access control for objects within a Kubernetes cluster. Subjects must also be granted an Cloud IAM role to authenticate to the cluster:

Platform administrators should be granted the roles/container.admin Cloud IAM role. This will allow them to install, upgrade, and delete Kf as well as create, and delete cluster scoped Kf objects like Spaces or ClusterServiceBrokers.
Kf end-users should be granted the roles/container.viewer Cloud IAM role. This role will allow them to authenticate to a cluster with limited permissions that can be expanded using Kf roles.

Google Cloud IAM offers additional predefined Roles for GKE to solve more advanced use cases.

Map Cloud Foundry roles to Kf

Cloud Foundry provides roles are similar to Kf’s predefined roles. Cloud Foundry has two major types of roles:

Roles assigned by the User Account and Authentication (UAA) subsystem that provide coarse-grained OAuth scopes applicable to all Cloud Foundry API endpoints.
Roles granted within the Cloud Controller API (CAPI) that provide fine-grained access to API resources.

UAA roles

Roles provided by UAA are most similar to project scoped Google Cloud IAM roles:

Admin users in Cloud Foundry can perform administrative activities for all Cloud Foundry organizations and spaces. The role is most similar to the roles/container.admin Google Cloud IAM role.
Admin read-only users in Cloud Foundry can access all Cloud Foundry API endpoints. The role is most similar to the roles/container.admin Google Cloud IAM role.
Global auditor users in Cloud Foundry have read access to all Cloud Foundry API endpoints except for secrets. There is no equivalent Google Cloud IAM role, but you can create a custom role with similar permissions.

Cloud Controller API roles

Roles provided by CAPI are most similar to Kf roles granted within a cluster to subjects that have the roles/container.viewer Google Cloud IAM role on the owning project:

Space auditors in Cloud Foundry have read-access to resources in a CF space. The role is most similar to the space-auditor Kf role.
Space developers in Cloud Foundry have the ability to deploy and manage applications in a CF space. The role is most similar to the space-developer Kf role.
Space managers in Cloud Foundry can modify settings for the CF space and assign users to roles. The role is most similar to the space-manager Kf role.

What’s next

Learn more about GKE security in the Security Overview.
Make sure you understand the GKE shared responsibility model.
Learn more about access control in GKE.
Read the GKE multi-tenancy overview.
Learn about hardening your GKE cluster.
Understand the Kubernetes permissions that make up each Kf predefined role.

3.8.3 - Configure role-based access control

Learn to grant users different roles on a cluster.

The following steps guide you through configuring role-based access control (RBAC) in a Kf Space.

Before you begin

Please follow the GKE RBAC guide before continuing with the following steps.

Configure Identity and Access Management (IAM)

In addition to permissions granted through Kf RBAC, users, groups, or service accounts must also be authenticated to view GKE clusers at the project level. This requirement is the same as for configuring GKE RBAC, meaning users/groups must have at least the container.clusters.get IAM permission in the project containing the cluster. This permission is included by the container.clusterViewer role, and other more privilleged roles. For more information, review Interaction with Identity and Access Management.

Assign container.clusterViewer to a user or group.

gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
  --role="container.clusterViewer" \
  --member="${MEMBER}"

Example member values are:

user:test-user@gmail.com
group:admins@example.com
serviceAccount:test123@example.domain.com

Manage Space membership as SpaceManager

The cluster admin role, or members with SpaceManager role, can assign role to a user, group or service account.

kf set-space-role MEMBER -t [Group|ServiceAccount|User]

The cluster admin role, or members with SpaceManager role, can remove a member from a role.

kf unset-space-role MEMBER -t [Group|ServiceAccount|User]

You can view members and their roles within a Space.

kf space-users

Examples

Assign SpaceDeveloper role to a user.

kf set-space-role alice@example.com SpaceDeveloper

Assign SpaceDeveloper role to a group.

kf set-space-role devs@example.com SpaceDeveloper -t Group

Assign SpaceDeveloper role to a Service Account.

kf set-space-role sa-dev@example.domain.com SpaceDeveloper -t ServiceAccount

App development as SpaceDeveloper

Members with SpaceDeveloper role can perform Kf App development operations within the Space.

To push an App:

kf push app_name -p [PATH_TO_APP_ROOT_DIRECTORY]

To view logs of an App:

kf logs app_name

SSH into a Kubernetes Pod running the App:

kf ssh app_name

View available service brokers:

kf marketplace

View Apps as SpaceManager or SpaceAuditor

Members with SpaceManager or SpaceAuditor role could view available Apps within the Space:

kf apps

View Kf Spaces within a cluster

All roles (SpaceManager, SpaceDeveloper, and SpaceAuditor) can view available Kf Spaces within a cluster:

kf spaces

View Space members and their roles within a Space.

kf space-users

Impersonation flags

To verify a member’s permission, a member with more priviliaged permission can test another member’s permissions using the impersonation flags: --as and --as-group.

For example, as a cluster admin, you can verify if a user (username: bob) has permission to push an App.

kf push APP_NAME --as bob

Verify a group (manager-group@example.com) has permission to assign permission to other members.

kf set-space-role bob SpaceDeveloper --as-group manager-group@example.com

3.8.4 - Enable compute isolation

Isolate the underlying nodes certain apps or builds are scheuled onto.

Kf Apps can be deployed on dedicated nodes in the cluster. This feature is required if you have the circumstances where you might want more control on a node where an App Pod lands. For example:

If you are sharing the same cluster for different Apps but want dedicated nodes for a particular App.
If you want dedicated nodes for a given organization (Kf Space).
If you want to target a specific operating system like Windows.
If you want to co-locate Pods from two different services that frequently communicate.

To enable compute isolation, Kf uses the Kubernetes nodeSelector. To use this feature, first add labels on the nodes or node pools where you want your App Pods to land and then add the same qualifying labels on the Kf Space. All the Apps installed in this Space then land on the nodes with matching labels.

Kf creates a Kubernetes pod to execute each Kf Build, the buildNodeSelector feature can be used to isolate compute resources to execute only the Build pods. One use case is to isolate Build pods to run on nodes with SSD, while running the App pods on other nodes. The BuildNodeSelectors feature provides compute resource optimization and flexibility in the cluster. Please refer to chapter ‘Configure BuildNodeSelectors and a build node pool’ on this page.

Configure nodeSelector in a Kf cluster

By default, compute isolation is disabled. Use the following procedure to configure labels and nodeSelector.

Add a label (distype=ssd) on the node where you want your application pods to land.
```
kubectl label nodes nodeid disktype=ssd
```
Add the same label on the Kf Space. All Apps deployed in this Space will then land on the qualifying nodes.
```
kf configure-space set-nodeselector space-name disktype ssd
```
You can add multiple labels by running the same command again.

Check the label is configured.

kf configure-space get-nodeselector space-name

Delete the label from the space.

kf configure-space unset-nodeselector space-name disktype

Override nodeSelector for Kf stacks

Deployment of Kf Apps can be further targeted based on what stack is being used to build and package the App. For example, if you want your applications built with spaceStacksV2 to land on nodes with Linux kernel 4.4.1., nodeSelector values on a stack override the values configured on the Space.

To configure the nodeSelector on a stack:

Edit the config-defaults of your Kf cluster and add the labels.
```
$ kubectl -n kf edit configmaps config-defaults
```

Add nodeSelector to the stacks definition.

.....
.....
spaceStacksV2: |
- name:  cflinuxfs3
        image: cloudfoundry/cflinuxfs3
        nodeSelector:
              OS_KERNEL: LINUX_4.4.1 
.....
.....

Configure BuildNodeSelectors and a Build node pool

Build node selectors are only effective at overriding the node selectors for the Build pods, they do not affect App pods. For example, if you specify both the node selectors on the Space and the Build node selectors in Kfsystem, App pods will have the Space node selectors while the Build pods will have the Build node selectors from Kfsystem; if node selectors are only specified in the Space, both the App and Build pods will have the node selector from the Space.

Add labels (disktype:ssd for example) on the nodes that you want your Build pods to be assigned to.
```
kubectl label nodes nodeid disktype=ssd
```

Add/update Build node selectors (in the format of key:value pairs) by patching KfSystem CR.

kubectl patch kfsystem kfsystem --type='json' -p='[{'op': 'replace', 'path': '/spec/kf/config/buildNodeSelectors', 'value': {<key>:<value>}}]'

For example, to add disktype=ssd as the Build node selector:

kubectl patch kfsystem kfsystem --type='json' -p='[{'op': 'replace', 'path': '/spec/kf/config/buildNodeSelectors', 'value': {"disktype":"ssd"}}]'

3.8.5 - Kubernetes Roles

Understand how Kf uses Kubernetes’ RBAC to assign roles.

The following sections describe the Kubernetes ClusterRoles that are created by Kf and lists the permissions that are contained in each ClusterRole.

Space developer role

The Space developer role aggregates permissions application developers use to deploy and manage applications within a Kf Space.

You can retrieve the permissions granted to Space developers on your cluster using the following command.

kubectl describe clusterrole space-developer

Space auditor role

The Space auditor role aggregates read-only permissions that auditors and automated tools use to validate applications within a Kf Space.

You can retrieve the permissions granted to Space auditors on your cluster using the following command.

kubectl describe clusterrole space-auditor

Space manager role

The Space manager role aggregates permissions that allow delegation of duties to others within a Kf Space.

You can retrieve the permissions granted to Space managers on your cluster using the following command.

kubectl describe clusterrole space-manager

Note: Subjects bound to the space-manager ClusterRole within a Kf Space are also granted write access to that Space.

Dynamic Space manager role

Each Kf Space creates a ClusterRole with the name SPACE_NAME-manager, where SPACE_NAME-manager is called the dynamic manager role.

Kf automatically grants all subjects with the space-manager role within the Space the dynamic manager role at the cluster scope. The permissions for the dynamic manager role allow Space managers to update settings on the Space with the given name.

You can retrieve the permissions granted to the dynamic manager role for any Space on your cluster using the following command.

kubectl describe clusterrole SPACE_NAME-manager

Kf cluster reader role

Kf automatically grants the kf-cluster-reader role to all users on a cluster that already have the space-developer, space-auditor, or space-manager role within a Space.

You can retrieve the permissions granted to Space Kf cluster readers on your cluster using the following command.

kubectl describe clusterrole kf-cluster-reader

3.9 - Stacks and Buildpacks

Learn about configuring stacks and buildpacks for your platform.

3.9.1 - Customize stacks

The Stack configuration can be updated by editing the kfsystem Custom Resource:

kubectl edit kfsystem kfsystem

This example sets the Google Cloud buildpacks a V3 Stack:

spec:
  kf:
    config:
      spaceStacksV3:
      - name: google
        description: Google buildpacks (https://github.com/GoogleCloudPlatform/buildpacks)
        buildImage: gcr.io/buildpacks/builder:v1
        runImage: gcr.io/buildpacks/gcp/run:v1

This new Stack can now be pushed:

kf push myapp --stack google

This example configures the Ruby V2 buildpack and sets the build pipeline defaults to use V2 Stacks:

spec:
  kf:
    config:
      spaceDefaultToV3Stack: false
      spaceBuildpacksV2:
      - name: ruby_buildpack
        url: https://github.com/cloudfoundry/ruby-buildpack
      spaceStacksV2:
      - name: cflinuxfs3
        image: cloudfoundry/cflinuxfs3@sha256:5219e9e30000e43e5da17906581127b38fa6417f297f522e332a801e737928f5

3.9.2 - Customize stacks and buildpacks

Buildpacks are used by Kf to turn an application’s source code into an executable image. Cloud Native buildpacks use the latest Buildpack API v3. Companies are actively adding v3 support to existing buildpacks.

Kf supports buildpacks that conform to both V2 and V3 of the Buildpack API specification.

Compare V2 and V3 buildpacks

	V2 buildpacks	V3 buildpacks
Alternate names	Cloud Foundry buildpacks	Cloud Native buildpacks (CNB), Builder Images
Status	Being replaced	Current
Ownership	Cloud Foundry	Buildpacks.io
Stack	Shared by builder and runtime	Optionally different for builder and runtime
Local development	Not possible	Yes, with the `pack` CLI
Custom buildpacks	Available at runtime	Must be built into the builder

Buildpack lifecycle

Step	Cloud Foundry	Kf with buildpacks V2	Kf with buildpacks V3
Source location	BITS service	Container registry	Container registry
Buildpack location	BOSH/HTTP	HTTP	Container registry
Stack location	BOSH	Container registry	Container registry
Result	Droplet (App binary without stack)	Image (Droplet on a stack)	Image
Runtime	Droplet glued on top of stack and run	Run produced image	Run produced image

Kf always produces a full, executable image as a result of its build process. Cloud Foundry, on the other hand, produces parts of an executable image at build time and the rest is added at runtime.

Kf chose to follow the model of always producing a full image for the following reasons:

Images can be exported, run locally, and inspected statically
Better security and auditing with tools like binary authorization
App deployments are reproducible

Kf and buildpacks

Kf stores its global list of buildpacks and stacks in the config-defaults ConfigMap in the kf Namespace. Modification of the buildpacks and stacks properties should be done at the kfsystem Custom Resource, the Kf operator automatically updates the config-defaults ConfigMap based on the values set at kfsystem.

Each Space reflects these buildpacks in its status field. For a Space named buildpack-docs you could run the following to see the full Space configuration:

kf space buildpack-docs

Getting Space buildpack-docs
API Version:  kf.dev/v1alpha1
Kind:         Space
Metadata:
  Creation Timestamp:  2020-02-14T15:09:52Z
  Name:                buildpack-docs
  Self Link:           /apis/kf.dev/v1alpha1/spaces/buildpack-docs
  UID:                 0cf1e196-4f3c-11ea-91a4-42010a80008d
Status:
  Build Config:
    Buildpacks V2:
    - Name:      staticfile_buildpack
      URL:       https://github.com/cloudfoundry/staticfile-buildpack
      Disabled:  false
    - Name:      java_buildpack
      URL:       https://github.com/cloudfoundry/java-buildpack
      Disabled:  false
    Stacks V2:
    - Image:  cloudfoundry/cflinuxfs3
      Name:   cflinuxfs3
    Stacks V3:
    - Build Image:  cloudfoundry/cnb:cflinuxfs3
      Description:  A large Cloud Foundry stack based on Ubuntu 18.04
      Name:         org.cloudfoundry.stacks.cflinuxfs3
      Run Image:    cloudfoundry/run:full-cnb

Under the Build Config section there are three fields to look at:

Buildpacks V2 contains a list of V2 compatible buildpacks in the order they’ll be run
Stacks V2 indicates the stacks that can be chosen to trigger a V2 buildpack build
Stacks V3 indicates the stacks that can be chosen to trigger a V3 buildpack build

You can also list the stacks with kf stacks:

kf stacks

Getting stacks in Space: buildpack-docs
Version  Name                                Build Image                  Run Image                  Description
V2       cflinuxfs3                          cloudfoundry/cflinuxfs3      cloudfoundry/cflinuxfs3
V3       org.cloudfoundry.stacks.cflinuxfs3  cloudfoundry/cnb:cflinuxfs3  cloudfoundry/run:full-cnb  A large Cloud Foundry stack based on Ubuntu 18.04

Because V3 build images already have their buildpacks built-in, you must use kf buildpacks to get the list:

kf buildpacks

Getting buildpacks in Space: buildpack-docs
Buildpacks for V2 stacks:
  Name                   Position  URL
  staticfile_buildpack   0         https://github.com/cloudfoundry/staticfile-buildpack
  java_buildpack         1         https://github.com/cloudfoundry/java-buildpack
V3 Stack: org.cloudfoundry.stacks.cflinuxfs3:
  Name                                        Position  Version     Latest
  org.cloudfoundry.jdbc                       0         v1.0.179    true
  org.cloudfoundry.jmx                        1         v1.0.180    true
  org.cloudfoundry.go                         2         v0.0.2      true
  org.cloudfoundry.tomcat                     3         v1.1.102    true
  org.cloudfoundry.distzip                    4         v1.0.171    true
  org.cloudfoundry.springboot                 5         v1.1.2      true
  ...

Customize V3 buildpacks

You can customize the buildpacks that are available to your developers by creating your own builder image with exactly the buildpacks they should have access to. You can also use builder images published by other authors.

Use a third-party builder image

A list of published CNB stacks is available from the Buildpack CLI pack. As of this writing, pack suggest-stacks outputs:

pack suggest-stacks

Stacks maintained by the community:

    Stack ID: heroku-18
    Description: The official Heroku stack based on Ubuntu 18.04
    Maintainer: Heroku
    Build Image: heroku/pack:18-build
    Run Image: heroku/pack:18

    Stack ID: io.buildpacks.stacks.bionic
    Description: A minimal Cloud Foundry stack based on Ubuntu 18.04
    Maintainer: Cloud Foundry
    Build Image: cloudfoundry/build:base-cnb
    Run Image: cloudfoundry/run:base-cnb

    Stack ID: org.cloudfoundry.stacks.cflinuxfs3
    Description: A large Cloud Foundry stack based on Ubuntu 18.04
    Maintainer: Cloud Foundry
    Build Image: cloudfoundry/build:full-cnb
    Run Image: cloudfoundry/run:full-cnb

    Stack ID: org.cloudfoundry.stacks.tiny
    Description: A tiny Cloud Foundry stack based on Ubuntu 18.04, similar to distroless
    Maintainer: Cloud Foundry
    Build Image: cloudfoundry/build:tiny-cnb
    Run Image: cloudfoundry/run:tiny-cnb

To modify Kf to use the stack published by Heroku, edit kfsystem Custom Resource, which automatically updates the config-defaults ConfigMap in the kf Namespace. Add an entry to the spaceStacksV3 key like the following:

kubectl edit kfsystem kfsystem

spaceStacksV3: |
  - name: org.cloudfoundry.stacks.cflinuxfs3
    description: A large Cloud Foundry stack based on Ubuntu 18.04
    buildImage: cloudfoundry/cnb:cflinuxfs3
    runImage: cloudfoundry/run:full-cnb
  - name: heroku-18
    description: The official Heroku stack based on Ubuntu 18.04
    buildImage: heroku/pack:18-build
    runImage: heroku/pack:18

Then, run stacks again:

kf stacks

Getting stacks in Space: buildpack-docs
Version  Name                                Build Image                  Run Image                  Description
V2       cflinuxfs3                          cloudfoundry/cflinuxfs3      cloudfoundry/cflinuxfs3
V3       org.cloudfoundry.stacks.cflinuxfs3  cloudfoundry/cnb:cflinuxfs3  cloudfoundry/run:full-cnb  A large Cloud Foundry stack based on Ubuntu 18.04
V3       heroku-18                           heroku/pack:18-build         heroku/pack:18             The official Heroku stack based on Ubuntu 18.04

Create your own builder image

The Buildpack CLI pack is used to create your own builder image. You can follow pack’s Working with builders using create-builder documentation to create your own builder image. After it’s created, push it to a container registry and add it to the kfsystem Custom Resource.

Set a default stack

Apps will be assigned a default stack if one isn’t supplied in their manifest. The default stack is the first in the V2 or V3 stacks list. Unless overridden, a V2 stack is chosen for compatibility with Cloud Foundry.

You can force Kf to use a V3 stack instead of a V2 by setting the spaceDefaultToV3Stack field in the kfsystem Custom Resource to be "true" (kfsystem automatically updates corresonding spaceDefaultToV3Stack field in the config-defaults ConfigMap):

kubectl edit kfsystem kfsystem

spaceDefaultToV3Stack: "true"

This option can also be modified on a per-Space basis by changing setting the spec.buildConfig.defaultToV3Stack field to be true or false. If unset, the value from the config-defaults ConfigMap is used.

`config-defaults` value for `spaceDefaultToV3Stack`	Space’s `spec.buildConfig.defaultToV3Stack`	Default stack
unset	unset	V2
`"false"`	unset	V2
`"true"`	unset	V3
any	`false`	V2
any	`true`	V3

4 - Kf CLI Reference

CLI Reference

4.1 - kf about

Print information about Kf’s terms of service.

Name

kf about - Print information about Kf’s terms of service.

Synopsis

kf about [flags]

Examples

kf about

Flags

-h, --help: help for about

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.2 - kf app

Print information about the given App.

Name

kf app - Print information about the given App.

Synopsis

kf app NAME [flags]

Examples

kf app my-app

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for app
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.3 - kf apps

List Apps in the targeted Space.

Name

kf apps - List Apps in the targeted Space.

Synopsis

kf apps [flags]

Examples

kf apps

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for apps
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.4 - kf bind-route-service

Bind a route service instance to an HTTP route.

Name

kf bind-route-service - Bind a route service instance to an HTTP route.

Synopsis

kf bind-route-service DOMAIN [--hostname HOSTNAME] [--path PATH] SERVICE_INSTANCE [-c PARAMETERS_AS_JSON] [flags]

Description

PREVIEW: this feature is not ready for production use. Binding a service to an HTTP route causes traffic to be processed by that service before the requests are forwarded to the route.

Examples

  kf bind-route-service company.com --hostname myapp --path mypath myauthservice

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for bind-route-service
--hostname=string: Hostname for the Route.
-c, --parameters=string: JSON object or path to a JSON file containing configuration parameters. (default "{}")
--path=string: URL path for the Route.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.5 - kf bind-service

Grant an App access to a service instance.

Name

kf bind-service - Grant an App access to a service instance.

Synopsis

kf bind-service APP_NAME SERVICE_INSTANCE [-c PARAMETERS_AS_JSON] [--binding-name BINDING_NAME] [flags]

Description

Binding a service injects information about the service into the App via the VCAP_SERVICES environment variable.

Examples

  kf bind-service myapp mydb -c '{"permissions":"read-only"}'

Flags

--async: Do not wait for the action to complete on the server before returning.
-b, --binding-name=string: Name of the binding injected into the app, defaults to the service instance name.
-h, --help: help for bind-service
-c, --parameters=string: JSON object or path to a JSON file containing configuration parameters. (default "{}")
--timeout=duration: Amount of time to wait for the operation to complete. Valid units are "s", "m", "h". (default 30m0s)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.6 - kf bindings

List bindings in the targeted Space.

Name

kf bindings - List bindings in the targeted Space.

Synopsis

kf bindings [flags]

Examples

kf bindings

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for bindings
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.7 - kf build

Print information about the given Build.

Name

kf build - Print information about the given Build.

Synopsis

kf build NAME [flags]

Examples

kf build my-build

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for build
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.8 - kf build-logs

Get the logs of the given Build.

Name

kf build-logs - Get the logs of the given Build.

Synopsis

kf build-logs BUILD_NAME [flags]

Examples

kf build-logs build-12345

Flags

-h, --help: help for build-logs

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.9 - kf buildpacks

List buildpacks in the targeted Space.

Name

kf buildpacks - List buildpacks in the targeted Space.

Synopsis

kf buildpacks [flags]

Description

List the buildpacks available in the Space to Apps being built with buildpacks.

The buildpacks available to an App depend on the Stack it uses. To ensure reproducibility in Builds, Apps should explicitly declare the Stack they use.

Examples

kf buildpacks

Flags

-h, --help: help for buildpacks

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.10 - kf builds

List Builds in the targeted Space.

Name

kf builds - List Builds in the targeted Space.

Synopsis

kf builds [flags]

Examples

kf builds

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
--app=string: Sets a filter for the "app.kubernetes.io/name" label.
-h, --help: help for builds
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.11 - kf configure-cluster

Set configuration for a cluster.

Name

kf configure-cluster - Set configuration for a cluster.

Synopsis

kf configure-cluster [subcommand] [flags]

Description

The configure-cluster sub-command allows operators to configure individual fields on a cluster.

Flags

-h, --help: help for configure-cluster

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.12 - kf configure-cluster get-feature-flags

Get the values for feature flags set on the cluster.

Name

kf configure-cluster get-feature-flags - Get the values for feature flags set on the cluster.

Synopsis

kf configure-cluster get-feature-flags [flags]

Examples

# Configure the cluster.
kf configure-cluster get-feature-flags

Flags

-h, --help: help for get-feature-flags

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.13 - kf configure-cluster set-feature-flag

Set a feature flag on the cluster.

Name

kf configure-cluster set-feature-flag - Set a feature flag on the cluster.

Synopsis

kf configure-cluster set-feature-flag FEATURE-FLAG-NAME BOOL [flags]

Examples

# Configure the cluster.
kf configure-cluster set-feature-flag enable_route_services true

Flags

-h, --help: help for set-feature-flag

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.14 - kf configure-cluster unset-feature-flag

Unset a feature flag on the cluster. Resets feature flag value to default.

Name

kf configure-cluster unset-feature-flag - Unset a feature flag on the cluster. Resets feature flag value to default.

Synopsis

kf configure-cluster unset-feature-flag FEATURE-FLAG-NAME [flags]

Examples

# Configure the cluster.
kf configure-cluster unset-feature-flag enable_route_service

Flags

-h, --help: help for unset-feature-flag

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.15 - kf configure-space

Set configuration for a Space.

Name

kf configure-space - Set configuration for a Space.

Synopsis

kf configure-space [subcommand] [flags]

Description

The configure-space sub-command allows operators to configure individual fields on a Space.

In Kf, most configuration can be overridden at the Space level.

NOTE: The Space is reconciled every time changes are made using this command. If you want to configure Spaces in automation it’s better to use kubectl.

Flags

-h, --help: help for configure-space

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.16 - kf configure-space append-domain

Append a domain for a Space.

Name

kf configure-space append-domain - Append a domain for a Space.

Synopsis

kf configure-space append-domain [SPACE_NAME] DOMAIN [flags]

Examples

# Configure the Space "my-space"
kf configure-space append-domain my-space myspace.mycompany.com
# Configure the targeted Space
kf configure-space append-domain myspace.mycompany.com

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for append-domain

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.17 - kf configure-space get-build-service-account

Get the service account that is used when building containers in the Space.

Name

kf configure-space get-build-service-account - Get the service account that is used when building containers in the Space.

Synopsis

kf configure-space get-build-service-account [SPACE_NAME] [flags]

Examples

# Configure the Space "my-space"
kf configure-space get-build-service-account my-space
# Configure the targeted Space
kf configure-space get-build-service-account

Flags

-h, --help: help for get-build-service-account

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.18 - kf configure-space get-buildpack-env

Get the environment variables for Builds in a Space.

Name

kf configure-space get-buildpack-env - Get the environment variables for Builds in a Space.

Synopsis

kf configure-space get-buildpack-env [SPACE_NAME] [flags]

Examples

# Configure the Space "my-space"
kf configure-space get-buildpack-env my-space
# Configure the targeted Space
kf configure-space get-buildpack-env

Flags

-h, --help: help for get-buildpack-env

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.19 - kf configure-space get-container-registry

Get the container registry used for Builds.

Name

kf configure-space get-container-registry - Get the container registry used for Builds.

Synopsis

kf configure-space get-container-registry [SPACE_NAME] [flags]

Examples

# Configure the Space "my-space"
kf configure-space get-container-registry my-space
# Configure the targeted Space
kf configure-space get-container-registry

Flags

-h, --help: help for get-container-registry

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.20 - kf configure-space get-domains

Get domains associated with the Space.

Name

kf configure-space get-domains - Get domains associated with the Space.

Synopsis

kf configure-space get-domains [SPACE_NAME] [flags]

Examples

# Configure the Space "my-space"
kf configure-space get-domains my-space
# Configure the targeted Space
kf configure-space get-domains

Flags

-h, --help: help for get-domains

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.21 - kf configure-space get-execution-env

Get the Space wide App environment variables.

Name

kf configure-space get-execution-env - Get the Space wide App environment variables.

Synopsis

kf configure-space get-execution-env [SPACE_NAME] [flags]

Examples

# Configure the Space "my-space"
kf configure-space get-execution-env my-space
# Configure the targeted Space
kf configure-space get-execution-env

Flags

-h, --help: help for get-execution-env

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.22 - kf configure-space get-nodeselector

Get the node selector associated with the Space.

Name

kf configure-space get-nodeselector - Get the node selector associated with the Space.

Synopsis

kf configure-space get-nodeselector [SPACE_NAME] [flags]

Examples

# Configure the Space "my-space"
kf configure-space get-nodeselector my-space
# Configure the targeted Space
kf configure-space get-nodeselector

Flags

-h, --help: help for get-nodeselector

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.23 - kf configure-space remove-domain

Remove a domain from a Space.

Name

kf configure-space remove-domain - Remove a domain from a Space.

Synopsis

kf configure-space remove-domain [SPACE_NAME] DOMAIN [flags]

Examples

# Configure the Space "my-space"
kf configure-space remove-domain my-space myspace.mycompany.com
# Configure the targeted Space
kf configure-space remove-domain myspace.mycompany.com

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for remove-domain

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.24 - kf configure-space set-app-egress-policy

Set the egress policy for Apps in the Space.

Name

kf configure-space set-app-egress-policy - Set the egress policy for Apps in the Space.

Synopsis

kf configure-space set-app-egress-policy [SPACE_NAME] POLICY [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-app-egress-policy my-space DenyAll
# Configure the targeted Space
kf configure-space set-app-egress-policy DenyAll

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-app-egress-policy

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.25 - kf configure-space set-app-ingress-policy

Set the ingress policy for Apps in the Space.

Name

kf configure-space set-app-ingress-policy - Set the ingress policy for Apps in the Space.

Synopsis

kf configure-space set-app-ingress-policy [SPACE_NAME] POLICY [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-app-ingress-policy my-space DenyAll
# Configure the targeted Space
kf configure-space set-app-ingress-policy DenyAll

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-app-ingress-policy

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.26 - kf configure-space set-build-egress-policy

Set the egress policy for Builds in the Space.

Name

kf configure-space set-build-egress-policy - Set the egress policy for Builds in the Space.

Synopsis

kf configure-space set-build-egress-policy [SPACE_NAME] POLICY [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-build-egress-policy my-space DenyAll
# Configure the targeted Space
kf configure-space set-build-egress-policy DenyAll

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-build-egress-policy

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.27 - kf configure-space set-build-ingress-policy

Set the ingress policy for Builds in the Space.

Name

kf configure-space set-build-ingress-policy - Set the ingress policy for Builds in the Space.

Synopsis

kf configure-space set-build-ingress-policy [SPACE_NAME] POLICY [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-build-ingress-policy my-space DenyAll
# Configure the targeted Space
kf configure-space set-build-ingress-policy DenyAll

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-build-ingress-policy

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.28 - kf configure-space set-build-service-account

Set the service account to use when building containers.

Name

kf configure-space set-build-service-account - Set the service account to use when building containers.

Synopsis

kf configure-space set-build-service-account [SPACE_NAME] SERVICE_ACCOUNT [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-build-service-account my-space myserviceaccount
# Configure the targeted Space
kf configure-space set-build-service-account myserviceaccount

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-build-service-account

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.29 - kf configure-space set-buildpack-env

Set an environment variable for Builds in a Space.

Name

kf configure-space set-buildpack-env - Set an environment variable for Builds in a Space.

Synopsis

kf configure-space set-buildpack-env [SPACE_NAME] ENV_VAR_NAME ENV_VAR_VALUE [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-buildpack-env my-space JDK_VERSION 11
# Configure the targeted Space
kf configure-space set-buildpack-env JDK_VERSION 11

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-buildpack-env

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.30 - kf configure-space set-container-registry

Set the container registry used for Builds.

Name

kf configure-space set-container-registry - Set the container registry used for Builds.

Synopsis

kf configure-space set-container-registry [SPACE_NAME] REGISTRY [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-container-registry my-space gcr.io/my-project
# Configure the targeted Space
kf configure-space set-container-registry gcr.io/my-project

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-container-registry

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.31 - kf configure-space set-default-domain

Set or create a default domain for a Space.

Name

kf configure-space set-default-domain - Set or create a default domain for a Space.

Synopsis

kf configure-space set-default-domain [SPACE_NAME] DOMAIN [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-default-domain my-space myspace.mycompany.com
# Configure the targeted Space
kf configure-space set-default-domain myspace.mycompany.com

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-default-domain

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.32 - kf configure-space set-env

Set a Space wide environment variable for all Apps.

Name

kf configure-space set-env - Set a Space wide environment variable for all Apps.

Synopsis

kf configure-space set-env [SPACE_NAME] ENV_VAR_NAME ENV_VAR_VALUE [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-env my-space ENVIRONMENT production
# Configure the targeted Space
kf configure-space set-env ENVIRONMENT production

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-env

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.33 - kf configure-space set-nodeselector

Set a Space wide node selector for all Apps.

Name

kf configure-space set-nodeselector - Set a Space wide node selector for all Apps.

Synopsis

kf configure-space set-nodeselector [SPACE_NAME] NS_VAR_NAME NS_VAR_VALUE [flags]

Examples

# Configure the Space "my-space"
kf configure-space set-nodeselector my-space DiskType ssd
# Configure the targeted Space
kf configure-space set-nodeselector DiskType ssd

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-nodeselector

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.34 - kf configure-space unset-buildpack-env

Unset an environment variable for Builds in a Space.

Name

kf configure-space unset-buildpack-env - Unset an environment variable for Builds in a Space.

Synopsis

kf configure-space unset-buildpack-env [SPACE_NAME] ENV_VAR_NAME [flags]

Examples

# Configure the Space "my-space"
kf configure-space unset-buildpack-env my-space JDK_VERSION
# Configure the targeted Space
kf configure-space unset-buildpack-env JDK_VERSION

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for unset-buildpack-env

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.35 - kf configure-space unset-env

Unset a Space wide environment variable for all Apps.

Name

kf configure-space unset-env - Unset a Space wide environment variable for all Apps.

Synopsis

kf configure-space unset-env [SPACE_NAME] ENV_VAR_NAME [flags]

Examples

# Configure the Space "my-space"
kf configure-space unset-env my-space ENVIRONMENT
# Configure the targeted Space
kf configure-space unset-env ENVIRONMENT

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for unset-env

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.36 - kf configure-space unset-nodeselector

Unset a Space wide node selector for all Apps.

Name

kf configure-space unset-nodeselector - Unset a Space wide node selector for all Apps.

Synopsis

kf configure-space unset-nodeselector [SPACE_NAME] NS_VAR_NAME [flags]

Examples

# Configure the Space "my-space"
kf configure-space unset-nodeselector my-space DiskType
# Configure the targeted Space
kf configure-space unset-nodeselector DiskType

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for unset-nodeselector

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.37 - kf create-autoscaling-rule

Create autoscaling rule for App.

Name

kf create-autoscaling-rule - Create autoscaling rule for App.

Synopsis

kf create-autoscaling-rule APP RULE_TYPE MIN_THRESHOLD MAX_THRESHOLD [flags]

Description

Create an autoscaling rule for App.

The only supported rule type is CPU. It is the target percentage. It is calculated by taking the average of MIN_THRESHOLD and MAX_THRESHOLD.

The range of MIN_THRESHOLD and MAX_THRESHOLD is 1 to 100 (percent).

Examples

# Scale myapp based on CPU load targeting 50% utilization (halfway between 20 and 80)
kf create-autoscaling-rule myapp CPU 20 80

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for create-autoscaling-rule
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.38 - kf create-job

Create a Job on the App.

Name

kf create-job - Create a Job on the App.

Synopsis

kf create-job APP_NAME JOB_NAME COMMAND [flags]

Description

The create-job sub-command lets operators create a Job that can be run on a schedule or ad hoc.

Examples

kf create-job my-app my-job "sleep 100"

Flags

--async: Do not wait for the action to complete on the server before returning.
-c, --concurrency-policy=string: Specifies how to treat concurrent executions of a Job: Always (default), Replace, or Forbid. (default "Always")
--cpu-cores=string: Amount of dedicated CPU cores to give the Task (for example 256M, 1024M, 1G).
-k, --disk-quota=string: Amount of dedicated disk space to give the Task (for example 256M, 1024M, 1G).
-h, --help: help for create-job
-m, --memory-limit=string: Amount of dedicated memory to give the Task (for example 256M, 1024M, 1G).
-s, --schedule=string: Cron schedule on which to execute the Job.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.39 - kf create-route

Create a traffic routing rule for a host+path pair.

Name

kf create-route - Create a traffic routing rule for a host+path pair.

Synopsis

kf create-route DOMAIN [--hostname HOSTNAME] [--path PATH] [flags]

Description

Creating a Route allows Apps to declare they want to receive traffic on the same host/domain/path combination.

Routes without any bound Apps (or with only stopped Apps) will return a 404 HTTP status code.

Kf doesn’t enforce Route uniqueness between Spaces. It’s recommended to provide each Space with its own subdomain instead.

Examples

kf create-route example.com --hostname myapp # myapp.example.com
kf create-route --space myspace example.com --hostname myapp # myapp.example.com
kf create-route example.com --hostname myapp --path /mypath # myapp.example.com/mypath
kf create-route --space myspace myapp.example.com # myapp.example.com

# Using SPACE to match 'cf'
kf create-route myspace example.com --hostname myapp # myapp.example.com
kf create-route myspace example.com --hostname myapp --path /mypath # myapp.example.com/mypath

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for create-route
--hostname=string: Hostname for the Route.
--path=string: URL path for the Route.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.40 - kf create-service

Create a service instance from a marketplace template.

Name

kf create-service - Create a service instance from a marketplace template.

Synopsis

kf create-service SERVICE PLAN SERVICE_INSTANCE [-c PARAMETERS_AS_JSON] [-b service-broker] [-t TAGS] [flags]

Description

Create service creates a new ServiceInstance using a template from the marketplace.

Examples

# Creates a new instance of a db-service with the name mydb, plan silver, and provisioning configuration
kf create-service db-service silver mydb -c '{"ram_gb":4}'

# Creates a new instance of a db-service from the broker named local-broker
kf create-service db-service silver mydb -c ~/workspace/tmp/instance_config.json -b local-broker

# Creates a new instance of a db-service with the name mydb and override tags
kf create-service db-service silver mydb -t "list, of, tags"

Flags

--async: Do not wait for the action to complete on the server before returning.
-b, --broker=string: Name of the service broker that will create the instance.
-h, --help: help for create-service
-c, --parameters=string: JSON object or path to a JSON file containing configuration parameters. (default "{}")
-t, --tags=string: User-defined tags to differentiate services during injection.
--timeout=duration: Amount of time to wait for the operation to complete. Valid units are "s", "m", "h". (default 30m0s)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.41 - kf create-service-broker

Add a service broker to the marketplace.

Name

kf create-service-broker - Add a service broker to the marketplace.

Synopsis

kf create-service-broker NAME USERNAME PASSWORD URL [flags]

Examples

  kf create-service-broker mybroker user pass http://mybroker.broker.svc.cluster.local

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for create-service-broker
--space-scoped: Only create the broker in the targeted space.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.42 - kf create-space

Create a Space with the given name.

Name

kf create-space - Create a Space with the given name.

Synopsis

kf create-space NAME [flags]

Examples

# Create a Space with custom domains.
kf create-space my-space --domain my-space.my-company.com

# Create a Space that uses unique storage and service accounts.
kf create-space my-space --container-registry gcr.io/my-project --build-service-account myserviceaccount

# Set running and staging environment variables for Apps and Builds.
kf create-space my-space --run-env=ENVIRONMENT=nonprod --stage-env=ENVIRONMENT=nonprod,JDK_VERSION=8

Flags

--build-service-account=string: Service account that Builds will use.
--container-registry=string: Container registry built Apps and source code will be stored in.
--domain=stringArray: Sets the valid domains for the Space. The first provided domain is the default.
-h, --help: help for create-space
--run-env=stringToString: Sets the running environment variables for all Apps in the Space.
--stage-env=stringToString: Sets the staging environment variables for all Builds in the Space.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.43 - kf create-user-provided-service

Create a standalone service instance from existing credentials.

Name

kf create-user-provided-service - Create a standalone service instance from existing credentials.

Synopsis

kf create-user-provided-service SERVICE_INSTANCE [-p CREDENTIALS] [-t TAGS] [flags]

Description

Creates a standalone service instance from existing credentials. User-provided services can be used to inject credentials for services managed outside of Kf into Apps.

Credentials are stored in a Kubernetes Secret in the Space the service is created in. On GKE these Secrets are encrypted at rest and can optionally be encrypted using KMS.

Examples

# Bring an existing database service
kf create-user-provided-service db-service -p '{"url":"mysql://..."}'

# Create a service with tags for autowiring
kf create-user-provided-service db-service -t "mysql,database,sql"

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for create-user-provided-service
--mock-class=string: Mock class name to use in VCAP_SERVICES rather than 'user-provided'.
--mock-plan=string: Mock plan name to use in VCAP_SERVICES rather than blank.
-p, --parameters=string: JSON object or path to a JSON file containing configuration parameters. (default "{}")
--params=string: JSON object or path to a JSON file containing configuration parameters. DEPRECATED: use --parameters instead. (default "{}")
-r, --route=string: URL to which requests for bound routes will be forwarded. Scheme must be https. NOTE: This is a preivew feature.
-t, --tags=string: User-defined tags to differentiate services during injection.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.44 - kf debug

Print debugging information useful for filing a bug report.

Name

kf debug - Print debugging information useful for filing a bug report.

Synopsis

kf debug [flags]

Examples

  kf debug

Flags

-h, --help: help for debug

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.45 - kf delete

Delete the App with the given name in the targeted Space.

Name

kf delete - Delete the App with the given name in the targeted Space.

Synopsis

kf delete NAME [flags]

Description

Deletes the App with the given name and wait for it to be deleted.

Kubernetes will delete the App once all child resources it owns have been deleted. Deletion may take a long time if any of the following conditions are true:

There are many child objects.
There are finalizers on the object preventing deletion.
The cluster is in an unhealthy state.

Deleting an App will remove associated deployment history, cluster logs, and Builds.

Things that won’t be deleted:

source code
container images
Routes
ServiceInstances

Examples

kf delete my-app

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for delete
--retries=int: Number of times to retry execution if the command isn't successful. (default 5)
--retry-delay=duration: Set the delay between retries. (default 1s)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.46 - kf delete-autoscaling-rules

Delete all autoscaling rules for App and disable autoscaling.

Name

kf delete-autoscaling-rules - Delete all autoscaling rules for App and disable autoscaling.

Synopsis

kf delete-autoscaling-rules APP_NAME [flags]

Examples

kf delete-autoscaling-rules myapp

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for delete-autoscaling-rules
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.47 - kf delete-job

Delete the Job with the given name in the targeted Space.

Name

kf delete-job - Delete the Job with the given name in the targeted Space.

Synopsis

kf delete-job NAME [flags]

Description

Deletes the Job with the given name and wait for it to be deleted.

Kubernetes will delete the Job once all child resources it owns have been deleted. Deletion may take a long time if any of the following conditions are true:

There are many child objects.
There are finalizers on the object preventing deletion.
The cluster is in an unhealthy state.

Note: Deleting the Job will delete all associated executions, even executions which are still running.

Examples

kf delete-job my-job

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for delete-job
--retries=int: Number of times to retry execution if the command isn't successful. (default 5)
--retry-delay=duration: Set the delay between retries. (default 1s)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.48 - kf delete-job-schedule

Delete the schedule for a Job.

Name

kf delete-job-schedule - Delete the schedule for a Job.

Synopsis

kf delete-job-schedule JOB_NAME [flags]

Description

The delete-job-schedule sub-command lets operators suspend the Job by deleting the schedule.

Examples

kf delete-job-schedule my-job

Flags

-h, --help: help for delete-job-schedule

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.49 - kf delete-network-policy

Delete the NetworkPolicy with the given name in the targeted Space.

Name

kf delete-network-policy - Delete the NetworkPolicy with the given name in the targeted Space.

Synopsis

kf delete-network-policy NAME [flags]

Description

Deletes the NetworkPolicy with the given name and wait for it to be deleted.

Kubernetes will delete the NetworkPolicy once all child resources it owns have been deleted. Deletion may take a long time if any of the following conditions are true:

There are many child objects.
There are finalizers on the object preventing deletion.
The cluster is in an unhealthy state.

Note: Deleting the last network policy targeting a Service will cause it to default to whatever policy is set for the cluster.

Examples

kf delete-network-policy my-networkpolicy

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for delete-network-policy
--retries=int: Number of times to retry execution if the command isn't successful. (default 5)
--retry-delay=duration: Set the delay between retries. (default 1s)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.50 - kf delete-orphaned-routes

Delete Routes with no App bindings.

Name

kf delete-orphaned-routes - Delete Routes with no App bindings.

Synopsis

kf delete-orphaned-routes [flags]

Description

Deletes Routes in the targeted Space that are fully reconciled and don’t have any bindings to Apps.

Examples

kf delete-orphaned-routes

Flags

-h, --help: help for delete-orphaned-routes

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.51 - kf delete-route

Delete a Route in the targeted Space.

Name

kf delete-route - Delete a Route in the targeted Space.

Synopsis

kf delete-route DOMAIN [--hostname HOSTNAME] [--path PATH] [flags]

Examples

# Delete the Route myapp.example.com
kf delete-route example.com --hostname myapp
# Delete a Route on a path myapp.example.com/mypath
kf delete-route example.com --hostname myapp --path /mypath

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for delete-route
--hostname=string: Hostname for the Route.
--path=string: URL path for the Route.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.52 - kf delete-service

Delete the ServiceInstance with the given name in the targeted Space.

Name

kf delete-service - Delete the ServiceInstance with the given name in the targeted Space.

Synopsis

kf delete-service NAME [flags]

Description

Deletes the ServiceInstance with the given name and wait for it to be deleted.

Kubernetes will delete the ServiceInstance once all child resources it owns have been deleted. Deletion may take a long time if any of the following conditions are true:

There are many child objects.
There are finalizers on the object preventing deletion.
The cluster is in an unhealthy state.

You should delete all bindings before deleting a service. If you don’t, the service will wait for that to occur before deleting.

Examples

kf delete-service my-serviceinstance

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for delete-service
--retries=int: Number of times to retry execution if the command isn't successful. (default 5)
--retry-delay=duration: Set the delay between retries. (default 1s)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.53 - kf delete-service-broker

Remove a service broker from the marketplace.

Name

kf delete-service-broker - Remove a service broker from the marketplace.

Synopsis

kf delete-service-broker NAME [flags]

Examples

  kf delete-service-broker mybroker

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for delete-service-broker
--space-scoped: Set to delete a space scoped service broker.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.54 - kf delete-space

Delete the Space with the given name.

Name

kf delete-space - Delete the Space with the given name.

Synopsis

kf delete-space NAME [flags]

Description

Deletes the Space with the given name and wait for it to be deleted.

Kubernetes will delete the Space once all child resources it owns have been deleted. Deletion may take a long time if any of the following conditions are true:

There are many child objects.
There are finalizers on the object preventing deletion.
The cluster is in an unhealthy state.

Deleting a Space will also delete its:

Apps
Build history
Service bindings
Service instances
Routes
The backing Kubernetes Namespace
Kubernetes resources in the Namespace

You will be unable to make changes to resources in the Space once deletion has begun.

Examples

kf delete-space my-space

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for delete-space
--retries=int: Number of times to retry execution if the command isn't successful. (default 5)
--retry-delay=duration: Set the delay between retries. (default 1s)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.55 - kf disable-autoscaling

Disable autoscaling for App.

Name

kf disable-autoscaling - Disable autoscaling for App.

Synopsis

kf disable-autoscaling APP_NAME [flags]

Examples

kf disable-autoscaling APP_NAME

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for disable-autoscaling
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.56 - kf doctor

Run validation tests against one or more components.

Name

kf doctor - Run validation tests against one or more components.

Synopsis

kf doctor [(COMPONENT|TYPE/NAME)...] [flags]

Description

Doctor runs tests on one or more components or objects to validate their desired status.

If no arguments are supplied, then all component tests are ran. If one or more arguments are suplied then only the test for those components or objects are ran.

Possible components are:

cluster
istio

Possible object types are:

app
build
clusterservicebroker
route
servicebroker
serviceinstance
serviceinstancebinding
sourcepackage
space
task

Examples

# Run doctor against all components.
kf doctor
# Run doctor against server-side components.
kf doctor cluster
# Run doctor for a Kf App named my-app.
kf doctor app/my-app
# Run doctor for a Kf Service named my-service.
kf doctor serviceinstance/my-service
# Run doctor for a Kf Binding named my-binding.
kf doctor serviceinstancebinding/my-binding
# Run doctor for the Kf Operator.
kf doctor operator

Flags

--delay=duration: Set the delay between executions. (default 5s)
-h, --help: help for doctor
--retries=int: Number of times to retry doctor if it isn't successful. (default 1)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.57 - kf domains

List domains that can be used in the targeted Space.

Name

kf domains - List domains that can be used in the targeted Space.

Synopsis

kf domains [flags]

Examples

kf domains

Flags

-h, --help: help for domains

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.58 - kf enable-autoscaling

Enable autoscaling for App.

Name

kf enable-autoscaling - Enable autoscaling for App.

Synopsis

kf enable-autoscaling APP_NAME [flags]

Description

Enabling autoscaling creates a HorizontalPodAutoscaler (HPA) for an App. HPA controls the number of replicas the App should have based on CPU utilization.

Autoscaler will only take effect after autoscaling limits are set and autoscaling rules are added.

Set autoscaling limits by running: kf update-autoscaling-limits APP_NAME MIN_INSTANCES MAX_INSTANCES

Add rules by running: kf create-autoscaling-rule APP_NAME RULE_TYPE MIN_THRESHOLD MAX_THRESHOLD

Examples

kf enable-autoscaling myapp

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for enable-autoscaling
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.59 - kf env

Print information about an App’s environment variables.

Name

kf env - Print information about an App’s environment variables.

Synopsis

kf env APP_NAME [flags]

Description

The env command gets the names and values of developer managed environment variables for an App.

Environment variables are evaluated in the following order with later values overriding earlier ones with the same name:

Space (set by administrators)
App (set by developers)
System (set by Kf)

Environment variables containing variable substitution “$(…)” are replaced at runtime by Kubernetes. Kf provides the following runtime environment variables:

CF_INSTANCE_ADDR: The cluster-visible IP:PORT of the App instance.
CF_INSTANCE_GUID: The UUID of the App instance.
INSTANCE_GUID: Alias of CF_INSTANCE_GUID
CF_INSTANCE_INDEX: The index number of the App instance, this will ALWAYS be 0.
INSTANCE_INDEX: Alias of CF_INSTANCE_INDEX
CF_INSTANCE_IP: The cluster-visible IP of the App instance.
CF_INSTANCE_INTERNAL_IP: Alias of CF_INSTANCE_IP
VCAP_APP_HOST: Alias of CF_INSTANCE_IP
CF_INSTANCE_PORT: The cluster-visible port of the App instance. In Kf this is the same as PORT.
DATABASE_URL: The first URI found in a VCAP_SERVICES credential.
DISK_LIMIT: The maximum amount of disk storage in MB the App can use.
LANG: Required by buildpacks to ensure consistent script load order.
MEMORY_LIMIT: The maximum amount of memory the App can consume.
MEMORY_LIMIT_IN_MB: The maximum amount of memory in MB the App can consume.
PORT: The port the App should listen on for requests.
VCAP_APP_PORT: Alias of PORT
VCAP_APPLICATION: A JSON structure containing app metadata.
VCAP_SERVICES: A JSON structure specifying bound services.

Examples

kf env myapp

Flags

-h, --help: help for env

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.60 - kf fix-orphaned-bindings

Fix bindings without an app owner in a space.

Name

kf fix-orphaned-bindings - Fix bindings without an app owner in a space.

Synopsis

kf fix-orphaned-bindings [flags]

Description

Fix bindings with a missing owner reference to an App.

Examples

# Identify broken bindings in the targeted space.
kf fix-orphaned-bindings

# Fix bindings in the targeted space.
kf fix-orphaned-bindings --dry-run=false

Flags

--dry-run: Run the command without applying changes. (default true)
-h, --help: help for fix-orphaned-bindings

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.61 - kf job-history

List the execution history of a Job.

Name

kf job-history - List the execution history of a Job.

Synopsis

kf job-history JOB_NAME [flags]

Description

The job-history sub-command lets operators view the execution history of a Job.

Examples

kf job-history my-job

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for job-history
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.62 - kf job-schedules

List job schedules in the targeted Space.

Name

kf job-schedules - List job schedules in the targeted Space.

Synopsis

kf job-schedules [flags]

Examples

kf job-schedules

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for job-schedules
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.63 - kf jobs

List Jobs in the targeted Space.

Name

kf jobs - List Jobs in the targeted Space.

Synopsis

kf jobs [flags]

Examples

kf jobs

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for jobs
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.64 - kf logs

Show logs for an App.

Name

kf logs - Show logs for an App.

Synopsis

kf logs APP_NAME [flags]

Description

Logs are streamed from the Kubernetes log endpoint for each running App instance.

If App instances change or the connection to Kubernetes times out the log stream may show duplicate logs.

Logs are retained for App instances as space permits on the cluster, but will be deleted if space is low or past their retention date. Cloud Logging is a more reliable mechanism to access historical logs.

If you need logs for a particular instance use the kubectl CLI.

Examples

# Follow/tail the log stream
kf logs myapp

# Follow/tail the log stream with 20 lines of context
kf logs myapp -n 20

# Get recent logs from the App
kf logs myapp --recent

# Get the most recent 200 lines of logs from the App
kf logs myapp --recent -n 200

# Get the logs of Tasks running from the App
kf logs myapp --task

Flags

-h, --help: help for logs
-n, --number=int: Show the last N lines of logs. (default 10)
--recent: Dump recent logs instead of tailing.
--task: Tail Task logs instead of App.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.65 - kf map-route

Grant an App access to receive traffic from the Route.

Name

kf map-route - Grant an App access to receive traffic from the Route.

Synopsis

kf map-route APP_NAME DOMAIN [--hostname HOSTNAME] [--path PATH] [--weight WEIGHT] [flags]

Description

Mapping an App to a Route will cause traffic to be forwarded to the App if the App has instances that are running and healthy.

If multiple Apps are mapped to the same Route they will split traffic between them roughly evenly. Incoming network traffic is handled by multiple gateways which update their routing tables with slight delays and route independently. Because of this, traffic routing may not appear even but it will converge over time.

Examples

kf map-route myapp example.com --hostname myapp # myapp.example.com
kf map-route myapp myapp.example.com # myapp.example.com
kf map-route myapp example.com --hostname myapp --weight 2 # myapp.example.com, myapp receives 2x traffic
kf map-route --space myspace myapp example.com --hostname myapp # myapp.example.com
kf map-route myapp example.com --hostname myapp --path /mypath # myapp.example.com/mypath

Flags

--async: Do not wait for the action to complete on the server before returning.
--destination-port=int32: Port on the App the Route will connect to.
-h, --help: help for map-route
--hostname=string: Hostname for the Route.
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.
--path=string: URL path for the Route.
--weight=int32: Weight for the Route. (default 1)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.66 - kf marketplace

List service classes available in the cluster.

Name

kf marketplace - List service classes available in the cluster.

Synopsis

kf marketplace [-s SERVICE] [flags]

Examples

# Show service classes available in the cluster
kf marketplace

# Show the plans available to a particular service class
kf marketplace -s google-storage

Flags

-h, --help: help for marketplace
-s, --service=string: List plans for the service class.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.67 - kf network-policies

List network policies in the targeted Space.

Name

kf network-policies - List network policies in the targeted Space.

Synopsis

kf network-policies [flags]

Examples

kf network-policies

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for network-policies
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.68 - kf network-policy

Print information about the given NetworkPolicy.

Name

kf network-policy - Print information about the given NetworkPolicy.

Synopsis

kf network-policy NAME [flags]

Examples

kf network-policy my-network-policy

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for network-policy
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.69 - kf proxy

Start a local reverse proxy to an App.

Name

kf proxy - Start a local reverse proxy to an App.

Synopsis

kf proxy APP_NAME [flags]

Description

Proxy creates a reverse HTTP proxy to the cluster’s gateway on a local port opened on the operating system’s loopback device.

The proxy rewrites all HTTP requests, changing the HTTP Host header and adding an additional header X-Kf-App to ensure traffic reaches the specified App even if multiple are attached to the same route.

Proxy does not establish a direct connection to the App.

For proxy to work:

The cluster’s gateway must be accessible from your local machine.
The App must have a public URL

If you need to establish a direct connection to an App, use the port-forward command in kubectl. It establishes a proxied connection directly to a port on a pod via the Kubernetes cluster. port-forward bypasses all routing.

Examples

kf proxy myapp

Flags

--gateway=string: IP address of the HTTP gateway to route requests to.
-h, --help: help for proxy
--port=int: Local port to listen on. (default 8080)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.70 - kf proxy-route

Start a local reverse proxy to a Route.

Name

kf proxy-route - Start a local reverse proxy to a Route.

Synopsis

kf proxy-route ROUTE [flags]

Description

Proxy route creates a reverse HTTP proxy to the cluster’s gateway on a local port opened on the operating system’s loopback device.

The proxy rewrites all HTTP requests, changing the HTTP Host header to match the Route. If multiple Apps are mapped to the same route, the traffic sent over the proxy will follow their routing rules with regards to weight. If no Apps are mapped to the route, traffic sent through the proxy will return a HTTP 404 status code.

Proxy route DOES NOT establish a direct connection to any Kubernetes resource.

For proxy to work:

The cluster’s gateway MUST be accessible from your local machine.
The Route MUST have a public URL

Examples

kf proxy-route myhost.example.com

Flags

--gateway=string: IP address of the HTTP gateway to route requests to.
-h, --help: help for proxy-route
--port=int: Local port to listen on. (default 8080)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.71 - kf push

Create a new App or apply updates to an existing one.

Name

kf push - Create a new App or apply updates to an existing one.

Synopsis

kf push APP_NAME [flags]

Examples

kf push myapp
kf push myapp --buildpack my.special.buildpack # Discover via kf buildpacks
kf push myapp --env FOO=bar --env BAZ=foo
kf push myapp --stack cloudfoundry/cflinuxfs3 # Use a cflinuxfs3 runtime
kf push myapp --health-check-http-endpoint /myhealthcheck # Specify a healthCheck for the app

Flags

--app-suffix=string: Suffix to append to the end of every pushed App.
--args=stringArray: Override the args for the image. Can't be used with the command flag.
-b, --buildpack=string: Use the specified buildpack rather than the built-in.
-c, --command=string: Startup command for the App, this overrides the default command specified by the web process.
--container-registry=string: Container registry to push images to.
--cpu-cores=string: Number of dedicated CPU cores to give each App instance (for example 100m, 0.5, 1, 2). For more information see https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/.
-k, --disk-quota=string: Size of dedicated ephemeral disk attached to each App instance (for example 512M, 2G, 1T).
--docker-image=string: Docker image to deploy rather than building from source.
--dockerfile=string: Path to the Dockerfile to build. Relative to the source root.
--entrypoint=string: Overwrite the default entrypoint of the image. Can't be used with the command flag.
-e, --env=stringArray: Set environment variables. Multiple can be set by using the flag multiple times (for example, NAME=VALUE).
--health-check-http-endpoint=string: HTTP endpoint to target as part of the health-check. Only valid if health-check-type is http.
-u, --health-check-type=string: App health check type: http, port (default) or process.
-h, --help: help for push
-i, --instances=int32: If set, overrides the number of instances of the App to run, -1 represents non-user input. (default -1)
-f, --manifest=string: Path to the application manifest.
-m, --memory-limit=string: Amount of dedicated RAM to give each App instance (for example 512M, 6G, 1T).
--no-manifest: Do not read the manifest file even if one exists.
--no-route: Prevents the App from being reachable once deployed.
--no-start: Build but do not run the App.
-p, --path=string: If specified, overrides the path to the source code.
--random-route: Create a random Route for this App if it doesn't have one.
--route=stringArray: Use the routes flag to provide multiple HTTP and TCP routes. Each Route for this App is created if it does not already exist.
-s, --stack=string: Base image to use for to use for Apps created with a buildpack.
--task: Push an App to execute Tasks only. The App will be built, but not run. It will not have a route assigned.
-t, --timeout=int: Amount of time the App can be unhealthy before declaring it as unhealthy.
--var=stringToString: Manifest variable substitution. Multiple can be set by using the flag multiple times (for example NAME=VALUE).
--vars-file=stringArray: JSON or YAML file to read variable substitutions from. Can be supplied multiple times.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.72 - kf restage

Rebuild and redeploy an App without downtime.

Name

kf restage - Rebuild and redeploy an App without downtime.

Synopsis

kf restage APP_NAME [flags]

Description

Restaging an App will re-run the latest Build to produce a new container image, and if successful will replace each running instance with the new image.

Instances are replaced one at a time, always ensuring that the desired number of instances are healthy. This property is upheld by running one additional instance of the App and swapping it out for an old instance.

The operation completes once all instances have been replaced.

Examples

kf restage myapp

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for restage

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.73 - kf restart

Restart each running instance of an App without downtime.

Name

kf restart - Restart each running instance of an App without downtime.

Synopsis

kf restart APP_NAME [flags]

Description

Restarting an App will replace each running instance of an App with a new one.

The operation completes once all instances have been replaced.

Examples

kf restart myapp

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for restart

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.74 - kf routes

List routes in the targeted Space.

Name

kf routes - List routes in the targeted Space.

Synopsis

kf routes [flags]

Examples

kf routes

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for routes
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.75 - kf run-job

Run the Job once.

Name

kf run-job - Run the Job once.

Synopsis

kf run-job JOB_NAME [flags]

Description

The run-job sub-command lets operators run a Job once.

Examples

kf run-job my-job

Flags

--cpu-cores=string: Amount of dedicated CPU cores to give the Task (for example 256M, 1024M, 1G).
-k, --disk-quota=string: Amount of dedicated disk space to give the Task (for example 256M, 1024M, 1G).
-h, --help: help for run-job
-m, --memory-limit=string: Amount of dedicated memory to give the Task (for example 256M, 1024M, 1G).

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.76 - kf run-task

Run a short-lived Task on the App.

Name

kf run-task - Run a short-lived Task on the App.

Synopsis

kf run-task APP_NAME [flags]

Description

The run-task sub-command lets operators run a short-lived Task on the App.

Examples

kf run-task my-app --command "sleep 100" --name my-task

Flags

-c, --command=string: Command to execute on the Task.
--cpu-cores=string: Amount of dedicated CPU cores to give the Task (for example 256M, 1024M, 1G).
-k, --disk-quota=string: Amount of dedicated disk space to give the Task (for example 256M, 1024M, 1G).
-h, --help: help for run-task
-m, --memory-limit=string: Amount of dedicated memory to give the Task (for example 256M, 1024M, 1G).
--name=string: Display name to give the Task (auto generated if omitted).

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.77 - kf scale

Change the horizontal or vertical scale of an App without downtime.

Name

kf scale - Change the horizontal or vertical scale of an App without downtime.

Synopsis

kf scale APP_NAME [flags]

Description

Scaling an App will change the number of desired instances and/or the requested resources for each instance.

The operation completes once all instances have been replaced.

Examples

# Display current scale settings
kf scale myapp
# Scale to exactly 3 instances
kf scale myapp --instances 3

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for scale
-i, --instances=int32: Number of instances, must be >= 1. (default -1)
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.78 - kf schedule-job

Schedule the Job for execution on a cron schedule.

Name

kf schedule-job - Schedule the Job for execution on a cron schedule.

Synopsis

kf schedule-job JOB_NAME SCHEDULE [flags]

Description

The schedule-job sub-command lets operators schedule a Job for execution on a cron schedule.

Examples

kf schedule-job my-job "* * * * *"

Flags

-h, --help: help for schedule-job

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.79 - kf service

Print information about the given ServiceInstance.

Name

kf service - Print information about the given ServiceInstance.

Synopsis

kf service NAME [flags]

Examples

kf service my-service

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for service
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.80 - kf services

List services in the targeted Space.

Name

kf services - List services in the targeted Space.

Synopsis

kf services [flags]

Examples

kf services

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for services
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.81 - kf set-env

Create or update an environment variable for an App.

Name

kf set-env - Create or update an environment variable for an App.

Synopsis

kf set-env APP_NAME ENV_VAR_NAME ENV_VAR_VALUE [flags]

Description

Sets an environment variable for an App. Existing environment variable(s) on the App with the same name will be replaced.

Apps will be updated without downtime.

Examples

# Set an environment variable on an App.
kf set-env myapp ENV production

# Don't wait for the App to restart.
kf set-env --async myapp ENV production

# Set an environment variable that starts with a dash.
kf set-env myapp -- JAVA_OPTS -Dtest=sometest

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for set-env
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.82 - kf set-space-role

Assgin a Role to a Subject (User|Group|ServiceAccount).

Name

kf set-space-role - Assgin a Role to a Subject (User|Group|ServiceAccount).

Synopsis

kf set-space-role SUBJECT_NAME ROLE [flags]

Examples

# Assign a User to a Role
kf set-space-role john@example.com SpaceDeveloper

# Assign a Group to a Role
kf set-space-role my-group SpaceAuditor -t Group

# Assign a ServiceAccount to a Role
kf set-space-role my-sa SpaceAuditor -t ServiceAccount

Flags

-h, --help: help for set-space-role
-t, --type=string: Type of subject, valid values are Group|ServiceAccount|User(default). (default "User")

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.83 - kf space

Print information about the given Space.

Name

kf space - Print information about the given Space.

Synopsis

kf space NAME [flags]

Examples

kf space my-space

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for space
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.84 - kf space-users

List users and their roles in a Space.

Name

kf space-users - List users and their roles in a Space.

Synopsis

kf space-users [flags]

Examples

kf space-users

Flags

-h, --help: help for space-users

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.85 - kf spaces

List Spaces in the cluster.

Name

kf spaces - List Spaces in the cluster.

Synopsis

kf spaces [flags]

Examples

kf spaces

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for spaces
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.86 - kf ssh

Open a shell on an App instance.

Name

kf ssh - Open a shell on an App instance.

Synopsis

kf ssh APP_NAME [flags]

Description

Opens a shell on an App instance using the Pod exec endpoint.

This command mimics CF’s SSH command by opening a connection to the Kubernetes control plane which spawns a process in a Pod.

The command connects to an arbitrary Pod that matches the App’s runtime labels. If you want a specific Pod, use the pod/ notation.

NOTE: Traffic is encrypted between the CLI and the control plane, and between the control plane and Pod. A malicious Kubernetes control plane could observe the traffic.

Examples

# Open a shell to a specific App
kf ssh myapp

# Open a shell to a specific Pod
kf ssh pod/myapp-revhex-podhex

# Start a different command with args
kf ssh myapp -c /my/command -c arg1 -c arg2

Flags

-c, --command=stringArray: Command to run for the shell. Subsequent definitions will be used as args. (default [/bin/bash])
--container=string: Container to start the command in. (default "user-container")
-T, --disable-pseudo-tty: Don't use a TTY when executing.
-h, --help: help for ssh

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.87 - kf stacks

List stacks in the targeted Space.

Name

kf stacks - List stacks in the targeted Space.

Synopsis

kf stacks [flags]

Description

Stacks contain information about how to build and run an App. Each stack contains:

A unique name to identify it.
A build image, the image used to build the App, this usually contains things like compilers, libraries, sources and build frameworks.
A run image, the image App instances will run within. These images are usually lightweight and contain just enough to run an App.
A list of applicable buildpacks available via the bulidpacks command.

Examples

kf stacks

Flags

-h, --help: help for stacks

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.88 - kf start

Deploy a stopped App and route traffic to it once healthy.

Name

kf start - Deploy a stopped App and route traffic to it once healthy.

Synopsis

kf start APP_NAME [flags]

Description

Starting an App will create a Kubernetes Deployment and Service. The Deployment will use the supplied health checks to validate liveness. While the Deployment is sclaing up, Routes will be modified to send traffic to live instances of the App.

Examples

kf start myapp

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for start

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.89 - kf stop

Remove instances of a running App and stop network traffic.

Name

kf stop - Remove instances of a running App and stop network traffic.

Synopsis

kf stop APP_NAME [flags]

Description

Stopping an App will remove the associated Kubernetes Deployment and Service. This will also delete any historical revisions associated with the App. HTTP traffic being routed to the App will instead be routed to other Apps bound to the Route or will error with an 404 status code if no other Apps are bound.

Examples

kf stop myapp

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for stop

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.90 - kf target

Set the default Space to run commands against.

Name

kf target - Set the default Space to run commands against.

Synopsis

kf target [flags]

Examples

# See the current Space
kf target
# Target a Space
kf target -s my-space

Flags

-h, --help: help for target
-s, --target-space=string: Target the given space.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.91 - kf tasks

List Tasks in the targeted Space.

Name

kf tasks - List Tasks in the targeted Space.

Synopsis

kf tasks [APP] [flags]

Examples

kf tasks

Flags

--allow-missing-template-keys: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. (default true)
-h, --help: help for tasks
-o, --output=string: Output format. One of: go-template|go-template-file|json|jsonpath|jsonpath-as-json|jsonpath-file|name|template|templatefile|yaml.
--show-managed-fields: If true, keep the managedFields when printing objects in JSON or YAML format.
--template=string: Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.92 - kf terminate-task

Terminate a running Task.

Name

kf terminate-task - Terminate a running Task.

Synopsis

kf terminate-task {TASK_NAME | APP_NAME TASK_ID} [flags]

Description

Allows operators to terminate a running Task on an App.

Examples

# Terminate Task by Task name
kf terminate-task my-task-name

# Terminate Task by App name and Task ID
kf terminate-task my-app 1

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for terminate-task
--retries=int: Number of times to retry execution if the command isn't successful. (default 5)
--retry-delay=duration: Set the delay between retries. (default 1s)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.93 - kf unbind-route-service

Unbind a route service instance from an HTTP route.

Name

kf unbind-route-service - Unbind a route service instance from an HTTP route.

Synopsis

kf unbind-route-service DOMAIN [--hostname HOSTNAME] [--path PATH] SERVICE_INSTANCE [flags]

Description

PREVIEW: this feature is not ready for production use. Unbinding a route service from an HTTP route causes requests to go straight to the HTTP route, rather than being processed by the route service first.

Examples

kf unbind-route-service company.com --hostname myapp --path mypath myauthservice

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for unbind-route-service
--hostname=string: Hostname for the Route.
--path=string: URL path for the Route.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.94 - kf unbind-service

Revoke an App’s access to a service instance.

Name

kf unbind-service - Revoke an App’s access to a service instance.

Synopsis

kf unbind-service APP_NAME SERVICE_INSTANCE [flags]

Description

Unbind removes an App’s access to a service instance.

This will delete the credential from the service broker that created the instance and update the VCAP_SERVICES environment variable for the App to remove the reference to the instance.

Examples

kf unbind-service myapp my-instance

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for unbind-service

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.95 - kf unmap-route

Revoke an App’s access to receive traffic from the Route.

Name

kf unmap-route - Revoke an App’s access to receive traffic from the Route.

Synopsis

kf unmap-route APP_NAME DOMAIN [--hostname HOSTNAME] [--path PATH] [flags]

Description

Unmapping an App from a Route will cause traffic matching the Route to no longer be forwarded to the App.

The App may still receive traffic from an unmapped Route for a small period of time while the traffic rules on the gateways are propagated.

The Route will re-balance its routing weights so other Apps mapped to it will receive the traffic. If no other Apps are bound the Route will return a 404 HTTP status code.

Examples

# Unmap myapp.example.com from myapp in the targeted Space
kf unmap-route myapp example.com --hostname myapp

# Unmap the Route in a specific Space
kf unmap-route --space myspace myapp example.com --hostname myapp

# Unmap a Route with a path
kf unmap-route myapp example.com --hostname myapp --path /mypath

Flags

--async: Do not wait for the action to complete on the server before returning.
--destination-port=int32: Port on the App the Route will connect to.
-h, --help: help for unmap-route
--hostname=string: Hostname for the Route.
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.
--path=string: URL path for the Route.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.96 - kf unset-env

Delete an environment variable on an App.

Name

kf unset-env - Delete an environment variable on an App.

Synopsis

kf unset-env APP_NAME ENV_VAR_NAME [flags]

Description

Removes an environment variable by name from an App. Any existing environment variable(s) on the App with the same name will be removed.

Environment variables that are set by Kf or on the Space will be unaffected.

Apps will be updated without downtime.

Examples

kf unset-env myapp FOO

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for unset-env
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.97 - kf unset-space-role

Unassign a Role to a Subject.

Name

kf unset-space-role - Unassign a Role to a Subject.

Synopsis

kf unset-space-role SUBJECT_NAME ROLE [flags]

Examples

# Unassign a User to a Role
kf unset-space-role john@example.com SpaceDeveloper

# Unassign a Group to a Role
kf unset-space-role my-group SpaceAuditor -t Group

# Unassign a ServiceAccount to a Role
kf unset-space-role my-sa SpaceAuditor -t ServiceAccount

Flags

-h, --help: help for unset-space-role
-t, --type=string: Type of subject, valid values are Group|ServiceAccount|User(default). (default "User")

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.98 - kf update-autoscaling-limits

Update autoscaling limits for App.

Name

kf update-autoscaling-limits - Update autoscaling limits for App.

Synopsis

kf update-autoscaling-limits APP_NAME MIN_INSTANCE_LIMIT MAX_INSTANCE_LIMIT [flags]

Examples

# Set min instances to 1, max instances to 3 for myapp
kf update-autoscaling-limits myapp 1 3

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for update-autoscaling-limits
--no-short-circuit-wait: Allow the CLI to skip waiting if the mutation does not impact a running resource.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.99 - kf update-user-provided-service

Update a standalone service instance with new credentials.

Name

kf update-user-provided-service - Update a standalone service instance with new credentials.

Synopsis

kf update-user-provided-service SERVICE_INSTANCE [-p CREDENTIALS] [-t TAGS] [flags]

Description

Updates the credentials stored in the Kubernetes Secret for a user-provided service. These credentials will be propagated to Apps.

Apps may need to be restarted to receive the updated credentials.

Examples

# Update an existing database service
kf update-user-provided-service db-service -p '{"url":"mysql://..."}'

# Update a service with tags for autowiring
kf update-user-provided-service db-service -t "mysql,database,sql"

Flags

--async: Do not wait for the action to complete on the server before returning.
-h, --help: help for update-user-provided-service
-p, --params=string: Valid JSON object containing service-specific configuration parameters, provided in-line or in a file. (default "{}")
-t, --tags=string: Comma-separated tags for the service instance.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.100 - kf vcap-services

Print the VCAP_SERVICES environment variable for an App.

Name

kf vcap-services - Print the VCAP_SERVICES environment variable for an App.

Synopsis

kf vcap-services APP_NAME [flags]

Examples

kf vcap-services my-app

Flags

-h, --help: help for vcap-services

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.101 - kf version

Print the CLI version.

Name

kf version - Print the CLI version.

Synopsis

kf version [flags]

Examples

  kf version

Flags

-h, --help: help for version

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.102 - kf wrap-v2-buildpack

Create a V3 buildpack that wraps a V2 buildpack.

Name

kf wrap-v2-buildpack - Create a V3 buildpack that wraps a V2 buildpack.

Synopsis

kf wrap-v2-buildpack NAME V2_BUILDPACK_URL|PATH [flags]

Description

Creates a V3 buildpack that wraps a V2 buildpack.

The resulting buildpack can then be used with other V3 buildpacks by creating a builder. See https://buildpacks.io/docs/operator-guide/create-a-builder/ for more information.

A V3 buildpack is packaged as an OCI container. If the –publish flag is provided, then the container will be published to the corresponding container repository.

This command uses other CLIs under the hood. This means the following CLIs need to be available on the path:

go
git
pack
cp
unzip

We recommend using Cloud Shell to ensure these CLIs are available and the correct version.

Examples

# Download buildpack from the given git URL. Uses the git CLI to
# download the repository.
kf wrap-v2-buildpack gcr.io/some-project/some-name https://github.com/some/buildpack

# Creates the buildpack from the given path.
kf wrap-v2-buildpack gcr.io/some-project/some-name path/to/buildpack

# Creates the buildpack from the given archive file.
kf wrap-v2-buildpack gcr.io/some-project/some-name path/to/buildpack.zip

Flags

--builder-repo=string: Builder repo to use. (default "github.com/poy/buildpackapplifecycle/builder")
--buildpack-stacks=stringArray: Stack(s) this buildpack will be compatible with. (default [google])
--buildpack-version=string: Version of the resulting buildpack. This will be used as the image tag. (default "0.0.1")
-h, --help: help for wrap-v2-buildpack
--launcher-repo=string: Launcher repo to use. (default "github.com/poy/buildpackapplifecycle/launcher")
--output-dir=string: Output directory for the buildpack data (before it's packed). If left empty, a temp dir will be used.
--publish: Publish the buildpack image.

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

4.103 - kf xargs-apps

Run a command for every App.

Name

kf xargs-apps - Run a command for every App.

Synopsis

kf xargs-apps [flags]

Description

Run a command for every App in targeted spaces.

Examples

# Example: restart all apps in all spaces
kf xargs-apps --all-namespaces -- kf restart {{.Name}} --space {{.Space}}

# Example: restage all apps in all spaces
kf xargs-apps --all-namespaces -- kf restage {{.Name}} --space {{.Space}}

# Example: stop all apps in spaces 'space1' and 'space2'
kf xargs-apps --space space1,space2 -- kf stop {{.Name}} --space {{.Space}}

# Example: use kubectl to label all apps in the default space
kf xargs-apps -- kubectl label apps -n {{.Space}} {{.Name}} environment=prod

Flags

--all-namespaces: Enables targeting all spaces in the cluster.
--dry-run: Enables dry-run mode, commands are printed but will not be executed. (default true)
-h, --help: help for xargs-apps
--resource-concurrency=int: Number of apps within a space that may be operated on in parallel. Total concurrency will be upto space-concurrency * app-concurrency. -1 for no limit. (default 1)
--space-concurrency=int: Number of spaces that may be operated on in parallel. -1 for no limit. (default -1)

Inherited flags

These flags are inherited from parent commands.

--as=string: Username to impersonate for the operation.
--as-group=strings: Group to impersonate for the operation. Include this flag multiple times to specify multiple groups.
--config=string: Path to the Kf config file to use for CLI requests.
--kubeconfig=string: Path to the kubeconfig file to use for CLI requests.
--log-http: Log HTTP requests to standard error.
--space=string: Space to run the command against. This flag overrides the currently targeted Space.

5 - Migrate to Kubernetes

Learn how to migrate off Kf to Kubernetes or another container runtime.

Kf is designed to be used as a waypoint on a full migration to Kubernetes or another container runtime.

Once your platform team is comfortable using and managing Kubernetes you can use the guides through this section to plan a full migration to Kubernetes while you upskill application developers.

5.1 - Build Container Images

Learn how to build container images that can be used with Kf and Kubernetes.

The first step you’ll want to take when migrating from Kf to another container runtime is to change the way your software is built so you build containers rather than relying on Kf to do it on your behalf.

Best practices for container runtimes differ from Cloud Foundry’s application best practices so you may need to change some organizational processes as well as technical processes.

Best practices

Images should be small. Images need to be pulled over a network potentially every time the container starts. Lager images cost more to store, transfer, and require bigger disks on the machines you run the image from.
Images should have few dependencies. More dependencies creates larger images and a larger attack surface. A base image like distroless or alpine can reduce your size while maintaining critical components like security certificates.
Rebuild rather than patch images. Images are built in layers so patching either increases the size of the image by applying the patches to a new layer or replaces lower layers which were used to produce upper layers which is unsafe.
Images should be built reproducibly. Reproducible builds not only give confidence in supply chain security, but also reduce storage cost by allowing better deduplication of layers shared between images.
Containers should be non-root. A root process in a container will have the same privilege as root on the host if it escapes the container. Non-root containers mean both a privilege escalation exploit and a container escape exploit are necessary to compromise the host.
Reference images by SHA, not tag. Referencing images by tag, or the implicit tag latest, means that your builds or runtimes may not be reproducible.

Differences from Cloud Foundry and Kf

In Cloud Foundry and Kf it’s common to push in every environment which produces a separate build and image. This is a common source of errors if an application development team hasn’t pinned the dependencies for their application correctly.

In Cloud Foundry and Kf this pushing provides an additional purpose, a platform operations team can change buildpacks or base images in an environment to patch applications on the fly. This has always been risky, but large base images like cflinuxfs3, come with many security vulnerabilities.

Things to consider:

Applications should be built with the dependencies they need for all environments. They shouldn’t depend on the environment like the Spring Autoreconfiguration buildpack.
Teams should be able to patch and promote an image quickly up through environments as the preferred method of patching.
Building on smaller base images than cflinuxfs3 may leave teams without tools they expect to use when using ssh to debug containers or may expose hidden dependencies.
Cloud Foundry and Kf add a launcher process responsible for reading Procfiles and certain environment variables. Applications will need to stop using these, or you should create a drop-in replacement launcher.
Operations teams should monitor deployed images for vulnerabilities.

Approaches

There are two major approaches to move from V2 buildpacks to building containers.

Cloud Native Buildpacks

Cloud Native Buildpacks (CNBs) are a joint effort by Pivotal and Heroku to bring a similar buildpack based development experience to containers. The project belongs to the CNCF and has a wide variety of buildpacks available.

The Cloud Foundry Foundation created Paketo a set of buildpacks that work together which are the next generation of the v2 buildpacks that Cloud Foundry uses. They’re also based on Ubuntu meaning migration should be less risky for CF applications.

It’s possible for an operations team to customize buildpack pipelines with company specific base images or buildpacks.

Images built with buildpacks are designed to be rebased on newer operating system layers, similar to what is done in Cloud Foundry. If building new images and promoting them is too much of an organizational hurdle, this may be a good option for patching large-scale vulnerabilities across a fleet of deployed applications.

Cloud Native Buildpacks can also achieve similar performance to Cloud Foundry builds using caches, which are separate container images created by the pipeline.

Developers can run the Cloud Native Buildacks locally to build and test images as they’d appear in production to reduce the number of failed builds.

Dockerfile

Dockerfiles are the second major option for producing container images. Dockerfiles specify a set of commands that are executed in a container. Each command creates a new layer that captures the delta from the layer before. The end result is an image.

Dockerfiles are very flexible and can be used to reliably reproduce nearly any build process. In fact, the same process Kf uses to execute v2 buildpacks can be turned into a Dockerfile.

Historically, Dockerfiles needed a root daemon on the host machine and additional permissions for each user to connect to it, but there are now alternatives that can run in userspace rootless like Podman.

Building Dockerfiles that rely on untrusted images with a Docker daemon running as root is risky. You should either restrict the network on build machines to only allow pulling trusted images and/or use a rootless container build engine that can execute Dockerfiles like Kaniko or Buildah. Kf uses Kaniko, but in certain circumstances it can be slow if it has to handle many files.

To reduce the need for developers to author Dockerfiles, a platform operations team may create steps in their build pipeline that containerize the existing output that would normally be fed to Cloud Foundry or Kf.

Rebasing containers created by Dockerfiles is possible with crane if you know which image was used to produce the container, but it’s risky unless you can guarantee all items in the new image are ABI compatible with the previous layer.

Migration path

Kf supports deploying applications from container images. You can migrate to building images rather than intermediate formats like JARs and change your deployment pipelines to use those images instead.

This will also reduce the resources consumed on your Kf clusters by build processes and the need to store source containers.

Additional resources

Learn how to pull large images faster on GKE using image streaming.
Learn about how reproducible builds help your security posture.
Learn how to attach debugging tools to existing Kubernetes pods to reduce the footprint of your images.

6 - Troubleshooting

Troubleshoot Kf resources or installation.

6.1 - Temporarily stop reconciliation

Stop resource reconciliation to change values for debugging.

Sometimes it may be necessary to stop the Kf controller or Kf operator from changing settings on a Kubernetes resource if you need to change values on it for debugging.

Operator managed resources

Resources managed by the Kf operator, usually found in the kf namespace, can have reconciliation disabled using the operator.knative.dev/mode annotation:

Value	Behavior
`EnsureExists`	The resource will be created if it doesn’t exist, but values won’t be overwritten.
`Reconcile` or blank	The resource will be created if it doesn’t exist, and values will be overwritten.

To change values on operator managed resources for testing:

Make a local copy of the resource so you can revert it later.
Disable reconciliation by setting the annotation to EnsureExists.
Example disabling reconciliation on the Kf controller
```
kubectl annotate --overwrite -n kf deployment controller operator.knative.dev/mode=EnsureExists
```
Update the resource as needed for testing.
When done, restore the annotation to the original value and optionally restore the changed values.

Kf managed resources

Kf managed resources are child resources created in response to configuration set on Spaces, Apps, Builds, etc. These usually have the label app.kubernetes.io/managed-by: kf.

You can disable reconciliation by removing the metadata.ownerReference that references the parent Kf resource. Save a copy of the field so you can add it back when you’re done.

Make a local copy of the resource so you can revert it later.

Disable reconciliation by removing the metadata.ownerReference field.

Example editing a Namespaced owned by a Kf Space:

$ kubectl edit namespace test

Empty the metadata.ownerReference field:

apiVersion: v1
kind: Namespace
metadata:
    creationTimestamp: "2023-04-19T20:34:50Z"
    name: test
    ownerReferences: []

Update the resource as needed for testing.
When done, restore the resource to the original value.

6.2 - Troubleshoot Apps

Use these steps to troubleshoot various issues that can occur when using Kf Apps.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get apps.kf.dev -n SPACE_NAME APP_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME app/APP_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get apps.kf.dev -n SPACE_NAME APP_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME app/APP_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get apps.kf.dev -n SPACE_NAME APP_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME app/APP_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

To understand the root cause, user can read the longer error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe serviceinstance my-service -n my-space.

Object has ChildNotOwned error (Name conflicts)

This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists.

To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACEcommand. For example, kf delete-space my-space or kubectl delete space my-space.

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

6.3 - Troubleshoot Builds

Use these steps to troubleshoot various issues that can occur when using Kf Builds.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get builds.kf.dev -n SPACE_NAME BUILD_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME build/BUILD_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get builds.kf.dev -n SPACE_NAME BUILD_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME build/BUILD_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get builds.kf.dev -n SPACE_NAME BUILD_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME build/BUILD_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

6.4 - Troubleshoot ClusterServiceBrokers

Use these steps to troubleshoot various issues that can occur when using Kf ClusterServiceBrokers.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get clusterservicebrokers.kf.dev CLUSTERSERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor clusterservicebroker/CLUSTERSERVICEBROKER_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get clusterservicebrokers.kf.dev CLUSTERSERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor clusterservicebroker/CLUSTERSERVICEBROKER_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get clusterservicebrokers.kf.dev CLUSTERSERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor clusterservicebroker/CLUSTERSERVICEBROKER_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

6.5 - Troubleshoot Routes

Use these steps to troubleshoot various issues that can occur when using Kf Routes.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get routes.kf.dev -n SPACE_NAME ROUTE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME route/ROUTE_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get routes.kf.dev -n SPACE_NAME ROUTE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME route/ROUTE_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get routes.kf.dev -n SPACE_NAME ROUTE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME route/ROUTE_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

6.6 - Troubleshoot ServiceBrokers

Use these steps to troubleshoot various issues that can occur when using Kf ServiceBrokers.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get servicebrokers.kf.dev -n SPACE_NAME SERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME servicebroker/SERVICEBROKER_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get servicebrokers.kf.dev -n SPACE_NAME SERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME servicebroker/SERVICEBROKER_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get servicebrokers.kf.dev -n SPACE_NAME SERVICEBROKER_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME servicebroker/SERVICEBROKER_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

6.7 - Troubleshoot ServiceInstanceBindings

Use these steps to troubleshoot various issues that can occur when using Kf ServiceInstanceBindings.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstancebindings.kf.dev -n SPACE_NAME SERVICEINSTANCEBINDING_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstancebinding/SERVICEINSTANCEBINDING_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstancebindings.kf.dev -n SPACE_NAME SERVICEINSTANCEBINDING_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstancebinding/SERVICEINSTANCEBINDING_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstancebindings.kf.dev -n SPACE_NAME SERVICEINSTANCEBINDING_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstancebinding/SERVICEINSTANCEBINDING_NAME

Possible Cause Solution

Object has TemplateError

Possible Cause	Solution
Object has TemplateError	This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.
Object has ChildNotOwned error (Name conflicts)	This error usually means that the object(s) the controller is trying to create already exists. This happens if the user created a K8s resource that has the same name as what the controller is trying to create; but more often it happens if user deletes a resource then Kf controller tries to re-create it. If a child resource is still hanging around, its owner will be the old resource that no longer exists. To recover from the error, it is recommended that user deletes the impacted resource and then recreates it. To delete the object, use a Kf deletion command or use the `kubectl delete RESOURCE_TYPE RESOURCE_NAME -n SPACE`command. For example, `kf delete-space my-space` or `kubectl delete space my-space`. To recreate a resource, use a Kf command. For example: `kf create-space my-space`.
Object has ReconciliationError	This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource. To understand the root cause, user can read the longer error message in the object’s `status.conditions` using the command:`kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space`. For example: `kubectl describe serviceinstance my-service -n my-space`.

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

6.8 - Troubleshoot ServiceInstances

Use these steps to troubleshoot various issues that can occur when using Kf ServiceInstances.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstances.kf.dev -n SPACE_NAME SERVICEINSTANCE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstance/SERVICEINSTANCE_NAME

Possible Cause Solution

Deletion timestamp is in the future.

Possible Cause	Solution
Deletion timestamp is in the future.	With clock skew the `metadata.deletionTimestamp` may still be in the future. Wait a few minutes to see if the object is deleted.
Finalizers exist on the object.	Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted. If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the `metadata.finalizers` array. To remove the finalizer from an object, use the `kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space` command. See using finalizers to control deletion to learn more. Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.
Dependent objects may exist.	The object may be waiting on dependents to be deleted before it is deleted. See the Kubernetes garbage collection guide to learn more. Have an administrator check all objects in the namespace and cluster to see if one of them is blocking deletion. If you need to remove the object without waiting for dependents, use `kubectl delete` with the cascade flag set to: `--cascade=orphan`.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstances.kf.dev -n SPACE_NAME SERVICEINSTANCE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstance/SERVICEINSTANCE_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstances.kf.dev -n SPACE_NAME SERVICEINSTANCE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstance/SERVICEINSTANCE_NAME

Possible Cause Solution

Object has TemplateError

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

Backing resource reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get serviceinstances.kf.dev -n SPACE_NAME SERVICEINSTANCE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME serviceinstance/SERVICEINSTANCE_NAME

Possible Cause Solution

Backing resource DeprovisionFailed error.

This error usually occurs when backing resources (MySQL database hosted at an external OSB server) fails to be deprovisioned. Kf can not safely determine if the dependent resource is deprovisioned.

To recover from the error, it is recommended that user reads the detail error message in the object’s status.conditions using the command:kubectl describe RESOURCE_TYPE RESOURCE_NAME -n space. For example: kubectl describe servipropogatesceinstance my-service -n my-space.

Once the error message is confirmed, have an administrator check the backing resource and clean it up manually. Once the backing resource is determined to be safely released, the impacted Kf resource can be reconciled successfully by manually removing the Finalizer from the object spec, use the kubectl edit serviceinstance my-service -n my-space command.

6.9 - Troubleshoot SourcePackages

Use these steps to troubleshoot various issues that can occur when using Kf SourcePackages.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get sourcepackages.kf.dev -n SPACE_NAME SOURCEPACKAGE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME sourcepackage/SOURCEPACKAGE_NAME

Possible Cause Solution

Deletion timestamp is in the future.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get sourcepackages.kf.dev -n SPACE_NAME SOURCEPACKAGE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME sourcepackage/SOURCEPACKAGE_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get sourcepackages.kf.dev -n SPACE_NAME SOURCEPACKAGE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME sourcepackage/SOURCEPACKAGE_NAME

Possible Cause Solution

Object has TemplateError

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

6.10 - Troubleshoot Spaces

Use these steps to troubleshoot various issues that can occur when using Kf Spaces.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get spaces.kf.dev SPACE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor space/SPACE_NAME

Possible Cause Solution

Deletion timestamp is in the future.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get spaces.kf.dev SPACE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor space/SPACE_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get spaces.kf.dev SPACE_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor space/SPACE_NAME

Possible Cause Solution

Object has TemplateError

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

6.11 - Troubleshoot Tasks

Use these steps to troubleshoot various issues that can occur when using Kf Tasks.

Object is stuck deleting.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get tasks.kf.dev -n SPACE_NAME TASK_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME task/TASK_NAME

Possible Cause Solution

Deletion timestamp is in the future.

With clock skew the metadata.deletionTimestamp may still be in the future. Wait a few minutes to see if the object is deleted.

Finalizers exist on the object.

Finalizers are present on the object, they must be removed by the controller that set them before the object is deleted.

If you want to force a deletion without waiting for the finalizers, edit the object to remove them from the metadata.finalizers array.

To remove the finalizer from an object, use the kubectl edit RESOURCE_TYPE RESOURCE_NAME -n my-space command.

See using finalizers to control deletion to learn more.

Warning: Removing finalizers without allowing the controllers to complete may cause errors, security issues, data loss, or orphaned resources.

Dependent objects may exist.

If you need to remove the object without waiting for dependents, use kubectl delete with the cascade flag set to: --cascade=orphan.

Object generation state drift.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get tasks.kf.dev -n SPACE_NAME TASK_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME task/TASK_NAME

Possible Cause	Solution
Object has generation version drift.	This error usually occurs Kf controller did not read the latest version of the object, this error is usually self-recovered once Kubernetes replicas reach eventual consistency, and it usually does not require action from users.

Object reconciliation failed.

Run the following command to get the resource information, then check for the causes listed below:

kubectl get tasks.kf.dev -n SPACE_NAME TASK_NAME -o yaml

The kf CLI can help check for some of the issues:

kf doctor --space SPACE_NAME task/TASK_NAME

Possible Cause Solution

Object has TemplateError

This error usually occurs if user has entered an invalid property in the custom resource Spec, or the configuration on the Space/Cluster is bad.

Object has ChildNotOwned error (Name conflicts)

To recreate a resource, use a Kf command. For example: kf create-space my-space.

Object has ReconciliationError

This error usually means that something has gone wrong with the HTTP call made (by Kf controller) to the Kubernetes API servier to create/update resource.

7 - Examples

Exapmles and demonstrations of special use-cases.

7.1 - Add X-VCAP-REQUEST-ID HTTP Headers.

Learn how to add additional Cloud Foundry style headers to your gateways.

Kf limits the headers it returns for security and network cost purposes.

If you have applications that need the X-VCAP-REQUEST-ID HTTP header and can’t be upgraded, then you can use Istio to add it to requests and responses to mimic CLoud Foundry’s gorouter.

To mimic this header, we can create an EnvoyFilter that does the following:

Watches for HTTP traffic coming into the gateway (make sure you target the same ingress gateway Kf is using).
Saves Envoy’s built-in request ID.
Copies that ID to the request.
Mutates the response with the same request ID.

You may need to make changes to this filter if:

You rely on the HTTP/1.1 Upgrade header.
You need these headers for mesh (East-West) traffic.
You only want to target a subset of applications.

Example

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: vcap-http-header
  # Set the namespace to match the gateway.
  namespace: asm-gateways
spec:
  # Set the workload selector to match the Istio ingress gateway
  # your domain targets and/or your workload
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
            subFilter:
              name: "envoy.filters.http.router"
    patch:
      operation: INSERT_BEFORE
      value:
       name: envoy.lua
       typed_config:
         "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
         inlineCode: |
          function envoy_on_request(request)
             local metadata = request:streamInfo():dynamicMetadata()
             -- Get Envoy's internal request ID
             local request_id = request:headers():get("x-request-id")

             if request_id ~= nil then
               -- Save the request ID for later and set it on the request
               -- for the application to conusme.
               metadata:set("envoy.filters.http.lua", "req.x-request-id", request_id)
               request:headers():add("x-vcap-request-id", request_id)
             end
           end

           function envoy_on_response(response)
             local metadata = response:streamInfo():dynamicMetadata():get("envoy.filters.http.lua")
             local request_id = metadata["req.x-request-id"]

             -- Set the value on the outbound response as well.
             if request_id ~= nil then
               response:headers():add("x-vcap-request-id", request_id)
             end
           end

7.2 - Deploy Docker apps with NFS UID/GID mapping.

Learn how to deploy Docker applications with NFS UID/GID mapping.

This document outlines how to do UID/GID mapping for NFS volumes within a Docker container. This may be necessary if your application uses NFS because Kubernetes assumes the UID and GID of the NFS volume map directly into the UID/GID namespace of your container.

To get around this limitation, Kf adds the mapfs binary to all continers it builds. The mapfs binary creates a FUSE filesystem that maps the UID and GID of a host container into the UID and GID of an NFS volume.

Prerequisites

In order for these operations to work:

Your container’s OS must be Linux.
Your container must have the coreutils timeout, sh, and wait installed.
Your container must have fusermount installed.

Update your Dockerfile

First, you’ll need to update your Dockerfile to add the mapfs binary to your application:

# Get the mapfs binrary from a version of Kf.
FROM gcr.io/kf-releases/fusesidecar:v2.11.14 as builder
COPY --from=builder --chown=root:vcap /bin/mapfs /bin/mapfs

# Allow users other than root to use fuse.
RUN echo "user_allow_other" >> /etc/fuse.conf
RUN chmod 644 /etc/fuse.conf

RUN chmod 750 /bin/mapfs
# Allow setuid so the mapfs binary is run as root.
RUN chmod u+s /bin/mapfs

Set manifest attributes

Next, you’ll have to update manifest attributes for your application. You MUST set args and entrypoint because they’ll be used by mapfs to launch the application.

Set args to be your container’s CMD
Set entrypoint to be your container’s ENTRYPOINT

applications:
- name: my-docker-app
  args: ["-jar", "my-app"]
  entrypoint: "java"
  dockerfile:
    path: gcr.io/my-application-with-mapfs

Note: If no entrypoint is specified in the manifest, Kf will use /lifecycle/entrypoint.bash.

Deploy your application

Once your Docker image and manifest are updated, you can deploy your application and check that your NFS volume mounting correctly in the container.

If something has gone wrong, you can debug it by getting the Deployment in Kubernetes with the same name as your application:

kubectl get deployment my-docker-app -n my-space -o yaml

Validate the command and args for the container named user-container look valid.

7.3 - Deploy Spring Cloud Config

Learn how to deploy Spring Cloud Config as a configuration server.

This document shows how to deploy Spring Cloud Config in a Kf cluster.

Spring Cloud Config provides a way to decouple application code from its runtime configuration. The Spring Cloud Config configuration server can read configuration files from Git repositories, the local filesystem, HashiCorp Vault servers, or Cloud Foundry CredHub. Once the configuration server has read the configuration, it can format and serve that configuration as YAML, Java Properties, or JSON over HTTP.

Before you begin

You will need a cluster with Kf installed and access to the Kf CLI.

Additionally, you will need the following software:

git: Git is required to clone a repository.

Download the Spring Cloud Config configuration server

To download the configuration server source:

Open a terminal.

Clone the source for the configuration server:

git clone --depth 1 "https://github.com/google/kf"

Configure and deploy a configuration server

To update the settings for the instance:

Change directory to spring-cloud-config-server:
```
cd kf/spring-cloud-config-server
```
Open manifest.yaml.
Change the GIT_URI environment variable to the URI of your Git configuration server.
Optionally, change the name of the application in the manifest.
Optionally, configure additional properties or alternative property sources by editing src/main/resources/application.properties.
Deploy the configuration server without an external route. If you changed the name of the application in the manifest, update it here:
```
kf push --no-route spring-cloud-config
```

Note: The default configuration is not production ready. The README.md file contains additional steps to take if you want to deploy the application to production.

Bind applications to the configuration server

You can create a user provided service to bind the deployed configuration server to other Kf applications in the same cluster or namespace.

How you configure them will depend on the library you use:

Applications using Pivotal’s Spring Cloud services client library

Existing PCF applications that use Pivotal’s Spring Cloud Services client library can be bound using the following method:

Create a user provided service named config-server. This step only has to be done once per configuration server:
```
kf cups config-server -p '{"uri":"http://spring-cloud-config"}' -t configuration
```
Note: If you want to use a configuration server in a different space, change the URI to include the space.
For each application that needs get credentials, run:
```
kf bind-service application-name config-server

kf restart application-name
```
This will create an entry into the VCAP_SERVICES environment variable for the configuration server.

Other applications

Applications that can connect directly to a Spring Cloud Config configuration server should be configured to access it using its cluster internal URI:

http://spring-cloud-config

For Spring applications that use the Spring Cloud Config client library can set the spring.cloud.config.uri property in the appropriate location for your application. This is usually an application.properties or application.yaml file.
For other frameworks, see your library’s reference information.

Delete the configuration server

To remove a configuration server:

Remove all bindings to the configuration server running the following commands for each bound application:
```
kf unbind-service application-name config-server

kf restart application-name
```
Remove the service entry for the configuration server:
```
kf delete-service config-server
```
Delete the configuration server application:
```
kf delete spring-cloud-config
```

What’s next

Read more about the types of configuration sources Spring Cloud Config supports.
Learn about the structure of the VCAP_SERVICES environment variable to understand how it can be used for service discovery.

7.4 - Deploy Spring Music

Learn to deploy an app with backing services.

These instructions will walk you through deploying the Cloud Foundry Spring Music reference App using the Kf Cloud Service Broker.

Building Java Apps from source: The Spring Music source will be built on the cluster, not locally.
Service broker integration: You will create a database using the Kf Cloud Service Broker and bind the Spring Music App to it.
Spring Cloud Connectors: Spring Cloud Connectors are used by the Spring Music App to detect things like bound CF services. They work seamlessly with Kf.
Configuring the Java version: You will specify the version of Java you want the buildpack to use.

Prerequisites

Install and configure the Kf Cloud Service Broker.

Deploy Spring Music

Clone source

Clone the Spring Music repo.

git clone https://github.com/cloudfoundry-samples/spring-music.git spring-music

cd spring-music

Edit manifest.yml, and replace path: build/libs/spring-music-1.0.jar with stack: org.cloudfoundry.stacks.cflinuxfs3. This instructs Kf to build from source using cloud native buildpacks so you don’t have to compile locally.

---
applications:
- name: spring-music
  memory: 1G
  random-route: true
  stack: org.cloudfoundry.stacks.cflinuxfs3
  env:
    JBP_CONFIG_SPRING_AUTO_RECONFIGURATION: '{enabled: false}'
#    JBP_CONFIG_OPEN_JDK_JRE: '{ jre: { version: 11.+ } }'

Push Spring Music with no bindings

Create and target a Space.

kf create-space test

kf target -s test

Deploy Spring Music.
```
kf push spring-music
```
Use the proxy feature to access the deployed App.
Start the proxy:
```
kf proxy spring-music
```
Open http://localhost:8080 in your browser:
The deployed App includes a UI element showing which (if any) Spring profile is being used. No profile is being used here, indicating an in-memory database is in use.

Create and bind a database

Create a PostgresSQL database from the marketplace.
Note: You must set the COMPUTE_REGION and VPC_NAME variables so Kf Cloud Service Broker knows where to provision your instance, and authorize the VPC Kf Apps run on to access it.
```
kf create-service csb-google-postgres small spring-music-postgres-db -c '{"region":"COMPUTE_REGION","authorized_network":"VPC_NAME"}'
```

Bind the Service with the App.

kf bind-service spring-music spring-music-postgres-db

Restart the App to make the service binding available via the VCAP_SERVICES environment variable.
```
kf restart spring-music
```
(Optional) View the binding details.
```
kf bindings
```
Verify the App is using the new binding.
1. Start the proxy:
```
kf proxy spring-music
```
2. Open http://localhost:8080 in your browser:
  You now see the Postgres profile is being used, and we see the name of our Service we bound the App to.

Clean up

Unbind and delete the PostgreSQL service:

kf unbind-service spring-music spring-music-db

kf delete-service spring-music-db

Delete the App:
```
kf delete spring-music
```

7.5 - Deploy With Offline Java Buildpack

Learn how to install and use an offline Java Buildpack (provided by Cloud Foundry) to compile your Java apps.

This document shows how to use an offline Java Buildpack to deploy your applications.

Cloud Foundry’s Java buildpack uses a number of large dependencies. At the time of writing, ~800 MB of sources are pulled into the builder during the buildpack’s execution. Much of this data is brought in from the internet which is convenient as it keeps the buildpack itself small but introduces a great deal of data transfer.

Java builds can be optimized to reduce outside network ingress and improve performance by hosting your own Java buildpack compiled in an offline mode. In its offline mode, Cloud Foundry’s Java buildpack downloads packages that may be used into the cache when creating the builder. This avoids pulling dependencies from the internet at runtime and makes the builder image self contained.

Before You Begin

You will need a cluster with Kf installed and access to the Kf CLI

Additionally, you will need access to the following software:

git: Git is required to clone a repository.
ruby: Ruby is required to create the Java bulidpack.
bundle: Ruby package manager to install Java buildpack dependencies.

Compile the Java Buildpack in Offline mode

Follow Cloud Foundry’s instructions to compile the Java Buildpack in Offline mode: https://github.com/cloudfoundry/java-buildpack/blob/main/README.md#offline-package.

These instructions will generate a .zip extension file containing the buildpack and its dependencies.

Deploy the Java Buildpack

Self Hosted on Kf (Recommended)

Once you have built the Java Buildpack, you can host it on your Kf cluster using a staticfile buildpack.

Create a new directory for your static file Kf app i.e. java-buildpack-staticapp
Create a new manifest in that directory manifest.yml with the contents:

---
applications:
  - name: java-buildpack-static

Create an emptyfile named Staticfile (case sensitive) in the directory. This indicates that this is a static app to the staticfile buildpack which will create a small image containing the contents of the directory + an nginx installation to host your files.
Copy the buildpack you created in the previous step into the directory as java-buildpack-offline-<hash>.zip. Your directory structure at this point should resemble:

/
├── java-buildpack-offline-fe26136c.zip
├── manifest.yml
└── Staticfile

Run kf push to deploy your app, this will trigger a build.
When the push finishes, run kf apps and take note of the URL your app is running on. You will reference this in the next step.

See internal routing to construct an internal link to your locally hosted buildpack. The URL should resemble <your route>/java-buildpack-offline-<hash>.zip. Take note of this URL as you will reference it later in your application manifests or in your cluster’s buildpack configuration.

Served from Google Cloud Storage

See the Google Cloud documentation on creating public objects in a Cloud Storage bucket: https://cloud.google.com/storage/docs/access-control/making-data-public.

To find the URL where your buildpack is hosted and that you will reference in your manifests, see: https://cloud.google.com/storage/docs/access-public-data.

Using an Offline Buildpack

Whole Cluster

You can apply your buildpack to the whole cluster by updating the cluster’s configuration. See configuring stacks to learn how to register your buildpack URL for an entire cluster.

In App Manifest

To use your offline buildpack in a specific app, update your application manifest to specify the buildpack explicitly i.e.

---
applications:
- name: my-app
  buildpacks:
    - http://<your host>/java-buildpack-offline-<hash>.zip

Documentation

1 - Getting Started

1.1 - Getting Started Overview

Get started with a quickstart

Get an introduction to key concepts

1.2 - Quickstart

Push an application

Prerequisites

Prepare space

Push the Cloud Foundry test app

Successful app push on Kf

Clean up

1.3 - Compare Cloud Foundry and Kf services

2 - Develop Applications on Kf

2.1 - Build and deploy applications

2.1.1 - Deploy an application

Source code upload

Ignore files during push

Build

Run

Push timeouts

2.1.2 - Get started with buildpacks

Before you begin

Java (v2) buildpack

Java (v3) buildpack

Python (v2) buildpack

Python (v3) buildpack

Staticfile (v2) buildpack

2.1.3 - Reduce deployment risk with blue-green deployments

Push the initial App

Push the updated App

Add routes to the updated App

Shift traffic

Complete traffic shifting

Turn down the original App

2.1.4 - Compare V2 and V3 Buildpacks

V2 buildpacks

Pros

Cons

V3 buildpacks

Builder image

Run image

Layers

Pros

Cons

Kf Stacks

View Stacks

V2 to V3 Buildpack Migration

Option 1: Migrate Apps built with standard V2 buildpacks

Option 2: Migrate Apps built with custom V2 buildpacks

2.1.5 - App Manifest

Format

Application fields

Docker fields

Route fields

Port fields

Metadata fields

Probe fields

TCPSocketAction fields

HTTPGetAction fields

Examples

Minimal App

Simple App

Docker App

App with multiple ports

Health check types

Known differences

2.1.6 - App runtime

Environment variables

VCAPServices

VCAPService

VCAP_APPLICATION

2.1.7 - Build runtime

Environment variables

2.2 - Backing services

2.2.1 - Backing Services Overview

Backing services in Kf

2.2.2 - Use managed services

Find a service

Provision a service