Deploy Kf Cloud Service Broker

This page shows you how to deploy Kf Cloud Service Broker and use it to provision or deprovision backing resources. Read about the concepts and architecture to learn more about the Kf Cloud Service Broker.

Create environment variables

• Linux

  export PROJECT_ID=YOUR_PROJECT_ID
  export CLUSTER_PROJECT_ID=YOUR_PROJECT_ID
  export CSB_IMAGE_DESTINATION=YOUR_DOCKER_REPO_CSB_PATH
  export CLUSTER_NAME=kf-cluster
  export INSTANCE_NAME=cloud-service-broker
  export COMPUTE_REGION=us-central1
  

• Windows PowerShell

  Set-Variable -Name PROJECT_ID -Value YOUR_PROJECT_ID
  Set-Variable -Name CLUSTER_PROJECT_ID -Value YOUR_PROJECT_ID
  Set-Variable -Name CSB_IMAGE_DESTINATION=YOUR_DOCKER_REPO_CSB_PATH
  Set-Variable -Name CLUSTER_NAME -Value kf-cluster
  Set-Variable -Name INSTANCE_NAME -Value cloud-service-broker
  Set-Variable -Name COMPUTE_REGION -Value us-central1
  

Build the broker

First you’ll want to download and build the broker and push it to your container registry:

git clone --single-branch --branch main https://github.com/google/kf.git kf
cd kf/samples/cloud-service-broker
docker build --tag ${CSB_IMAGE_DESTINATION} .
docker push ${CSB_IMAGE_DESTINATION}

Set up the Kf Cloud Service Broker database

1. Create a MySQL instance.

   gcloud sql instances create ${INSTANCE_NAME} --cpu=2 --memory=7680MB --require-ssl --region=${COMPUTE_REGION}
   

2. Create a database named servicebroker in the MySQL instance.

   gcloud sql databases create servicebroker -i ${INSTANCE_NAME}

3. Create a username and password to be used by the broker.

   gcloud sql users create csbuser -i ${INSTANCE_NAME} --password=csbpassword

Set up a Google Service Account for the broker

1. Create a Google Service Account.

    gcloud iam service-accounts create csb-${CLUSTER_NAME}-sa \
        --project=${CLUSTER_PROJECT_ID} \
        --description="GSA for CSB at ${CLUSTER_NAME}" \
        --display-name="csb-${CLUSTER_NAME}"

2. Grant roles/cloudsql.client permissions to the Service Account. This is required to connect the service broker pod to the CloudSQL for MySQL instance through the CloudSQL Proxy.

    gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
        --member="serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
        --role="roles/cloudsql.client"

3. Grant additional Google Cloud permissions to the Service Account.

    gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
        --member="serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
        --role="roles/compute.networkUser"

    gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
        --member="serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
        --role="roles/cloudsql.admin"

    gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
        --member="serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
        --role="roles/redis.admin"

4. Verify the permissions.

    gcloud projects get-iam-policy ${CLUSTER_PROJECT_ID} \
        --filter='bindings.members:serviceAccount:"CSB_SERVICE_ACCOUNT_NAME"' \
        --flatten="bindings[].members"

Set up Workload Identity for the broker

1. Bind the Google Service Account with the Kubernetes Service Account.

    gcloud iam service-accounts add-iam-policy-binding "csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
        --project=${CLUSTER_PROJECT_ID} \
        --role="roles/iam.workloadIdentityUser" \
        --member="serviceAccount:${CLUSTER_PROJECT_ID}.svc.id.goog[kf-csb/csb-user]"

2. Verify the binding.

    gcloud iam service-accounts get-iam-policy "csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
        --project=${CLUSTER_PROJECT_ID}

Set up a Kubernetes Secret to share configuration with the broker

1. Create a config.yml file.

   cat << EOF >> ./config.yml
   gcp:
     credentials: ""
     project: ${CLUSTER_PROJECT_ID}
   
   db:
     host: 127.0.0.1
     password: csbpassword
     user: csbuser
     tls: false
   api:
     user: servicebroker
     password: password
   EOF
   

2. Create the kf-csb namespace.

   kubectl create ns kf-csb
   

3. Create the Kubernetes Secret.

   kubectl create secret generic csb-secret --from-file=config.yml -n kf-csb
   

Install the Kf Cloud Service Broker

1. Copy the kf-csb-template.yaml into kf-csb.yaml for working:

   cp kf-csb-template.yaml /tmp/kf-csb.yaml
   

2. Edit /tmp/kf-csb.yaml and replace placeholders with final values. In the example below, sed is used.

   sed -i "s|<GSA_NAME>|csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com|g" /tmp/kf-csb.yaml
   
   sed -i "s|<INSTANCE_CONNECTION_NAME>|${CLUSTER_PROJECT_ID}:${COMPUTE_REGION}:${INSTANCE_NAME}|g" /tmp/kf-csb.yaml
   
   sed -i "s|<DB_PORT>|3306|g" /tmp/kf-csb.yaml
   
   sed -i "s|<CSB_IMAGE_DESTINATION>|${CSB_IMAGE_DESTINATION}|g" /tmp/kf-csb.yaml
   

3. Apply yaml for Kf Cloud Service Broker.

   kubectl apply -f /tmp/kf-csb.yaml
   

4. Verify the Cloud Service Broker installation status.

   kubectl get pods -n kf-csb
   

Create a service broker

kf create-service-broker cloud-service-broker servicebroker password http://csb-controller.kf-csb/

Validate installation

Check for available services in the marketplace.

kf marketplace

If everything is installed and configured correctly, you should see the following:

$ kf marketplace

Broker                Name                          Namespace  Description
cloud-service-broker  csb-google-bigquery                      A fast, economical and fully managed data warehouse for large-scale data analytics.
cloud-service-broker  csb-google-dataproc                      Dataproc is a fully-managed service for running Apache Spark and Apache Hadoop clusters in a simpler, more cost-efficient way.
cloud-service-broker  csb-google-mysql                         Mysql is a fully managed service for the Google Cloud Platform.
cloud-service-broker  csb-google-postgres                      PostgreSQL is a fully managed service for the Google Cloud Platform.
cloud-service-broker  csb-google-redis                         Cloud Memorystore for Redis is a fully managed Redis service for the Google Cloud Platform.
cloud-service-broker  csb-google-spanner                       Fully managed, scalable, relational database service for regional and global application data.
cloud-service-broker  csb-google-stackdriver-trace             Distributed tracing service
cloud-service-broker  csb-google-storage-bucket                Google Cloud Storage that uses the Terraform back-end and grants service accounts IAM permissions directly on the bucket.

Clean up

1. Delete cloud-service-broker.

   kf delete-service-broker cloud-service-broker
   

2. Delete CSB components.

   kubectl delete ns kf-csb
   

3. Delete the broker’s database instance.

    gcloud sql instances delete ${INSTANCE_NAME} --project=${CLUSTER_PROJECT_ID}

4. Remove the IAM policy bindings.

    gcloud projects remove-iam-policy-binding ${CLUSTER_PROJECT_ID} \
    --member='serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com' \
    --role=roles/cloudsql.client

    gcloud projects remove-iam-policy-binding ${CLUSTER_PROJECT_ID} \
    --member='serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com' \
    --role=roles/compute.networkUser

    gcloud projects remove-iam-policy-binding ${CLUSTER_PROJECT_ID} \
    --member='serviceAccount:csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com' \
    --role=roles/redis.admin

5. Remove the GSA.

    gcloud iam service-accounts delete csb-${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com \
      --project=${CLUSTER_PROJECT_ID}