Provision PCF Healthwatch¶

In this lab, you augment the provision-sandbox pipeline with jobs to upload, stage, and configure PCF Healthwatch.

PCF Healthwatch helps monitor the health of your foundation.

Update the pipeline¶

Author two new jobs:

upload-and-stage-healthwatch
configure-healthwatch

Other than the fact that these jobs work with a different product (Healthwatch instead of PAS), these are identical to the existing PAS jobs, upload-and-stage-pas, and configure-pas.

Review ~/workspace/platform-config/download-product/healthwatch.yml.

Verify that the download product config file for healthwatch was augmented to include gcs blobstore details

Verify that the blobstore-stemcell-path matches the path in gcs where the healthwatch stemcell is stored.
Pattern the jobs after their existing PAS counterparts.
The trigger on upload-and-stage-healthwatch is conditional upon the presence of a new version of the Healthwatch product in the blobstore.
- Declare a Concourse resource for the healthwatch product that resides in the gcs blobstore.
- Configure the triggers on each job as a function of this healthwatch resource.
apply-changes needs to trigger not only when a new version of PAS has been configured, but also when a new version of healthwatch has been configured.
- In apply-changes, add to the in_parallel block a get step to get the healthwatch product, skipping the download, with a trigger: true, conditioned upon configure-healthwatch passing.

Consult the reference below if you need help.

Healthwatch Diff

Here are the additions to the provision pipeline presented in diff form:

index 477aa2f..ece1912 100644
--- a/provision-sandbox.yml
+++ b/provision-sandbox.yml
@@ -51,6 +51,13 @@ resources:
     json_key: ((json_key))
     regexp: srt-(.*).pivotal

+- name: healthwatch-product
+  type: gcs
+  source:
+    bucket: ((bucket))
+    json_key: ((json_key))
+    regexp: \[.*\]p-healthwatch-(.*).pivotal
+
 jobs:
 - name: install-opsman
   serial: true
@@ -293,6 +300,60 @@ jobs:
     params:
       ENV_FILE: sandbox/env.yml

+- name: upload-and-stage-healthwatch
+  serial: true
+  serial_groups: [ install ]
+  plan:
+  - in_parallel:
+    - get: platform-automation-tasks
+      params: { unpack: true }
+    - get: platform-automation-image
+      params: { unpack: true }
+    - get: configuration
+    - get: healthwatch-product
+      params: { skip_download: true }
+      trigger: true
+  - task: interpolate-env-creds
+    image: platform-automation-image
+    file: platform-automation-tasks/tasks/credhub-interpolate.yml
+    params:
+      CREDHUB_CLIENT: ((credhub-client))
+      CREDHUB_SECRET: ((credhub-secret))
+      CREDHUB_SERVER: ((credhub-server))
+      CREDHUB_CA_CERT: ((credhub-ca-cert.certificate))
+      PREFIX: '/concourse/sandbox'
+      SKIP_MISSING: false
+    input_mapping:
+      files: configuration
+    output_mapping:
+      interpolated-files: interpolated-configuration
+  - task: download-healthwatch-product
+    image: platform-automation-image
+    file: platform-automation-tasks/tasks/download-product.yml
+    params:
+      CONFIG_FILE: download-product/healthwatch.yml
+      SOURCE: gcs
+    input_mapping:
+      config: interpolated-configuration
+    output_mapping:
+      downloaded-product: healthwatch-tile
+  - task: upload-and-stage-healthwatch
+    image: platform-automation-image
+    file: platform-automation-tasks/tasks/upload-and-stage-product.yml
+    input_mapping:
+      product: healthwatch-tile
+      env: interpolated-configuration
+    params:
+      ENV_FILE: sandbox/env.yml
+  - task: upload-stemcell
+    image: platform-automation-image
+    file: platform-automation-tasks/tasks/upload-stemcell.yml
+    input_mapping:
+      stemcell: downloaded-stemcell
+      env: interpolated-configuration
+    params:
+      ENV_FILE: sandbox/env.yml
+
 - name: configure-pas
   serial: true
   serial_groups: [ install ]
@@ -331,6 +392,44 @@ jobs:
       CONFIG_FILE: sandbox/pas.yml
       ENV_FILE: sandbox/env.yml

+- name: configure-healthwatch
+  serial: true
+  serial_groups: [ install ]
+  plan:
+  - in_parallel:
+    - get: platform-automation-tasks
+      params: { unpack: true }
+    - get: platform-automation-image
+      params: { unpack: true }
+    - get: configuration
+    - get: healthwatch-product
+      params: { skip_download: true }
+      passed: [ upload-and-stage-healthwatch ]
+      trigger: true
+  - task: interpolate-env-creds
+    image: platform-automation-image
+    file: platform-automation-tasks/tasks/credhub-interpolate.yml
+    params:
+      CREDHUB_CLIENT: ((credhub-client))
+      CREDHUB_SECRET: ((credhub-secret))
+      CREDHUB_SERVER: ((credhub-server))
+      CREDHUB_CA_CERT: ((credhub-ca-cert.certificate))
+      PREFIX: '/concourse/sandbox'
+      SKIP_MISSING: false
+    input_mapping:
+      files: configuration
+    output_mapping:
+      interpolated-files: interpolated-configuration
+  - task: configure-healthwatch
+    image: platform-automation-image
+    file: platform-automation-tasks/tasks/configure-product.yml
+    input_mapping:
+      config: interpolated-configuration
+      env: interpolated-configuration
+    params:
+      CONFIG_FILE: sandbox/healthwatch.yml
+      ENV_FILE: sandbox/env.yml
+
 - name: apply-changes
   serial: true
   serial_groups: [ install ]
@@ -345,6 +444,10 @@ jobs:
       params: { skip_download: true }
       passed: [ configure-pas ]
       trigger: true
+    - get: healthwatch-product
+      params: { skip_download: true }
+      passed: [ configure-healthwatch ]
+      trigger: true
   - task: interpolate-env-creds
     image: platform-automation-image
     file: platform-automation-tasks/tasks/credhub-interpolate.yml

Aside

The two upload-and-stage jobs, and the two configure jobs are essentially duplicates of one another.

If this feels like a code smell, it's because it is. We just violated the DRY principle.

Imagine having to augment your pipeline further to configure additional tiles: redis, rabbitmq, mysql, pcf event alerts, etc..

How maintainable would this pipeline yaml be?

If you are looking for a better way to author your pipelines, have a look at this open source project.

Configure Healthwatch¶

Set the pipeline.

upload-and-stage-healtwatch passes, but configure-healthwatch fails for a simple reason: the healthwatch.yml product configuration file has not yet been authored.

Configure healthwatch manually per the instructions in the product documentation.
Use om staged-config to export the configuration for the product.

bash om staged-config -p p-healthwatch --include-placeholders > ../platform-config/sandbox/healthwatch.yml
In Ops Manager, revert the configuration changes.
Commit and push the configuration file to github.

Re-run configure-healthwatch, this time it passes.

Further, apply-changes will trigger and Ops Manager proceeds to install Healthwatch.

Alternative Configuration Generation Method¶

Another way to generate a tile configuration file is by way of the metadata present the tile itself.

Familiarize yourself with the usage of the om config-template command.
```
om config-template --help
```
This command generates a product configuration template given: a PivNet API token, a product slug (PivNet product identifier), a product version, and optionally, a product file glob.
Navigate to ~/workspace/artifacts

Run the om config-template command against the Pivotal Healthwatch tile:

om config-template \
    --output-directory . \
    --pivnet-api-token ${PIVNET_TOKEN} \
    --pivnet-product-slug p-healthwatch \
    --product-version 1.8.1

The command produces a directory structure resembling the following:

p-healthwatch
└── 1.8.1-build.1
    ├── default-vars.yml
    ├── errand-vars.yml
    ├── features
    │   ├── boshtasks-disable.yml
    │   ├── indicators_selector-active.yml
    │   ├── opsman-disable.yml
    │   ├── syslog_selector-active.yml
    │   └── syslog_selector-active_with_tls.yml
    ├── network
    │   ├── 2-az-configuration.yml
    │   └── 3-az-configuration.yml
    ├── optional
    │   ├── add-healthwatch-forwarder-foundation_name.yml
    │   └── add-healthwatch-forwarder-health_check_vm_type.yml
    ├── product.yml
    ├── required-vars.yml
    ├── resource
    │   ├── healthwatch-forwarder_additional_vm_extensions.yml
    │   ├── healthwatch-forwarder_elb_names.yml
    │   ├── healthwatch-forwarder_internet_connected.yml
    │   ├── healthwatch-forwarder_nsx_security_groups.yml
    │   ├── mysql_additional_vm_extensions.yml
    │   ├── mysql_elb_names.yml
    │   ├── mysql_internet_connected.yml
    │   ├── mysql_nsx_security_groups.yml
    │   ├── redis_additional_vm_extensions.yml
    │   ├── redis_elb_names.yml
    │   ├── redis_internet_connected.yml
    │   └── redis_nsx_security_groups.yml
    └── resource-vars.yml

5 directories, 26 files

Here is a breakdown of the main files that were just generated.

product.yml is the configuration template for the product.
The *-vars.yml files help interpolate many of the configuration options with default values.
The yaml files underneath features, network, and optional can be used to further modify the base configuration. These are meant to be applied to the product template using bosh ops files (see example below).
Additional values for parameterized fields in product.yml can be supplied with the --var option, or placed into a separate vars file.

The bosh interpolate command combines the above components to produce a tile configuration file.

The om tool provides the same interpolation capabilities via the om interpolate command.

Study the following command.

om interpolate --config ./product.yml \
    --ops-file ./features/boshtasks-disable.yml \
    --ops-file ./features/opsman-disable.yml \
    --vars-file ./default-vars.yml \
    --vars-file ./resource-vars.yml \
    --vars-file ./errand-vars.yml \
    --vars-file ./required-vars.yml

The command starts with the base configuration, disables two optionals features, and interpolates the configuration using defaults to produce the tile configuration.

Run the above command. It will complain that values for three variables are not supplied.

Create a file named my-vars.yml similar to the following.

network_name: pas
service_network_name: services
singleton_availability_zone: us-central1-a

Re-run the command adding your vars file in the mix, and pipe the output to a file named generated-healthwatch.yml
Study the generated output, and compare it with the healthwatch.yml file that you previously obtained via the om staged-config command.

The properties are cited in a different order but the settings are largely the same.

Verify Healthwatch is up and running¶

Visit the healthwatch dashboard at https://healthwatch.sys.sandbox.your-environment-name.pal4pe.com.
Log in using your UAA admin credentials.

Navigate the Healthwatch dashboard and familiarize yourself with its user interface.