Provision PAS¶
In this lab you build the automation to provision a Small Footprint PAS as your sandbox foundation and keep it up to date.
General Process¶
The procedure mirrors the steps we take when manually configuring a product:
- upload and stage the product, along with its stemcell
- configure the product
- apply changes
The fetch-products
pipeline runs daily and takes care of placing into the blobstore new versions of the PAS product as they are released on the Tanzu Network.
From a continuous deployment perspective, the availability of the new version in the blobstore should be the trigger to upload it, configure it, apply changes.
The successful completion of each job should trigger the next one in the sequence.
Platform Automation provides the tasks required: upload-and-stage-product
, upload-stemcell
, configure-product
, and apply-changes
.
Review these tasks in the Platform Automation task reference.
Download product configuration¶
Continue to use the download-product
task to fetch the product from gcs.
This task conveniently fetches both the product and its stemcell.
Navigate to ~/workspace/platform-config/download-product
, and
augment pas.yml
the same way you did opsman.yml
in the previous lab:
- Add the two gcs-specific properties needed to fetch the products from gcs.
- Review the download product configuration reference once more.
- The property
blobstore-stemcell-path
must also be supplied. This product has an accompanying stemcell and its location is not in the root path of the bucket.
Repeat the above configuration for healthwatch.yml
.
Commit the changes and push to GitHub.
Upload and Stage¶
Draft a new job named upload-and-stage-pas
.
It starts in the same manner that other jobs do:
- get platform automation image and tasks
- get the configuration repository
- interpolate the repository
- download the pas product and stemcell
Next, add two steps:
upload-and-stage-product
to upload the tileupload-stemcell
to upload the stemcell
Draft Job
---
jobs:
- name: upload-and-stage-pas
plan:
- in_parallel:
- get: platform-automation-tasks
params: { unpack: true }
- get: platform-automation-image
params: { unpack: true }
- get: configuration
- task: interpolate-env-creds
image: platform-automation-image
file: platform-automation-tasks/tasks/credhub-interpolate.yml
params:
CREDHUB_CLIENT: ((credhub-client))
CREDHUB_SECRET: ((credhub-secret))
CREDHUB_SERVER: ((credhub-server))
CREDHUB_CA_CERT: ((credhub-ca-cert.certificate))
PREFIX: '/concourse/sandbox'
SKIP_MISSING: false
input_mapping:
files: configuration
output_mapping:
interpolated-files: interpolated-configuration
- task: download-srt-product
image: platform-automation-image
file: platform-automation-tasks/tasks/download-product.yml
params:
CONFIG_FILE: download-product/pas.yml
SOURCE: gcs
input_mapping:
config: interpolated-configuration
output_mapping:
downloaded-product: srt-tile
- task: upload-and-stage-pas
image: platform-automation-image
file: platform-automation-tasks/tasks/upload-and-stage-product.yml
input_mapping:
product: srt-tile
env: interpolated-configuration
params:
ENV_FILE: sandbox/env.yml
- task: upload-stemcell
image: platform-automation-image
file: platform-automation-tasks/tasks/upload-stemcell.yml
input_mapping:
stemcell: downloaded-stemcell
env: interpolated-configuration
params:
ENV_FILE: sandbox/env.yml
Set the pipeline and trigger the job.
After the job passes, verify that PAS is staged.
- Navigate to
~/workspace/sandbox
- Use the
om staged-products
command to see what products are staged
Configure¶
Write another job, name it configure-pas
.
Use platform automation's configure-product
task.
An implementation of this job is presented below for reference.
Configure job
---
jobs:
- name: configure-pas
plan:
- in_parallel:
- get: platform-automation-tasks
params: { unpack: true }
- get: platform-automation-image
params: { unpack: true }
- get: configuration
- task: interpolate-env-creds
image: platform-automation-image
file: platform-automation-tasks/tasks/credhub-interpolate.yml
params:
CREDHUB_CLIENT: ((credhub-client))
CREDHUB_SECRET: ((credhub-secret))
CREDHUB_SERVER: ((credhub-server))
CREDHUB_CA_CERT: ((credhub-ca-cert.certificate))
PREFIX: '/concourse/sandbox'
SKIP_MISSING: false
input_mapping:
files: configuration
output_mapping:
interpolated-files: interpolated-configuration
- task: configure-pas
image: platform-automation-image
file: platform-automation-tasks/tasks/configure-product.yml
input_mapping:
config: interpolated-configuration
env: interpolated-configuration
params:
CONFIG_FILE: sandbox/pas.yml
ENV_FILE: sandbox/env.yml
The missing input is a configuration file.
A pragmatic way to generate this configuration file is to:
- Manually configure the tile in Ops Manager (see Configuring PAS).
- Use
om staged-config
to export the configuration to a yaml file. - With the configuration captured, in Ops Manager, revert the configuration changes you made manually.
- Parameterize the file as necessary to remove secrets.
- Commit and push the configuration file to github.
- Store the secrets in CredHub.
Steps 1-4 above have been done for you (for PAS version 2.8).
Inspect the pas configuration file below.
PAS Config
product-name: cf
product-properties:
.cloud_controller.allow_app_ssh_access:
value: true
.cloud_controller.apps_domain:
value: ((apps-domain))
.cloud_controller.default_app_memory:
value: 1024
.cloud_controller.default_app_ssh_access:
value: true
.cloud_controller.default_disk_quota_app:
value: 1024
.cloud_controller.default_quota_max_number_services:
value: 100
.cloud_controller.default_quota_memory_limit_mb:
value: 10240
.cloud_controller.enable_custom_buildpacks:
value: true
.cloud_controller.encrypt_key:
value: {}
.cloud_controller.max_disk_quota_app:
value: 2048
.cloud_controller.max_file_size:
value: 1024
.cloud_controller.max_package_size:
value: 2147483648
.cloud_controller.security_event_logging_enabled:
value: true
.cloud_controller.staging_timeout_in_seconds:
value: 900
.cloud_controller.system_domain:
value: ((system-domain))
.diego_brain.starting_container_count_maximum:
value: 200
.ha_proxy.skip_cert_verify:
value: false
.mysql.cli_history:
value: true
.mysql.max_connections:
value: 3500
.mysql.prevent_node_auto_rejoin:
value: false
.mysql.remote_admin_access:
value: false
.mysql_monitor.poll_frequency:
value: 30
.mysql_monitor.recipient_email:
value: esuez@pivotal.io
.mysql_monitor.write_read_delay:
value: 20
.mysql_proxy.enable_inactive_mysql_port:
value: false
.mysql_proxy.shutdown_delay:
value: 30
.mysql_proxy.startup_delay:
value: 0
.nfs_server.blobstore_internal_access_rules:
value: allow 10.0.0.0/8;,allow 172.16.0.0/12;,allow 192.168.0.0/16;
.properties.autoscale_api_disable_connection_pooling:
value: false
.properties.autoscale_api_instance_count:
value: 1
.properties.autoscale_enable_notifications:
value: true
.properties.autoscale_enable_verbose_logging:
value: false
.properties.autoscale_instance_count:
value: 3
.properties.autoscale_metric_bucket_count:
value: 120
.properties.autoscale_scaling_interval_in_seconds:
value: 35
.properties.cc_api_rate_limit:
selected_option: disable
value: disable
.properties.cc_logging_level:
value: info
.properties.ccdb_connection_validation_timeout:
value: 3600
.properties.ccdb_read_timeout:
value: 3600
.properties.cf_networking_database_connection_timeout:
value: 120
.properties.cf_networking_enable_space_developer_self_service:
value: false
.properties.cf_networking_internal_domains:
value:
- name: apps.internal
.properties.cloud_controller_completed_tasks_cutoff_age_in_days:
value: 31
.properties.cloud_controller_default_health_check_timeout:
value: 60
.properties.cloud_controller_temporary_disable_deployments:
value: false
.properties.container_networking:
selected_option: enable
value: enable
.properties.container_networking_interface_plugin:
selected_option: silk
value: silk
.properties.container_networking_interface_plugin.silk.enable_log_traffic:
value: false
.properties.container_networking_interface_plugin.silk.enable_policy_enforcement:
value: true
.properties.container_networking_interface_plugin.silk.iptables_accepted_udp_logs_per_sec:
value: 100
.properties.container_networking_interface_plugin.silk.iptables_denied_logs_per_sec:
value: 1
.properties.container_networking_interface_plugin.silk.network_mtu:
value: 1454
.properties.container_networking_interface_plugin.silk.vtep_port:
value: 4789
.properties.credhub_database:
selected_option: internal_mysql
value: internal_mysql
.properties.credhub_hsm_provider_client_certificate:
value: {}
.properties.credhub_hsm_provider_partition_password:
value: {}
.properties.credhub_internal_provider_keys:
value:
- key:
secret: ((credhub-key))
name: default
primary: true
.properties.diego_database_max_open_connections:
value: 100
.properties.diego_log_timestamp_format:
selected_option: rfc3339
value: rfc3339
.properties.enable_garden_containerd_mode:
value: true
.properties.enable_log_cache_syslog_ingestion:
value: false
.properties.enable_smb_volume_driver:
value: true
.properties.enable_tls_to_internal_pxc:
value: false
.properties.enable_v1_firehose:
value: true
.properties.enable_v2_firehose:
value: true
.properties.experimental_dynamic_egress_enforcement:
value: false
.properties.garden_disk_cleanup:
selected_option: reserved
value: reserved
.properties.garden_disk_cleanup.reserved.reserved_space_for_other_jobs_in_mb:
value: 15360
.properties.gorouter_ssl_ciphers:
value: ECDHE-RSA-AES128-GCM-SHA256:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
.properties.haproxy_client_cert_validation:
selected_option: none
value: none
.properties.haproxy_forward_tls:
selected_option: disable
value: disable
.properties.haproxy_hsts_support:
selected_option: disable
value: disable
.properties.haproxy_max_buffer_size:
value: 16384
.properties.haproxy_ssl_ciphers:
value: DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384
.properties.istio:
selected_option: disable
value: disable
.properties.locket_database_max_open_connections:
value: 200
.properties.log_cache_max_per_source:
value: 100000
.properties.metric_registrar_blacklisted_tags:
value: deployment,job,index,id
.properties.metric_registrar_enabled:
value: true
.properties.metric_registrar_scrape_interval_in_seconds:
value: 35
.properties.mysql_activity_logging:
selected_option: enable
value: enable
.properties.mysql_activity_logging.enable.audit_logging_events:
value: connect,query
.properties.networking_poe_ssl_certs:
value:
- certificate:
cert_pem: ((tls-cert.certificate))
private_key_pem: ((tls-cert.private_key))
name: default
.properties.networkpolicyserver_database_max_open_connections:
value: 200
.properties.networkpolicyserverinternal_database_max_open_connections:
value: 200
.properties.nfs_volume_driver:
selected_option: disable
value: disable
.properties.push_apps_manager_app_poll_interval:
value: 10
.properties.push_apps_manager_currency_lookup:
value: '{ "usd": "$", "eur": "€" }'
.properties.push_apps_manager_display_plan_prices:
value: false
.properties.push_apps_manager_enable_invitations:
value: true
.properties.push_apps_manager_nav_links:
value:
- href: https://docs.pivotal.io/pivotalcf/2-8/pas/intro.html
name: Docs
- href: /tools
name: Tools
.properties.push_apps_manager_poll_interval:
value: 30
.properties.push_usage_service_cutoff_age_in_days:
value: 365
.properties.route_integrity:
selected_option: tls_verify
value: tls_verify
.properties.route_services:
selected_option: enable
value: enable
.properties.route_services.enable.ignore_ssl_cert_verification:
value: false
.properties.route_services.enable.internal_lookup:
value: false
.properties.router_backend_max_conn:
value: 500
.properties.router_balancing_algorithm:
selected_option: round_robin
value: round-robin
.properties.router_client_cert_validation:
selected_option: request
value: request
.properties.router_enable_proxy:
value: false
.properties.router_keepalive_connections:
selected_option: enable
value: enable
.properties.routing_disable_http:
value: false
.properties.routing_log_client_ips:
selected_option: log_client_ips
value: log_client_ips
.properties.routing_minimum_tls_version:
selected_option: tls_v1_2
value: tls_v1_2
.properties.routing_tls_termination:
selected_option: load_balancer
value: load_balancer
.properties.saml_signature_algorithm:
value: SHA256
.properties.secure_service_instance_credentials:
value: false
.properties.security_acknowledgement:
value: X
.properties.silk_database_max_open_connections:
value: 200
.properties.smoke_tests:
selected_option: on_demand
value: on_demand
.properties.smtp_auth_mechanism:
value: plain
.properties.smtp_credentials:
value: {}
.properties.smtp_enable_starttls_auto:
value: false
.properties.syslog_drop_debug:
value: true
.properties.syslog_tls:
selected_option: disabled
value: disabled
.properties.syslog_use_tcp_for_file_forwarding_local_transport:
value: false
.properties.system_blobstore:
selected_option: internal
value: internal
.properties.system_blobstore_backup_level:
selected_option: all
value: all
.properties.system_blobstore_ccdroplet_max_staged_droplets_stored:
value: 2
.properties.system_blobstore_ccpackage_max_valid_packages_stored:
value: 2
.properties.system_database:
selected_option: internal_pxc
value: internal_pxc
.properties.tcp_routing:
selected_option: disable
value: disable
.properties.uaa:
selected_option: internal
value: internal
.properties.uaa.internal.password_expires_after_months:
value: 0
.properties.uaa.internal.password_max_retry:
value: 5
.properties.uaa.internal.password_min_length:
value: 0
.properties.uaa.internal.password_min_lowercase:
value: 0
.properties.uaa.internal.password_min_numeric:
value: 0
.properties.uaa.internal.password_min_special:
value: 0
.properties.uaa.internal.password_min_uppercase:
value: 0
.properties.uaa_database:
selected_option: internal_mysql
value: internal_mysql
.properties.uaa_session_cookie_max_age:
value: 28800
.properties.uaa_session_idle_timeout:
value: 1800
.router.disable_insecure_cookies:
value: false
.router.drain_timeout:
value: 900
.router.drain_wait:
value: 20
.router.enable_isolated_routing:
value: false
.router.enable_write_access_logs:
value: true
.router.enable_zipkin:
value: true
.router.frontend_idle_timeout:
value: 900
.router.lb_healthy_threshold:
value: 20
.router.request_timeout_in_seconds:
value: 900
.uaa.apps_manager_access_token_lifetime:
value: 3600
.uaa.cf_cli_access_token_lifetime:
value: 7200
.uaa.cf_cli_refresh_token_lifetime:
value: 1209600
.uaa.customize_password_label:
value: Password
.uaa.customize_username_label:
value: Email
.uaa.enable_uri_encoding_compatibility_mode:
value: true
.uaa.proxy_ips_regex:
value: 10\.\d{1,3}\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3}|169\.254\.\d{1,3}\.\d{1,3}|127\.\d{1,3}\.\d{1,3}\.\d{1,3}|172\.1[6-9]{1}\.\d{1,3}\.\d{1,3}|172\.2[0-9]{1}\.\d{1,3}\.\d{1,3}|172\.3[0-1]{1}\.\d{1,3}\.\d{1,3}
.uaa.service_provider_key_credentials:
value:
cert_pem: ((tls-cert.certificate))
private_key_pem: ((tls-cert.private_key))
.uaa.service_provider_key_password:
value: {}
network-properties:
network:
name: pas
other_availability_zones:
- name: us-central1-c
- name: us-central1-b
- name: us-central1-a
singleton_availability_zone:
name: us-central1-c
resource-config:
backup_restore:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: automatic
internet_connected: true
persistent_disk:
size_mb: automatic
swap_as_percent_of_memory_size: automatic
blobstore:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: automatic
internet_connected: true
persistent_disk:
size_mb: automatic
swap_as_percent_of_memory_size: automatic
compute:
max_in_flight: 4%
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: 3
internet_connected: true
swap_as_percent_of_memory_size: automatic
control:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names:
- tcp:sandbox-ssh-lb
instance_type:
id: automatic
instances: automatic
internet_connected: true
swap_as_percent_of_memory_size: automatic
database:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: automatic
internet_connected: true
persistent_disk:
size_mb: automatic
swap_as_percent_of_memory_size: automatic
ha_proxy:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: automatic
internet_connected: true
swap_as_percent_of_memory_size: automatic
istio_control:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: 0
internet_connected: true
swap_as_percent_of_memory_size: automatic
istio_router:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: 0
internet_connected: true
swap_as_percent_of_memory_size: automatic
mysql_monitor:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: automatic
internet_connected: true
swap_as_percent_of_memory_size: automatic
route_syncer:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: 0
internet_connected: true
swap_as_percent_of_memory_size: automatic
router:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names:
- tcp:sandbox-websocket-lb
- http:sandbox-http-lb
instance_type:
id: automatic
instances: automatic
internet_connected: true
swap_as_percent_of_memory_size: automatic
tcp_router:
max_in_flight: 1
additional_networks: []
additional_vm_extensions: []
elb_names: []
instance_type:
id: automatic
instances: 0
internet_connected: true
persistent_disk:
size_mb: automatic
swap_as_percent_of_memory_size: automatic
errand-config:
deploy-autoscaler:
post-deploy-state: false
deploy-notifications:
post-deploy-state: false
deploy-notifications-ui:
post-deploy-state: false
metric_registrar_smoke_test:
post-deploy-state: false
nfsbrokerpush:
post-deploy-state: false
push-apps-manager:
post-deploy-state: true
push-usage-service:
post-deploy-state: false
rotate_cc_database_key:
post-deploy-state: false
smbbrokerpush:
post-deploy-state: false
smoke_tests:
post-deploy-state: true
test-autoscaling:
post-deploy-state: false
Copy the file to ~/workspace/platform-config/sandbox/pas.yml
.
Add to git, commit and push to GitHub.
CredHub¶
Draft a credentials.yml
file to set three of the variables from the above pas.yml
file.
---
credentials:
- name: /concourse/sandbox/apps-domain
type: value
value: apps.sandbox.your-environment-name-goes-here.pal4pe.com
- name: /concourse/sandbox/system-domain
type: value
value: sys.sandbox.your-environment-name-goes-here.pal4pe.com
- name: /concourse/sandbox/tls-cert
type: certificate
value:
certificate: |
paste the contents of the terraform output
variable named ssl_certificate in here in my place
be sure it's indented two spaces
private_key: |
paste the contents of the terraform output
variable named ssl_private_key in here in my place
be sure it's indented two spaces
To display the pas terraform output, use:
yq read --colors --prettyPrint pas-terraform-output.yml
Generate the fourth item, /concourse/sandbox/credhub-key
using the credhub generate
command.
The item should be of type password and should at least be 20 characters long.
Test
Set the pipeline, run the job, and troubleshoot as necessary to make it pass.
Visit the Ops Manager and spot-check the tile configuration and satisfy yourself that PAS is configured.
Apply Changes¶
Add a job to apply changes. Name it apply-changes
.
Apply Changes Job
---
jobs:
- name: apply-changes
plan:
- in_parallel:
- get: platform-automation-tasks
params: { unpack: true }
- get: platform-automation-image
params: { unpack: true }
- get: configuration
- task: interpolate-env-creds
image: platform-automation-image
file: platform-automation-tasks/tasks/credhub-interpolate.yml
params:
CREDHUB_CLIENT: ((credhub-client))
CREDHUB_SECRET: ((credhub-secret))
CREDHUB_SERVER: ((credhub-server))
CREDHUB_CA_CERT: ((credhub-ca-cert.certificate))
PREFIX: '/concourse/sandbox'
SKIP_MISSING: false
input_mapping:
files: configuration
output_mapping:
interpolated-files: interpolated-configuration
- task: apply-changes
image: platform-automation-image
file: platform-automation-tasks/tasks/apply-changes.yml
input_mapping:
env: interpolated-configuration
params:
ENV_FILE: sandbox/env.yml
Update the pipeline, and trigger the job.
The PAS installation takes approximately 45 minutes.
After the installation finishes, retrieve the UAA Admin Credentials from your Ops Manager:
- Drill down into the PAS tile
- select the Credentials tab
- Click the link to the Credential under UAA -> Admin Credentials
Log in to the apps manager at https://apps.sys.sandbox.your-environment-name.pal4pe.com/
Job dependencies and triggers¶
There is a clear relationship between the three jobs: upload-and-stage-pas, configure-pas, and apply-changes. How do we ensure that they run automatically and in succession?
Job dependencies are modeled in Concourse via a common resource. That common resource in our case is the PAS product.
This product is not modeled as a resource in our pipeline. It is fetched with the download-product task.
-
Model the pas product as a resource in your pipeline.
Add the following resource:
- name: pas-product type: gcs source: bucket: ((bucket)) json_key: ((json_key)) regexp: \[.*\]srt-(.*).pivotal
-
The
trigger
parameter in Concourse is always associated with a resource in aget
step.In the job
upload-and-stage-pas
, add a step to get thepas-product
. Place this step inside thein_parallel
step, as illustrated by this diff:diff @@ -233,6 +240,9 @@ jobs: - get: platform-automation-image params: { unpack: true } - get: configuration + - get: pas-product + params: { skip_download: true } + trigger: true - task: interpolate-env-creds image: platform-automation-image file: platform-automation-tasks/tasks/credhub-interpolate.yml
The
skip_download
option ensures that the product is not downloaded, since we download it via other means.The
trigger
option ensures the job will be triggered each time a new version of the product appears in the blobstore. -
The next job in the sequence is
configure-pas
. Make a similar addition, but with an additional line that conditions the triggering of the job onupload-and-stage-pas
passing:@@ -282,6 +292,10 @@ jobs: - get: platform-automation-image params: { unpack: true } - get: configuration + - get: pas-product + params: { skip_download: true } + passed: [ upload-and-stage-pas ] + trigger: true - task: interpolate-env-creds image: platform-automation-image file: platform-automation-tasks/tasks/credhub-interpolate.yml
-
Make the same change for the subsequent job,
apply-changes
, but conditioned onconfigure-pas
passing:@@ -314,6 +328,10 @@ jobs: - get: platform-automation-image params: { unpack: true } - get: configuration + - get: pas-product + params: { skip_download: true } + passed: [ configure-pas ] + trigger: true - task: interpolate-env-creds image: platform-automation-image file: platform-automation-tasks/tasks/credhub-interpolate.yml
Set the pipeline.
The Concourse dashboard will reflect the dependencies between the three jobs.
Another trigger¶
Consider triggering the upgrade-opsman
job after a successful run of export-installation
.
How would you modify your pipeline?
Serial Jobs¶
The pipeline has another problem: it does not prevent to jobs such as install-opsman
and upgrade-opsman
from taking place at the same time.
Uploading and staging products, configuring products, and applying changes should likewise not run concurrently.
For each job in your pipeline, insert the following below the job name (above plan
):
serial: true
serial_groups: [ install ]