Skip to content

Job dependencies

So far, all of the pipelines we have looked at have been single-job pipelines. We have been studying the specifics of jobs and tasks.

In this lab, we look at the mechanism to configure job dependencies, the glue that binds jobs together into a pipeline.

How it works

In Concourse, a pipeline is represented by jobs and their dependencies on other jobs.

When defining job dependencies, the main concept is that those dependencies are expressed via common resources:

  • One job's output resource may become the input of another job.
  • Or perhaps two jobs use the same input resource but the two must run in sequence.

When configuring a job, we use the passed field to indicate one or more jobs that must have passed (succeeded) before this particular job can run.

But more interestingly, the passed field is specified on the resource that is common between the two jobs.

The two jobs are linked via the resource, and each pair of builds uses the same version of the resource.

For example, if a compile job produces version 1.1 of an artifact, Concourse arranges for a subsequent deploy job to use the same artifact, version 1.1. The next time compile runs, the new version, say 1.2, is again used by subsequent jobs in the pipeline.

Exercise

Make a copy of the git-triggered.yml pipeline. Name the new file job-deps.yml. A job uses zero or more resources as inputs and produces zero or more output resources.

The pipeline already defines a resource and a job.

  • Rename job to job-one
  • Add a second job named job-next that displays (cat) the contents of the README file from the repository. If your repository does not contain a README file, then please create and add such a file.

Set the pipeline. View the pipeline in the Concourse dashboard. Two jobs will be rendered.

  • In job-next, use the passed field to define a dependency on job-one. This must be specified on the repository resource in the get step.
  • Now re-fly the pipeline and note how the pipeline is now rendered to specifically show a dependency between the two jobs.
  • Add a trigger: true declaration alongside the passed declaration, re-fly the pipeline, and observe the dashed line turn to a solid line. This means that when job-one passes, job-next will automatically run afterwards.

Go ahead and trigger the first job, either manually or via a commit, and watch the two jobs run serially.