Build and deploy

Last updated: 1 March 2016

The IBM® Bluemix® DevOps Services Build & Deploy feature, also known as the pipeline, automates the continuous deployment of your projects. In a project's pipeline, sequences of stages retrieve input and run jobs, such as builds, tests, and deployments.

Stages

Stages organize input and jobs as your code is built, deployed, and tested. Stages accept input from either source control repositories or build jobs in other stages. When you create your first stage, the default settings are set for you on the INPUT tab.

Input from the stage is passed to its jobs, and each job runs in a clean container. The jobs in a stage can't pass artifacts to each other. However, stage environment properties can be used in all jobs. For example, you might define a TEST_URL property that passes a URL to deploy and test jobs in a stage. The deploy job deploys to that URL, and the test job tests the running app at that URL.

To learn how to add a stage, see Adding a stage.

By default in a stage, builds and deployments are triggered automatically every time changes are delivered to a project's source control repository. Stages and jobs run serially; they enable flow control for your work. For example, you might place a test stage before a deployment stage. If the tests in the test stage fail, the deployment stage won't run.

You might want tighter control of a specific stage. If you do not want a stage to run every time a change occurs at its input, you can disable the capability. On the INPUT tab, in the Stage Trigger section, click Run jobs only when this stage is run manually.

The INPUT tab

Jobs

A job is an execution unit within a stage. A stage can contain multiple jobs, and the jobs in a stage run sequentially. By default, if a job fails, subsequent jobs in the stage do not run.

Build and test jobs within a stage

Jobs run in discrete working directories within Docker containers that are created for each pipeline run. Before a job is run, its working directory is populated with input that is defined at the stage level. For example, you might have a stage that contains a test job and a deploy job. If you install dependencies on one job, they are not available to the other job. However, if you make the dependencies available in the stage's input, they are available to both jobs.

After a job runs, the container that was created for it is discarded. The results of a job run can persist, but the environment in which it ran does not.

Note: Jobs can run for up to 60 minutes. When jobs exceed that limit, they fail. If a job is exceeding the limit, break it into multiple jobs. For example, if a job performs three tasks, you might break it into three jobs: one for each task.

To learn how to add a job to a stage, see Adding a job to a stage.

Build jobs

Build jobs compile your project in preparation for deployment. They generate artifacts that can be sent to a build archive directory, although by default, the artifacts are placed in the project's root directory.

Jobs that take input from build jobs must reference build artifacts in the same structure that they were created in. For example, if a build job archives build artifacts to an output directory, a deploy script would refer to the output directory rather than the project root directory to deploy the compiled project.

Note: If you select the Simple builder type for a build job, you skip the build process. In that case, your code is not compiled, but is sent to the deployment stage as is. To both build and deploy, select a builder type other than Simple.

Environment properties for build scripts

You can include environment properties within a build job's build shell commands. The properties provide access to information about the job's execution environment. For more information, see Environment properties and resources for the Build & Deploy pipeline.

Deploy jobs

Deploy jobs upload your project to Bluemix as an app and are accessible from a URL. After a project is deployed, you can find the deployed app on your Bluemix Dashboard. You can configure the build and deploy jobs as separate stages or add them to the same stage to run automatically.

Deploy jobs can deploy new apps or update existing apps. Even if you first deployed an app by using another method, such as the Cloud Foundry command line interface or the run bar in the Web IDE, you can update the app by using a deploy job. To update an app, in the deploy job, use that app's name.

Environment properties for deployment scripts

You can include environment properties within a deploy job's deployment script. These properties provide access to information about the job's execution environment. For more information, see Environment properties and resources for the Build & Deploy pipeline.

Test jobs

If you want to require that certain conditions are met, include test jobs before or after your build and deploy jobs. Test jobs are highly customizable. For example, you might run tests on your project code and a deployed instance of your app.

Manifest files

Manifest files, which are named manifest.yml and stored in a project's root directory, control how your project is deployed to Bluemix. For information about creating manifest files for a project, see the Bluemix documentation about application manifests. To integrate with Bluemix, your project must have a manifest file in its root directory. However, you are not required to deploy based on the information in the file.

In the pipeline, you can specify everything that a manifest file can by using cf push command arguments. The cf push command arguments are helpful in projects that have multiple deployment targets. If multiple deploy jobs all try to use the route that is specified in the project manifest file, a conflict occurs.

To avoid conflicts, you can specify a route by using cf push followed by the host name argument, -n, and a route name. By modifying the deployment script for individual stages, you can avoid route conflicts when you deploy to multiple targets.

To use the cf push command arguments, open the configuration settings for a deploy job and modify the Deploy Script field. For more information, see the Cloud Foundry Push documentation.

An example pipeline

A simple pipeline might contain three stages:

  1. A build stage that compiles, runs, or compiles and runs build processes on an app.
  2. A test stage that deploys a instance of the app and then runs tests on it.
  3. A prod stage that deploys a production instance of the tested app.

A conceptual diagram of stages and jobs in a pipeline

A conceptual model of a three-stage pipeline

Stages receive input from repositories and build jobs. The jobs in a stage run sequentially and independently of each other. In the example pipeline, the stages run sequentially even though the test and prod stages receive input from the build stage.

Adding a stage

  1. On the Build & Deploy Pipeline page, click ADD STAGE. The Stage Configuration page opens.
  2. Configure the stage.
    1. On the INPUT tab, select an input for the stage.
    2. On the JOBS tab, add and configure at least one job. The first stage usually has at least a build job.
  3. Click SAVE.

Adding a stage to a pipeline

Adding a job to a stage

  1. On the stage, click the Stage Configuration icon, and then click Configure Stage.
  2. Click the JOBS tab.
  3. Click ADD JOB. Select the type of job to add.
  4. Configure the job.
  5. Click SAVE.

Adding a job to a stage

Running a stage

You can manually run a stage by clicking the Run Stage icon on the Build & Deploy Pipeline page.

Clicking the Run Stage icon on a stage

You can also request on-demand builds and deployments from the build history page in one of two ways:

  • Drag a build to the box that is under a configured stage.
  • Next to a build, click the Send to icon and then select a space to deploy to. The Execute stage with this build icon

To cancel a running stage, on the stage, click View logs and history. In the list on the left, click the running job's number and then click CANCEL. You can also cancel jobs individually by clicking a job and then clicking CANCEL, or by clicking the Stop icon next to a job on its stage.

Deploying an app

A properly configured deploy job deploys your app to your target whenever the job is run. To manually run a deploy job, click the Run Stage icon of the stage that the job is in.

Input revisions

When you run a stage manually, or if it runs because the stage before it is completed, the running stage selects its input revision. Usually, the input revision is a build number. To select the input revision, the stage follows this process:

  1. If a specific revision is selected, use it.
  2. If a specific revision is not specified, search previous stages until a stage is found that uses the same input. Find and use the last successfully run revision of that input.
  3. If a specific revision is not specified and no other stages use the specified source as input, use the latest revision of the input.

Tip: You can deploy a previous build. On the stage that contains the build, click View logs and history. On the page that opens, select the build. Click SEND TO, and select a target.

Adding services to apps

You can add services to your apps and manage those services from your Bluemix Dashboard or the Cloud Foundry command line interface (CLI). You can also issue Cloud Foundry CLI commands in scripts for DevOps Services pipeline jobs. For example, you can add a service to an app in the script of a deploy job. For more information about adding services, see see Adding a service to your application.

Viewing logs

You can view the logs for jobs and view stages as they are running on the Stage History page.

To view a job's log, click the job. Alternatively, on a stage, click View logs and history.

To view the runtime log, click View runtime log.

Areas in a stage tile that can be clicked to open relevant logs

In addition to job logs, you can view unit test results, generated artifacts, and code changes for any build job.

You can also run, cancel, or configure a stage from the Stage History page. At the top of the page, click RUN to run a stage or CONFIGURE to configure a stage. While a stage is running, you can cancel it by clicking the run number and then clicking CANCEL.

Clicking a stage run number to select it on the Stage History page

Controlling access

You can restrict who is able to run stages or modify a pipeline. To do so, go to the Pipeline Settings page, which you can reach by clicking the Stage Configuration icon on the Pipeline: All Stages page. The pipeline settings gear icon

Environment properties and resources

You can use environment properties and pre-installed resources to interact with the Build & Deploy pipeline. For example, you might incorporate them into a job script or test command. For more information, see Environment properties and resources for the Build & Deploy pipeline.

You can add your own environment properties to a stage from its ENVIRONMENT PROPERTIES tab. Environment properties are available to every job in a stage.

You can add four types of properties from the Environment Properties tab:

  • Text: A property key with a single-line value.
  • Text Area: A property key with a multi-line value.
  • Secure: A property key with a single-line value. The value is displayed as asterisks.
  • Properties: A file in the project's repository. This file can contain multiple properties. Each property must be on its own line. To separate key-value pairs, use the equals sign (=).

Extending the capabilities of your pipeline

You can extend the capabilities of your Build & Deploy pipeline by configuring your jobs to use supported services. For example, test jobs can run static code scans and build jobs can globalize strings.

For more information on extending pipeline capabilities, see Extending the capabilities of your Build & Deploy pipeline.